ADAF API

API for working with the ADAF type.

Import this module like this:

from sympathy.api import adaf

The ADAF structure

An ADAF consists of three parts: meta data, results, and timeseries.

Meta data contains information about the data in the ADAF. Stuff like when, where and how it was measured or what parameter values were used to generated it. A general guideline is that the meta data should be enough to (at least in theory) reproduce the data in the ADAF.

Results and timeseries contain the actual data. Results are always scalar whereas the timeseries can have any number of values.

Timeseries can come in several systems and each system can contain several rasters. Each raster in turn has one basis and any number of timeseries. So for example an experiment where some signals are sampled at 100Hz and others are sampled only once per second would have (at least) two rasters. A basis doesn’t have to be uniform but can have samples only every now and then.

Accessing the data

The adaf.File object has two members called meta and res containing the meta data and results respectively. Both are Group objects.

Example of how to use meta (res is completely analogous):

>>> from sympathy.api import adaf
>>> import numpy as np
>>> f = adaf.File()
>>> f.meta.create_column(
...     'Duration', np.array([3]), {'unit': 'h'})
>>> f.meta.create_column(
...     'Relative humidity', np.array([63]), {'unit': '%'})
>>> print(f.meta['Duration'].value())
[3]
>>> print(f.meta['Duration'].attr['unit'])

Timeseries can be accessed in two different ways. Either via the member sys or via the member ts. Using sys is generally recommended since ts handles multiple timeseries with the same name across different rasters poorly.

Example of how to use sys:

>>> f.sys.create('Measurement system')
>>> f.sys['Measurement system'].create('Raster1')
>>> f.sys['Measurement system']['Raster1'].create_basis(
...     np.array([0.01, 0.02, 0.03]),
...     {'unit': 's'})
>>> f.sys['Measurement system']['Raster1'].create_signal(
...     'Amount of stuff',
...     np.array([1, 2, 3]),
...     {'unit': 'kg'})
>>> f.sys['Measurement system']['Raster1'].create_signal(
...     'Process status',
...     np.array(['a', 'b', 'c']),
...     {'description': 'a=awesome, b=bad, c=critical'})
>>> f.sys.keys()
['Measurement system']
>>> f.sys['Measurement system'].keys()
['Raster1']
>>> f.sys['Measurement system']['Raster1'].keys()
['Signal1', 'Signal2']
>>> print(f.sys['Measurement system']['Raster1']['Signal1'].t)
[ 0.01  0.02  0.03]
>>> print(f.sys['Measurement system']['Raster1']['Signal1'].y)
[1 2 3]
>>> print(f.sys['Measurement system']['Raster1']['Signal1'].unit())
kg

The rasters are of type RasterN.

Timeseries and raster attributes

Attributes can be added to timeseries, rasters, etc. Attributes work the same way as they do for Tables.

Name restrictions

The naming restrictions for Tables apply to ADAFs too.

Class adaf.File

class sympathy.typeutils.adaf.File(fileobj=None, data=None, filename=None, mode='r', scheme='hdf5', source=None, managed=False, import_links=False)

File represents the top level of the ADAF format.

Any node port with the ADAF type will produce an object of this kind.

Use the members meta, res and sys to access the data. See Accessing the data for an example.

hjoin(other_adaf)

HJoin ADAF with other ADAF. See also node HJoin ADAF.

classmethod icon()

Return full path to svg icon.

init()

Perform any initialization, such as, defining local fields.

names(kind=None, **kwargs)

The names that can be automatically adjusted from an adaf.

kind should be one of ‘cols’ (all column names from meta, res, and all rasters), ‘ts’ (all signal names from all rasters), or ‘rasters’ (all raster names including system names).

package_id()

Get the package identifier string.

set_source_id(source_id)

Set the source identifier string.

source(other_adaf, shallow=False)

Use the data from other_adaf as source for this file.

source_id()

Get the source identifier string. If the source identifier has not been set, it will default to an empty string.

sync()

Synchronize data fields that are kept in memory against self._data.

Called before data is written to disk and must be re-implemented by subclasses that define custom storage fields.

timestamp()

Get the time string.

types(kind=None, **kwargs)

Return types associated with names.

user_id()

Get the user identifier string.

version()

Return the version as a string. This is useful when loading existing files from disk.

New in version 1.2.5.

classmethod viewer()

Return viewer class, which must be a subclass of sympathy.api.typeutil.ViewerBase

vjoin(other_adafs, input_index, output_index, fill, minimum_increment, include_rasters=False, use_reference_time=False)

VJoin ADAF with other ADAF. See also node VJoin ADAF.

vsplit(other_adafs, input_index, remove_fill, require_index, include_rasters=False)

VSplit ADAF, appending the resulting ADAFs onto other_adafs list.

Class Group

class sympathy.typeutils.adaf.Group(data, name=None)

Class representing a group of scalars. Used for meta and res. Supports dictionary-like __getitem__ interface for data retrieval. To write a column use create_column().

create_column(name, data, attributes=None)

Create and add a new, named, data column to the group. Return created column.

delete_column(name)

Delete named data column from the group.

from_table(table)

Set the content to that of table. This operation replaces the columns of the group with the content of the table.

get_attributes()

Return a dictionary of all attributes on this group.

hjoin(other_group)

HJoin Group with other Group.

items()

Return the current group items.

keys()

Return the current group keys.

number_of_rows()

Return the number of rows in the Group.

New in version 1.2.6.

rename_column(old_name, new_name)

Rename the named data column.

set_attribute(key, value)

Add an attribute to this Group. If the attribute already exists it will be updated.

to_table()

Export table containing the data.

values()

Return the current group values.

vjoin(other_groups, input_index, output_index, fill, minimum_increment)

VJoin Group with other Group.

Class RasterN

class sympathy.typeutils.adaf.RasterN(record, system, name)

Represents a raster with a single time basis and any number of timeseries columns.

attr

Raster level attributes.

basis_column()

Return the time basis for this raster. The returned object is of type Column.

create_basis(data, attributes=None, **kwargs)

Create and add a basis. The contents of the dictionary attributes are added as attributes on the signal.

Changed in version 1.2.1: Added the attributes parameter. Using kwargs to set attributes is now considered obsolete and will result in a warning.

create_signal(name, data, attributes=None, **kwargs)

Create and add a new signal. The contents of the dictionary attributes are added as attributes on the signal.

Changed in version 1.2.1: Added the attributes parameter. Using kwargs to set attributes is now considered obsolete and will result in a warning.

delete_signal(name)

Delete named signal.

from_table(table, basis_name=None, use_basis_name=True)

Set the content to that of table.

This operation replaces the signals of the raster with the content of the table.

When basis_name is used, that column will be used as basis, otherwise it will not be defined after this operation and needs to be set using create_basis.

items()

Return a list of tuples, each with the name of a timeseries and the corresponding Timeseries object.

keys()

Return a list of names of the timeseries.

number_of_columns()

Return the number of signals including the basis.

number_of_rows()

Return the number of rows (length of a time basis/timeseries) in the raster.

to_table(basis_name=None)

Export all timeseries as a Table.

When basis_name is given, the basis will be included in the table and given the basis_name, otherwise it will not be included in the table.

update_basis(other_raster)

Updates the basis from the basis of RasterN other_raster.

New in version 1.4.3.

update_signal(signal_name, other_raster, other_name=None)

Updates a signal from a signal in another raster.

The signal other_name from other_raster will be copied into signal_name. If signal_name already exists it will be replaced.

If other_name is not specified, signal_name will be used instead.

New in version 1.4.3.

values()

Return a list of all signal items.

vjoin(other_groups, input_index, output_index, fill, minimum_increment)

VJoin Group with other Group.

Class Timeseries

class sympathy.typeutils.adaf.Timeseries(node, data, name)

Class representing a timeseries. The values in the timeseries can be accessed as a numpy array via the member y. The timeseries is also connected to a time basis whose values can be accessed as a numpy array via the property t.

The timeseries can also have any number of attributes. The methods unit() and description() retrieve those two attributes. To get all attributes use the method get_attributes().

basis()

Return the timeseries data basis as a Column.

description()

Return the description attribute or an empty string if it is not set.

dtype

dtype of timeseries.

get_attributes()

Return all attributes (including unit and description).

raster()

Return the associated raster.

raster_name()

Return the name of the associated raster.

signal()

Return the timeseries data signal as a Column.

signal_name()

Return the name of the timeseries data signal.

system_name()

Return the name of the associated system.

t

Time basis values as a numpy array.

unit()

Return the unit attribute or an empty string if it is not set.

y

Timeseries values as a numpy array.

Class Column

class sympathy.typeutils.adaf.Column(attributes, data, name)

Class representing a named column with values and attributes. Get attributes with attr member.

dtype

dtype of column.

name()

Return the column name.

size()

Return the size of the column.

value(kind='numpy')

Return the column value.

Return type is numpy.array when kind is ‘numpy’ (by default) and dask.array.Array when kind is ‘dask’.

Dask arrays can be used to reduce memory use in locked subflows by handling data more lazily.