ADAF API

API for working with the ADAF type.

Import this module like this:

from sympathy.api import adaf

The ADAF structure

An ADAF consists of three parts: meta data, results, and timeseries.

Meta data contains information about the data in the ADAF. Stuff like when, where and how it was measured or what parameter values were used to generated it. A general guideline is that the meta data should be enough to (at least in theory) reproduce the data in the ADAF.

Results and timeseries contain the actual data. Results are always scalar whereas the timeseries can have any number of values.

Timeseries can come in several systems and each system can contain several rasters. Each raster in turn has one basis and any number of timeseries. So for example an experiment where some signals are sampled at 100Hz and others are sampled only once per second would have (at least) two rasters. A basis doesn’t have to be uniform but can have samples only every now and then.

Accessing the data

The adaf.ADAF object has two members called meta and res containing the meta data and results respectively. Both are Group objects.

Example of how to use meta (res is completely analogous):
>>> from sympathy.api import adaf
>>> import numpy as np
>>> f = adaf.ADAF()
>>> f.meta.create_column(
...     'Duration', np.array([3]), {'unit': 'h'})
>>> f.meta.create_column(
...     'Relative humidity', np.array([63]), {'unit': '%'})
>>> print(f.meta['Duration'].value())
[3]
>>> print(f.meta['Duration'].attr['unit'])

Timeseries can be accessed in two different ways. Either via the member sys or via the member ts. Using sys is generally recommended since ts handles multiple timeseries with the same name across different rasters poorly.

Example of how to use sys:
>>> f.sys.create('Measurement system')
>>> f.sys['Measurement system'].create('Raster1')
>>> f.sys['Measurement system']['Raster1'].create_basis(
...     np.array([0.01, 0.02, 0.03]),
...     {'unit': 's'})
>>> f.sys['Measurement system']['Raster1'].create_signal(
...     'Amount of stuff',
...     np.array([1, 2, 3]),
...     {'unit': 'kg'})
>>> f.sys['Measurement system']['Raster1'].create_signal(
...     'Process status',
...     np.array(['a', 'b', 'c']),
...     {'description': 'a=awesome, b=bad, c=critical'})
>>> f.sys.keys()
['Measurement system']
>>> f.sys['Measurement system'].keys()
['Raster1']
>>> f.sys['Measurement system']['Raster1'].keys()
['Signal1', 'Signal2']
>>> print(f.sys['Measurement system']['Raster1']['Signal1'].t)
[ 0.01  0.02  0.03]
>>> print(f.sys['Measurement system']['Raster1']['Signal1'].y)
[1 2 3]
>>> print(f.sys['Measurement system']['Raster1']['Signal1'].unit())
kg

The rasters are of type RasterN.

Timeseries and raster attributes

Attributes can be added to timeseries, rasters, etc. Attributes work the same way as they do for Tables.

Name restrictions

The naming restrictions for Tables apply to ADAFs too.

Class adaf.ADAF

class sympathy.api.adaf.ADAF(filename: Optional[str] = None, mode: str = 'r', **kwargs)

ADAF represents the top level of the ADAF format.

Any node port with the ADAF type will produce an object of this kind.

Use the members meta, res and sys to access the data. See Accessing the data for an example.

completions(**kwargs)

Return completions builder for this object.

classmethod icon()

Return full path to svg icon.

index(limit=None)

INTERNAL use only!

Return READ-ONLY index of internal storage including typealiases. Caller may view but not modify returned structure.

Limit can be used, for performance reasons, to exclude certain items from the output.

info() → dict

Return index information about the content.

init()

Perform any initialization, such as, defining local fields.

names(kind=None, fields=None, **kwargs)

The names that can be automatically adjusted from an adaf.

kind should be one of ‘cols’ (all column names from meta, res, and all rasters), ‘ts’ (all signal names from all rasters), or ‘rasters’ (all raster names including system names).

package_id() → str

Get the package identifier string.

set_index(index)

INTERNAL use only!

Set index to provided index (produced by matching index()). Does nothing unless implemented. Provided index need to match the internal storage data-structure exactly and can therefore not be set after modifications.

Caller hands over ownership of index and may not modify the argument structure.

set_source_id(source_id: str)

Set the source identifier string.

source(other_adaf, shallow=False)

Use the data from other_adaf as source for this file.

source_id() → str

Get the source identifier string. If the source identifier has not been set, it will default to an empty string.

sync()

Synchronize data fields that are kept in memory to self._data.

Called before data is written to disk and must be re-implemented by subclasses that introduce additional fields to ensure that the fields will be written through self._data.

timestamp() → str

Get the time string.

user_id() → str

Get the user identifier string.

version() → str

Return the version as a string. This is useful when loading existing files from disk.

New in version 1.2.5.

classmethod viewer()

Return viewer class, which must be a subclass of sympathy.api.typeutil.ViewerBase

Class Group

class sympathy.api.adaf.Group(data, name: str)

Class representing a group of scalars. Used for meta and res. Supports dictionary-like __getitem__ interface for data retrieval. To write a column use create_column().

create_column(name: str, data: ndarray, attributes: Optional[Dict[str, Union[int, bool, float, complex, str]]] = None)

Create and set/add a new column. If a column with the same name already exists, it will be replaced.

Parameters:

  • data – Column data. Data must be of ndarray type (numpy ndarray). Needs to match the existing number of rows in the raster.

  • attributes – Column attributes added as attributes on the column.

Return type:

Created column.

delete_column(name: str)

Delete named data column from the group.

from_table(table: Table)

Set the content to that of table. This operation replaces the columns of the group with the content of the table.

get_attributes() → Dict[str, Union[int, bool, float, complex, str]]

Return a dictionary of all attributes on this group.

hjoin(other_group: Group)

HJoin Group with other Group.

items() → Iterable[Tuple]

Return the current group items.

keys() → Collection[str]

Return the current group keys.

number_of_rows() → int

Return the number of rows in the Group.

New in version 1.2.6.

rename_column(old_name: str, new_name: str)

Rename the named data column.

set_attribute(key: str, value: Union[int, bool, float, complex, str])

Add an attribute to this Group. If the attribute already exists it will be updated.

to_table() → Table

Export table containing the data.

values() → Collection[Column]

Return the current group values.

vjoin(other_groups: Iterable[Group], input_index: str = '', output_index: str = '', fill: bool = False, minimum_increment: int = 1)

VJoin Group with other Group.

Class RasterN

class sympathy.api.adaf.RasterN(record, system: str, name: str)

Represents a raster with a single time basis and any number of timeseries columns.

property attr: Attributes

Raster level attributes.

basis_column() → Column

Return the time basis for this raster. The returned object is of type Column.

create_basis(data: ndarray, attributes: Optional[Dict[str, Union[int, bool, float, complex, str]]] = None, **kwargs)

Create the raster basis. If a basis already exists, it will be replaced.

Parameters:

  • data – Basis data. Data must be of ndarray type (numpy ndarray). Needs to match the existing number of rows in the raster.

  • attributes – Basis attributes added as attributes on the basis.

  • kwargs – Using kwargs to set attributes is OBSOLETE and will result in a warning.

Changed in version 1.2.1: Added the attributes parameter and made kwargs obsolete.

See also, create_signal() for creating signals.

create_signal(name: str, data: ndarray, attributes: Optional[Dict[str, Union[int, bool, float, complex, str]]] = None, **kwargs)

Create and set/add a new signal. If a signal with the same name already exists, it will be replaced.

Parameters:

  • name – Signal name.

  • data – Signal data. Data must be of ndarray type (numpy ndarray). Needs to match the existing number of rows in the raster.

  • attributes – Signal attributes added as attributes on the signal.

  • kwargs – Using kwargs to set attributes is OBSOLETE and will result in a warning.

Changed in version 1.2.1: Added the attributes parameter and made kwargs obsolete.

See also, create_basis() for creating the basis.

delete_signal(name: str)

Delete named signal.

from_table(table: Table, basis_name: Optional[str] = None, use_basis_name: bool = True)

Set the content to that of table.

This operation replaces the signals of the raster with the content of the table.

When basis_name is used, that column will be used as basis, otherwise it will not be defined after this operation and needs to be set using create_basis.

items() → Collection[Tuple[str, Timeseries]]

Return a list of tuples, each with the name of a timeseries and the corresponding Timeseries object.

keys() → Collection[str]

Return a list of names of the timeseries.

number_of_columns() → int

Return the number of signals including the basis.

number_of_rows() → int

Return the number of rows (length of a time basis/timeseries) in the raster.

to_table(basis_name: Optional[str] = None) → Table

Export all timeseries as a Table.

When basis_name is given, the basis will be included in the table and given the basis_name, otherwise it will not be included in the table.

update_basis(other_raster: RasterN)

Updates the basis from the basis of RasterN other_raster.

New in version 1.4.3.

update_signal(signal_name: str, other_raster: RasterN, other_name: Optional[str] = None)

Updates a signal from a signal in another raster.

The signal other_name from other_raster will be copied into signal_name. If signal_name already exists it will be replaced.

If other_name is not specified, signal_name will be used instead.

New in version 1.4.3.

values() → Collection[Timeseries]

Return a list of all signal items.

vjoin(other_groups: Iterable[RasterN], input_index: str = '', output_index: str = '', fill: bool = False, minimum_increment: int = 1)

VJoin Group with other Group.

Class Timeseries

class sympathy.api.adaf.Timeseries(node, data, name: str)

Class representing a timeseries. The values in the timeseries can be accessed as a numpy array via the member y. The timeseries is also connected to a time basis whose values can be accessed as a numpy array via the property t.

The timeseries can also have any number of attributes. The methods unit() and description() retrieve those two attributes. To get all attributes use the method get_attributes().

basis() → Column

Return the timeseries data basis as a Column.

description() → str

Return the description attribute or an empty string if it is not set.

property dtype: dtype

dtype of timeseries.

get_attributes() → Dict[str, Union[int, bool, float, complex, str]]

Return all attributes (including unit and description).

raster() → RasterN

Return the associated raster.

raster_name() → str

Return the name of the associated raster.

signal() → Column

Return the timeseries data signal as a Column.

signal_name() → str

Return the name of the timeseries data signal.

system_name() → str

Return the name of the associated system.

property t: ndarray

Time basis values as a numpy array.

unit() → str

Return the unit attribute or an empty string if it is not set.

property y: ndarray

Timeseries values as a numpy array.

Class Column

class sympathy.api.adaf.Column(attributes, data, name: str)

Class representing a named column with values and attributes. Get attributes with attr member.

property dtype: dtype

dtype of column.

name() → str

Return the column name.

size() → int

Return the size of the column.

value(kind: str = 'numpy') → ndarray

Return the column value.

Return type is numpy.array when kind is ‘numpy’ (by default) and dask.array.Array when kind is ‘dask’.

Dask arrays can be used to reduce memory use in locked subflows by handling data more lazily.