Node overview

Here follows lists of most of the nodes in the standard library of Sympathy for Data. Nodes originating from toolkits are not yet listed here.

Analysis

The Analysis folder holds mathematical nodes for data analysis. Statistics and various algorithms for time-series analysis. More nodes are available in the Timeseries toolkit.

Statistics

• Descriptive statistics

• Heatmap calculation

• Histogram calculation

Data Processing

In Data Processing you find nodes for formatting and structuring data, commonly used in a pre-processing step for clean up. The operations inculde: converting between data formats, naming, setting attributes, replacing values etc.

Attributes

• Set/Get Table(s) Name

• Set/Get Table(s) Attributes

• Set/Get column attributes in Table

Calculate

The Calculate tag holds special nodes which manipulate the data by applying functions.

• Calculator

  Apply pre-defined Python functions (arithmetics, comparators, logics etc.) to any data type.

• Calculator List

• F(x)

  The most versitile node in the Library. This node applies any user-defined Python function to any datatype.

• F(x) List

• Json Calculator

  Apply functions to JSON files, keeping the JSON format.

• Matlab Table

  Similar to F(x) but functions are written in Matlab instead of Python.

Convert

Nodes that convert between different Sympathy data formats.

• ADAF to Table/Table to ADAF

• Table to Datasources/Datasource to Table

• JSON to Dict/List/Text/Table

• Table to Text/Text to Table

• etc..

Data

Nodes that edit data.

• Convert columns in Table

  Change the datatype of a table column, e.g integer to floating point.

• Hold value Table

  Replace nan values with the previous non-nan.

• Replace values in Table

  Find and replace any value in a Table.

• Replace values in Table with Table

  Use a Table with find-replace pairs to replace several values in target Table.

• Rename datasource with Regex

  Use regular expression to edit the path of a Datasource.

Index

• ADAF structure to Table

• Create Index Table

  Adds an index column with new indices for unique values in the Table.

Select

Nodes that selects/discards parts of data.

• Drop NaN Table/Drop NaN Tables

• Drop masked values in Table

• Fill masked values in Table

• Filter rows in Table/Filter rows in Tables

• Mask values in Table

• Remove Json

• Select Json

• Select categories in ADAFs

• Select columns by type in Table/Select columns by type in Tables

• Select columns in ADAF with structure Table

• Select columns in ADAFs with structure Table/Select columns in ADAFs with structure Tables

• Select columns in Table/Select columns in Tables

• Select columns in Table with Regex

• Select columns in Table with Table

• Select columns in Tables with Table

• Select keys in Json

• Select rows in Table/Select rows in Tables

• Select rows in Table with Table

• Select rows in Tables with Table

• Slice List

• Slice columns in Table/Slice columns in Tables

• Slice rows in Table/Slice rows in Tables

• Unique Table/Unique Tables

Structure

Nodes that perform non-mutating transformations such as transposing, sorting, joining and splitting.

• Cartesian Product Table/Cartesian Product Tables

• Ensure columns in Tables with Table

• Get column names in Table

• HJoin Table/HJoin Tables

• HJoin Tables pairwise

• HSplit Table/HSplit Tables

• Index columns in Table/Index columns in Tables

• Index rows in Table/Index rows in Tables

• Index table

• Lookup Table/Lookup Tables

• Match Table lengths/Match Tables lengths

• Merge Table/Merge Tables

• Pivot Table/Pivot Tables

• Rename columns in ADAF with Table/Rename columns in ADAFs with Table

• Rename columns in Table/Rename columns in Tables

• Rename columns in Tables with Table

• Reverse columns in Table/Reverse columns in Tables

• Reverse rows in Table/Reverse rows in Tables

• Set column names in Table with Table/Set column names in Tables with Table

• Sort columns in Table

• Sort rows in Table/Sort rows in Tables

• Transpose Table/Transpose Tables

• Unpivot Table/Unpivot Tables

• VJoin Table/VJoin Tables

• VJoin Tables pairwise

• VSplit Table/VSplit Tables

Text

• Concatenate text/Concatenate texts

• Jinja2 template

• Split text

Development

Nodes with the main purpose of aiding Sympathy development and testing.

Debug

Export/Import the internal datastructure of any datatype.

• Debug Export

• Debug Import

Example

Various examples of nodes for help in node creation are found under the Example tag.

Test

• Assert Equal ADAF

• Assert Equal Table

• Conditional error/warning

  Raises errors/warnings based on custom lambda expression.

Disk

Under the Disk tag are a collection of nodes that do system operations such as copying, moving and deleting files.

• Archive files

• Copy file/Copy files

• Delete file/Delete files

• Download URL to file

• Move File/Move Files

Generic

Configuration

Create and edit JSONs for configuration of other nodes

• Configure Json Parameters

  Configure a given set of Json parameters in this nodes configuration GUI

• Create Json

  Manually create a Json object.

• Create Json Parameters

  Create a Sympathy parameter structure with Python code.

• Update Configuration with Table

  Update the configuration parameters with values from a Table.

Control

Here you find nodes for controlling the execution of the flow.

• Conditional Propagate

  Propagate one of two inputs based on custom if condition.

• Propagate First Input

  Always propagate the first input, useful for waiting until different parts have finished before advancing.

• Propagate First Input (Same Type)

Dict

Nodes that do operations on Sympathy dict datatype.

• Dict items

  Get key value pairs as Sympathy list of Sympathy tuples

• Dict keys

  Get keys in list

• Dict values

  Get values in list

• Get Item Dict

  Get value based on key

• Insert Dict

  Insert key, value pair

• Items to Dict

  Convert list of key value pairs to dict

• Update Dict

  Combine two dicts

Lambda

Nodes that work with the special datatype Lambda. Which are unexecuted flows packaged for later use.

• Apply

  Apply (execute) the lambda flow on an argument.

• Extract Flows as Lambdas

  Extract all top-level flows as Lambdas from a .syx file.

• Extract Lambdas

  Extract all Lambdas from a .syx file.

• Map

  Map (execute) the lambda flow on each element in a list.

• Parallel Map 2

  Map in 2 parallel processes.

• Parallel Map 4

  Map in 4 parallel processes.

• Parallel Map 4 Configurable

  Map in 1-4 parallel processes.

List

Nodes that do operations on Sympathy dict datatype.

• Append List

  Append an item to a list.

• Bisect List

  Split list into equally big parts.

• Chunk List

  Split list into a defined number of parts.

• Extend List

  Combine two lists.

• Filter List Predicate

  Filter items that meets custom condition.

• Filter List with Table

  Select items based on values (True/False) in a Table.

• Flatten List

  Flattens nested list.

• Get Item List

  Get item at index.

• Group List

  Create groups of nested lists.

• Index List

  Select items based on index.

• Insert List

  Insert an item in a list.

• Item to List

  Convert item to list of length 1

• Pad List

  Pad a list to match length of other list.

• Pad List with Item

  Pad a list to match length of other list with given item.

• Partition List Predicate

  Divide list into two based on custom condition.

• Reverse List

• Sort List

• Transpose List

  Swap the first and second layer of a nested list.

Tuple

Nodes that do operations on Sympathy tuple datatype.

• Cartesian Product Tuple

  Create a list of tuples with all combinations of elements from multiple lists.

• First Tuple2

  Get first item in tuple with two items.

• Second Tuple2

  Get second item in tuple with two items.

• Tuple

  Create a tuple from two or more items.

• Untuple

  Get items from a tuple.

• Unzip Tuple

  Get items in separate lists from list of tuples.

• Zip Tuple

  Create a list of tuples from two or more lists.

Image Processing

In the Image Processing folder are all nodes that handle the image port type. More nodes are available in the Image Analysis toolkit.

Extract statistics

• Extract Image Data

  Extract data from image from specified pixel regions in a Table.

• Image Statistics

  Extracts statistics with one of a selection of algorithms.

Image Manipulation

Nodes that operate on images, drawing, editing filtering etc.

• Color range conversion

• Color space conversion

• Corner detection

• Draw on Image

• Edge detection

• Filter Image, Dual Output

• Filter image

• Morphological Image Operations

• Morphology (single input)

• Transform image

Input/Output

Nodes that create, import and export images.

• Convert 2D Table to Image

• Convert Image to 2D Table

• Convert Image to Table

• Convert Table to Image

• Example Image

• Generate Image

• Load Image

• Load Image List

• Save Image

Layer operations

Nodes that operate on the different color channels of an image, overlaying, merging and splitting them.

• Image to List

• List to Image

• Merge Image Channels

• Overlay Images

• Overlay Images List

• Split Image Channels

Segmentation

• Label image

  Create integer image with labels for each connected region (same pixel values).

• Threshold image

  Boolean output from thresholding the pixel values.

Input

Environment

• Get Environment to Table

  Choose between a number of environment variables to display in a Table.

Generate

Under the Generate tag are nodes that can generate data in various Sympathy formats.

• Create Table

• Create Text

• Empty

• Generate Signal Table

• Generate Signal Tables

• Random ADAF

• Random ADAFs

• Random Table

• Random Tables

Import

Nodes used for importing data into the Sympathy workflow. All import nodes are preceeded by the Datasource node that specify the location to import from.

• ADAF

• ADAFs

• Datasource

• File Datasources

• Json

• Jsons

• Table

• Tables

• Text

• Texts

Machine Learning

The Machine Learning tag holds nodes that work with the Model datatype. The models include classifiers, regressors, dimensionality reduction etc. More models are available in the Avanced Machine Learning extension.

Apply model

Nodes that apply, trained or un-trained Models to data.

• Decision Function

  Apply a trained Models decision function (if avaliable).

• Extract Attributes

  Extract Model attributes into Table.

• Fit

  Fit a Model to Table data.

• Fit Texts

  Fit a Model to Text data.

• Fit Transform

  Fit a transformation Model and apply it to the data.

• Fit Transform Text

• Inverse Transform

  If possible, reverse data transformation from a Model.

• Pipeline

  Create a pipline of sucessive Models.

• Pipeline decomposition

• Predict

  Use Model to predict Y given X.

• Predict Probabilities

  Use Model to predict Y given X, and return estimated probabilities.

• Select Features from Model

• Set Input and Output Names

• Transform

  Apply a transformation Model to data.

• Transform Text

  Apply a transformation Model to text.

• Voting Classifier

  Use voting by multiple classifier Models.

Dimensionality reduction

Models that transform data by projecting onto a space of lower dimension.

• Kernel Principal Component Analysis (KPCA)

• Partial Least Squares cross-decomposition (PLS regression)

• Principal Component Analysis (PCA)

IO

Import/export Models and generate ML datasets.

• Example datasets

• Export Model

• Features to Images

• Generate classification dataset

• Generate dataset blobs

• Generate dataset blobs from table

• Images to Features

• Import Model

Metrics

Nodes that produce quantitative measures used to evaluate and compare the performance of the Machine Learning Models.

• Conditional Probabilty from Categories

• Confusion Matrix

• Learning Curve

• ROC from Probabilities

• R² regression score (R2)

• Score

Parameters

Nodes used for setting and extracting Model hyperparameters.

• Extract Parameters

• Grid Parameter Search

• Parameter Distribution

• Randomized Parameter Search

• Set Parameters

• Simulated Annealing Parameter Search

Partitioning and validation

Nodes used for partitioning data, cross-validataion and train-test split.

• Group K-fold Cross Validation

• K-fold Cross Validation

• Leave One Group out Cross Validation

• Score Cross Validation

• Simple Train-Test Split

• Split Data for Cross Validation

• Stratified K-fold cross validation

• Time Series K-fold Based Cross Validation

Processing

Nodes for pre-processing data for ML purposes. Scaling, normalizing, encoding labels etc.

• Binarizer

• Categorical Encoder

• Imputer

• Label Binarizer

• Label Encoder

• Max Abs Scaler

• Normalizer

• One-Hot Encoder

• Polynomial Features

• Robust Scaler

• Standard Scaler

• Text Count Vectorizer

Regression

Regressor Models

• Epsilon Support Vector Regression

• Kernel Ridge Regression

• Linear Regression

• Multi-output Regressor

Supervised

Models for supervised learning, that require labeled data.

• Decision Tree Classifier

• k-Nearest Neighbors Classifier

• Logistic Regression

• Multi-Layer Perceptron Classifier

• Multi-Layer Perceptron Regressor

• Multi-output Classifier

• Random Forest Classifier

• Support Vector Classifier

Unsupervised

Models that work unsupervised, without labels on the data.

• Isolation Forest

• K-means Clustering

• Mini-batch K-means Clustering

• One Class Support Vector Machines

Output

Export

Export data from Sympathy to external sources.

• Export ADAFs

• Export Bokeh Figures

• Export Figures

• Export Tables

• Export Texts

Visual

In the Visual folder you find nodes for visualization of the data.

Figure

Nodes that create and work with the Figure datatype. These are the recommended nodes for creating plots from your data.

• Bokeh Components

  Export html components for embedding a Bokeh figure.

• Bokeh Figure

  Create a Bokeh Figure

• Colormap lookup

  Map input into colors.

• Figure

  Create a Figure.

• Figure Compressor

  Combine several Figures into one, overlaying the plots.

• Figures

  Create a list of Figures.

• Layout Bokeh Figures in Subplots

• Layout Figures in Subplots

  Create a subplot layout with several figures.

Html

• HTML to Text

  Convert html code to text.

• Html Report

  Import HTML code from datasource.

Plot

• Scatter 3D Table

  Create a 3D plot, saved to disk.

Report

Under the Report tag are nodes for creating reports in PDF format. The reports can be customized to include plots and data from the Sympathy flow. These nodes are not actively maintained. Consider using either Figure or Bokeh Figure instead.

• Merge Reports

• Report Apply ADAFs

• Report Apply ADAFs with Datasources

• Report Apply Tables

• Report Template ADAFs

• Report Template Tables

• Select Report Pages