.. This file is part of Sympathy for Data. .. Copyright (c) 2010-2017 Combine Control Systems AB .. .. SYMPATHY FOR DATA COMMERCIAL LICENSE .. You should have received a link to the License with Sympathy for Data. News in 1.3.5 ------------- Node/plugin changes: ^^^^^^^^^^^^^^^^^^^^ * Calculations in :ref:`Calculator` can be deselected for output enabling better support for intermediary calculations. This also enables intermediary calculations to have different lengths from output columns. * The input table(s) in :ref:`Calculator` can be easily copied over to the output table(s) with the new *Copy Input* parameter. Calculations with the same column name override columns from the input table(s). * :ref:`MATLAB nodes` and ``Matlab Calculator`` have gotten better cross-platform compatibility. * ``Matlab Calculator`` has been updated with the same GUI and (almost) the same functionality as :ref:`Calculator List`. * :ref:`Matlab Table` and :ref:`Matlab Tables` have gotten a new simplified format. See the documentation for details on how to use that. This format can also be imported and exported in :ref:`Table` and :ref:`Export Tables` respectively. A Table-like API is planned for a future release. The API that currently resides in Sympathy/Matlab will also be deprecated in a later release, in favor of the new format. The old nodes are left for compatibility, so current flows and scripts will still work. * The generic :ref:`Empty` node allows to specify the data type of the output port. The previous, specific, Empty-nodes have been deprecated. * :ref:`Rename column nodes` have more consistent priority rules when more that one column are renamed to the same name. * :ref:`Extract lambda nodes` are more robust with regard to corrupt flows. One corrupt flow should no longer stop the nodes from extracting other lambdas. * New node: :ref:`Heatmap calculation` useful for feeding the heatmap in :ref:`Figure(s) from Table(s)
`. * New features for heatmaps in :ref:`Figure(s) from Table(s)
`: logarithmic color scales and Z labels. * :ref:`Datasource` and other nodes where you specify a file path can specify paths relative to its own workflow or the top workflow. This can make a difference when working with linked subflows. * :ref:`org.sysess.sympathy.datasources.filedatasourcemultiple` GUI is no longer slowed down when searching large folder structures. If the search takes to long it is aborted, and to get the full results the node has to be executed. * The table name used for the output in :ref:`HJoin Table` can now be selected. * Fixes to extract flows as lambdas so that workflow environment variables and flow name are set correctly. * Timesync ADAFs can now use integer timebases and correctly displays datetimes in the plot. * :ref:`Assert equal table` now treats NaNs as equal. * Improved config gui and handling of NaN values, masked values and non-ascii binary data in :ref:`VSplit Table(s)`. * A new node has been added: HJoin ADAFs pairwise. * When zooming and panning in Plot Table and using datetime as X axis, the current time span in the plot is displayed. * SQL importer plugin can use SQLAlchemy and provide betters autodetection of existing tables. * SQL exporter plugin can use SQLAlchemy. * Improved documentation generation with support for libraries on different drives or on unicode paths. Platform: ^^^^^^^^^ * Nodes have gotten dynamic port icons that display the actual types. * Color of textfields can now be changed. * A textfields can be moved by dragging on any part of it. It is now edited by double clicking it or by right clicking and choosing "Edit". * The table viewer and any viewer which uses that component (i.e. ADAF viewer) can now be transposed for better viewing of long column names and tables with few rows but many columns. * Table viewer now supports copying values and/or column names as a table or as text. * The viewer can now show histograms for more types of data. * Allow maximizing subflow configurations. * Linked flows can now be placed on a different drive than their parent flows. * combo_editor for string parameters can now have an empty list of options. * Invalid subflows are more reliably shown as invalid (gray). Now any subflow which looks executable should be executable. * Subflows show an error indicator if they contain any nodes that are not found in the node library. This should make such nodes much easier to find. * Better feedback when trying to open a non-existing or corrupt workflow. * The platform can handle a larger number of linked files without running into the OS limit. * An Advanced tab has been added to Preferences, with one option to limit the number of concurrent nodes that may be executed, and one option to display warnings about deprecated nodes. * New preference option to set number of concurrent worker processes. This may help with performance for heavily branched flows. * Python 3 support for files created with the node and function wizards. * Library wizard can create subdirectories. * Spyder can't handle files on file paths contaning non-ascii characters, and will fail to start when trying to debug nodes. An error message is now displayed to notify the user of this. * Improved stability of type inference. * File datasources always store absolute paths. * Database datasources can use SQLalchemy in addition to ODBC. News in 1.3.4 ------------- Sympathy for Data version 1.3.4 offers improvements to existing nodes, including several new plot types for the figure nodes and overall polish. Node/plugin changes: ^^^^^^^^^^^^^^^^^^^^ * :ref:`Figure nodes
` have been massively improved with several new plot types (scatter/bar plots/histograms/heatmaps), improved gui, etc * Extended :ref:`figure export node` with plugin exporter structure as for other types and choice of specifying image size in mm and dpi * :ref:`Reporting Nodes` have been improved with rulers in layout window, pdf exporting and auto creation of tree structures * :ref:`Calculator`, allows accessing the input table directly under the name "table" allowing for a way to test if a column exists. The node was also extended with the json module in the execution context * ca.changed now correctly returns empty array for empty input * Added functions ca.global_min and ca.global_max to standard calculator plugin. These handle empty input as you would expect * :ref:`Interpolate ADAF` nodes have improved handling of missing values and resampling of zero-length signals * :ref:`Datasource` and :ref:`exporter`/:ref:`importer` of SQL can use SQLAlchemy * :ref:`Pad List` input can be different types of lists * :ref:`Predicate` nodes have new editors for writing code * :ref:`VJoin` nodes can mask missing values * MDF importer creates MDF_datetime metadata * :ref:`Assert Equal Table` allows approximate comparison of floats * Added documentation for internal nodes (:ref:`Apply`, :ref:`Map`, etc.) APIs: ^^^^^ * Made it possible to specify viewer and icon for custom types (TypeAlias). For details, see :ref:`create_type` * Only scanning Libraries for plugins, PYTHONPATH is no longer included * Scalar parameters can use the new combobox editor. See :ref:`All parameters example` for an example * Code parameter editor for string parameters. See :ref:`parameter_editors` for details and :ref:`All parameters example` for an example * Allow :ref:`controllers` to trigger on user-specified value. For an example of this see :ref:`Controller example` * Implemented ``cols()`` and added documentation for col/cols and Column class. See :ref:`Table API` * Added ``attrs`` property to :ref:`Table API` * Expose dtypes module in sympathy.api New nodes: ^^^^^^^^^^ * :ref:`Histogram calculation` * :ref:`Bisect list` * :ref:`Empty` * :ref:`Extract Flows as Lambdas` * Export Figures with Datasources * :ref:`Concatenate texts` * :ref:`Jinja2 template` * :ref:`Select columns in Table with Regex` UI: ^^^ * Improved look and feel of wizards * Library wizard has new examples * Node wizard can select tags * Show filename in flow tab unless flow label has been explicitly set by user. This means that a flow created in 1.3.4 will have no flow label when opened in older versions. Platform: ^^^^^^^^^ * More robust checks of port types * Masked arrays Deprecated nodes: ^^^^^^^^^^^^^^^^^ * Raw Tables nodes * Scatter 3D ADAF News in 1.3.3 ------------- Sympathy for Data version 1.3.3 offers improvements to existing nodes, the table viewer and automatic parameter validation when configuring nodes. GUI: ^^^^ * Behaviour change of “?” wildcard in :ref:`Table viewer` :ref:`search bar ` to match single character only * General improvements of Table viewer GUI * General improvements of parameter validation New nodes/plugins: ^^^^^^^^^^^^^^^^^^ * New node: :ref:`Conditional error/warning` * New node: :ref:`Cartesian product tuple2` Changes in nodes/plugins: ^^^^^^^^^^^^^^^^^^^^^^^^^ * Allow unicode characters in :ref:`Calculator` node * Improved default behaviour of Calculator node * Improved rescaling of preview plot in :ref:`Filter ADAFs` node * Improved :ref:`XLSX export` output compatibility * :ref:`Extract Lambdas` can be configured when connected * Improved performance of :ref:`VSplit Table` * Improved bounds checking for calculator functions ``shift_seq_start`` and ``shift_seq_end`` * Improve gui in :ref:`Create Table`. Now allows removing selected rows/columns as well as changing name and datatype of existing columns * Improved handling of bad timebases in :ref:`interpolation nodes ` APIs: ^^^^^ * Added ``value_changed`` propagation to parameters * Made :ref:`verify_parameters ` validate every change to configured parameters, for nodes with generated configurations Miscellaneous: ^^^^^^^^^^^^^^ * Fixed update method for tuple type * :reF:`data_viewer` can once again be run stand alone * Updated icons News in 1.3.2 ------------- Sympathy for Data version 1.3.2 offers several new and prominent features, such as the ability to specify libraries used by workflows, new window handling which brings open, but minimized, configurations and viewers into focus, a reworked save dialog that properly detects changes in subflows and many improvements to existing nodes. GUI ^^^ * Raise open Configuration/Settings/Viewer windows on consecutive clicks * Improved save confirmation for workflows * Improvements to the function wizard. Including updating it to work with the new generic :ref:`F(x)` nodes New features ^^^^^^^^^^^^ * Flows can now specify libraries and python paths in the Info dialog. These are added to the global library/python paths when loading the flow * New error message box for node dialogs for showing validation errors/messages in node configurations * Support for storing masked arrays, but not every node can handle them correctly New nodes/plugins ^^^^^^^^^^^^^^^^^ * Figure nodes with support for Tables * New version of :ref:`Transpose Table(s)`. These handle multiple rows and columns * :ref:`Assert Equal Table`: for checking if two tables are equal. Mostly useful for testing purposes * Generic :ref:`F(x)` nodes replacing all the previous type-specific f(x) nodes * ATFX importer plugin for :ref:`ADAF` * Set and Get nodes for :ref:`Table attributes` and :ref:`Table column attributes` * :ref:`Propagate First Input (Same Type)`. Can be used for constraining type if needed. Changes in nodes/plugins ^^^^^^^^^^^^^^^^^^^^^^^^ * Renamed Plot to Figure for nodes using the Figure type * :ref:`Figure Compressor`, :ref:`Layout Figures in Subplots`: added auto recolor and auto rescale * Improved datetime handling in Figure nodes * MDF :ref:`exporter` plugin: encode unicode columns instead of ignoring them * :ref:`Convert columns in Table(s)`: converts string dates to either UTC or Naive datetimes. Choosing UTC, localized times will be converted to UTC. Choosing naive, the time zone info in the input is simply ignored. Old nodes will automatically use UTC * Improved performance of :ref:`Select rows in Table(s)` better handles values without explicit type annotation * Improved error handling in :ref:`lookup nodes` * :ref:`Calculator` plugin: Make sure that result is always correct length in changed_up, changed_down, and shift_array * :ref:`Filter ADAFs`: added parameter validation and error messages. Filter design is computed and shown on parameter changes * Changed the visible name for importer and exporter plugins for ADAF and Table to SyData * Removes matlab settings from :ref:`Matlab Table`, ``Matlab Calculator`` and put them into global Preferences dialog * Renamed calculator nodes to :ref:`Calculator(s)` * CSV :ref:`Exporter` plugin: improved writing of datetime columns * Improve handling of missing units in :ref:`interpolate` nodes APIs ^^^^ * Extended :ref:`Table API` and added :class:`Column` object * Change default value for attribute ``'unit'`` to always be empty string in ADAFs * Added ParameterView base class for generated and custom GUIs to API. Custom GUIs can override the methods and properties to customize the behavior. Inheriting from ParameterView will be required in the future versions Miscellaneous ^^^^^^^^^^^^^ * Added support for signing the Installer/Uninstaller * Extended :ref:`searchbar` functionality for the :ref:`Table viewer` * Always write generated files in the right directory * Fix overrides not saved in syx files * Non-linked subflows inherit their parents $SY_FLOW_FILEPATH and $SY_FLOW_DIR * Improve performance of type inference News in 1.3.1 ------------- Sympathy for Data version 1.3.1 offers several new and prominent features such as an improved data viewer with embedded plot, a new figure datatype and many new nodes as well as improved performance and stability. New features ^^^^^^^^^^^^ * Improved :ref:`data_viewer` with embedded plotting of signals. * Overhaul of :ref:`subflow configuration`: Split into settings and configuration. Removed grouping. Only allow selecting shallow nodes/flows. Added Wizard configuration mode. Optionally override parameters of linked subflows. Should be somewhat backwards compatible * Added :ref:`Figure-type`. Passes serialized matplotlib figures between nodes * Added tuple-type * Better handling of broken links/nodes missing from library and changed port types due to subflow changes * F(x) function wizard * Allow setting flow name, description, version, author, and copyright information in flow info dialog. Also improved handling of flow labels all around * Expose more :ref:`environment variables` from workflow * New :ref:`command-line option`: ``--nocapture`` for debugging New nodes ^^^^^^^^^ * Figure-type nodes: Figure from Table with Table, :ref:`Figure Compressor`, :ref:`Layout Figures in Subplots`, :ref:`Export Figures` * :ref:`Calculator` for a single Table added to Library * New :ref:`Filter ADAFs` node with preview plots and improved configuration gui * :ref:`Create Table` * Signal generator nodes for generating Table(s) of sinus, cosines or tangents * :ref:`Matlab Tables` node * :ref:`Hold value Table(s)` * :ref:`Flatten list` * ``Propagate Input`` and :ref:`Propagate First Input`. These can be used to implement some workarounds and for determining execution order in a flow * :ref:`Interpolate ADAFs with Table` * :ref:`Report Apply ADAFs with Datasources` * :ref:`Filter rows in Tables`. This is the multiple Table version of existing Filter rows in Table * Tuple nodes * :ref:`Delete file`, which deletes a specified file from the file system Node changes ^^^^^^^^^^^^ * Allow selection of multiple columns in :ref:`Unique Table` * Allow choosing specific rasters in :ref:`Select categories in ADAFs` * Table attributes are merged for the :ref:`HJoin` nodes * Allow setting fixed width/height for TextBoxes in :ref:`Report Template ` * Easier date settings in Plot Table * Rewrote :ref:`Matlab Tables` and ``Matlab Calculator`` nodes Exporters/Importers changes ^^^^^^^^^^^^^^^^^^^^^^^^^^^ * ADAF Importer was extended with option to link to imported content * MDF Importer can handle zip-files that include a single MDF-file as input * Gzip Exporter binary writes files correctly * ATF Importer supports a wider range of files * Export tables nodes will now create output folders if necessary * Increased compression for exported sydata-files produces smaller files Optimizations ^^^^^^^^^^^^^ * Faster reading of writing of intermediate files * Faster ADAF copy methods * Improved length handling for tables * Faster execution of :ref:`Select rows in Table(s)
`/:ref:`export`. Especially handling of dates, times, and mixed types. Cells formatted as Time are now imported as timedeltas. * Renamed Sort Table(s) to :ref:`Sort rows in Table(s)` * :ref:`Calculator List`: chooses columns case-sensitively on Windows too. * :ref:`Calculator List`: shows number of output rows in preview in calculator gui. * :ref:`VSplit Table`: Removed constraint that the index should be sorted. The elements will be grouped by the first occurrence of each unique value. * :ref:`Convert columns in Table`: Added conversion path between datetime and float. * :ref:`Select columns in ADAF with structure Table` now works as expected when *Remove selected* has been checked. * :ref:`Select rows in Table with Table` offers a choice of reduction function between rows in config table. Previously it only read first row of the config table. * "Slice List of ADAFs/Tables": Basic integer indexing now works as expected. * Improve handling of one sample signals in :ref:`Interpolate ADAF(s) ` * :ref:`Report Apply nodes ` output datasources to created files * Improved :ref:`CSV import
`. Can now handle empty input, input with only one row, with or without trailing newline, and files with errors towards the end. It also features a new option for how to handle errors when importing a file. Header row has been made independent of the other input boxes, and no longer affects the data row. When read to end of file is selected, the number of footer rows is ignored. Delimiter detection was improved * Fixed issues with nesting of higher order functions (:ref:`Map `, :ref:`Lambda` and :ref:`Apply`) * Improvements to :ref:`reporting `: Improved bin placement and x-axis extent of 1d histograms. Automatically set axes labels from data source if they are empty. Added option "Lift pen when x decreases" to line graph layer. Added vline layer in reporting tool. * Several nodes are better at forwarding attributes, table names, etc. to output `Slice data Table (deprecated)`, :ref:`Select columns in ADAF(s) with structure Table(s)`, :ref:`Interpolate ADAF(s) `, and :ref:`Rename columns nodes ` * Many nodes are better at handling missing or incomplete input data: :ref:`Filter Rows in Table`, :ref:`Replace values in Tables`, :ref:`Detrend ADAF(s) `, :ref:`ADAF(s) to Table(s) `, :ref:`Select Report Pages`, :ref:`Scatter nodes `. * Added 'calculation' attribute on all output columns from :ref:`Calculator List` node * :ref:`Export Tables` and :ref:`Archive files` create missing folders * Fixed :ref:`Export Texts` Other improvements ^^^^^^^^^^^^^^^^^^ * Added :ref:`default workflow environment variables ` ``SY_FLOW_FILEPATH``, ``SY_FLOW_DIR`` and ``SY_FLOW_AUTHOR``. All flows have these and they can't be set or deleted. * Subflows can define :ref:`workflow variables `. Each subflow specializes the variables of its parent flow, so that the parent flows vars are accessible in the subflow but not vice versa. * Improve performance by skipping validation of any nodes that don’t implement :meth:`verify_parameters` * Improve performance by changing compression settings for sydata files, compression is faster but compresses slightly less * Pretty print workflow xml files, making diffs possible New requirements: * Requiring pandas version 0.15 for the CSV import, for versions before 0.15 down to 0.13 it will still work but may behave slightly differently in edge cases with blank rows