.. This file is part of Sympathy for Data.
..
.. Copyright (c) 2010-2017 System Engineering Software Society
..
.. Sympathy for Data is free software: you can redistribute it and/or modify
.. it under the terms of the GNU General Public License as published by
.. the Free Software Foundation, either version 3 of the License, or
.. (at your option) any later version.
..
.. Sympathy for Data is distributed in the hope that it will be useful,
.. but WITHOUT ANY WARRANTY; without even the implied warranty of
.. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
.. GNU General Public License for more details.
.. You should have received a copy of the GNU General Public License
.. along with Sympathy for Data. If not, see .
What's new
==========
News in 1.6.0
-------------
Gui changes
^^^^^^^^^^^
* Panning the flow view is now mapped to middle mouse button (scroll wheel
click). There are also toolbar buttons for panning/selection.
* New icons from Font Awesome in toolbar, menus and context menus.
* Color theme based on the standard palette in Windows 10, is used for all OSs.
* Changed keyboard shortcuts for Zoom to selection and Zoom fit all.
* Flow Overview look and feel similar to that of Library View.
* New splash screen on startup of GUI.
* Removed redundant groupings in parameter GUIs, this makes some configuration
GUIs more compact.
* More clear highlight of invalid parameter values, on Windows.
* Double click on node executes or does nothing, will never trigger configure.
* Plots in Table viewer shows current resampling.
* Figure viewer now supports mouse interactions for 3D axes.
* Deprecation warnings are on by default. They can be disabled by unchecking
*Display warnings for deprecated nodes* in the *Advanced* section of
*Preferences*.
Platform changes:
^^^^^^^^^^^^^^^^^
* Supports Python 3.6+.
* Supports PySide 2 (Qt for Python). This results in smoother rendering,
various improvements.
* Asyncio is used in place of twisted, this will make it easier to install
Sympathy on some OSs.
* Support for installing Sympathy in a Python virtual env, on Windows.
* Improved platform stability, undeterministic process crashes when running and
on shutdown caused by PySide do not seem to happen anymore.
* Improved structure of menus, with new groupings and consistent ordering.
* Windows installer produces different package structure, similar to a wheel
install.
* Added json parameter type.
* Added datetime parameter type.
* Added dict port type.
* Added Html port type.
* Added GeoJSON port type.
* Added Table editor for json parameter type.
* General preview which can easily be added to nodes.
* list_node_decorator replaces list_node_factory.
* Reduce size of syx files by not writing descriptions etc for all parameters.
Pretty print parameter structures for increased legibility.
* Selections can be expanded while holding down Ctrl (Cmd on OSX). Ctrl + click
toggles selection of individual nodes.
* List parameters with static lists are now automatically updated from their
definition.
* Automatically fix minor inconsistencies between .value and .value_names during
update_parameters. Warn about parameters that are still inconsistent during
execute/configure.
* Documentation improvements:
* New instructions for wheel install, now also for Windows.
See :ref:`installation`.
* New instructions for debugging using PyCharm. See :ref:`pycharm_debug`.
* Show labels for parameters in node documentation.
* Improved documentation about node libraries and their structure.
* No longer supported:
* Python 2. Python 3.5 and older.
* PySide.
* Built-in debugging and editing using Spyder.
* Built-in benchmark of the platform.
* Add maintainer information to library.ini metadata. Useful when maintainer and
original author are not the same.
* Node copyright notice is now inherited from library.ini metadata if
unspecified in the node. Allows less repetition when creating nodes.
Node/plugin changes:
^^^^^^^^^^^^^^^^^^^^
* Datasource can supply URL to importer nodes.
* Support for URL datasource in import nodes and new
:ref:`org.sysess.sympathy.files.downloadfile`.
* VJoin, Transpose: improved feedback when constructing unsupported columns
* Select columns in Table uses new preview feature.
* New :ref:`org.sysess.sympathy.visualize.figure`:
* Many different improvements to the configuration gui and to the code.
* Allow setting major/minor ticks.
* Support for legend outside of axes.
* Enabling and disabling individual plots.
* Uses new preview.
* The node no longer outputs its configuration in the form of a Table.
* Improved support for plotting datetimes
* Improved documentation.
* The old figure node can still be used, but will be deprecated at some point
in the near future.
* Transpose can set column names even when there are no rows
* Consistently using "regex" to refer to regular expressions in node configurations.
* Calculator should be able to detect res-dependencies that use other quoting than ''.
* Improved reading of typed input value:
* Mask nodes.
* Replace value in Table
* Select rows in Table
* :ref:`F(x)` can be configured without external file.
New nodes/plugins:
^^^^^^^^^^^^^^^^^^
* :ref:`org.sysess.sympathy.html.report`
* :ref:`org.sysess.sympathy.html.dicttogeojson`
* :ref:`org.sysess.sympathy.html.htmltotext`
* :ref:`org.sysess.sympathy.datasources.renames`
* :ref:`org.sysess.sympathy.files.downloadfile`
* :ref:`org.sysess.sympathy.data.json.selectkeyjson`
* :ref:`org.sysess.sympathy.data.json.removekeyjson`
* :ref:`org.sysess.sympathy.data.json.splitonkeyjson`
* :ref:`org.sysess.sympathy.dict.insert`
* :ref:`org.sysess.sympathy.dict.update`
* :ref:`org.sysess.sympathy.dict.getitem`
* :ref:`org.sysess.sympathy.dict.fromitems`
* :ref:`org.sysess.sympathy.dict.values`
* :ref:`org.sysess.sympathy.dict.keys`
* :ref:`org.sysess.sympathy.dict.items`
* :ref:`org.sysess.sympathy.json.jsontodict`
* :ref:`org.sysess.sympathy.json.dicttojson`
* :ref:`org.sysess.sympathy.json.jsontolist`
* :ref:`org.sysess.sympathy.json.listtojson`
Removed nodes:
^^^^^^^^^^^^^^
* Propagate Input
* Repeat Item to List
* Elementwise ADAFs to Tables
* Filter Image (deprecated)
Deprecated nodes/plugins:
^^^^^^^^^^^^^^^^^^^^^^^^^^
* Convert specific columns in Table(s)
* Copy files with Datasources
* Rename file(s)
* Rename Files with Table
* Jinja2 Template (deprecated)
* Plot Table(s)
* Export Figures with Datasources
* Matlab Calculator
* DIVA import plugin
* LAA import plugin
API changes:
^^^^^^^^^^^^
* sympathy.api.dtypes includes handling for reading typed values from text
input
* Deprecated ManagedNode base class.
* Removed deprecated methods from table, adaf, table_wrapper and adaf_wrapper.
* New (compatible) format for F(x) script, see :ref:`F(x)`.
* Deprecate support for ${signal0} syntax in calculator plugins.
* Deprecate imports() from calculator plugins.
News in 1.5.3
-------------
Platform changes:
^^^^^^^^^^^^^^^^^
* Improved behavior of locked subflows:
* Child nodes that have finished executing will give feedback immediately
instead of after the whole locked flow has completed.
* Child nodes that has executed successfully, in memory, will show up in
purple color to indicate that their output port data is unavailable.
* Output from locked child nodes will be shown like for other nodes, in the
Messages view.
* Modifying locked flows will no longer reload all nodes.
* Showing progress from child nodes.
* Locked subflows and lambdas (Apply, Map) will continue executing as much
as possible instead of aborting after the first node has failed.
* Subflow configuration wizard copes with nodes executed in locked subflows
and executes them when needed to ensure that progress can be made.
* Toggle-able filter for list parameters with mode selection saves vertical
space.
* Included brief description of current node state in tooltips.
* Documentation improvements:
* Links to example flows in the node documentation for many nodes, see
:ref:`example_flows`.
* Documented how to create custom library tags for third party libraries, see
:ref:`library_tags`.
* Documented how to create plugins for calculators, importer and exporter
nodes, see :ref:`pluginwriting`.
* Improved documentation for Tables with more modern APIs and descriptions of
attributes etc.
* Windows installer generates links and file associations that ignore local user
site - this makes the bundled Python more stand-alone and helps avoid issues
in case of package conflicts.
Figure and plotting updates:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
* Figure(s) can now take any sympathy object or lists of object instead of just
tables. The configuration output table is now optional and need to be added
explicitly to be used. Arguments can now be accessed using "arg" instead of
"table" in expressions, eg. arg['X'] gives column X if the argument is a
table.
* Improved the Figure node with new plot types and drawing primitives:
* Boxplots calculating and visualizing median, average, quantiles and outliers.
* Pie-charts.
* Timeline plots for drawing events or states.
* Sets of unconnected lines based on starting/ending XY points.
* Rectangles and Ellipses.
* Text and arrows (annotations) to point out or label parts of data.
* Images (using the generic Figure inputs).
* Added extended example "Figure Gallery" with 12 plots to show how
the new plot types and options can be used.
* Added options for drawing error-bars on scatter-plots and
bar-plots.
* Frames and axes can now toggle visibility and the XY spines of the
axes can be drawn at a predetermined XY data coordinate (eg. with
the axis through origo), or at a given point on the screen
(eg. center).
* Fixed bug showing tooltips for plot parameters.
Node/plugin changes:
^^^^^^^^^^^^^^^^^^^^
* Added error strategies to :ref:`org.sysess.sympathy.list.getitemlist`, when
the list is empty.
* Add different strategies for handling lists of different lengths in
:ref:`org.sysess.sympathy.tuple.ziptuple2`.
* Added median filtering to :ref:`Overlay Images` and :ref:`Overlay Images List`
* Added :ref:`Colormap lookup` to explicitly create colors from
values, useful when creating many figures where exact control of the
colors are needed. New color-maps added for discrete categories of
objects.
* Fixed bug in :ref:`Image to List` missing one of the extracted objects.
* Improved :ref:`Generic Calculator` with:
* More robust handling of dependencies between calculations.
* Compacted layout with a larger portion available for preview.
Input signals are shown in in the tree where functions
are shown.
* Drag and drop of calculations.
* Ability to set custom attributes for each calculation.
* Error handling for :ref:`Extract lambda nodes` with choice
between skip and error.
* :ref:`Match Tables Nodes`:
support for masked arrays.
* :ref:`Filter ADAFs`: will only auto-refresh when it has been enabled.
* :ref:`Adaf(s) to Table(s)`:
shows preview of available signals.
* Many nodes have toggle-able filter for their column selection editors.
* :ref:`org.sysess.sympathy.examples.daskvisualize`: improved detection
of Graphviz installation.
* :ref:`org.sysess.sympathy.data.table.ensuretablecolumns`: can create
type-dependent zero elements for missing columns.
* MDF importer plugin has better support for reading partially unsupported
or broken files.
* Avoid infinite loop when loading scikit-learn 0.20.
* Excel importer plugin copes with date values outside of supported range by
assigning masked values.
New nodes/plugins:
^^^^^^^^^^^^^^^^^^
* :ref:`Rename datasource with regex`: helps to simplify automatically
creating new datasource names based on existing file names.
* ADAF importer plugin for LabVIEW TDMS files.
* :ref:`org.sysess.sympathy.list.transposelist` swaps the outer two layers of a
nested list.
* :ref:`org.sysess.sympathy.keyvaluecalculation`: calculates simple descriptive
statistics for all columns in a table.
API changes:
^^^^^^^^^^^^
* Generalized interface for import and export nodes and their plugins.
* Opt-in support for toggle-able filter for single-select parameters with
combo box editors.
* Public APIs for creating importer and exporter plugins.
* Datasource: added getitem. Example use: ``datasource['type']``.
* Table: added setter for name. Example use: ``table.name = 'some name'``.
* New exception: SyColumnTypeError, raised when creating table columns of
unsupported type.
News in 1.5.2
-------------
Platform changes:
^^^^^^^^^^^^^^^^^
* Fixed problem causing the OK button to be disabled (grey) when configuring
subflows containing certain nodes (for example,
:ref:`org.sysess.sympathy.data.adaf.interpolateadaf` and
:ref:`org.sysess.sympathy.data.table.converttablecolumns`)
using the Wizard or Tabbed dialog.
News in 1.5.1
-------------
Platform changes:
^^^^^^^^^^^^^^^^^
* Improved performance overhead of nodes that simply select some columns of the
incoming data and pass them to the output.
* Significantly reduced file sizes of lambdas on disk.
* More careful disconnection of signals to improve stability and performance.
* Added option to *Unlink* linked subflows.
* Confirmation dialog when canceling subflow configurations with unsaved changes.
* Better feedback and handling of node errors when configuring subflows.
* Improved ability to open (somewhat corrupt) flows with duplicate identifiers,
instead of failing early.
* Cleared up separation between flows linked from library and normal linked
subflows.
* Disabled automatic documentation generation.
* Cleaned up command line options, "-" is used to separate words instead of
"_".
* Added *Recent* libraries, to simplify switching between libraries in use.
* Restructured documentation format for Nodes.
* New installation instructions for Linux and Mac OS.
* Saving warning filter between node executions to avoid interference.
* Improved deprecation warnings for nodes, on by default.
* Fixed some regressions causing library view not to update.
Node/plugin changes:
^^^^^^^^^^^^^^^^^^^^
* Improved feedback and validation for many scalar parameters. Incorrect values
are displayed in red and the tooltip shows why.
* Added progress and current list index, in case of error - to many list
nodes.
* In some cases, improved performance for :ref:`Replace values in Table`.
* Improved documentation regarding handling of unmatched values for
:ref:`Lookup Table`.
* :ref:`Select category in ADAFs` now respects the choice made in the combo box
for raster selection.
* :ref:`Datasource` nodes store UNIX paths in their configuration, but show
paths in native format.
* MDF Importer, improved performance when importing unsorted files and added
option to attempt to read incomplete files without error.
* ADAF exporter, improved GUI.
* CSV exporter, improved performance.
* @ca.changed()@, @ca.changed_up()@, and @ca.changed_down()@ can now all handle
masked arrays with scalar masks.
* Filtered several expected warnings produced by nodes.
New nodes:
^^^^^^^^^^^
* :ref:`org.sysess.sympathy.create.configureparameters`
* :ref:`org.sysess.sympathy.create.createparameters`
* :ref:`org.sysess.sympathy.data.table.createindextable`
New flows:
^^^^^^^^^^
* :ref:`org.sysess.lambda.map2flow`
* :ref:`org.sysess.lambda.map4flow`
* :ref:`org.sysess.lambda.configurablemap4flow`
News in 1.5.0
-------------
Sympathy for Data version 1.5.0 offers several improvements and new features,
such as workflows in the library and routing points for connections.
Nodes and flows from 1.3 and 1.4 should for the most part be compatible with
1.5.0.
.. * Connections are rendered as lines. This can be changed to *Spline* in
.. *Preferences -> General: connection shape* if you prefer the original look.
Platform changes:
^^^^^^^^^^^^^^^^^
* Markdown_ is now supported in text fields.
* Mark unavailable items in list views.
* list views and combobox views can be edited when customized with editor
attribute edit=True. This makes it possible to create selections that
includes columns that do not exist in the input data or without executing
previous nodes.
* Shorter tab labels for flows, only showing parent flow names as necessary.
* Only nodes that are in the current libraries can be used. Current libraries
depend on the configured libraries for the current flow and the global
selection from preferences.
* Allow subflows to specify custom svg icons.
* Route points for connections, see :ref:`route_points`.
* 0 based indexing is used more consistently, that is, 0 is the index of the
first element, etc. This behavior is more consistent with python code.
* English/US locale is used regardless of the host language settings.
* Generalized text-output port on all nodes.
* Flows can be added to library, see :ref:`flows_in_library`.
.. _Markdown: http://daringfireball.com/projects/markdown
API changes:
^^^^^^^^^^^^
* Improved backwards compatibility of ParameterLists during execution.
* ParameterLists return copies of its internal state disallowing direct
mutation.
* Ability to get and set dask arrays in adaf and table APIs.
See example nodes: :ref:`Dask max example`, :ref:`Dask stack example` and
:ref:`Dask tail example`.
* Made getitem, setitem usable as a shorthand way of working with arrays
columns in tables. For example table1['x'] = table2['y'].
Node/plugin changes:
^^^^^^^^^^^^^^^^^^^^
* Improved masked arrays support in input data to
:ref:`Select rows nodes