.. This file is part of Sympathy for Data.
.. Copyright (c) 2020 Combine Control Systems AB
..
.. Sympathy for Data is free software: you can redistribute it and/or modify
.. it under the terms of the GNU General Public License as published by
.. the Free Software Foundation, version 3 of the License.
..
.. Sympathy for Data is distributed in the hope that it will be useful,
.. but WITHOUT ANY WARRANTY; without even the implied warranty of
.. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
.. GNU General Public License for more details.
..
.. You should have received a copy of the GNU General Public License
.. along with Sympathy for Data. If not, see .
.. _reporting-layers:
Creating New Layers
===================
The reporting tool is based on a concept where complicated plots can be
constructed from relatively simple specialized layers. Layers are put on top
of each other to produce a total richer content.
.. image:: layers3d.png
Layers are rendered using a given backend. Currently there is only a backend
based on `Matplotlib `_ available.
Writing New Layers
==================
Writing new layers can be a difficult process depending on the knowledge of
the details of the underlying backend. A layer is responsible for creating
and updating elements which should be added to its parent graph. The
reporting tool has a data binding system which should be utilized to update
layer properties properly.
In the standard library of Sympathy for Data below
``Library/Common/sylib/report/backends`` there are two folders, one for
layers and one for systems (backends). For each layer there should be an
ordinary ``__init__.py`` file, one ``layer.py`` containing miscellaneous
definitions and ``renderer_{backend}.py`` where ``renderer_mpl.py`` is the
renderer for the layer for the mpl (Matplotlib) backend.
Any new icons used must be added to the folder ``report/svg_icons`` and later
added to ``report/icons.py``.
Tour of Scatter Layer
---------------------
The scatter layer is a common plot type where graphical symbols represent a
coordinate pair in a two dimensional space. Its basic definition is based on
``layer.py``. The definition contains two dictionaries within a class;
``meta`` and ``property_definitions``. The first definition of meta data
below defines the visual icon of the layer, its label and default property
data. The number of items in ``data`` specifies the number of dimensions of
the layer, in this case two dimensions. To create your own two
dimensional layer it is safe to copy this structure and replace icon, label
and type with your own values.
.. image:: scatter_tree.png
.. code-block:: python
meta = {
'icon': icons.SvgIcon.scatter, # Shown in toolbar
'label': 'Scatter Plot', # Can be seen in tree
'default-data': {
'type': 'scatter', # Internal identifier
'data': [ # Appears in tree as dimension 1 and
{ # dimension 2 below Scatter Plot.
'source': '',
'axis': ''
},
{
'source': '',
'axis': ''
}
]
}
}
The next block called ``property_definitions`` defines all properties which can
be modified by the user. The code which takes care of changes is implemented in
the renderer, i.e. ``renderer_mpl.py`` for Matplotlib. For the scatter plot
example we have six different properties defined: ``name``, ``symbol``,
``size``, ``face-color``, ``edge-color`` and ``alpha``. Each property is
defined by a dictionary whose fields depend on the type of data it contains.
All properties require four fields: ``type``, ``label``, ``icon`` and
``default``.
The currently allowed values for the ``type`` field giving property editors
specialized for each type are:
* string
* integer
* float
* color
* boolean
* list
* datasource
* colorscale
* image
The ``label`` is a free text label shown to the left of the editor in the
property editor as seen in the picture below.
.. image:: scatter_properties.png
The ``icon`` field is a special icon for the given property. It is currently
not having any effect.
The default value of the property editor is given in ``default``. Any new
property editor without any previous value is given the default value.
For lists an extra field called ``options`` must be given which contains a
list of available choices. The ``default`` field must contain a value equal
to one of the options.
For numeric types like integer and float there is a field called ``range``
which defines the behavior of the
`Qt spin box `_ in the property editor.
The ``range`` field is a dictionary with three fields called ``min`` (minimum
allowed value), ``max`` (maximum allowed value)
and ``step`` (step size when clicking arrow buttons).
For some numeric and color values it is possible to bind a data source. The
binding behavior is not automatic since it must be implemented in the
renderer of the layer. The complexity of implementing a binding differs
depending on the plotting framework used for the backend. To indicate that a
property is possible to bind to a data source a field called
``scale_bindable`` should be set to true.
The entire code for a scatter plot definition in ``layer.py`` becomes:
.. code-block:: python
from sylib.report import layers
from sylib.report import icons
class Layer(layers.Layer):
"""
ScatterLayer
"""
meta = {
'icon': icons.SvgIcon.scatter,
'label': 'Scatter Plot',
'default-data': {
'type': 'scatter',
'data': [
{
'source': '',
'axis': ''
},
{
'source': '',
'axis': ''
}
]
}
}
property_definitions = {
'name': {'type': 'string',
'label': 'Name',
'icon': icons.SvgIcon.blank,
'default': 'Scatter Plot'},
'symbol': {'type': 'list',
'label': 'Symbol',
'icon': icons.SvgIcon.blank,
'options': ('point', 'circle', 'square'),
'default': 'circle'},
'size': {'type': 'float',
'label': 'Size',
'icon': icons.SvgIcon.blank,
'scale_bindable': True,
'range': {'min': 10, 'max': 1000, 'step': 25},
'default': 50.0},
'face-color': {'type': 'color',
'label': 'Face Color',
'icon': icons.SvgIcon.blank,
'scale_bindable': True,
'default': '#809dd5'},
'edge-color': {'type': 'color',
'label': 'Edge Color',
'icon': icons.SvgIcon.blank,
'default': '#000000'},
'alpha': {'type': 'float',
'label': 'Alpha',
'range': {'min': 0.0, 'max': 1.0, 'step': 0.1},
'icon': icons.SvgIcon.blank,
'default': 1.0}
}
To implement a Matplotlib renderer for the scatter layer we have to write
some code. The file should be called ``renderer_mpl.py``. First we need some
definitions.
.. code-block:: python
import functools
from sylib.report import plugins
from sylib.report import editor_type
mpl_backend = plugins.get_backend('mpl') # get backend for matplotlib
# Mapping between symbol name to symbol used in MPL.
SYMBOL_NAME_TO_SYMBOL = {
'point': '.',
'circle': 'o',
'square': 's'
}
SYMBOL_TO_MARKER_NAME = {v: k for k, v in SYMBOL_NAME_TO_SYMBOL.iteritems()}
def create_layer(binding_context, parameters):
"""
Build layer for MPL and bind properties using binding context.
:param binding_context: Binding context.
:param parameters: Dictionary containing:
'layer_model': a models.GraphLayer instance,
'axes': the MPL-axes to add layer to,
'canvas': current canvas (Qt-widget) we are rendering to,
'z_order': Z-order of layer.
"""
A common pattern is to bundle parameters which are often used in callback
functions to make code shorter.
.. code-block:: python
context = {
'binding_context': binding_context,
'path_collection': None,
'layer_model': parameters['layer_model'],
'axes': parameters['axes'],
'canvas': parameters['canvas'],
'z_order': parameters['z_order'],
'properties': [],
'drawing': False
}
Depending on the plotting framework different strategies needs to be
developed to handle any property or data updates properly. The strategy might
have to differ between different plot types within the same framework since
plots can behave very different. Here we are using a single callback for
updating data which takes the context as its first argument and ignoring the
value sent.
There are two ways to update an MPL-plot. Either rebuild everything from
scratch or only update the specific objects involved. The latter method
generally gives quicker response but might be difficult to get to work properly.
For cases when the plotting framework does not want to do what you expect a
good fallback solution is to redraw everything. How to do this is shown later
in this text.
.. code-block:: python
def update_data(context_, _):
properties_ = context_['layer_model'].properties_as_dict()
# Remove old path collection first.
if context_['path_collection'] is not None:
context_['path_collection'].remove()
context_['path_collection'] = None
For convenience there is a method in the layer data model (defined in
``models.py``) which extracts all data and data properties for you.
Matplotlib gives errors when the length of data in x and y does not match so
we cannot do anything until those lengths match and are not zero.
.. code-block:: python
(x_data_, y_data_), _ = \
context_['layer_model'].extract_data_and_properties()
if len(x_data_) != len(y_data_) or len(x_data_) == 0:
return
Next we extract all property values needed to be able to generate the plot.
Some of the values need to be scaled and we are using a function from the
backend to help perform those calculations. Such utility functions are
specific to each backend since each plotting framework needs to be treated
differently. If no scale is present the scale function only returns the
scalar value. For ``edge-color`` we did not activate any data binding so we
could have omitted the scale function, but in this case it does not make any
difference. The alternative is to fetch the property value directly as done
for ``marker``.
.. code-block:: python
scale = functools.partial(mpl_backend.calculate_scaled_value,
context_['layer_model'])
size = scale(properties_['size'])
face_color = scale(properties_['face-color'])
edge_color = scale(properties_['edge-color'])
alpha = scale(properties_['alpha'])
marker = properties_['symbol'].get()
Here we just generate a scatter plot and store the resulting objects such
that we can remove them later on. For Matplotlib we have focused on using
existing plotting routines as far as possible. If performance is to be
optimized it is probably more efficient to write each plotting routine from
scratch using low level components of Matplotlib.
.. code-block:: python
context_['path_collection'] = context_['axes'].scatter(
x_data_, y_data_, s=size, c=face_color, alpha=alpha,
marker=SYMBOL_NAME_TO_SYMBOL.get(marker, 'o'),
edgecolors=edge_color,
zorder=context_['z_order'])
Using ``draw_idle`` postpones rendering until the event loop is free. This
gives better responsibility of the GUI.
.. code-block:: python
context_['canvas'].draw_idle()
Back to the code running before the callback is called. First we have to
extract data and data source properties and perform the initial rendering of
the plot.
.. code-block:: python
(x_data, y_data), data_source_properties = context[
'layer_model'].extract_data_and_properties()
if len(x_data) != len(y_data) and len(x_data) == 0:
update_data(context, None)
The reporting framework contains a simple data binding system which
automatically calls callbacks of bound targets such that necessary actions can
take place on write. Wrapping and binding properties is so common that we
implemented a utility function for this in the backend for matplotlib. The
following code makes sure that the ``update_data`` callback gets called each
time the data source is changed.
.. code-block:: python
if data_source_properties is not None:
mpl_backend.wrap_and_bind(binding_context,
parameters['canvas'],
data_source_properties[0],
data_source_properties[0].get,
functools.partial(update_data, context))
mpl_backend.wrap_and_bind(binding_context,
parameters['canvas'],
data_source_properties[1],
data_source_properties[1].get,
functools.partial(update_data, context))
In order to have the axes updated properly we add a tag to the property
editor to force an entire rebuild of the plots.
.. code-block:: python
# This is used to force update of axis range.
data_source_properties[0].editor.tags.add(
editor_type.EditorTags.force_rebuild_after_edit)
data_source_properties[1].editor.tags.add(
editor_type.EditorTags.force_rebuild_after_edit)
And for the rest of the properties we only need to call ``update_data`` on
any changes.
.. code-block:: python
# Bind stuff.
properties = parameters['layer_model'].properties_as_dict()
for property_name in ('symbol', 'size', 'face-color', 'edge-color',
'alpha'):
mpl_backend.wrap_and_bind(binding_context,
parameters['canvas'],
properties[property_name],
properties[property_name].get,
functools.partial(update_data, context))
To learn more about how layers can be implemented you are encouraged to study
all renderers of layers and the backend code.