Creating New Layers

The reporting tool is based on a concept where complicated plots can be constructed from relatively simple specialized layers. Layers are put on top of each other to produce a total richer content.


Layers are rendered using a given backend. Currently there is only a backend based on Matplotlib available.

Writing New Layers

Writing new layers can be a difficult process depending on the knowledge of the details of the underlying backend. A layer is responsible for creating and updating elements which should be added to its parent graph. The reporting tool has a data binding system which should be utilized to update layer properties properly.

In the standard library of Sympathy for Data below Library/Common/sylib/report/backends there are two folders, one for layers and one for systems (backends). For each layer there should be an ordinary file, one containing miscellaneous definitions and renderer_{backend}.py where is the renderer for the layer for the mpl (Matplotlib) backend.

Any new icons used must be added to the folder report/svg_icons and later added to report/

Tour of Scatter Layer

The scatter layer is a common plot type where graphical symbols represent a coordinate pair in a two dimensional space. Its basic definition is based on The definition contains two dictionaries within a class; meta and property_definitions. The first definition of meta data below defines the visual icon of the layer, its label and default property data. The number of items in data specifies the number of dimensions of the layer, in this case two dimensions. To create your own two dimensional layer it is safe to copy this structure and replace icon, label and type with your own values.

meta = {
    'icon': icons.SvgIcon.scatter,  # Shown in toolbar
    'label': 'Scatter Plot',        # Can be seen in tree
    'default-data': {
        'type': 'scatter',          # Internal identifier
        'data': [                   # Appears in tree as dimension 1 and
            {                       # dimension 2 below Scatter Plot.
                'source': '',
                'axis': ''
                'source': '',
                'axis': ''

The next block called property_definitions defines all properties which can be modified by the user. The code which takes care of changes is implemented in the renderer, i.e. for Matplotlib. For the scatter plot example we have six different properties defined: name, symbol, size, face-color, edge-color and alpha. Each property is defined by a dictionary whose fields depend on the type of data it contains. All properties require four fields: type, label, icon and default. The currently allowed values for the type field giving property editors specialized for each type are:

  • string
  • integer
  • float
  • color
  • boolean
  • list
  • datasource
  • colorscale
  • image

The label is a free text label shown to the left of the editor in the property editor as seen in the picture below.


The icon field is a special icon for the given property. It is currently not having any effect.

The default value of the property editor is given in default. Any new property editor without any previous value is given the default value.

For lists an extra field called options must be given which contains a list of available choices. The default field must contain a value equal to one of the options.

For numeric types like integer and float there is a field called range which defines the behavior of the Qt spin box in the property editor. The range field is a dictionary with three fields called min (minimum allowed value), max (maximum allowed value) and step (step size when clicking arrow buttons).

For some numeric and color values it is possible to bind a data source. The binding behavior is not automatic since it must be implemented in the renderer of the layer. The complexity of implementing a binding differs depending on the plotting framework used for the backend. To indicate that a property is possible to bind to a data source a field called scale_bindable should be set to true.

The entire code for a scatter plot definition in becomes:

from import layers
from import icons

class Layer(layers.Layer):
    meta = {
        'icon': icons.SvgIcon.scatter,
        'label': 'Scatter Plot',
        'default-data': {
            'type': 'scatter',
            'data': [
                    'source': '',
                    'axis': ''
                    'source': '',
                    'axis': ''
    property_definitions = {
        'name': {'type': 'string',
                 'label': 'Name',
                 'icon': icons.SvgIcon.blank,
                 'default': 'Scatter Plot'},

        'symbol': {'type': 'list',
                   'label': 'Symbol',
                   'icon': icons.SvgIcon.blank,
                   'options': ('point', 'circle', 'square'),
                   'default': 'circle'},

        'size': {'type': 'float',
                 'label': 'Size',
                 'icon': icons.SvgIcon.blank,
                 'scale_bindable': True,
                 'range': {'min': 10, 'max': 1000, 'step': 25},
                 'default': 50.0},

        'face-color': {'type': 'color',
                       'label': 'Face Color',
                       'icon': icons.SvgIcon.blank,
                       'scale_bindable': True,
                       'default': '#809dd5'},

        'edge-color': {'type': 'color',
                       'label': 'Edge Color',
                       'icon': icons.SvgIcon.blank,
                       'default': '#000000'},

        'alpha': {'type': 'float',
                  'label': 'Alpha',
                  'range': {'min': 0.0, 'max': 1.0, 'step': 0.1},
                  'icon': icons.SvgIcon.blank,
                  'default': 1.0}

To implement a Matplotlib renderer for the scatter layer we have to write some code. The file should be called First we need some definitions.

import functools
from import plugins
from import editor_type
mpl_backend = plugins.get_backend('mpl') # get backend for matplotlib

# Mapping between symbol name to symbol used in MPL.
    'point':  '.',
    'circle': 'o',
    'square': 's'
SYMBOL_TO_MARKER_NAME = {v: k for k, v in SYMBOL_NAME_TO_SYMBOL.iteritems()}

def create_layer(binding_context, parameters):
    Build layer for MPL and bind properties using binding context.
    :param binding_context: Binding context.
        :param parameters: Dictionary containing:
                           'layer_model': a models.GraphLayer instance,
                           'axes': the MPL-axes to add layer to,
                           'canvas': current canvas (Qt-widget) we are rendering to,
                           'z_order': Z-order of layer.

A common pattern is to bundle parameters which are often used in callback functions to make code shorter.

context = {
    'binding_context': binding_context,
    'path_collection': None,
    'layer_model': parameters['layer_model'],
    'axes': parameters['axes'],
    'canvas': parameters['canvas'],
    'z_order': parameters['z_order'],
    'properties': [],
    'drawing': False

Depending on the plotting framework different strategies needs to be developed to handle any property or data updates properly. The strategy might have to differ between different plot types within the same framework since plots can behave very different. Here we are using a single callback for updating data which takes the context as its first argument and ignoring the value sent.

There are two ways to update an MPL-plot. Either rebuild everything from scratch or only update the specific objects involved. The latter method generally gives quicker response but might be difficult to get to work properly. For cases when the plotting framework does not want to do what you expect a good fallback solution is to redraw everything. How to do this is shown later in this text.

def update_data(context_, _):
    properties_ = context_['layer_model'].properties_as_dict()
    # Remove old path collection first.
    if context_['path_collection'] is not None:
        context_['path_collection'] = None

For convenience there is a method in the layer data model (defined in which extracts all data and data properties for you. Matplotlib gives errors when the length of data in x and y does not match so we cannot do anything until those lengths match and are not zero.

(x_data_, y_data_), _ = \
if len(x_data_) != len(y_data_) or len(x_data_) == 0:

Next we extract all property values needed to be able to generate the plot. Some of the values need to be scaled and we are using a function from the backend to help perform those calculations. Such utility functions are specific to each backend since each plotting framework needs to be treated differently. If no scale is present the scale function only returns the scalar value. For edge-color we did not activate any data binding so we could have omitted the scale function, but in this case it does not make any difference. The alternative is to fetch the property value directly as done for marker.

scale = functools.partial(mpl_backend.calculate_scaled_value,
size = scale(properties_['size'])
face_color = scale(properties_['face-color'])
edge_color = scale(properties_['edge-color'])
alpha = scale(properties_['alpha'])
marker = properties_['symbol'].get()

Here we just generate a scatter plot and store the resulting objects such that we can remove them later on. For Matplotlib we have focused on using existing plotting routines as far as possible. If performance is to be optimized it is probably more efficient to write each plotting routine from scratch using low level components of Matplotlib.

context_['path_collection'] = context_['axes'].scatter(
    x_data_, y_data_, s=size, c=face_color, alpha=alpha,
    marker=SYMBOL_NAME_TO_SYMBOL.get(marker, 'o'),

Using draw_idle postpones rendering until the event loop is free. This gives better responsibility of the GUI.


Back to the code running before the callback is called. First we have to extract data and data source properties and perform the initial rendering of the plot.

(x_data, y_data), data_source_properties = context[

if len(x_data) != len(y_data) and len(x_data) == 0:
    update_data(context, None)

The reporting framework contains a simple data binding system which automatically calls callbacks of bound targets such that necessary actions can take place on write. Wrapping and binding properties is so common that we implemented a utility function for this in the backend for matplotlib. The following code makes sure that the update_data callback gets called each time the data source is changed.

if data_source_properties is not None:
                              functools.partial(update_data, context))
                              functools.partial(update_data, context))

In order to have the axes updated properly we add a tag to the property editor to force an entire rebuild of the plots.

# This is used to force update of axis range.

And for the rest of the properties we only need to call update_data on any changes.

# Bind stuff.
properties = parameters['layer_model'].properties_as_dict()

for property_name in ('symbol', 'size', 'face-color', 'edge-color',
                              functools.partial(update_data, context))

To learn more about how layers can be implemented you are encouraged to study all renderers of layers and the backend code.