debugging - troubleshootingTroubleshooting and error tracing is greatly simplified by this improvement. Finding out why a field is not formatted as wanted or a result value is not what was expected has cost a lot of time in the past. This tedious task can now be substantially reduced using the new single-step processing (Debugging of Definitions) feature.

If a Definition does not run as expected you now only need to set a stopping point (Breakpoint) in the related Heater (function) and when the Definition is run in test mode processing it will stop at precisely this point and you can calmly inspect and analyse the input and output parameters, data types involved and further information about the process. Moreover, it is possible to proceed and inspect the next invocation of the Heater (function) for further analysis.

To activate a Breakpoint, hover the mouse over the Heater in the Designer, open the context menu with a right click and select the Breakpoint option. Thereafter the Heater is displayed with a dashed red circle in the Designer. Check it out for yourself. You will learn to love this function.

FlowHeater Server is now enhanced with another very useful function. There is now an option to record invocation history for each task the server performs. This makes it much easier to trace problems even in the case of tasks that were executed some time ago. It records the date and time at the start and end of each task invoked, as well as the details of each individual step, including run duration, result of the run (success, warning, error) and there is also an option to record the console output generated. The retention period for history logging can be individually configured in the server settings.

InMemory Adapter - for fast data processing/enrichmentThe new InMemory Adapter is considered an absolute highlight. The new InMemory Adapter was developed to take full advantage of the trend of increasingly powerful computers with more and more main memory, making the processing of intermediate data possible, completely in memory.

With the InMemory Adapter as many “internal” tables can be created as required, which can then be populated, enriched and transformed while running a Definition and of course subsequently used as input.

Tasks that previously required a temporary database table can be easily converted to using InMemory tables. As a demonstration the example “Merging CSV and Excel data using an SQLite database” is adapted to use InMemory tables, incl. video example.

Advantage: As processing is completely performed in main memory it is significantly faster than was previously possible.

The use of InMemory tables is in principle very simple. Insert the Adapter and if necessary define a name for the table. Fields can either be included in advance or further fields simply drawn from the field list in the Adapter by drag & drop. Optionally the data types can be modified for specific needs, although normally the default FlowHeater data type STRING should suffice.

On the READ side the data source of InMemory tables presented can also be filtered by a WHERE clause in a similar way to SQL tables. Naturally, the data can also be ordered in an alternative sort sequence.

Several processing steps in a DefinitionAs of this version FlowHeater now supports several processing steps within a single Definition (.fhd file). Any number of processing steps can be added to a Definition, which are then run consecutively in the sequence given.

Tip: FlowHeater Parameters are “globally” available within a Definition. This makes a number of interesting new options available for processing import/export data. For example, it is possible to record specific data from an upstream processing step in FlowHeater Parameters and use these values in a downstream processing step, e.g. as filter criteria.

Note: Heaters/Functions that are configured to fire “at the start” or “at the end” of the run now apply to each processing step! For example, if a script or SQL command must only be run at the very end of a Definition then this script (.NET Heater) or SQL command (SQL Heater) should only be defined in the last processing step.

Restrictions: At present several processing steps is only supported when using “Mass Data Mode”. The use of several processing steps is not presently supported with “Memory Mode”, and any attempt to use it this way will terminate with an error report.

Preview of later development: We offer here a brief outlook on future functionality that will profit from this new feature. The development of an “internal” memory-like Adapter is planned, using which will simplify the assembly, merging and processing of data from several sources that can then finally be exported.

FlowHeater Server - Exit CodesFlowHeater Server is enhanced with some powerful new functions by this version. Not only is it possible to monitor folders for the appearance of new files, but also now for modified files. Processing starts as soon as the file is no longer open in any other process.

In addition, it is now possible to check “Exit Codes” to determine if tasks executed were “Successful” or terminated with “Warnings” or with “Errors”. Formerly, only a fixed value of “0” could be checked, indicating success. Particular exit codes can be specified, as well as ranges like “0-10” or combinations such as “0, 5-10, 200”. Also new is that exit codes are given for warnings. The most recent run of a task is now highlighted in color in the task list. Yellow indicating the task ended with warnings and red when the task last ended with an error.

Notification options have also been extended. Email notifications can now be sent for warnings as well. Previously emails were only possible for successful runs or fatal errors.