News
Improved troubleshooting in the FlowHeater Server
In the opening status tab window, the FlowHeater Server Manager now displays an alert when errors or warnings have occurred since the last opening. The alert remains until explicitly reset by the user. In addition, while the Server Manager is open the alert is dynamically updated. There are also different colors that distinguish between warnings and error messages: errors are highlighted with a red button and warnings with a yellow button. The totals of errors and warnings that have occurred are also displayed.
Furthermore, the display of history data has been extended with extensive filters and limitation criteria. It is now possible to search explicitly across all tasks, e.g. only for errors or warnings, and the search can be more precisely limited in time. This means that even errors and warnings from earlier periods can be analyzed without any problems.
Improved debugging with single-step processing
debugging - troubleshootingTroubleshooting and error tracing is greatly simplified by this improvement. Finding out why a field is not formatted as wanted or a result value is not what was expected has cost a lot of time in the past. This tedious task can now be substantially reduced using the new single-step processing (Debugging of Definitions) feature.
If a Definition does not run as expected you now only need to set a stopping point (Breakpoint) in the related Heater (function) and when the Definition is run in test mode processing it will stop at precisely this point and you can calmly inspect and analyse the input and output parameters, data types involved and further information about the process. Moreover, it is possible to proceed and inspect the next invocation of the Heater (function) for further analysis.
To activate a Breakpoint, hover the mouse over the Heater in the Designer, open the context menu with a right click and select the Breakpoint option. Thereafter the Heater is displayed with a dashed red circle in the Designer. Check it out for yourself. You will learn to love this function.
FlowHeater Server logs invocation history
FlowHeater Server is now enhanced with another very useful function. There is now an option to record invocation history for each task the server performs. This makes it much easier to trace problems even in the case of tasks that were executed some time ago. It records the date and time at the start and end of each task invoked, as well as the details of each individual step, including run duration, result of the run (success, warning, error) and there is also an option to record the console output generated. The retention period for history logging can be individually configured in the server settings.
New InMemory Adapter for fast data processing/enrichment
InMemory Adapter - for fast data processing/enrichmentThe new InMemory Adapter is considered an absolute highlight. The new InMemory Adapter was developed to take full advantage of the trend of increasingly powerful computers with more and more main memory, making the processing of intermediate data possible, completely in memory.
With the InMemory Adapter as many “internal” tables can be created as required, which can then be populated, enriched and transformed while running a Definition and of course subsequently used as input.
Tasks that previously required a temporary database table can be easily converted to using InMemory tables. As a demonstration the example “Merging CSV and Excel data using an SQLite database” is adapted to use InMemory tables, incl. video example.
Advantage: As processing is completely performed in main memory it is significantly faster than was previously possible.
The use of InMemory tables is in principle very simple. Insert the Adapter and if necessary define a name for the table. Fields can either be included in advance or further fields simply drawn from the field list in the Adapter by drag & drop. Optionally the data types can be modified for specific needs, although normally the default FlowHeater data type STRING should suffice.
On the READ side the data source of InMemory tables presented can also be filtered by a WHERE clause in a similar way to SQL tables. Naturally, the data can also be ordered in an alternative sort sequence.
Support for several processing steps in a Definition
Several processing steps in a DefinitionAs of this version FlowHeater now supports several processing steps within a single Definition (.fhd file). Any number of processing steps can be added to a Definition, which are then run consecutively in the sequence given.
Tip: FlowHeater Parameters are “globally” available within a Definition. This makes a number of interesting new options available for processing import/export data. For example, it is possible to record specific data from an upstream processing step in FlowHeater Parameters and use these values in a downstream processing step, e.g. as filter criteria.
Note: Heaters/Functions that are configured to fire “at the start” or “at the end” of the run now apply to each processing step! For example, if a script or SQL command must only be run at the very end of a Definition then this script (.NET Heater) or SQL command (SQL Heater) should only be defined in the last processing step.
Restrictions: At present several processing steps is only supported when using “Mass Data Mode”. The use of several processing steps is not presently supported with “Memory Mode”, and any attempt to use it this way will terminate with an error report.
Preview of later development: We offer here a brief outlook on future functionality that will profit from this new feature. The development of an “internal” memory-like Adapter is planned, using which will simplify the assembly, merging and processing of data from several sources that can then finally be exported.
