Real-time DataLogger reference

Available from firmware 2019.6

Sources and data types

The DataLogger can record data from any IN or OUT ports and variables. The following data sources are available:

• Global Data Space IN and OUT ports
  • Type real-time program (C++, IEC 61131-3 and MATLAB®/Simulink®) – task-synchronous mode
  • Type component
  • Global IEC 61131-3 variables

The following data types are supported:

• Elementary data types (according to Supported elementary data types);
  all STRING variables, regardless of their length, are supported and thus belong to the elementary data types.

Schematic overview

Find all details on the attributes in the tables and sections below.  

Configuration parameters explained visually

Sampling intervals, publishing intervals, and writing intervals? Why are there so many parameters for transporting the data?

Here's why this is the best way to log data in a real-time automation environment ‒ explained by Martin Boers, Technical Specialist in the Product Management PLCnext Runtime System at Phoenix Contact Electronics: 

 05m:26s     HDTV 720p     English     none

XML configuration file

An XML configuration file for the DataLogger is structured as shown in the following example:

<?xml version="1.0" encoding="utf-8"?>
  <DataLoggerConfigDocument
     xmlns="http://www.phoenixcontact.com/schema/dataloggerconfig"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
     xsi:schemaLocation="http://www.phoenixcontact.com/schema/dataloggerconfig.xsd">
    <General name="data-logger" samplingInterval="100ms" publishInterval="500ms" bufferCapacity="10"/>
    <Datasink type="db" dst="test.db" rollover="true" maxFiles="3" writeInterval="1000" 
       maxFileSize="4000000" storeChangesOnly="false" tsfmt="Iso8601"/>
    <Variables>
      <Variable name = "Arp.Plc.ComponentName/GlobalUint32Var"/>
      <Variable name = "Arp.Plc.ComponentName/PrgName.Uint32Var"/>
      <Variable name = "Arp.Plc.ComponentName/PrgName.StuctVar.IntElement"/>
      <Variable name = "Arp.Plc.ComponentName/PrgName.IntArrayVarElement[1]"/>
    </Variables>
  </DataLoggerConfigDocument>

As you can see in the structure, there are only a few XML elements to be configured.

Within the <DataLoggerConfigDocument> schema, there are always:

• <General>
• <Datasink>
• <Variables>, containing one or more <Variable> elements

Attributes for file-based configuration

Of course, there's a bunch of attributes adding some complexity. See the following 3 tables in which the attributes are grouped by the XML tags they belong to. Where there's much more to explain to an attribute you will find additional information below the tables.

<General>

Attribute	Description
name	Unique name of the logging session. Note: Must not begin with "PCWE_" which is reserved for the triggered Logic Analyzer of the PLCnext Engineer.
samplingInterval	Interval during which the data points of a variable are created, e.g. samplingInterval ="50ms".  The default value is500ms. The following suffixes can be used: ms, s, m, h.
taskContext	Available from firmware 2021.6 or newer. The name of an ESM task which samples the values of all variables of this session, e.g. taskContext="myTaskName".  The attribute is optional. If it is configured the samplingInterval attribute will be ignored.
publishInterval	Frequency for forwarding the collected data from the ring buffer to the data sink, e.g. publishInterval="1s". The default value is 500ms. The following suffixes can be used: ms, s, m, h. Note: This attribute will be ignored if a taskContext is also configured.
bufferCapacity	Configuration of the buffer capacity. Capacity of data sets for the internal buffer memory. The default value is 2.

<Datasink>

In each cycle, the values of all ports of a task are stored in a ring buffer. Therefore, the capacity of the ring buffer determines the maximum number of task cycles that can be recorded before data must be forwarded to the data sink. The data to be archived is written to an SQLite database.  For each configured DataLogger instance, a separate SQLite database is created.

Attribute	Description
type	Configuration of the data sink: db: (SQLite) Database file volatile: RAM only (Note: maxFileSize must be larger than 150000000 (approximately 150 MB)
dst	With file-based configuration, this is the file name and path under which the data sink is to be stored. If no specific path is given it will be placed in the working directory of the firmware which is /opt/plcnext/. Note: The DataLogger does not create folders. If you want to store a data sink under a specific path, you have to create it first. If the configuration is made with the PLCnext Engineer interface (available from software 2020.6), this attribute is set to dst="/opt/plcnext/logs/datalogger/<LogSessionName>.db" where  <LogSessionName> is what you have written into the "Name" field of the interface.
rollover	true: Once the maximum file size is reached, the file is closed and renamed with an index starting from 0 (e.g. database.db.0). Then a new file is created. Every file with an index is closed and can be copied for evaluation.  The current data is always logged in the database that is defined in the attribute dst. false:  When rollover is set to false and the maximum file size is reached, a configurable amount of the oldest data is deleted before the record proceeds. The amount of data to be deleted is configured with the attribute deleteRatio.
writeInterval	Number of data records the DataLogger collects and writes to the SD card. The default value is 1000 to keep write access operations to the SD card as low as possible. In other words, as soon as 1000 data records have been transferred to the data sink, they are grouped in a block and written to the SD card. When the data sink or the firmware is closed, all the values that have not yet been transferred are written to the SD card.  Note: If the value of the attribute writeInterval is low, the resulting high number of write operations to the SD card might cause performance problems. If a faster writeInterval is required, Phoenix Contact recommends to create the database in the RAM.  Otherwise, it is possible that the data cannot be written to the SD card in the required speed. This may result in the loss of data. If there is a loss of data it will be displayed in the database in the ConsistentDataSeries column (see details below at Data consistency check).
maxFileSize	Maximum memory size of the log file in bytes. (Note: maxFileSize must be larger than 150000000 (approximately 150 MB) in case a volatile sink is used.
maxFiles	Maximum number of rolling files (default value is 1).  The rollover attribute must be set to true. When the maximum number of files is reached, the oldest file will be deleted. The file index of the closed files will be counted up further.  If the maximum number of files is set to 0 (maxFiles="0") the behaviour corresponds to a deactivated rollover (rollover="false"). If the maximum number of files is set to a negative number (e.g. -1) the file number limitation is deactivated. This results in logging activity until the memory is full. The default value is -1.  Note: When the value for maxFiles is 1, rollover is set to true and the maximum file size is reached, a configurable amount (attribute deleteRatio) of the oldest data in the database is deleted. The deleteRatio is related to the maximum file size that is defined with the attribute maxFileSize.
storeChangesOnly	true: The values are only stored if they change. If a value stays the same, it is defined as NULL in the database. false: The values are always stored, even if they do not change. (see details below at Recording mode)
deleteRatio	(available from firmware 2020.0 LTS) Percentage of maximum memory size to be deleted for the logging of new data. Default is 30 %. This attribute defines the amount of data that is deleted before new data is written into the database. The old data is deleted when the value that is defined in maxFileSize is reached and the attribute rollover is set to false. The value for deleteRatio must be provided as an unsigned integer value (16 bit). It must be in the range from 1 to 100. The value corresponds to the percentage of old data to be deleted.  Examples: 5 = 5 % of old data is deleted. 30 = 30 % of old data is deleted. Note: For large data sinks (larger than 10 MB) the deleteRatio should be set to a value lower than the default to avoid data loss during the deletion process. E.g., to delete 15 MB (= 30 % of a 50 MB data sink) on an SD Card may take so much time that the limits of values being buffered and published to the RAM are exceded. In this case, some values get lost which you can detect in the interrupted series of timestamps.
tsfmt	(available from firmware 2020.0 LTS) Configuration of the timestamp format Raw: The timestamp is stored as 64 bit integer value. Iso8601: The timestamp is stored in the ISO 8601 format with microsecond accuracy. (see details below at Timestamp) Note: Logging with Iso8601 timestamps is on the expense of performance. In high-performance scenarios maybe converting the timestamp after logging is a better way.

<Variables>

Attribute	Description
Variable name	Complete name (URI) of a variable or a port whose values are to be recorded. Example:  Arp.Plc.ComponentName/PrgName.Uint32Var A maximum of 996 variables per session is allowed.

<TriggerCondition>

Available from firmware version 2020.3

From firmware version 2020.3 on you have the possibility to define trigger conditions for starting a DataLogger session via the XML configuration file. To do this, use the element <TriggerCondition> and the attributes described below.

This element starts a list of <RpnItem> where one item consists of an attribute type and a text that, depending on the type, either names a variable, a constant or an operation. The optional attributes postCycles and preCycles can be used to specify the amount of datasets recorded before and after the trigger condition is fulfilled.

Note: RPN (Reverse Polish Notation) is used for the configuration of the trigger.

<TriggerCondition>

Attribute	Description
postCycles	The postCycles attribute is optional. It defines the number of datasets that are recorded after a trigger event.
preCycles	The preCycles attribute is optional. It defines the number of datasets that are recorded before a trigger event.
taskContext	Name of the task in which the trigger condition is evaluated.

<RpnItem type>

List of trigger items. A trigger item can be a variable or a constant or an operation.

Attribute	Description
Variable	The attribute type Variable is used to define a single variable as a trigger condition item. It must contain the complete name (URI) of a variable or a port whose values are to be considered. Example: Arp.Plc.ComponentName/PrgName.Uint32Var
Constant	The attribute type Constant is used to define a constant as a trigger condition item. The item must contain the value of the constant, e.g. 5.
Operation	The attribute is used to define an operation. The following values are valid for the Operation attribute: None - No trigger condition, so recording starts immediately Equals - Starts recording if Variable/Constant 1 is equal to Variable/Constant 2 NotEqual - Start recording if Variable/Constant1 is not equal to Variable/Constant2 GreaterThan - Start recording if Variable/Constant1 is greater than Variable/Constant2 GreaterEqual - Start recording if Variable/Constant1 is greater than or equal to Variable/Constant2 LessThan - Start recording if Variable/Constant1 is less than Variable/Constant2 LessEqual - Start recording if Variable/Constant1 is less or equal to Variable/Constant2 Modified - Start recording when a modification of the Variable/Constant1 is detected RisingEdge - Start recording when a positive (rising) edge of the Variable/Constant1 is detected FallingEdge - Start recording when a negative (falling) edge of the Variable/Constant1 is detected And - Start recording if TriggerCondition1 and TriggerCondition2 is true Or - Start recording if TriggerCondition1 or TriggerCondition2 is true Not - Logical not

Example:

The trigger condition (Variable a > Variable b) & (Variable c > Variable d) can be configured using the following list (a, b, c, and d are used in this example instead of the complete names (URI) of the variables or ports for better readability):

<TriggerCondition postCycles="200" preCycles="100" taskContext="Cyclic100">
    <RpnItem type="Variable">a</RpnItem>
    <RpnItem type="Variable">b</RpnItem>
    <RpnItem type="Operation">Greater</RpnItem>
    <RpnItem type="Variable">c</RpnItem>
    <RpnItem type="Variable">d</RpnItem>
    <RpnItem type="Operation">Greater</RpnItem>
    <RpmItem type="Operation">And</RpnItem>
</TriggerCondition>

Database layout

The values of the configured variables are saved in a table inside the SQLite database. The default path for the database files on your controller is /opt/plcnext. The database files are saved as *.db files. The file system of the controller is accessed via the SFTP protocol. Use a suitable SFTP client software for this, e.g., WinSCP. 

Copy the *.db files to your PC and use a suitable software tool to open and evaluate the *.db files (e.g. DB Browser for SQLite). 

Depending on your configuration, a database table that is created by the DataLogger can consist of the following columns:

• Timestamp:
  Timestamp for the logged variable value (see details below at Timestamp).
• Consistent Data Series:
  This index shows if there is a inconsistency in the logged data (see details below at ConsistentDataSeries).
• Task/Variable:
  One column for each variable that is configured for data logging. The column name consists of the task name and the variable name (see details below at Variables).
• Task/Variable_change_count:
  In case of storeChangesOnly="true" (see details below at Recording_mode), this column serves as change counter. There is a change counter for every configured variable.

Time stamp

The DataLogger provides a time stamp for each value of a port. Only one time stamp is generated for ports from the same task because this time stamp is identical for all the values of the task. Time resolution has a precision of 100 ns.

• Firmware 2019.6 to 2019.9 :
  The time stamp is always displayed as raw 64 bit integer value.
• From firmware 2020.0 LTS:
  It is possible to configure the format of the time stamp inside the database. It can be displayed as ISO 8601 or as raw 64 bit integer value.

Despite the format, all time stamps are reported using the UTC timezone. The implementation and internal representation complies to the Microsoft® .NET DateTime class, see the documentation of the DateTime.ToBinary method on  docs.microsoft.com. 

The time stamp is created in the task cycle via the system time of the controller. It is set at the start of the task (task executing event) and maps exactly the cycle time of the task, so that the values of the following task cycles are always one interval apart.

Data consistency check

If recording gaps caused by performance problems or memory overflow occur this information will be saved in the data sink. If there is a loss of data it will be displayed in the database in the ConsistentDataSeries column.

This column can contain the values 0 or 1:

• Value 0:
  If the value is 0 a data gap occurred during the recording of the preceding data series. The first data series always has the value 0 because there is no preceding data series for referencing.
• Value 1: 
  If the value is 1 the data is recorded without a gap related to the preceding data series. Therefore the data series tagged with a 1 is consistent to the preceding data series. 

Example from a database with an indicated data gapClick to open an example from a database with an indicated data gap

Recording mode

The recording mode is set by the attribute storeChangesOnly. There are two recording modes available:

• Endless mode
  The DataLogger records the data in endless mode. All the ports and variables configured for recording are recorded without interruption (storeChangesOnly="false"). 
• Save on change
  The DataLogger only records the data when they change. If the value stays the same it is displayed in the data base with a NULL (storeChangesOnly="true"). 

Examples for storeChangesOnly configuration

Note: In these examples the time stamp is displayed in a readable format. In a *.db file generated by the DataLogger, the time stamp is UTC of typeArp::DateTime. It is displayed as 64 bit value in the database. The implementation and internal representation complies to the .NET DateTime class, refer to the documentation of DateTime struct at https://docs.microsoft.com to convert the time stamp into a readable format.

Attribute storeChangesOnly="false"

show this exampleshow this example

Attribute storeChangesOnly="true"

show this exampleshow this example

Attribute storeChangesOnly="false" and variables from different tasks

show this exampleshow this example

 

 

• Published/reviewed: 2024-09-24   ☀  Revision 073 •