How to use rsync to back up files from a PLCnext Control device

The latest release of PLCnext Control firmware (2020.3.1) introduces the Linux command-line utility “rsync”. Rsync can be used to back up or synchronise files between devices on the same network, or across remote networks. This article demonstrates how to use rsync to synchronise files between a PLCnext Control device and a PC.

Background

Many automation applications have a requirement to record time-series process data such as temperatures, pressures, flow rates, levels, and valve positions. PLCs can usually record time-stamped data with higher frequency and precision than devices that are further from the process (e.g. OPC UA clients), so the PLC is often given this task.

There are generally two places where time-series process data, generated by the PLC, can be stored:

1. On the PLC itself. Applications that store data directly on a PLCnext Control device include:
   • The real-time datalogger that comes with the PLCnext Runtime.
   • The datalogger app from Phoenix Contact in the PLCnext Store.
   • The data logger app from FourZero in the PLCnext Store.
   • Your own custom data logger, like the one described in these Makers Blog posts from Sven Lemmens in the PLCnext Community:
     • Datalogging in IEC 61131-3: The basic principle
     • Datalogging in IEC 61131-3: Create your own file manager
2. On an external device or server, perhaps with some limited local buffering in case of network interruption. Examples include:
   • Proficloud Time-Series Data (TSD) service.
   • Other cloud services such as AWS, Azure, Google Cloud or IXON Cloud.
   • MQTT services.
   • SQL databases populated directly using SQL INSERT queries.

This article will not discuss the relative benefits of each of these data storage methods, but will consider option 1 above, where data is stored directly on the PLC.

The problem

PLCs are not designed as mass-storage devices, and so the volume of data that can be stored on a PLCnext Control device is limited. All good data loggers provide mechanisms to ensure that the available storage capacity will not be exhausted, for example by deleting the oldest data once a certain time or data quantity threshold has been exceeded.

On the PLCnext Community Forum, there have been discussions about how the storage capacity of a PLCnext Control device could be increased, for example through the use of larger removeable SD cards, or possibly left-side extension modules containing large capacity solid-state memory. However, at some point the cost of storing more and more data directly on the PLC becomes an issue. In addition, this data is probably needed on other machines for analysis and/or long-term archiving.

So, the question becomes: how best to transfer data files from the PLC to another network device? It is possible to use standard Linux utilities like secure copy (scp) and secure file transfer protocol (sftp), however PLCnext Control devices now offer another solution: rsync.

What is rsync?

To quote from the rysnc man page (link to the delta-transfer algorithm added by me):

Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Similar to the way scp works, rsync clients can exchange data with any device that is set up as an rsync server. That rsync server could be:

• A Network Accessible Storage (NAS) device, perhaps mounted in the control cabinet alongside the PLC. Many NAS devices include rsync server capability as standard.
• An operator workstation running Windows or Linux.
• A secure, high-availability corporate server that is accessible from the PLC.
• A cloud server hosted (for example) on AWS, Azure, or Google Cloud.

Example: Synchronising files with a Linux PC

This example demonstrates how files on a PLC can be transferred to a PC running Linux. You will need a PC running a Linux distribution (e.g. Debian or Ubuntu) and a PLCnext Control device running firmware 2020.3.1 or later. Both devices need to be on the same Ethernet network.

1. Set up the Linux PC.
   • If not already installed, use apt to install openssh-server and rsync.$ sudo apt install openssh-server rsync
   • Create a directory that will be used to receive the data files from the PLC, e.g.$ mkdir ~/plcnext-archive
2. Set up the PLCnext Control device.
   • Check that the ssh server is running on the Linux PC by opening a secure shell session from the PLC:$ # Change the user name and IP address in the following command $ # to the user name and IP address of your Linux PC. $ ssh user@192.168.1.5After entering the password for the PC user, you should see the command prompt on the PC. Use exit to close the shell session and return to the PLC command prompt.
   • Create a directory that will contain the data files to be sent to the PC:$ mkdir ~/my-data-files
   • Create a file containing a few items of time-series process data:$ { date --rfc-3339=ns; echo "Temp: 42"; } | sed 'N;s/\n/ /' >> ~/my-data-files/data.txt $ { date --rfc-3339=ns; echo "Temp: 43"; } | sed 'N;s/\n/ /' >> ~/my-data-files/data.txt $ { date --rfc-3339=ns; echo "Temp: 44"; } | sed 'N;s/\n/ /' >> ~/my-data-files/data.txt
3. Transfer files from the PLC to the PC.
   • On the PLC, use rsync to send all the files in the source directory, to the destination directory:$ # Change the user name and IP address in the following command $ # to the user name and IP address of your Linux PC. $ rsync -av -e ssh ~/my-data-files/ user@192.168.1.5:~/plcnext-archive/The -a option uses “archive” mode (recurses into directories, copies symlinks as symlinks, preserves permissions, etc. Check the man page for full details).
     The -v option gives verbose output.
     The -e option specifies the remote shell to use, in this case ssh. This ensures that the rsync transfer is done securely, just like scp.
   • On the PC, check that the file was transferred:$ ls -lsah ~/plcnext-archive/ $ cat ~/plcnext-archive/data.txt
4. Explore various scenarios:
   • Add data to the file data.txt on the PLC.
     This simulates the addition of records to a log file after it has been archived to the PC.
     Repeat the rsync command.
     => The file on the PC is updated with the new data.
   • Create a new file in the my-data-files directory on the PLC
     This simulates the automatic “roll over” of logging to a new file, e.g. when the size of the current log file has exceeded a certain limit.
     Repeat the rsync command.
     => The new file is transferred to the PC.
   • Delete the original data.txt file on the PLC
     This simulates the automatic deletion of an old log file, e.g. when the number of log files has exceeded a certain limit.
     Repeat the rsync command.
     => The new file is transferred to the PC, but the original data.txt file remains on the PC.

Next steps

You can schedule one or more rsync commands to run automatically using Cron. The frequency of these commands will depend on (for example) how fast the log files on the PLC fill up, roll over, and are deleted; and how frequently the log files should be archived for offline analysis or for safe-keeping.

It is beyond the scope of this article to explain how to use Cron, but there are plenty of online resources that can help with that task.

If you are using ssh with rsync (and you should!), then you will also want to set up a trusted relationship between the PLC and the PC, so that a password is not needed when repeatedly executing rsync commands, particularly in a Cron job. Again, there are plenty of online resources to help with that task.

Potential applications

There are many potential applications for rsync on a PLCnext Control device – here are a few:

• Datalogger file backups.
• Control program backups.
• Use rsync in the other direction: Pull updates from a server to a PLC, just like app updates on your smartphone!

If you have other ideas for how to use rsync in a PLCnext Control project, please let us know in the comments.