Can AS fetch real-time data from a .CSV table?

Ask Questions
, ,
1

Sometimes the third-party equipment does not support the opening of some commonly used industrial protocols such as OPC UA, etc., and now the other party proposed a solution is to provide a real-time data refresh CSV form, I can use AS to find a way to take the data in it, and then add it to the variable group, there is no similar application cases can be referred to, thank you!

2

“Real time data” and “File based” basically exclude each other I’d say, but let’s find a way for you anyway.

Handling CSV data isn’t really anything special. One can open, read (and close) files just fine using the FileIO library

Additionally there are already others who had the usecase of handling CSV and were nice enough to share it, e.g. here - CSVFileLib.

But going into what I said first about real time data: You do not get notified whenever the file changed. Also there might be issues with data integrity since you will not know when the file has been written successfully and when the write access has been done by the other party (you wouldn’t want to read only half of what’s going to be there)

With that being said, feel free to try using the CSV with one of the approaches but keep in mind the caveats.

Best regards

1 Like
3

You are absolutely right, I don’t know how to ensure that after the CSV data is refreshed, AS can receive and update the variable data displayed in the mappview in a timely manner

4

Hi,

I’m not sure if I understood the use-case right, so some more questions here:

  • Where is the CSV file located? On the user partition, or on a CIFS network drive, or on a FTP server?
  • How often does the file content changes?
  • Does the file always have the same name?
  • How is the file changed by the data provider? By “always deleting and creating”, or by “extending”?
  • Does the file always have the same size?

I’m asking those questions, to get an impression how to determine if “something has changed”. There COULD be several ways to address it, depending on how the origin file is handled by the creator.
For example by cyclic (but slow!) :

  • checking the files datetime stamp (if it’s every time deleted and written) and size (using FileInfo())
  • or by reading a directory content (if the filename changes)
  • or by using a 2.nd file as “interface”/“trigger” (if possible by the data creator - e.g. a similar process to ‘.lock’ files known by some operating systems)
  • or by introducing a “change counter” inside the CSV file, best case at the beginning / a known position that can be read in read-only mode before processing the content itself

But of course, all of those ideas have disadvantages and aren’t “transaction safe” at all, like Michael already mentioned.
Nevertheless, strongly depending on how the file is processed by the origin/creator and where it’s located, maybe at least one of those methods is worth a 2nd look?

Best regards!

5

hello Alexander,

A FTP server

Once a second;Yes,alaways the same name;Cyclic write to the same cell, all cells are overwritten and written to the new value；

No,but almost the same.

6

Hi,

okay, thanks for the details!

Honestly spoken, it’s hard to imagine a solution (at least from my side :wink: ), the main reason for it is the location of the file on a FTP server, but also because of the way the data provider works.

Some limitations inside the FTP protocol itself avoid reading the files information like creation date and so on, and accessing the files content means you have to transfer the file before (explicitely by FileCopy(), but when using FTP this also happens implicitely e.g. by calling FileOpen()).

So when using FTP, detecting if something has chanced is not possible without getting the file and processing the content.

Best regards!

7

Hi,

Another way to determine whether a file has changed is to create a hash of it.

Of course, this means cyclically downloading the changed file from the server and determining the hash of it. If this has changed, the file can then be evaluated in detail.

djb2, for example, is an algorithm that can be implemented quickly.

cse.yorku.ca/~oz/hash.html

collisions are of course possible, so that in some cases the change to the file is not recognized. But at least it is quicker and easier than monitoring all values in the CSV.

There are also better hashing algorithms for this.

8

OK, got it,thank you :grinning: