Working with large strings

Share Info & Ideas
1

If you need to create a large STRING (such as STRING[1000000]) or a data object for user logging or other reasons, here are some tips.

General info

  1. AsBrStr library should be used because AsString library is obsolete.
  2. For large strings, DO NOT use starting address of the destination variable (like ADR(DestVar)) in string functions (brsstrcat, brsstrcpy). Because if the destination variable is a large string that is already filled with some data, there is always a loop with a huge number of iterations.
  3. For large string composition USE outputs nxt_adr of AsBrStr functions for the destination variable address. It can be used with either string functions or memory functions.

Examples and best practices
Let’s assume that LargeString is, for example, STRING[1000000] and StringToAdd is some small string that we want to add in different steps, for example, STRING[80].

  1. I recommend to always reset the memory first when initializing a string so that no data remains there from the past.
// Reset memory
brsmemset(ADR(LargeString), 0, SIZEOF(LargeString));
  1. Get the address of the string.
  • Just get address.
// Get address
NextAddress := ADR(LargeString);
  • Get the address of the string and directly write the initial value of the string.
// Get address and initiate value
NextAddress := brsstrcpy(ADR(LargeString), ADR(StringToAdd));
  1. Adding more data to the large string.
  • Add string from variable.
// Add string from variable
NextAddress := brsmemcpy(NextAddress, ADR(StringToAdd), brsstrlen(ADR(StringToAdd)));

OR

NextAddress := brsstrcat(NextAddress, ADR(StringToAdd));
  • Add hardcoded string. The brsmemcpy variant is a bit better because SIZEOF is evaluated during compilation, so no extra performance is needed to calculate the length of the string when the program is running. After SIZEOF there must be -1 because it returns size also with null character.
// Add hardcoded string
NextAddress := brsmemcpy(NextAddress, ADR('Hello'), SIZEOF('Hello') - 1);

OR

NextAddress := brsstrcat(NextAddress, ADR('Hello'));
  • Add integer/float value. Function brsitoa returns length of its output so it must be added to the NextAddress value.
// Add integer value
NextAddress := NextAddress + brsitoa(IntToAdd, NextAddress);

// Add float value
NextAddress := NextAddress + brsftoa(RealToAdd, NextAddress);

Example on a specific use case

  • Logging axis position, speed and temperature and logged user in CSV format every 100 ms. The task is in a cyclic class with a repetition of 100 ms, so you only need to log data every cycle. When the string is full, write to a file and stop logging.

Variables declaration

VAR
	State : StateEnum;
	FB : FBType;
	Axis : AxisType;
	Start : BOOL;
	StartAddress : UDINT;
	NextAddress : UDINT;
	LogData : STRING[1000000];
	LoggedUser : STRING[20] := 'John';
END_VAR

Data types

TYPE
	AxisType : 	STRUCT 
		Position : REAL;
		Velocity : REAL;
		Temperature : REAL;
	END_STRUCT;
	FBType : 	STRUCT 
		FileCreate_0 : FileCreate;
		FileWrite_0 : FileWrite;
		FileClose_0 : FileClose;
	END_STRUCT;
	StateEnum : 
		(
		stWAIT,
		stLOGGING,
		stFILE_CREATE,
		stFILE_WRITE,
		stFILE_CLOSE
		);
END_TYPE

Code

PROGRAM _CYCLIC
	
	CASE State OF
		stWAIT:	// Wait for commands
			
			// Command to start logging
			IF Start THEN
				Start	:= FALSE;
				
				// Reset memory
				brsmemset(ADR(LogData), 0, SIZEOF(LogData));
				
				// Get address
				StartAddress := NextAddress := ADR(LogData);
				
				// Initiate value (add CSV header)
				NextAddress := brsstrcpy(NextAddress, ADR('Position;Velocity;Temperature;LoggedUser'));
				
				State	:= stLOGGING;
				
			END_IF
		
		stLOGGING:	// Logging
			
			// Check if the string is already full (lets say a string is full when it has only 100 characters left)
			IF ((NextAddress - StartAddress) > (SIZEOF(LogData) - 100)) THEN
				State	:= stFILE_CREATE;
				
			ELSE
				// Add line feed
				NextAddress	:= brsmemcpy(NextAddress, ADR('$n'), SIZEOF('$n') - 1);
				
				// Add position
				NextAddress	:= NextAddress + brsftoa(Axis.Position, NextAddress);
				
				// Add delimiter
				NextAddress	:= brsmemcpy(NextAddress, ADR(';'), SIZEOF(';') - 1);
				
				// Add velocity
				NextAddress	:= NextAddress + brsftoa(Axis.Velocity, NextAddress);
				
				// Add delimiter
				NextAddress	:= brsmemcpy(NextAddress, ADR(';'), SIZEOF(';') - 1);
				
				// Add temperature
				NextAddress	:= NextAddress + brsftoa(Axis.Temperature, NextAddress);
				
				// Add delimiter
				NextAddress	:= brsmemcpy(NextAddress, ADR(';'), SIZEOF(';') - 1);
				
				// Add logged user
				NextAddress	:= brsmemcpy(NextAddress, ADR(LoggedUser), brsstrlen(ADR(LoggedUser)));
				
			END_IF
			
		stFILE_CREATE:	// Create file for further write
			FB.FileCreate_0.enable	:= TRUE;
			FB.FileCreate_0.pDevice	:= ADR('USER');
			FB.FileCreate_0.pFile	:= ADR('LogData.csv');
			
			FB.FileCreate_0();
			
			IF (FB.FileCreate_0.status = ERR_OK) THEN
				State	:= stFILE_WRITE;
			END_IF
			
		stFILE_WRITE:	// Write data to a csv file on user partition
			FB.FileWrite_0.enable	:= TRUE;
			FB.FileWrite_0.ident	:= FB.FileCreate_0.ident;
			FB.FileWrite_0.pSrc		:= StartAddress;
			FB.FileWrite_0.offset	:= 0;
			FB.FileWrite_0.len		:= NextAddress - StartAddress;
			
			FB.FileWrite_0();
			
			IF (FB.FileWrite_0.status = ERR_OK) THEN
				State	:= stFILE_CLOSE;
			END_IF
		
		stFILE_CLOSE:	// Close file
			FB.FileClose_0.enable	:= TRUE;
			FB.FileClose_0.ident	:= FB.FileCreate_0.ident;
			
			FB.FileClose_0();
			
			IF (FB.FileClose_0.status = ERR_OK) THEN
				State	:= stWAIT;
			END_IF
			
	END_CASE
	
	// Simulation of value change
	Axis.Position		:= Axis.Position + 0.1;
	Axis.Velocity		:= Axis.Velocity + 0.2;
	Axis.Temperature	:= Axis.Temperature + 0.3;
	
END_PROGRAM
8 Likes
2

Hello Michal,

Thank you for sharing your experience!
Your examples are great and have a clear emphasis on avoiding unnecessary loops.

I’d like to add some suggestions from our projects on how to build the long strings at the PLC.

We mostly manipulate strings using the brsstrcpy() and brsstrcat() functions. The clear advantage of using them even for short strings (under 255 symbols) is that the internal loops to search for the end of the string can be avoided.

Here is part of your “mem” example overwritten in “str” style.

// Add line feed
p	:= brsstrcat(p, ADR('$n'));

// Add position
p	:= p + brsftoa(Axis.Position, p);
				
// Add delimiter
p	:= brsstrcat(p, ADR(';'));
				
// Add velocity
p	:= p + brsftoa(Axis.Velocity, p);
		
// Add delimiter
p	:= brsstrcat(p, ADR(';'));
				
// Add temperature
p	:= p + brsftoa(Axis.Temperature, p);
				
// Add delimiter
p	:= brsstrcat(p, ADR(';'));
				
// Add logged user
p	:= brsstrcat(p, ADR(LoggedUser));

Our string builders are in most cases separate functions with minimal additional logic. So we use a short name variable “p” as a pointer to the end of the string buffer (‘\0’). This makes the line of code shorter and focuses more on the variables (Axis.Velocity) or string constants.

The advantage of the “str” functions is that the string constant (‘$n’, ‘;’ ) is referenced once per call. We get rid of SIZEOF() which can have some nasty refactoring problems if the string is changed at only one position.

We use concatenation of several previously prepared “long strings” into one “very long string” and should call brsstrlen() in the “mem” function, which does a double loop over the “long string”.

Here is part of a function that creates a JSON string to send to a visualisation software as an example:

VAR_INPUT:
  dst: UDINT // addresse of the output string buffer
  pName: UDINT // pointer to the string with some name
  description: UDINT; // pointer to a description (may be a long string)
  isFixed: BOOL; // some boolean property
  offset: REAL; // another property
  image: UDINT; // pointer to a very long string with encoded image
END_VAR
VAR
  p: UDINT
END_VAR

FUNCTION BuildJsonResponseString 
  p := brsstrcpy(dst, ADR('{"name":"'));p := brsstrcat(p, pName);
  p := brsstrcat(p, ADR('","description":"'));
  p := brsstrcat(p, description);
  p := brsstrcat(p, ADR('","isFixed":'));
  p := brsstrcat(p, SEL(isFixed, ADR('false'), ADR('true')));
  p := brsstrcat(p, ADR(',"offset":'));
  p := p + brsftoa(offset, p);
  p := brsstrcat(p, ADR(',"image":{"format":"BASE64","content":"'));
  p := brsstrcat(p, image);
  p := brsstrcat(p, ADR('"}}'));

  // The return value depends on your upper layer logic.
  // Possible values:
  // p -> pointer to the end of string
  // (p - dst) -> length of created string
  // constant to avoid compiler warning
  BuildJsonResponseString := TRUE; // to avoid compiler warning
END_FUNCTION

The first call is a brsstrcpy() to (over-) write a string from the beginning of the buffer. This allows us to skip a call of brsmemset() for the long buffer.

The next function could also be “cpy”, as the pointer to the end of the string is given, but we have decided to use “cat” here.

If you look at the declaration of the input variables - it does not matter how long the buffers are, so if the contents grow, the function should not be changed.

Happy Coding!

1 Like
4

Hello Pavel, thanks for your comment.

I agree with “refactoring problems” if the hardcoded string is changed. I’ve had more than one occasion where I needed to change it, I changed it only in one place and then it generated incorrectly. That’s why I started using constants for these strings. This way I avoid the possibility that the strings would be different.

Example:

// Data header
Data.NextAddress	:= brsmemcpy(Data.NextAddress, ADR(DATA_HEADER), brsstrlen(ADR(DATA_HEADER)));

I don’t understand how you said that internal loops can be avoided in brsstrcpy() and brsstrcat() functions. How can you avoid looping to the null character with these functions?

And about your function, it looks good except this one line:

p := brsstrcat(p, image);

This one line is looping in the very long string “image”. This is the case I am trying to avoid. What is the advantage to use brsstrcat() here instead of brsmemcpy()?

However, it mostly depends on the use case whether it is a problem to do it this way or not. In your case, if it is a function that is used once in a long time, it probably doesn’t matter that it iterates through a long string once. The biggest problem is in the case like the one I have in the example. There, data is added to the end of the long string in each cycle, and if there was brsstrcat(), the entire large string is iterated through in each cycle, and that already poses a problem.

Best regards,
Michal

5

Hello Michal!

Using the constants is a possible solution to avoid this type of error. Even better if they are used in several places. Do not forget to change the name if a new “DATA_HERADER_2” appears :slight_smile:
This is why I prefer the str functions to work with strings. The calculation of the string length is done in the function.

If you want to use brsmemcpy for string concatenation, I’d write a wrapper over it like

FUNCTION memstrcpy

  nextAddress := brsmemcpy(dest, data, brsstrlen(data)); 
  memstrcpy := nextAddress;

END_FUNCTION

And call it like:

Data.NextAddress := memstrcpy(Data.NextAddress, ADR(DATA_HEADER));

brsstr…() returns a pointer to the end of the string.
Note: This is different from the glibc/str…-functions, which return a pointer to the destination string.
So even if the string is long, just store the output value (your “NextAddress”, my “p”) and that is the address of the ‘\0’ - end of string.

p := brsstrcpy(ADR(dest), ADR(SomeLongString));
// Here p points to the end of "SomeLongString" copied to the dest.
// The same as p := ADR(dest) + brsstrlen(ADR(SomeLongString));

// Use the pointer to the '\0' as the destination 
// to avoid long search for the end of the string
p := brsstrcat(p, ADR(AnotherLongString));

After this two lines of code, the dest contains the very long string consisting of two long string chunks. And the p contains the address of the end of the string in dest.
So there are no any long loops compared to brsmemcpy.

We use this functions to create files of measurements (JSON, several kilobytes each, 2-5 seconds between measurements). The results are stored as files locally and transferred over the network.

Best regards,
Pavel

2 Likes
6

I see what you mean now. I’m surprised I didn’t notice that, somehow I didn’t realize that brsstrcat also has nxt_adr output. Thanks for the tip, I’ll edit the post accordingly.

1 Like