UNIX (Shell) Q: How to dump multi-line data into a buffer

Factual Questions
1

I have a shell program that reads data from several sources, and I need to store them in a holding spot until they’re used later on. The way I’m doing it now is to append each line as it is read into a file, which is, obviously, slow. Is there a way to hold the data in memory instead? AWK arrays look like a possibility, but I’d prefer to not do that because (so it looks, anyway) that means I’ll have to do a massive re-write of my entire shell to accommodate it.

Thanks.

2

You haven’t said what shell you’re using. sh, csh, bash, ksh(either the 88 or 93 flavors, it makes a difference), zsh, etc. The techniques are different per shell.

ksh, both 88 and 93, has a built-in array function. Arrays of up to 1024 elements are supported. Look up the ‘set’ command to find the syntax. ksh93 has associative arrays which, if you’re optimising for performance, would be the array of choice if it’s available and the data is more than a handful of lines.

You can also concatenate variables to make a big variable with the data from both. VAR3={VAR1}{VAR2} in ksh notation. The limit of the size of data VAR3 can hold will depend on your shell and your system. Typically there is a decent amount of info(meaning large number of characters) which can be stored in a single shell variable. Several k worth of characters in modern systems IIRC.

Hope this helps,
Steven

3

Use a Perl script. It’s not that much different than csh/tcsh, and it’s perfect for this type of application. The command you’ll use is:

@output = whatever your command is;

the first line of output is stored in $output[0], the second line in $output[1], etc.

4

To elaborate, you don’t need to rewrite your entire shell script in Perl, you can just write a simple Perl script that you call from your original program.

5

I’m using ksh93. Actually, I’m running it in UWIN, which is David Korn’s implementation of ksh under MS Windows. I’m not sure that perl is available here, though – at least there was no man page for it in UWIN.

Doesn’t using a variable mean that all the data will get strung out as a single line (e.g., when I do “echo $VAR3”)? Or am I doing something wrong?

Didn’t realize ksh93 came with associative arrays – will look into that. Thanks.

6

Just so you know, Perl is available for Windows (http://www.activeperl.com). But it sounds like that might not be the easier solution for you.