# Linux Bash Script Question **URL:** \ **Category:** Factual Questions\ **Created:** [November 11, 2012, 8:30pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712 "2012-11-11T20:30:34Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![drewtwo99](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@drewtwo99](https://boards.straightdope.com/u/drewtwo99)\ **Post date:** [November 11, 2012, 8:30pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/1 "2012-11-11T20:30:34Z")
Hi linux scripting gurus. I have a very simple script I need to make, but I just can’t figure out how to do it. Here’s the deal: I need to know how I can make a script to take a file that looks something like this: 12345678 91234567 56789012 34751021 … and turn it into a file that looks like this: 1 12345678 12345678 03 2 91234567 91234567 03 3 56789012 56789012 03 4 34751021 34751021 03 … With a couple of stipulations. I need to be able to start the numbering at any arbitrary value, so it would be a variable I would pass the script when running it. I know how to handle variables, and so I know if my variable was $1, the numbering would just be something like $1 $1+1 $1+2 $1+3 … The other stipulation is that I don’t know how many lines are necessarily going to be in each file. So I need to get the line count first and probably store that as a variable and do a for loop. How do I get just a line count for a file to store as a variable? I’m hazy on the syntax on for loops in bash scripts though. And perhaps most embarrassingly, I’m not sure of how to actually copy the text of each line and then append it so that I have two copies of the number like I need on each line. And lastly, the 03 was just an example, but this would actually be a variable entered when running the script, so I need to tack on that at the end as well. So, basically, here’s my pseudocode for what I think I need to do. $1 = line\_number $2 = batch\_number linecount{filename} = line\_count for i=0; i\<=line\_count; i++ concatenate {line\_number+i," ",line i of filename, " ", line i of filename, " ", batch\_number} \>\> newfile I hope that makes sense. If you need clarification on anything please let me know. ---
**Author:** ![septimus](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/septimus/32/410_2.png) [@septimus](https://boards.straightdope.com/u/septimus)\ **Post date:** [November 11, 2012, 8:58pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/2 "2012-11-11T20:58:14Z")
I would just an _awk_ filter: awk 'BEGINFILE { kk = 1} { print kk, $0, $0, “03”; kk +=1 } ’ Here I’ve just hardcoded the variables (1 and 03) . ---
**Author:** ![Derleth](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Derleth](https://boards.straightdope.com/u/Derleth)\ **Post date:** [November 11, 2012, 9:01pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/3 "2012-11-11T21:01:27Z")
> [@septimus](#): > > I would just an _awk_ filter: > > awk 'BEGINFILE { kk = 1} { print kk, $0, $0, “03”; kk +=1 } ’ > > Here I’ve just hardcoded the variables (1 and 03) . There are other ways to do this if you don’t have awk for some bizarre reason, but you almost certainly will. Be sure you note the difference between comma and semicolon here. ---
**Author:** ![Rysto](https://avatars.discourse-cdn.com/v4/letter/r/ecccb3/32.png) [@Rysto](https://boards.straightdope.com/u/Rysto)\ **Post date:** [November 11, 2012, 9:09pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/4 "2012-11-11T21:09:52Z")
It’s not helpful here because of the additional output required, but I would note that cat -n will prepend a line number to every line. ---
**Author:** ![drewtwo99](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@drewtwo99](https://boards.straightdope.com/u/drewtwo99)\ **Post date:** [November 11, 2012, 9:27pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/5 "2012-11-11T21:27:54Z")
> [@septimus](#): > > I would just an _awk_ filter: > > awk 'BEGINFILE { kk = 1} { print kk, $0, $0, “03”; kk +=1 } ’ > > Here I’ve just hardcoded the variables (1 and 03) . Thanks. This almost doing what I want… however, it’s not printing the line number on the first line (it starts printing kk on the 2nd line). And also, it doesn’t seem to matter what I’m putting in for kk=… it always seems to start at whatever number I put in for kk += #. I think the kk+=# is overriding whatever kk is getting set to at the beginning for some reason, because when I set it to kk+=2, it starts at 2 and increases by 2. ---
**Author:** ![septimus](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/septimus/32/410_2.png) [@septimus](https://boards.straightdope.com/u/septimus)\ **Post date:** [November 11, 2012, 9:33pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/6 "2012-11-11T21:33:40Z")
Oops. I think BEGINFILE is a “gawkism.” Try BEGIN instead. ---
**Author:** ![drewtwo99](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@drewtwo99](https://boards.straightdope.com/u/drewtwo99)\ **Post date:** [November 11, 2012, 9:43pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/7 "2012-11-11T21:43:50Z")
Ok, BEGIN works. When I try out that awk command in the terminal, with hardcoded numbers, it works beautifully. Now, the trouble I’m having is trying to pass my variables into the awk command within my script. It doesn’t seem to want to use them. For example: line\_number=$2 batch\_number=$3 awk ‘BEGIN{kk=$line\_number}{print kk,$0,$0,$batch\_number;kk+=1}’ filename That’s what I tried coding in my script, but the awk command isn’t seeing those variables at all so it’s not printing things correctly. How do I accurately pass the variables from my script into the awk commands/parameters? ---
**Author:** ![drewtwo99](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@drewtwo99](https://boards.straightdope.com/u/drewtwo99)\ **Post date:** [November 11, 2012, 9:50pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/8 "2012-11-11T21:50:40Z")
Nevermind. I figured out how to get the variables. It’s all working now except one minor point. I need to be able to enter the batch numbers with a leading 0 sometimes, but when I pass the script a “03” it converts it to a 3 automatically. How do I make sure it accepts that as a string with the leading 0 in my script? I can’t just prepend a 0 no matter what is entered, because if the batch number is 12 for example, I don’t want a leading 0. ---
**Author:** ![Derleth](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Derleth](https://boards.straightdope.com/u/Derleth)\ **Post date:** [November 11, 2012, 9:53pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/9 "2012-11-11T21:53:05Z")
> [@drewtwo99](#): > > line\_number=$2 > batch\_number=$3 > > awk ‘BEGIN{kk=$line\_number}{print kk,$0,$0,$batch\_number;kk+=1}’ filename Try: awk “BEGIN { kk=$2; batch\_number=$3 } { print kk, $0, $0, batch\_number; kk += 1 }” Don’t forget the backslashes! ---
**Author:** ![drewtwo99](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@drewtwo99](https://boards.straightdope.com/u/drewtwo99)\ **Post date:** [November 11, 2012, 10:07pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/10 "2012-11-11T22:07:12Z")
Thanks Derleth, I actually got it working a slightly different way… now I’m just trying to figure out how to make it keep and print a 03 if that’s what I enter, instead of converting it to a 3. ---
**Author:** ![Noone\_Special](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/noone_special/32/2863_2.png) [@Noone\_Special](https://boards.straightdope.com/u/Noone_Special)\ **Post date:** [November 11, 2012, 10:19pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/11 "2012-11-11T22:19:57Z")
> [@drewtwo99](#): > > Thanks Derleth, I actually got it working a slightly different way… now I’m just trying to figure out how to make it keep and print a 03 if that’s what I enter, instead of converting it to a 3. I don’t remember the exact syntax offhand, but you’ll want to use sprintf instead of print – which will allow you to specify a printing format. You should use the “%s” format for your printouts of $0. It’s midnight here, so I’m going to sleep 🙂 If nobody has come up with the exact syntax by morning I’ll check the sprintf syntax when I get into the office and have access to a Unix machine. You can probably figure it out using google, though. ---
**Author:** ![drewtwo99](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@drewtwo99](https://boards.straightdope.com/u/drewtwo99)\ **Post date:** [November 11, 2012, 10:46pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/12 "2012-11-11T22:46:16Z")
Yeah I can’t seem to make heads or tails out of the sprintf command syntax :\ So close yet so far away! ---
**Author:** ![drewtwo99](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@drewtwo99](https://boards.straightdope.com/u/drewtwo99)\ **Post date:** [November 11, 2012, 10:53pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/13 "2012-11-11T22:53:12Z")
Ok I figured out a workaround. I’m all done with my script now guys! Thank you all for the help. ---
**Author:** ![Senegoid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/senegoid/32/6606_2.png) [@Senegoid](https://boards.straightdope.com/u/Senegoid)\ **Post date:** [November 11, 2012, 11:23pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/14 "2012-11-11T23:23:43Z")
For this sort of task, a one-line awk script would have been my first idea too. But it could also be done entirely with bash commands without using awk. I’ll leave it to OP to look up the details of how these commands work and their syntax. But to help OP further his already-estimable bash skills, here are some useful points: 1. There is a _read_ built-in command that reads one line from standard input. It can put the entire line into one shell variable, or it can put separate words into several shell variables. 2. This _read_ command, IIRC, also sets the last-command exit status as usual: 0 (for success), 1 (for failure, like when it hits end-of-file). 3. Because _read_ returns an exit status, it can be used to control a _while_ loop. Thus, you can build a loop that reads successive lines from standard input, doing something with each line, and the loop will quit at end-of-file. 4. You can group several consecutive shell commands in parentheses, which causes that group of commands to run kinda-sorta like it was a separate shell script. This group of commands, as a whole, can have its own standard input and standard output, separate from that of the rest of the script. This enables you to read a file (other than the overall script’s standard input) line by line, and write output to a different file (other than the overall script’s standard output). 5. bash also has several kinds of syntax for doing arithmetic with shell variables that have numeric values. That enables you to do the arithmetic you need to increment the line counter variable. 6. I’m not sure how you would force that other variable to be written as a two-digit number (with leading zero) even when its value is less than 10. But if you enter it as a parameter on the original command line where you start the script, and you enter it there with a leading zero, I think it should be treated as a string value rather than a numeric value. And since you are never doing any arithmetic with that number, I think the leading zero should never be lost, as long as you enter it with a leading zero in the first place. These are all just given as suggestion for useful things for **drewtwo99** to learn about, as they allow all sorts of useful stuff to be done in shell scripts. For the immediate problem, as shown in all the posts above, that one-line awk script is the simplest. (I’ve been away from this kind of work for several years and thus have forgotten most of the nitty-gritty command syntax details, so I’ll leave that as an exercise for the OP.) ---
**Author:** ![drewtwo99](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@drewtwo99](https://boards.straightdope.com/u/drewtwo99)\ **Post date:** [November 11, 2012, 11:29pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/15 "2012-11-11T23:29:01Z")
Thanks Senegoid. The print command does in fact only print out “4” even if the variable was stored as “04”. There is evidently a way to pad a number with 0’s using sprintf but I couldn’t figure out the syntax. Instead I just used an if then statement to check to see how big the value of the batch number was, and to print an extra 0 in front of it in the awk command if it was less than 10. It’s working nicely now that way 🙂 ---
**Author:** ![goldmund](https://avatars.discourse-cdn.com/v4/letter/g/a4c791/32.png) [@goldmund](https://boards.straightdope.com/u/goldmund)\ **Post date:** [November 11, 2012, 11:39pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/16 "2012-11-11T23:39:15Z")
Perl one-liner: ```auto X=5 perl -lne 'printf "%02d $_ $_ 03 ", $ENV{X}++' filename.txt ``` You can change the X= value to start the numbering at whatever number you choose. This will print the numbers with a minimum of 2 digits: 01, 02, 03, etc. If you would like 3, change the “%02d” to “%003d”. Why did you want to get an initial line count for the file? ---
**Author:** ![Senegoid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/senegoid/32/6606_2.png) [@Senegoid](https://boards.straightdope.com/u/Senegoid)\ **Post date:** [November 11, 2012, 11:48pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/17 "2012-11-11T23:48:49Z")
For **drewtwo99** or anyone else who is working with programs or shell scripts, I have a trivially simple program I wrote that has been immensely helpful in working out arcane details about how command-line arguments are parsed and passed to the program. Like the _echo_ command, it simply prints all the arguments it got. But in a somewhat more informative format: It prints each argument on a separate line. The lines are numbered. And the argument itself in enclosed in \<…\> characters, which helps you to see if there are any leading or trailing blank spaces in the argument. I call the program _parg_ – It’s source file, parg.c, is trivially simple. I recommend that every programmer alive should write such a program. Or I could send my source to anyone who wants it. Example usage and output: ```auto $ parg What have we here parg: Argument count: 5 0: 1: 2: 3: 4: ``` You can play around, typing command with arguments having all kinds of combinations of shell variables, single quotes, double quotes, back-quotes, wild-cards, and any other shell syntax you want to investigate. The output will show you _exactly_ what arguments the program actually sees. It’s instructive to do this under Unix and Linux, and also in DOS command windows, and compare the results. In particular, DOS handles wild-cards on a command-line entirely differently than Unix and Linux shells do. ---
**Author:** ![Senegoid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/senegoid/32/6606_2.png) [@Senegoid](https://boards.straightdope.com/u/Senegoid)\ **Post date:** [November 11, 2012, 11:52pm UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/18 "2012-11-11T23:52:55Z")
What the heck, I’ll just post the entire source code for parg.c here, since it’s so short. As I wrote above, I’ve found this simple program immensely helpful when I write shell scripts (both for Unix/Linux and DOS), in order to study exactly how command-line arguments are parsed and transmitted to a program. (This particular source code is dated June 5, 2010, but actually I’ve had some version of this around since I was a Unix sysadmin in the mid-1980’s.) ```auto /* parg.c Print arguments (one per line, numbered) */ /* June 5, 2010 */ #include int main ( int argn, char* argc[] ) { int iarg ; printf("parg: Argument count: %d ", argn) ; for ( iarg = 0 ; iarg < argn ; iarg++ ) { printf("%4d: <%s> ", iarg, argc[iarg]) ; } printf("----------------------------------------------------------- ") ; } ``` ---
**Author:** ![Senegoid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/senegoid/32/6606_2.png) [@Senegoid](https://boards.straightdope.com/u/Senegoid)\ **Post date:** [November 12, 2012, 12:05am UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/19 "2012-11-12T00:05:21Z")
**More thunks about printing numbers with leading zeros.** A character string that is a number with a leading zero will typically lose the leading zero if it gets converted from a string to a numeric value and then back to a string to be printed. So I guess that’s what’s happening with your number inside the awk script. This typically happens when you actually do some arithmetic with the number. You never did any arithmetic with that number, but I suppose awk must have done a string-\>number-\>string conversion for you anyway. You might have been able to prevent that by putting some kind of quotes around it. Here’s a general plan I’ve used to print numbers with leading zeros. You may be able to do something equivalent in just about any language. Say you have a number that could be anything from 0 to 9999, and you want to print it with exactly four digits (included leading zeros as needed). Simply add 10000 to the number, convert that to a character string, and then print out the right-most four characters of that string. Most languages have enough functionality that you can do all these steps one way or another. ---
**Author:** ![Derleth](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Derleth](https://boards.straightdope.com/u/Derleth)\ **Post date:** [November 12, 2012, 12:07am UTC](https://boards.straightdope.com/t/linux-bash-script-question/640712/20 "2012-11-12T00:07:30Z")
**Senegoid** : Add a ‘return 0;’ to that and it’s golden. You can also declare iarg in the for loop preamble in the modern C standard (since C99) and as a nonstandard extension to the standard since a lot longer in compilers like gcc. That would look like ‘for (int iarg = 1; iarg \< argc; iarg++)’ in this case. This helps document that the loop counter is limited to that loop and does not get reused, which is enforced by the scoping rules. [Next page](https://boards.straightdope.com/t/linux-bash-script-question/640712.md?page=2)