Regular expression, find and replace using Unix tools

June 20, 2010

I need to add some text to a lot of text files. Previously, I used jEdit to perform regular expression for several files. But currently, I need to add the text for about 120 files. In order to do this, I tried using Unix tools, “sed”, “grep”, and “xargs”.

Firstly, I tried MSYS, yet it fails. Because the “sed” version was too old. Then, I used Cygwin.

My problem was to add the following text to the text files:

Background { skyColor 1 1 1 }

Since the text files are computer generated, all the text files have following text:

DEF Camera

And the text only occurred once for each file.

Therefore, the best solution is to use regular expression for search and replace to produce the following result:

[sourcecode language=“text”] Background { skyColor 1 1 1 }

DEF Camera [/sourcecode]

Therefore, I used the following command to solve the problem. (Do not copy paste, because the user must type the “newline”).

$ grep -l "DEF Camera" *.wrl | xargs -l sed -i -e 's/\(DEF Camera\)/Background { skyColor 1 1 1 } \
> \
> \1/g'

First of all, all the files are in .wrl format. So, I used

grep -l "DEF Camera" *.wrl

to get all the file names where the files contain “DEF Camera”. The option “-l” indicates the result is the file name. Then I used xargs and sed to search and replace the text (actually I am not familiar with “xargs” command). Then,

sed -i -e 's/\(DEF Camera\)/Background { skyColor 1 1 1 } \
\
\1/g'

“sed” command cannot write newline like C, such as ‘\n’. Thus, I need to type ‘\’ followed by ’enter’ key. This will produce exact newline to the I/O stream. Unlike jEdit or PHP regular expression, one needs to use ‘$’ and ‘$’ instead of ‘(’ and ‘)’ for replacement by reference. And the ‘\1’ is used instead of ‘$1’ for the reference.

Finally, using this command helps me to add the text to about 120 files with several seconds. Good!