hqjma hqjma - 1 year ago 53
Bash Question

How do I use shell variables in an awk script?

I found some ways to pass external shell variables to an

awk
script, but I'm confused about
'
and
"
.

First, I tried with a shell script:

$ v=123test
$ echo $v
123test
$ echo "$v"
123test


Then tried awk:

$ awk 'BEGIN{print "'$v'"}'
$ 123test
$ awk 'BEGIN{print '"$v"'}'
$ 123


Why is the difference?

Lastly I tried this:

$ awk 'BEGIN{print " '$v' "}'
$ 123test
$ awk 'BEGIN{print ' "$v" '}'
awk: cmd. line:1: BEGIN{print
awk: cmd. line:1: ^ unexpected newline or end of string


I'm confused about this.

Answer Source

Getting shell variables into awk may be done in several ways. Some are better than others.


This is the best way to do it. It uses the -v option: (P.S. use a space after -v or it will be less portable. E.g., awk -v var= not awk -vvar)

variable="line one\nline two"
awk -v var="$variable" 'BEGIN {print var}'
line one
line two

This should be compatible with most awk and variable is available in the BEGIN block as well:


You can use a variable within the awk code, but it's messy and hard to read, and as Charles Duffy points out, this version may also be a victim of code injection. If someone adds bad stuff to the variable, it will be executed as part of the awk code, so DO NOT USE.

variable="line one\nline two"
awk 'BEGIN {print "'"$variable"'"}'
line one
line two

Here is an example of code injection:

variable='line one\nline two" ; for (i=1;i<=1000;++i) print i"'
awk 'BEGIN {print "'"$variable"'"}'
line one
line two
1
2
3
.
.
1000

You can add lots of commands to awk this way. Even make it crash with non valid commands.


Here we get the variable after the awk code. This will work fine as long as you do not need the variable in the BEGIN block:

variable="line one\nline two"
echo "input data" | awk '{print var}' var="$variable"
or
awk '{print var}' var="$variable" file

Variable can also be added to awk using here string

awk '{print $0}' <<< "$variable"
test

This is the same as:

echo "$variable" | awk '{print $0}'

It's always good to double quote variable "$variable"
If not, multiple lines will be added as a long single line.

Example:

var="Line one
This is line two"

echo $var
Line one This is line two

echo "$var"
Line one
This is line two

Other errors you can get without double quote:

variable="line one\nline two"
awk -v var=$variable 'BEGIN {print var}'
awk: cmd. line:1: one\nline
awk: cmd. line:1:    ^ backslash not last character on line
awk: cmd. line:1: one\nline
awk: cmd. line:1:    ^ syntax error

And with single quote, it does not expand the value of the variable:

awk -v var='$variable' 'BEGIN {print var}'
$variable
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download