marshallm marshallm - 7 months ago 8
Bash Question

How to populate a bash array with a file's lines

Populating an array from a file should be basic, but I can't get it to work.

#!/bin/bash
declare -a a
i=0
cat "file.txt" | while read line; do
a[$i]="$line"
i=$(($i + 1))
done
echo "${a[0]}"


This prints an empty line. For a file that contains the lines "Foo" and "Bar", here's the output from bash -x:

+ declare -a a
+ i=0
+ cat file.txt
+ read line
+ a[$i]=Foo
+ i=1
+ read line
+ a[$i]=Bar
+ i=2
+ read line
+ echo ''


I can't even get the readarray builtin working:

#!/bin/bash
declare -a b
cat "file.txt" | readarray b
echo "${b[0]}"

+ declare -a b
+ cat file.txt
+ readarray b
+ echo ''


What am I doing wrong here?

Answer

This is a basic bash pitfall: all segments of a pipeline execute in subshells by default, which means that whatever variables they create will go out of scope when these subshells terminate (the current shell won't see them).

Depending on what version of bash you're using, you have two options:

  • [Bash v4.2+] Execute shopt -s lastpipe before you use the pipeline to ensure that the last pipeline segment runs in the current shell.

  • Use input redirection (<) instead of a pipeline (or, in more complex cases, process substitution (<(...)) to ensure that your while loop doesn't run in a subshell:

 a=()
 while IFS= read -r line; do
   a+=( "$line" )
   # ... 
 done < "file.txt" 

In the simple case of reading lines from a file, you can read directly into an array ("${a[@]}", in this example):

  • Bash 4.x:

    • readarray -t a < "file.txt"
  • Earlier versions:

    • IFS=$'\n' read -r -d '' -a a < "file.txt"