Jetse Jetse - 2 months ago 40
Linux Question

How can I get unique values from an array in linux bash?

I've got almost the same question as here.

I have an array which contains

aa ab aa ac aa ad
, etc.
Now I want to select all unique elements from this array.
Thought, this would be simple with
sort | uniq
or with
sort -u
as they mentioned in that other question, but nothing changed in the array...
The code is:

echo `echo "${ids[@]}" | sort | uniq`


What am I doing wrong?

Answer

A bit hacky, but this should do it:

echo "${ids[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' '

To save the sorted unique results back into an array, do:

sorted_unique_ids=$(echo "${ids[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')

If your shell supports herestrings (bash should), you can spare an echo process by altering it to:

tr ' ' '\n' <<< "${ids[@]}" | sort -u | tr '\n' ' '

Input:

ids=(aa ab aa ac aa ad)

Output:

aa ab ac ad

Explanation:

  • "${ids[@]}" - Syntax for working with shell arrays, whether used as part of echo or a herestring. The @ part means "all elements in the array"
  • tr ' ' '\n' - Convert all spaces to newlines. Because your array is seen by shell as elements on a single line, separated by spaces; and because sort expects input to be on separate lines.
  • sort -u - sort and retain only unique elements
  • tr '\n' ' ' - convert the newlines we added in earlier back to spaces.
  • $(...) - Command Subsitution
  • Aside: tr ' ' '\n' <<< "${ids[@]}" is a more efficient way of doing: echo "${ids[@]}" | tr ' ' '\n'