muffel muffel - 4 months ago 8x
Bash Question

how to parse csv-like definition and build dependency set from shell script?

Consider a definitions of roles, where each line is either empty, a comment (prefixed with hash sign) or a role definitions. A role definitions consists of a name followed by a colon and none, one or multiple, space-separated role dependencies:

# this is a comment

somerole: role2 role3
role2: role3
# 'empty' roles should be supported as well
anotherrole: somerole role2

I want to get all dependent roles in a bash function in a way I can iterate over them. They need to be listed only once and in the right order, e.g. if role1 depends on role2, and role2 depends on role3, the order need to be role3, role2, role1, not role1, role2, role3. The definition file is guaranteed to not contain any circular dependencies.

I can parse the role file (quite naively, for sure) by iterating over each entry like

for role in $(cat "$ROLEFILE" |grep -v -e '^$' -e '#'); do
# test if name ends with colon
echo $role|grep -qe ':$'
if [ $? -eq 0 ]; then
# remove last character
role=$(echo $role|sed 's/.$//')
echo "new role definition: $role"
echo "role depenency: $role"

But how can I build unique sets of dependencies, and collect the dependencies for a given input role?

This is how an example invocation could look like. Each dependency need to be printed before the role which depends on it. The final role is always the role we've asked for.

$ ./testroles somerole
role3 role2 somerole
$ ./testroles roled
$ ./testroles anotherrole
role3 role2 somerole anotherrole

The resulting script should work with Ubuntu 14.04 and OS X.


I created the script bellow, it prints the dependency tree just like you described. It uses shell_map to create a Map of roles-dependencies.



# create a new map where roles are keys and deps are values
shell_map new roles_map

while read line; do 
    #skip comments
    [[ $line =~ ^#.*$ ]] && continue;

    #skip blank lines
    [[ "$line" =~ ^[[:alpha:]]+ ]] || continue;

    deps=$(echo $line | cut -d ":" -f2)
    role=$(echo $line | cut -d ":" -f1)

    # put in the map
    roles_map put $role "$deps"
done < $ROLEFILE

# this map will store the dependency tree without repetitions
shell_map new deps_tree

function role_deps {
    local role=$1
    local my_deps=`roles_map get $role`

    [ -z "$my_deps" ] && return 0

    for dep in $my_deps; do
        deps_tree put $dep ""
        for inherited_dep in `role_deps $dep`; do
            deps_tree put $inherited_dep ""

    # echoes the dependencies stored in the map
    deps_tree keys

echo `role_deps $ROLE_ARG` $ROLE_ARG