Mark Galeck Mark Galeck - 2 months ago 5x
Bash Question

does quote removal happen after command substitution in a POSIX shell?

The POSIX shell standard at

says in Section 2.6:

command substitution (...) shall be performed


Quote removal (...) shall always be performed last.

It appears to me that quote removal is not performed after command substitution:

$ echo "#"
$ echo '"'

as expected, but

$ echo $(echo '"')#"

What am I not understanding?

Added after reading answer/comments:

From what everybody is saying, the consideration of quotes happens at the very beginning of parsing, for example, to decide if a command is even "acceptable". Then why does the standard bother to emphasise, that the quote removal is performed late in the process??


"then the outer command becomes echo "#" and is balanced"

That is not 'balanced' because the first double-quote does not count. Quotes are only meaningful as quotes if they appear unencumbered on the command line.

To verify, let's look at this:

$ echo $(echo '"')#

That is balanced because the shell does considers that " to be just another character.

By contrast, this is unbalanced because it has one and only one shell-active ":

$ echo $(echo '"')#"

Similar example 1

Here we show the same thing but using parameter expansion instead of command substitution:

$ q='"'; echo $q

Once the shell has substituted " for $q, one might think that there was an unbalanced double-quote. But, that double-quote was the results of parameter expansion and is therefore not a shell-active quote.

Similar example 2

Let's consider a directory containing file:

$ ls 
$ ls "file"

As you can see above, quote removal is perfomed before ls is run.

But, consider this command:

$ echo ls $(echo '"file"')
ls "file"

As you can see ls $(echo '"file"') expands to ls "file" which is the command which ran successfully above. Now, let's try running that:

$ ls $(echo '"file"')
ls: cannot access '"file"': No such file or directory

As you can see, the shell does not treat the double-quotes that remain after command substitution. This is because those quotes are not considered to be shell-active. As a consequence, they are treated as normal characters and passed on to ls which complains that the file whose name begins and ends with " does not exist in the directory.

The same is happening here:

$ cmd='ls "file"'
$ $cmd
ls: cannot access '"file"': No such file or directory

POSIX standard

From the POSIX standard:

Enclosing characters in single-quotes ( ' ' ) shall preserve the literal value of each character within the single-quotes

In other words, once the double-quote appears inside single quotes, it has no special powers: it is just another character.

The standard also mentions escaping and double-quotes as methods of preserving "the literal value" of a character.

Practical consequences

People new to shell often want to store a command in a variable as in the cmd='ls "file"' example above. But, because quotes and other shell-active characters cease to be shell active once they are stored in a variable, the complex cases always fail. This leads to a classic essay:

"I'm trying to put a command in a variable, but the complex cases always fail!"