wuethrich44 wuethrich44 - 1 year ago 78
PHP Question

PHPUnit should fail if no test is executed

I use PHPUnit to execute unit tests on my Gitlab CI Server. When no tests are executed the build pass altough there could be an error (e.g. autoloader issue).

Is there a way to tell PHPUnit Command Line Runner it should exit with code 1 if no test is executed?

Answer Source

If there's no tests, there's nothing PHPUnit can mark as failed. PHPUnit, therefore, can't do this. What you can use is another one of Sebastian Bergmann's tools phploc.

This has a --count-tests option. Run that before running phpunit, process the result and if the count is 0, just exit 1 from a script.

Update, I just realized I missed the obvious thing to do a bit earlier, the simple command to get the exit status you're looking for can be obtained using this command:

 count=$(./phploc.phar --count-tests src | awk '/Tests/{getline; print}' | awk '{print $NF}') && [ "${count}" -eq "0" ] && exit 1 || exit 0

A simple way to do this is would be:

 count=$(./phploc.phar --count-tests src | awk '/Tests/{getline; print}' | awk '{print $NF}') && [ "${count}" -ne "0" ]
exit $?

Most of the command breakdown below applies, so I'm leaving that as-is. The main difference is at the end:

[ "${count}" -ne "0" ]
exit $?

This basically means: compare ${count} to 0, if it doesn't match 0, the comparison will exit with status 0, if it's 0, the exit status will be 1. We then simple reuse the exit status in our own exit statement: exit $? is basically saying: exit with the same exit status as the last thing we did.

Breaking it down a bit:

./phploc.phar --count-tests src

This basically runs phploc on the src directory, and the output will contain a section that looks something like:

    classes      123
    methods      1234

What we're after is the line after Tests. so the output of the command is passed to awk, which will match Tests, get the next line and print that (ie: classes 123). This happens here:

| awk '/Tests/{getline; print}'

That string is passed to awk yet again, to extract the last part (the actual number of tests found):

awk '{print $NF}'

That value will be assigned to count, because we're running these commands like so:

count=$(phploc | awk | awk)

So the final output is assigned to count. Next, we want to see if the test count is zero or not, and based on that we need to exit with a zero, or non-zero status. This is something we can easily do like so:

 [ "${count}" -eq "0" ] && exit 1 || exit 0

This is basically shorthand for:

if [ "${count}" -eq "0" ]; then
    exit 1
    exit 0

That's thanks to short-circuit evaluation: if [ "${count}" -eq "0" ] evaluates to true, the && clause needs to be evaluated, which is an exit 1 statement, returning the desired non-zero exit code

If we omit the || exit 0, the last status code would be the result of the [ "${count}" -eq "0" ] && exit 1 evaluation, which is false, resulting in an status code of 1. To avoid that, we have to add the || exit 0.

You can easily check this by writing something like this on your CLI:

foo="bar" && [ "${foo}" -eq "123" ] && echo 'foo matched 123'
//run this, then run
echo $?

The output will be 1 (last command's exit status)