Jon Raynor Jon Raynor - 1 month ago 17
Groovy Question

How to get the test run assertion result (status and name) from a SOAP UI TestSuite Teardown Script?

I am having trouble getting the assertion results from the the test steps at the testsuite teardown level in SOAP UI. I looked at the Javadoc and attempted to use

getAssertionList()
but calling that is causing an error. Here's my Groovy script so far:

def failedTestCases = 0
def successTestCases = 0
def failedTestSteps = 0
def successTestSteps = 0

runner.results.each { testCaseResult ->
def caseName = testCaseResult.testCase.name
def caseStatus = testCaseResult.status.toString()
if(caseStatus == 'FAILED'){failedTestCases ++}
else {successTestCases ++}

log.info "Test Case: $caseName $caseStatus"

testCaseResult.results.each{ testStepResults ->
testStepResults.messages.each() { msg -> log.info msg }
def stepName = testStepResults.testStep.name
def stepStatus = testStepResults.status.toString()
if(stepStatus == 'FAILED'){failedTestSteps ++}
else{successTestSteps ++}

log.info "Test Step: $stepName $stepStatus"

def testStep = testStepResults.testStep
log.info "XX" + testStep.name
testStep.getAssertionList().each{
log.info "$it.label - $it.status"
}
}
}

log.info "Total test cases:" + (successTestCases + failedTestCases).toString()
log.info "Total succeeded: $successTestCases"
log.info "Total failed: $failedTestCases"
log.info "Total test step count:" + (successTestSteps + failedTestSteps).toString()
log.info "Total succeeded: $successTestSteps"
log.info "Total failed: $failedTestSteps"


I'm able to see XX[TestStepName] but no assertion or results from the log entry, but I am getting this error:

Fri Nov 06 10:13:36 EST 2015:ERROR:An error occurred [No signature of method: com.eviware.soapui.impl.wsdl.teststeps.WsdlPropertiesTestStep.getAssertionList() is applicable for argument types: () values: []], see error log for details

Answer

For SOAP UI test steps, the type of step matters. I am not all that familiar with the SOAP UI api, but some test cases don't seem to support getAssertionList() call. So we check the test type before getting the assertions and that seems to work out well.

Here's my final script. Seem to be working out well. Now I can get the test cases, test steps, and assertions for a home grown report. I am using SOAP UI free edition so I will be using this to output an XML report.

def failedTestCases = 0
def successTestCases = 0
def failedTestSteps = 0
def successTestSteps = 0
def failedAssertions = 0
def successAssertions = 0

runner.results.each { testCaseResult ->
    def caseName = testCaseResult.testCase.name
        def caseStatus = testCaseResult.status.toString()
        if(caseStatus == 'FAILED'){failedTestCases ++}
     else {successTestCases ++}

     log.info "Test Case: $caseName $caseStatus" 

     testCaseResult.results.each{ testStepResults ->
        testStepResults.messages.each() { msg -> log.info msg }
        def stepName = testStepResults.testStep.name
        def stepStatus =  testStepResults.status.toString()
        if(stepStatus == 'FAILED'){failedTestSteps ++}
        else{successTestSteps ++}

        log.info "Test Step: $stepName $stepStatus"

          //Some test steps don't have assertions, so calling getAssertionList() blows up
          //We want WSDLTestStep (request)
        if (testStepResults.testStep.config.type == 'request')
        {
            testStepResults.testStep.getAssertionList().each{
                def assertLabel = it.label
                def assertStatus = it.status
                if (assertStatus == 'VALID') { successAssertions ++ }
                else {failedAssertions ++ }
                    log.info "$it.label - $it.status"   
            }
        }
    }
}

log.info "Total test case count:" + (successTestCases + failedTestCases).toString()
log.info "Total test cases succeeded: $successTestCases"
log.info "Total test cases failed: $failedTestCases"

log.info "Total test step count:" + (successTestSteps + failedTestSteps).toString()
log.info "Total test steps succeeded: $successTestSteps"
log.info "Total test steps failed: $failedTestSteps"

log.info "Total test assertion count:" + (successAssertions + failedAssertions).toString()
log.info "Total test assertions succeeded: $successAssertions"
log.info "Total test assertions failed: $failedAssertions"
Comments