matandked matandked - 4 months ago 26
C# Question

Display tested URL when running Selenium test using vstest.console.exe tool

Context

I configured Jenkins job which executes Selenium C# tests.
When building a job, I must provide two values:


  • branch to build my solution

  • URL which should be tested



Detailed description

Jenkins performs the following steps:


  • checkout selected branch

  • replace URL stored in AppSettings with URL provided by the user ("TestedURL")

  • build solution

  • execute vstest.console.exe tool (which comes with Visual Studio 2012, exists in the following path: "Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow") as follows:

    vstest.console.exe "c:\myProject\bin\Debug\myProject.Test.dll"



Question

I would like to ensure that proper URL is tested: I would like to display URL in console output of the Jenkins/vstest.console.exe output.

I amended my code following different answers on StackOverflow.

URL is visible in test output directly from Visual Studio. Unfortunately, I still don't see any testing URL in vstest.console.exe/Jenkins output.

How to display URL in output (how to modify printMessage method)?

My code is as follows:

using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using System;
using System.Configuration;
using System.Diagnostics;

namespace My.Test
{
[TestClass]
public class MyTests
{
IWebDriver driver;
string baseURL;

[TestInitialize]
public void Initialize()
{
System.Configuration.Configuration config = ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel.None);
// URL is replaced by the value provided in Jenkins
baseURL = config.AppSettings.Settings["TestedURL"].Value;
driver = new ChromeDriver();
driver.Navigate().GoToUrl(baseURL);
printMessage(string.Format("Tested URL: '{0}'", baseURL));
}

private TestContext testContextInstance;
/// <summary>
///Gets or sets the test context which provides
///information about and functionality for the current test run.
///</summary>
public TestContext TestContext
{
get
{
return testContextInstance;
}
set
{
testContextInstance = value;
}
}

private void printMessage(string message)
{
// Method should display custom message in both Visual Studio output and Jenkins (vstest.console.exe) output
Trace.Listeners.Add(new TextWriterTraceListener(Console.Out));
Trace.Listeners.Add(new ConsoleTraceListener());
Trace.WriteLine(message);
Debug.WriteLine(message);
TestContext.WriteLine(message);
Console.WriteLine(message);
}
}
}

Answer

Since I can't find direct solution, I've found a workaround.

vstest.console.exe has /logger:trx option which generates file with .trx extension. Mentioned file is an xml which contains test results together with Std output. I noticed that when I use my printmessage within test methods, messages are there (in Std output in trx file).

Let me provide part of generated .trx file:

<Output>
    <StdOut>Tested URL was 'http://someURL'

Debug Trace:
Tested URL: 'http://someURL'
Trying to log into as 'Incorrect account'

TestContext Messages:
Tested URL: 'http://someURL'
Trying to log into as 'Incorrect account'</StdOut>
      </Output>
    </UnitTestResult>

There are some tool which allows conversion of .trx to .html. Thus Jenkins can show in console output link to the test results.


BUT

I have problem, because I can't find tool which works properly and shows Std output in generated html file. Since that, I wrote following Powershell code:

# set $inputTRXFile, $outputHTMLFile before running following lines
$results = ([xml] (Get-Content $inputTRXFile)).TestRun.Results.UnitTestResult
$results | select testname,outcome,duration, @{Name="Output";Expression={$_.Output.InnerText}} | ConvertTo-HTML -head $a -title "Test results" | Out-File $outputHTMLFile