Vinayaka Vinayaka - 3 years ago 102
Objective-C Question

XCUI Test: How does test collection happen

The test collection in XCUI Test happens in alphabetical order

So, if there are test like


  • test_A

  • test_X

  • test_1A

  • DISABLE_test



the order of execution is


  • test_1A

  • test_A

  • test_X



and the DISABLE_test is not picked as it doesn't begin with test

I was wondering how do I intelligently name the tests to run them in order.

First solution i thought was use numbers, but


  • test_1a

  • test_2b

  • test_10c



the tests execute in order,


  • test_1a

  • test_10c

  • test_2b



which is not the intended order.

What is the better design?

Answer Source

Test collection is done by using reflection on the object to get all the method names. Im not sure if this is done via the Objective-C runtime in Swift test cases but its not important.

You are close with your ordering but you can observe that this is the correct alphabetical order for your test names.

let things:[String] = ["test-2","test-1","test-10"]
let sorted = things.sorted() 
//["test-1", "test-10", "test-2"]

Pad out the names to get your desired order.

let things:[String] = ["test-02","test-01","test-10"]
let sorted = things.sorted()
//["test-01", "test-02", "test-10"]

At a more structural level the requirement to run test in a particular order might indicate that the tests are not decoupled. i.e test-10 should not depend on state setup by test-01 or test-02.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download