Dan Beaulieu Dan Beaulieu - 4 years ago 103
Swift Question

How to test generic performance with whole module optimization

In WWDC 2015 Session 409 near the 18 minute mark. The discussion at hand leads me to believe that generics can be optimized through Generic Specialization by enabling whole module optimization mode. Unfortunately my tests, which I'm not confident in, revealed nothing useful.

I ran some very simple tests between the following two methods to see if the performance was similar:

func genericMax<T : Comparable>(x:T, y:T) -> T {
return y > x ? y : x
}

func intMax(x:Int, y:Int) -> Int {
return y > x ? y : x
}


Simple XCTest:

func testPerformanceExample() {

self.measureBlock {

let x: Int = Int(arc4random_uniform(9999))
let y: Int = Int(arc4random_uniform(9999))

for _ in 0...1000000 {
// let _ = genericMax(x, y: y)
let _ = intMax(x, y: y)
}
}
}


What happened



Without optimization the following tests were reasonably different:


  • genericMax: 0.018 sec

  • intMax: 0.005 sec



However with Whole Module Optimization the following tests weren't similar:


  • genericMax: 0.014 sec

  • intMax: 0.004 sec



What I Expected



With whole module optimization enabled I expected similar times between the two methods calls. This leads me to believe that my test is flawed.

Question



Assuming my tests are flawed / poor. How could I better measure how Whole Module Optimization mode optimizes generics through Generic Specialization?

Answer Source

Your tests are flawed because they measure test performance, not app performance. Tests live in a separate executable file, and they do not benefit from whole-module optimizations themselves. Because of this, your test always uses the generic, non-specialized implementation even in places where your program doesn't.

If you want to see that whole-module optimizations are enabled in your executable, you need to test a function from your executable. (You also need to make sure that your tests either use the Release build, or that you have WMO enabled in the debug build.)

Adding this function to the executable:

func genericIntMax(x: Int, y: Int) -> Int {
    return genericMax(x, y: y)
}

and using it from the tests in place of genericMax in the tests, I get identical performance.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download