Ari Ari - 1 month ago 19
Ruby Question

How do you specify the exact order of Minitest tests?

Minitest lets you run tests in order by overriding

test_order
to
alpha
. (You can also use the
i_suck_and_my_tests_are_order_dependent!
method.)

After doing this, how do you control the order the tests are run across multiple files?

Is there a way to run some tests from one file and then switch to another file?

Looking at the source code, it seems one should be able to control how methods are sorted. But how do you specify that order?

Answer

i_suck_and_my_tests_are_order_dependent! (or def self.test_order; :alpha;end define an alphabetic order.

Beside the alphabetic order there is only a random order defined (at least in 'minitest', '>5.5.1', '<=5.6.1'.

But you can patch MiniTest::Test.runnable_methods to get another order.

gem 'minitest'
require 'minitest/autorun'

class MiniTest::Test
      #Add a test order :defined 
      def self.runnable_methods
      methods = methods_matching(/^test_/)

      case self.test_order
      when :random, :parallel then
        max = methods.size
        methods.sort.sort_by { rand max }
      when :defined then  # <-new
        methods
      when :alpha, :sorted then
        methods.sort
      else
        raise "Unknown test_order: #{self.test_order.inspect}"
      end
    end
end

class TestOrder < MiniTest::Test
  def self.test_order; :defined; end

  #Alphabetic order
  #~ def self.test_order; :alpha;end
  #~ i_suck_and_my_tests_are_order_dependent!

  def test_4;    p __method__;  end
  def test_3;    p __method__;  end
  def test_2;    p __method__;  end
  def test_1;    p __method__;  end
end

But the test order is only defined per Test-subclass, not global for all tests. So this gives you not access to the order of test methods in multiple test classes.

I estimate you have different test-classes in your files, so this would correspond to your problem. (Not the files is the criteria, but the Test-class.)

If you define only one Test class in your test files then you have the possibility to define your own order.