JUnit 5 Test Execution Order

By mkyong | Updated: October 23, 2019

Viewed: 8,463 (+36 pv/w)

Tags: junit 5 order test test order

This article shows you how to control the JUnit 5 test execution order via the following MethodOrderer classes:

• Alphanumeric
• OrderAnnotation
• Random
• Custom Order

P.S Tested with JUnit 5.5.2

1. Alphanumeric

1.1 It sorts test methods alphanumerically.

package com.mkyong.order;

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.Alphanumeric.class)
public class MethodAlphanumericTest {

    @Test
    void testZ() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void testA() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void testY() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void testE() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void testB() {
        assertEquals(2, 1 + 1);
    }

}

Output

testA()
testB()
testE()
testY()
testZ()

2. OrderAnnotation

2.1 It sorts test methods based on the @Order values.

package com.mkyong.order;

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class MethodOrderTest {

    @Test
    void test0() {
        assertEquals(2, 1 + 1);
    }

    @Test
    @Order(3)
    void test1() {
        assertEquals(2, 1 + 1);
    }

    @Test
    @Order(1)
    void test2() {
        assertEquals(2, 1 + 1);
    }

    @Test
    @Order(2)
    void test3() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void test4() {
        assertEquals(2, 1 + 1);
    }

}

Output

test2()
test3()
test1()
test0()
test4()

3. Random

3.1 It sorts test methods pseudo-randomly and supports configuration of a custom seed:

junit.jupiter.execution.order.random.seed

package com.mkyong.order;

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

import static org.junit.jupiter.api.Assertions.assertEquals;

@TestMethodOrder(MethodOrderer.Random.class)
public class MethodRandomTest {

    @Test
    void testZ() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void testA() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void testY() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void testE() {
        assertEquals(2, 1 + 1);
    }

    @Test
    void testB() {
        assertEquals(2, 1 + 1);
    }

}

Output, random.

# Run 1
testA()
testZ()
testE()
testY()
testB()

# Run 2
testY()
testE()
testZ()
testA()
testB()

# Run 3
testA()
testB()
testY()
testE()
testZ()

3.2 Configure a custom seed junit.jupiter.execution.order.random.seed to create a repeatable test builds

In properties file.

junit.jupiter.execution.order.random.seed=99

In Maven, configuration parameters

	<plugin>
		<groupId>org.apache.maven.plugins</groupId>
		<artifactId>maven-surefire-plugin</artifactId>
		<version>3.0.0-M3</version>
		<configuration>
			<properties>
				<configurationParameters>
					junit.jupiter.execution.order.random.seed=99
				</configurationParameters>
			</properties>
		</configuration>
	</plugin>

In Gradle, configuration parameters

test {

	useJUnitPlatform()

	systemProperties = [
			'junit.jupiter.execution.order.random.seed': 99
	]

}

Run it again with custom seed.

# Run 1 - seed 99
testA()
testZ()
testE()
testY()
testB()

# Run 2 - seed 99
testA()
testZ()
testE()
testY()
testB()

# Run 3 - seed 99
testA()
testZ()
testE()
testY()
testB()

4. Custom Order

4.1 Implements MethodOrderer to create a custom test order.

4.2 Below example is order by parameter count.

package com.mkyong.order;

import org.junit.jupiter.api.MethodDescriptor;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.MethodOrdererContext;

import java.util.Comparator;

public class ParameterCountOrder implements MethodOrderer {

    private Comparator<MethodDescriptor> comparator =
            Comparator.comparingInt(md1 -> md1.getMethod().getParameterCount());

    @Override
    public void orderMethods(MethodOrdererContext context) {

        context.getMethodDescriptors().sort(comparator.reversed());

    }

}

4.3 Test the above custom order with @ParameterizedTest

package com.mkyong.order;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.TestMethodOrder;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.junit.jupiter.params.provider.ValueSource;

import java.math.BigDecimal;

import static org.junit.jupiter.api.Assertions.assertTrue;

@TestMethodOrder(ParameterCountOrder.class)
public class MethodParameterCountTest {

    @DisplayName("Parameter Count : 2")
    @ParameterizedTest(name = "{index} ==> fruit=''{0}'', qty={1}")
    @CsvSource({
            "apple,         1",
            "banana,        2"
    })
    void test2(String fruit, int qty) {
        assertTrue(true);
    }

    @DisplayName("Parameter Count : 1")
    @ParameterizedTest(name = "{index} ==> ints={0}")
    @ValueSource(ints = {1, 2, 3})
    void test1(int num1) {
        assertTrue(num1 < 4);
    }

    @DisplayName("Parameter Count : 3")
    @ParameterizedTest(name = "{index} ==> fruit=''{0}'', qty={1}, price={2}")
    @CsvSource({
            "apple,         1,  1.99",
            "banana,        2,  2.99"
    })
    void test3(String fruit, int qty, BigDecimal price) {
        assertTrue(true);
    }

}

Output

4.4 Remove the comparator.reversed()

public class ParameterCountOrder implements MethodOrderer {

   //...

    @Override
    public void orderMethods(MethodOrdererContext context) {

        //context.getMethodDescriptors().sort(comparator.reversed());
        context.getMethodDescriptors().sort(comparator);
    }

}

Output

Download Source Code

References

• JUnit 5 Test Execution Order
• MethodOrderer.Random