Hanging test timeout is ignored by Symbolic execution

[alisevych]

Description

When a hanging test is generated by Symbolic execution, it doesn't have Timeout annotation.

To Reproduce

1. Install UTBot plugin on IntelliJ Idea
2. Check Symbolic execution is enabled and Hanging test timeout is 1000 (default)
3. Create a project and add the following class:

public class A {

    public int hangForSeconds(int seconds) {
        for (int i = 1; i < seconds; i++) {
            try {
                Thread.sleep(1001);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
        }
        return seconds;
    }
}

4. Open the generated test

Expected behavior

Tests whose generation lasts for more than 1000 millisec should fail with TimeoutException.

Actual behavior

Test generated by Symbolic execution, lasts for , it doesn't have Timeout annotation.
Tests generated by Fuzzing has correct annotation.

Visual proofs (screenshots, logs, images)

public class ATest {
    ///region Test suites for executable org.example.A.hangForSeconds

    ///region SYMBOLIC EXECUTION: SUCCESSFUL EXECUTIONS for method hangForSeconds(int)

    /**
     * <pre>
     * Test iterates the loop {@code for(int i = 1; i < seconds; i++) } once.
     * Test afterwards returns from: {@code return seconds; }
     * </pre>
     */
    @Test
    @DisplayName("hangForSeconds: ThreadSleep -> return seconds")
    public void testHangForSeconds_ThreadSleep() {
        A a = new A();

        int actual = a.hangForSeconds(2);

        assertEquals(2, actual);
    }
    ///endregion

    ///region FUZZER: SUCCESSFUL EXECUTIONS for method hangForSeconds(int)

    @Test
    @DisplayName("hangForSeconds: seconds > 0 -> return 1")
    public void testHangForSecondsReturnsOne() {
        A a = new A();

        int actual = a.hangForSeconds(1);

        assertEquals(1, actual);
    }
    ///endregion

    ///region FUZZER: EXPLICITLY_THROWN_UNCHECKED_EXCEPTIONS for method hangForSeconds(int)

    @Test
    @DisplayName("hangForSeconds: seconds = 134217729 (mutated from 1)")
    @Timeout(value = 1000L, unit = TimeUnit.MILLISECONDS)
    public void testHangForSeconds() {
        A a = new A();
        
        /* This execution may take longer than the 1000 ms timeout
         and therefore fail due to exceeding the timeout. */
        assertTimeoutPreemptively(Duration.ofMillis(1000L), () -> a.hangForSeconds(134217729));
    }
    ///endregion

    ///endregion
}

Environment

Windows 10 Pro
IntelliJ IDEA 2022.2.2 / 2022.1.4

Additional context

Also Fuzzing test generation mode throws exception in Concrete execution logs:

Exception in thread "executor" java.lang.IllegalMonitorStateException
	at java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
	at java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
	at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:406)
	at org.utbot.common.ThreadBasedExecutor$invokeWithTimeout$1.invoke(ThreadUtil.kt:46)
	at org.utbot.common.ThreadBasedExecutor$invokeWithTimeout$1.invoke(ThreadUtil.kt:43)
	at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)

[Damtev]

It looks like a corner case for the child process timeout - as such as the generated test takes 1000 + epsilon milliseconds to execute, a child process does not detect such overflow of hanging tests timeout. Tests are generated as expected in case of decreasing the hanging tests timeout setting or increasing sleeping time. @sergeypospelov what do you think?

[alisevych]

@Damtev @sergeypospelov Sorry, corrected description, I've used 1001 milliseconds.