Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sep'23 Regression testing - Part 1 #2582

Closed
22 of 30 tasks
tyuldashev opened this issue Sep 7, 2023 · 1 comment
Closed
22 of 30 tasks

Sep'23 Regression testing - Part 1 #2582

tyuldashev opened this issue Sep 7, 2023 · 1 comment
Assignees
@tyuldashev
Labels
ctg-qa Issue is related to QA process
Milestone
October Release

Comments

@tyuldashev
Copy link
Collaborator

tyuldashev commented Sep 7, 2023 •
edited
Loading

JDK <-> Framework

Project JDK Framework Status Comment Sample
Gradle 1.8 JUnit 5 x strings
Maven 1.8 TestNG x structures
Gradle 11 JUnit 5 x lambda
Maven 11 TestNG x objects
Maven 17 JUnit 5 x stdlib
Gradle 17 TestNG x algorithms/CorrectBracketSequences

JDK <-> Framework <-> Parametrization

IntelliJ build system

JDK Framework Parametrization Status Comment Sample
1.8 JUnit 4 On Not applicable
1.8 JUnit 5 On x arrays/ArrayOfObjects
1.8 TestNG On Not applicable
11 JUnit 4 On Not applicable
11 JUnit 5 On x arrays/FinalStaticFieldArray
11 TestNG On x enums/ClassWithEnum
17 JUnit 4 On Not applicable
17 JUnit 5 On x collections/QueueUsages
17 TestNG On x collections/Optionals

JDK <-> Fuzzing/Symbolic

JDK Fuzzing/Symbolic Symbolic 100% Fuzzing 100%
1.8 exceptions/JvmCrashExamples nested invokes
11 controlflow/CycleDependedCondition make.symbolic/ClassWithComplicatedMethods reflection
17 types/Generics natives stream

Fuzzing/Symbolic <-> Timeout

Test Generation Timeout Fuzzing/Symbolic Symbolic 100% Fuzzing 100%
60 arrays arrays arrays
20 primitives primitives primitives
10 math math math
  • timeout limits only test generation, building and reading classes phases are not included.

Mocks

  • No mocks

  • No mocks in settings, but forced mock for Random

  • Mock outside class

  • Mock outside package

  • Mock static methods

  • strings

  • structures

  • lambda

  • objects

  • stdlib

  • algorithms/CorrectBracketSequences

  • arrays/ArrayOfObjects

  • arrays/FinalStaticFieldArray

  • enums/ClassWithEnum

  • exceptions/JvmCrashExamples

  • collections/Optionals

  • collections/QueueUsages

  • controlflow/CycleDependedCondition - to check summaries

  • controlflow/Switch

  • types/Generics

  • nested

  • invokes

  • make.symbolic/ClassWithComplicatedMethods

  • mixed/LoggerExample - to check force mocking

  • natives

  • reflection

  • stream

  • thirdparty/numbers/ArithmeticUtils

  • threads

  • unsafe

  • wrappers

  • annotations/NotNullAnnotation

  • math

  • primitives

  • mocks
@tyuldashev tyuldashev added the ctg-qa Issue is related to QA process label Sep 7, 2023
@tyuldashev tyuldashev self-assigned this Sep 7, 2023
@tyuldashev
Copy link
Collaborator Author

Regression defects found:

@alisevych alisevych moved this from Todo to In Progress in UTBot Java Sep 11, 2023
@github-project-automation github-project-automation bot moved this from In Progress to Done in UTBot Java Sep 13, 2023
@alisevych alisevych added this to the October Release milestone Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tyuldashev tyuldashev

Labels
ctg-qa Issue is related to QA process
Projects
UTBot Java
Archived in project
Milestone
October Release
Development

No branches or pull requests
2 participants
@tyuldashev @alisevych