Testing Summaries with custom tags checklist

[alisevych]

Initial set-up

Check that the IntelliJ Idea UTBot plugin can be successfully installed

• Choose appropriate workflow from the list
• Download plugin
• Check downloaded zip-file size < 100 MB
• Open IntelliJ IDEA 2022.1.4
• Remove previously installed UTBot plugin
• Clone or reuse UTBot project (https://github.com/UnitTestBot/UTBotJava.git)
• Open the project in the IDE
• Install the downloaded plugin

Go through manual scenarios

Manual scenario #1

• Use default plugin setting / or turn Fuzzing off
• Check: File -> Settings -> Tools -> UnitTestBot -> Javadoc comment style is "Structured via custom Javadoc tags"
• Open the utbot-sample/src/main/java/org/utbot/examples/algorithms/ArraysQuickSort.java file
• Generate tests for the class
  blocked by Invalid line separators in a custom JavaDoc tag prevent test generation #1001
• Check Java docs, all tags are correct
• Render Java doc
  reproducing Duplicate Summaries with Tags in Rendered view #985

Manual scenario #2

• Use default plugin settings.
• Check: File -> Settings -> Tools -> UnitTestBot -> Javadoc comment style is "Structured via custom Javadoc tags"
• Open the utbot-sample/src/main/java/org/utbot/examples/mock/CommonMocksExample.java file
• Generate tests with different Mocking options combinations
• Check Java docs, all tags are correct
• Render Java doc
  reproducing Duplicate Summaries with Tags in Rendered view #985

Manual scenario #3

• Use default plugin settings.
• Check: File -> Settings -> Tools -> UnitTestBot -> Javadoc comment style is "Structured via custom Javadoc tags"
• Open the utbot-sample/src/main/java/org/utbot/examples/recursion/Recursion.java file
• Generate tests for the methods
• Check Java docs, all tags are correct
• Render Java doc
  new bug: @utbot.triggersRecursion tag is missing

Manual scenario #4

• Use default plugin settings.
• Check: File -> Settings -> Tools -> UnitTestBot -> Javadoc comment style is "Structured via custom Javadoc tags"
• Open the utbot-sample/src/main/java/org/utbot/examples/exceptions/ExceptionExamples.java
• Generate tests for the methods
• Check Java docs, all tags are correct, @utbot.caughtException should be present
• Render Java doc
• File -> Settings -> Tools -> UnitTestBot -> Javadoc comment style set to "Plain text"
• Generate tests for the class
• Check Java docs, all tags are correct
• Render Java doc
  reproducing Duplicate Summaries with Tags in Rendered view #985

Manual scenario #5

• Create a new Maven project with JDK 8
• Use default plugin settings.
• Check: File -> Settings -> Tools -> UnitTestBot -> Javadoc comment style is "Structured via custom Javadoc tags"
• Add a simple java file to test
• Generate a test with a new test root
• Check Java docs, all tags are correct
• Render Java doc

Manual scenario #6

• Create a new Idea project with JDK 11
• Use default plugin settings.
• Check: File -> Settings -> Tools -> UnitTestBot -> Javadoc comment style is "Structured via custom Javadoc tags"
• Add a simple java file to test
• Generate tests for several classes
• Check Java docs, all tags are correct
• Render Java doc