Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Manual testing of Release 2022.12.1 Part 3 #1864

Closed
8 of 9 tasks
alisevych opened this issue Feb 27, 2023 · 2 comments
Closed
8 of 9 tasks

Manual testing of Release 2022.12.1 Part 3 #1864

alisevych opened this issue Feb 27, 2023 · 2 comments
Assignees
@tyuldashev @alisevych
Labels
ctg-qa Issue is related to QA process lang-java Issue is related to Java support
Milestone
2023.3 Release

Comments

@alisevych
Copy link
Member

alisevych commented Feb 27, 2023 •
edited
Loading

Initial set-up

Add link to Plugin build #***

  • Sarif report + Problems view
  • Cancellation of generation (with Generate / Generate & Run)
  • Java docs styles
  • Sandboxing
  • Logging
  • Hanging test timeout, check with Fuzzing/Symbolic
  • Overflows, check Summaries
  • Context estimator, check nightly runs
  • Bugs list
@alisevych alisevych added the ctg-qa Issue is related to QA process label Feb 27, 2023
@alisevych alisevych added this to the 2023.03 Release milestone Feb 27, 2023
@alisevych alisevych added the lang-java Issue is related to Java support label Mar 14, 2023
@alisevych
Copy link
Member Author

alisevych commented Mar 17, 2023 •
edited
Loading

Security testing (sandboxing)

Basic security

  • Project with JDK 8 + manual scenario from PR Add SecurityManager support to block suspicious code #622 #625 + default settings
  • Fuzzing only
  • Symbolic execution only
  • Project with JDK 11 + default settings
  • Fuzzing only
  • Symbolic execution only
  • Concrete executor is invoked from Symbolic execution
  • Project with JDK 17 + default settings

With permissions

  • add PropertyPermission in ~.utbot\sandbox.policy - check test for property is not in sandbox, execute
  • add "permission java.io.FilePermission "<>", "read";" - check test for read not in sandbox, execute
  • add AllPermission in ~.utbot\sandbox.policy - check NO tests are in sandbox, try to execute (carefully!!!)
  • corrupt permission - check how UTBot fails - try scenario from Too many permissions file rereads - can slow down much #796
  • empty file
  • remove ~.utbot\sandbox.policy file - check generated tests are in sandbox

Test rendering

  • JUnit4
  • JUnit5
  • TestNG
  • JUnit5 with parametrization
  • TestNG with parametrization

Code structures

@alisevych
Copy link
Member Author

alisevych commented Mar 21, 2023 •
edited
Loading

Reproducing:

@github-project-automation github-project-automation bot moved this from Todo to Done in UTBot Java Mar 21, 2023
@alisevych alisevych changed the title Manual testing of Release 2023.03 Part 3 Manual testing of Release 2022.12.1 Part 3 Mar 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tyuldashev tyuldashev

@alisevych alisevych

Labels
ctg-qa Issue is related to QA process lang-java Issue is related to Java support
Projects
UTBot Java
Archived in project
Milestone
2023.3 Release
Development

No branches or pull requests
2 participants
@tyuldashev @alisevych