Coverage annotations are ignored/misinterpreted

[tim-bezhashvyly]

Q	A
php-code-coverage version	9.1.7
PHP version	7.3.8 and 7.4.10
Driver	Xdebug
Xdebug version	2.9.6
Installation Method	Composer
Usage Method	PHPUnit
PHPUnit version (if used)	9.3.8

Since upgrade from PHPUnit 9.2 to 9.3 there are few issues with code coverage:

1. "This test executed code that is not listed as code to be covered or used:" is thrown even if valid annotation is present

2. Code coverage calculation seems to take unproductive lines into calculation. Like empty exception classes or code surrounding productive lines. The code at a screenshot below has coverage of 80% even though all productive lines are covered:

My configuration file contains following non-default values:

• colors="true"
• forceCoversAnnotation="true"
• beStrictAboutCoversAnnotation="true"
• beStrictAboutOutputDuringTests="true"

<coverage> node contains no option overrides.

Test suite is executed using following command:

php -dxdebug.coverage_enable=1 vendor/phpunit/phpunit/phpunit --coverage-clover foo.xml --configuration tests/Unit/phpunit.xml.dist

I have set up a minimal, self-contained example reproducing both issues here https://github.com/LlZARD/phpunit-demo

I would gladly contribute and provide a PR if you will give me a hint what can be done here.

[sebastianbergmann]

I have set up a minimal, self-contained example reproducing both issues here https://github.com/LlZARD/phpunit-demo

I am afraid this is a bit too minimal as it is missing a phpunit.xml configuration file.

[tim-bezhashvyly]

Doch https://github.com/LlZARD/phpunit-demo/blob/master/tests/Unit/phpunit.xml.dist

[sebastianbergmann]

My bad. I have to say, though, that this is a rather unusual location. At least I expect phpunit.xml.dist or phpunit.xml in a project's root directory. That way I can clone a repository, enter the directory, and immediately Sorry for bothering you. run PHPUnit. But okay :) Sorry for bothering you.

[tim-bezhashvyly]

My reasoning for putting configuration XML under tests/Unit is that for most of the projects I have different types of PHPUnit tests (e.g. tests/Unit, tests/Integration, tests/Acceptance) and I would like those to be executed separately and possibly have different configurations. If there's a more common setup for a situation like this please let me know.

[sebastianbergmann]

This seems to have been already fixed by #807:

$ git clone [email protected]:LlZARD/phpunit-demo.git
Cloning into 'phpunit-demo'...
remote: Enumerating objects: 20, done.
remote: Counting objects: 100% (20/20), done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 20 (delta 2), reused 20 (delta 2), pack-reused 0
Receiving objects: 100% (20/20), done.
Resolving deltas: 100% (2/2), done.

$ cd phpunit-demo

$ composer install
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 34 installs, 0 updates, 0 removals
  - Installing sebastian/version (3.0.1): Loading from cache
  - Installing sebastian/type (2.2.1): Loading from cache
  - Installing sebastian/resource-operations (3.0.2): Loading from cache
  - Installing sebastian/recursion-context (4.0.2): Loading from cache
  - Installing sebastian/object-reflector (2.0.2): Loading from cache
  - Installing sebastian/object-enumerator (4.0.2): Loading from cache
  - Installing sebastian/global-state (5.0.0): Loading from cache
  - Installing sebastian/exporter (4.0.2): Loading from cache
  - Installing sebastian/environment (5.1.2): Loading from cache
  - Installing sebastian/diff (4.0.2): Loading from cache
  - Installing sebastian/comparator (4.0.3): Loading from cache
  - Installing sebastian/code-unit (1.0.5): Loading from cache
  - Installing sebastian/cli-parser (1.0.0): Loading from cache
  - Installing phpunit/php-timer (5.0.1): Loading from cache
  - Installing phpunit/php-text-template (2.0.2): Loading from cache
  - Installing phpunit/php-invoker (3.1.0): Loading from cache
  - Installing phpunit/php-file-iterator (3.0.4): Loading from cache
  - Installing nikic/php-parser (v4.9.1): Loading from cache
  - Installing sebastian/complexity (2.0.0): Loading from cache
  - Installing sebastian/lines-of-code (1.0.0): Loading from cache
  - Installing theseer/tokenizer (1.2.0): Loading from cache
  - Installing sebastian/code-unit-reverse-lookup (2.0.2): Loading from cache
  - Installing phpunit/php-code-coverage (9.1.7): Loading from cache
  - Installing phpdocumentor/reflection-common (2.2.0): Loading from cache
  - Installing symfony/polyfill-ctype (v1.18.1): Loading from cache
  - Installing webmozart/assert (1.9.1): Loading from cache
  - Installing phpdocumentor/type-resolver (1.3.0): Loading from cache
  - Installing phpdocumentor/reflection-docblock (5.2.1): Loading from cache
  - Installing doctrine/instantiator (1.3.1): Loading from cache
  - Installing phpspec/prophecy (1.11.1): Loading from cache
  - Installing phar-io/version (3.0.2): Loading from cache
  - Installing phar-io/manifest (2.0.1): Loading from cache
  - Installing myclabs/deep-copy (1.10.1): Loading from cache
  - Installing phpunit/phpunit (9.3.8): Loading from cache
sebastian/global-state suggests installing ext-uopz (*)
Writing lock file
Generating autoload files
26 packages you are using are looking for funding.
Use the `composer fund` command to find out more!

$ ./vendor/bin/phpunit --configuration tests/Unit --coverage-html /tmp/coverage 
PHPUnit 9.3.8 by Sebastian Bergmann and contributors.

R.                                                                  2 / 2 (100%)

Time: 00:00.842, Memory: 8.00 MB

There was 1 risky test:

1) Demo\DemoTest::testFoo
This test executed code that is not listed as code to be covered or used:
- /home/sb/phpunit-demo/src/Demo.php:21
- /home/sb/phpunit-demo/src/Exception/DemoException.php:11

OK, but incomplete, skipped, or risky tests!
Tests: 2, Assertions: 2, Risky: 1.

Generating code coverage report in HTML format ... done [00:00.046]

diff --git a/composer.json b/composer.json
index 0b36d27..d879ba8 100644
--- a/composer.json
+++ b/composer.json
@@ -3,7 +3,8 @@
     "php": "^7.1"
   },
   "require-dev": {
-    "phpunit/phpunit": "~9.3"
+    "phpunit/phpunit": "~9.3",
+    "phpunit/php-code-coverage": "dev-master"
   },
   "autoload" : {
     "psr-4": { "Demo\\": "src/" }

$ composer update 
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 0 installs, 1 update, 0 removals
  - Removing phpunit/php-code-coverage (9.1.7)
  - Installing phpunit/php-code-coverage (dev-master 57b333e): Cloning 57b333ea54 from cache
Writing lock file
Generating autoload files
26 packages you are using are looking for funding.
Use the `composer fund` command to find out more!

$ ./vendor/bin/phpunit --configuration tests/Unit --coverage-html /tmp/coverage
PHPUnit 9.3.8 by Sebastian Bergmann and contributors.

..                                                                  2 / 2 (100%)

Time: 00:00.112, Memory: 6.00 MB

OK (2 tests, 2 assertions)

Generating code coverage report in HTML format ... done [00:00.048]