Speedup #387

MajorCaiger · 2015-09-27T18:57:32Z

Here are the speedup changes so far. Not sure if there is anything else that needs changing if the whitelist is to be mandatory. If you want to give it a quick check over and let me know if there is anything missing.

Thanks

Rob

sebastianbergmann · 2015-09-28T05:13:28Z

ChangeLog-3.0.md

@@ -7,10 +7,16 @@ All notable changes of the PHP_CodeCoverage 3.0 release series are documented in
 ### Changed

 * It is now mandatory to configure a whitelist
+* Merged [#384](https://github.com/sebastianbergmann/php-code-coverage/pull/384): Performance improvements
+* It is now mandatory to configure a whitelist
+* Merged [#384](https://github.com/sebastianbergmann/php-code-coverage/pull/384): Performance improvements



These duplications need to be cleaned up

MajorCaiger · 2015-09-28T21:17:57Z

Hi @sebastianbergmann

I have now refactored the changes and re-introduced the processUncoveredFilesFromWhitelist. It now runs each test first to determine the covered lines/files, then it can use the coverage data to then determine uncovered files.

sebastianbergmann · 2015-10-06T19:46:33Z

Is this ready for me to look at, @MajorCaiger? Thanks!

MajorCaiger · 2015-10-06T20:02:22Z

Yes please

sebastianbergmann · 2015-10-06T20:10:30Z

There are test failures:

PHPUnit 5.0.3 by Sebastian Bergmann and contributors.

.........FFFF.........................FFF.....                    46 / 46 (100%)

Time: 515 ms, Memory: 9.25Mb

There were 7 failures:

1) PHP_CodeCoverage_Report_CloverTest::testCloverForBankAccountTest
Failed asserting that format description matches text.
--- Expected
+++ Actual
@@ @@
 <?xml version="1.0" encoding="UTF-8"?>
 <coverage generated="1444162129">
   <project timestamp="1444162129" name="BankAccount">
     <file name="/usr/local/src/php-code-coverage/tests/_files/BankAccount.php">
       <class name="BankAccount" namespace="global">
-        <metrics methods="4" coveredmethods="3" conditionals="0" coveredconditionals="0" statements="10" coveredstatements="5" elements="14" coveredelements="8"/>
+        <metrics methods="4" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="15" coveredstatements="5" elements="19" coveredelements="5"/>
       </class>
-      <line num="6" type="method" name="getBalance" crap="1" count="2"/>
+      <line num="6" type="method" name="getBalance" crap="1.12" count="2"/>
+      <line num="7" type="stmt" count="0"/>
       <line num="8" type="stmt" count="2"/>
       <line num="11" type="method" name="setBalance" crap="6" count="0"/>
+      <line num="12" type="stmt" count="0"/>
       <line num="13" type="stmt" count="0"/>
       <line num="14" type="stmt" count="0"/>
       <line num="15" type="stmt" count="0"/>
       <line num="16" type="stmt" count="0"/>
+      <line num="17" type="stmt" count="0"/>
       <line num="18" type="stmt" count="0"/>
-      <line num="20" type="method" name="depositMoney" crap="1" count="2"/>
+      <line num="20" type="method" name="depositMoney" crap="1.04" count="2"/>
+      <line num="21" type="stmt" count="0"/>
       <line num="22" type="stmt" count="2"/>
       <line num="24" type="stmt" count="1"/>
-      <line num="27" type="method" name="withdrawMoney" crap="1" count="2"/>
+      <line num="27" type="method" name="withdrawMoney" crap="1.04" count="2"/>
+      <line num="28" type="stmt" count="0"/>
       <line num="29" type="stmt" count="2"/>
       <line num="31" type="stmt" count="1"/>
-      <metrics loc="33" ncloc="33" classes="1" methods="4" coveredmethods="3" conditionals="0" coveredconditionals="0" statements="10" coveredstatements="5" elements="14" coveredelements="8"/>
+      <metrics loc="33" ncloc="33" classes="1" methods="4" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="15" coveredstatements="5" elements="19" coveredelements="5"/>
     </file>
-    <metrics files="1" loc="33" ncloc="33" classes="1" methods="4" coveredmethods="3" conditionals="0" coveredconditionals="0" statements="10" coveredstatements="5" elements="14" coveredelements="8"/>
+    <metrics files="1" loc="33" ncloc="33" classes="1" methods="4" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="15" coveredstatements="5" elements="19" coveredelements="5"/>
   </project>
 </coverage>

/usr/local/src/php-code-coverage/tests/PHP/CodeCoverage/Report/CloverTest.php:38

2) PHP_CodeCoverage_Report_CloverTest::testCloverForFileWithIgnoredLines
Failed asserting that format description matches text.
--- Expected
+++ Actual
@@ @@
       <line num="6" type="stmt" count="0"/>
-      <metrics loc="37" ncloc="25" classes="2" methods="0" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="2" coveredstatements="1" elements="2" coveredelements="1"/>
+      <line num="29" type="stmt" count="0"/>
+      <line num="31" type="stmt" count="0"/>
+      <metrics loc="37" ncloc="25" classes="2" methods="0" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="4" coveredstatements="1" elements="4" coveredelements="1"/>
     </file>
-    <metrics files="1" loc="37" ncloc="25" classes="2" methods="0" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="2" coveredstatements="1" elements="2" coveredelements="1"/>
+    <metrics files="1" loc="37" ncloc="25" classes="2" methods="0" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="4" coveredstatements="1" elements="4" coveredelements="1"/>
   </project>
 </coverage>

/usr/local/src/php-code-coverage/tests/PHP/CodeCoverage/Report/CloverTest.php:51

3) PHP_CodeCoverage_Report_CloverTest::testCloverForClassWithAnonymousFunction
Failed asserting that format description matches text.
--- Expected
+++ Actual
@@ @@
 <?xml version="1.0" encoding="UTF-8"?>
 <coverage generated="1444162129">
   <project timestamp="1444162129">
     <file name="/usr/local/src/php-code-coverage/tests/_files/source_with_class_and_anonymous_function.php">
       <class name="CoveredClassWithAnonymousFunctionInStaticMethod" namespace="global">
-        <metrics methods="2" coveredmethods="1" conditionals="0" coveredconditionals="0" statements="5" coveredstatements="4" elements="7" coveredelements="5"/>
+        <metrics methods="2" coveredmethods="1" conditionals="0" coveredconditionals="0" statements="6" coveredstatements="4" elements="8" coveredelements="5"/>
       </class>
-      <line num="5" type="method" name="runAnonymous" crap="1.04" count="1"/>
+      <line num="5" type="method" name="runAnonymous" crap="1.12" count="1"/>
+      <line num="6" type="stmt" count="0"/>

@@ @@
       <line num="18" type="stmt" count="1"/>
-      <metrics loc="19" ncloc="17" classes="1" methods="2" coveredmethods="1" conditionals="0" coveredconditionals="0" statements="8" coveredstatements="7" elements="10" coveredelements="8"/>
+      <metrics loc="19" ncloc="17" classes="1" methods="2" coveredmethods="1" conditionals="0" coveredconditionals="0" statements="9" coveredstatements="7" elements="11" coveredelements="8"/>
     </file>
-    <metrics files="1" loc="19" ncloc="17" classes="1" methods="2" coveredmethods="1" conditionals="0" coveredconditionals="0" statements="8" coveredstatements="7" elements="10" coveredelements="8"/>
+    <metrics files="1" loc="19" ncloc="17" classes="1" methods="2" coveredmethods="1" conditionals="0" coveredconditionals="0" statements="9" coveredstatements="7" elements="11" coveredelements="8"/>
   </project>
 </coverage>

/usr/local/src/php-code-coverage/tests/PHP/CodeCoverage/Report/CloverTest.php:64

4) PHP_CodeCoverage_Report_FactoryTest::testSomething
Failed asserting that 15 matches expected 10.

/usr/local/src/php-code-coverage/tests/PHP/CodeCoverage/Report/FactoryTest.php:42

5) PHP_CodeCoverageTest::testCollect
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
         32 => null
+        7 => Array ()
+        12 => Array ()
+        17 => Array ()
+        21 => Array ()
+        28 => Array ()
     )
 )

/usr/local/src/php-code-coverage/tests/PHP/CodeCoverageTest.php:269

6) PHP_CodeCoverageTest::testMerge
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
         32 => null
+        7 => Array ()
+        12 => Array ()
+        17 => Array ()
+        21 => Array ()
+        28 => Array ()
     )
 )

/usr/local/src/php-code-coverage/tests/PHP/CodeCoverageTest.php:300

7) PHP_CodeCoverageTest::testMerge2
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
         32 => null
+        7 => Array ()
+        12 => Array ()
+        17 => Array ()
+        21 => Array ()
+        28 => Array ()
     )
 )

/usr/local/src/php-code-coverage/tests/PHP/CodeCoverageTest.php:319

FAILURES!
Tests: 46, Assertions: 72, Failures: 7.

Generating code coverage report in Clover XML format ... done

Generating code coverage report in HTML format ... done

MajorCaiger · 2015-10-09T09:23:34Z

Yikes, looks like I'm not quite there. When I tried to run the tests on master they were failing but they aren't now. I'll investigate.

MajorCaiger · 2015-10-19T07:47:59Z

@sebastianbergmann

This merge request was started before version 3.0 was released and this was something you added when you tidied up the original merge request.

What would you like me to do about it?

sebastianbergmann · 2015-10-19T07:52:23Z

The blacklist is gone and the whitelist is now mandatory. This was not only done to make this speedup refactoring easier but most importantly to promote best practices.

MajorCaiger · 2015-10-19T10:50:50Z

Shall I just remote the line from the change log? Or should I create a new change log? If so what version?

sebastianbergmann · 2015-10-19T13:19:35Z

I think it would be best if the speedup branch did not touch the ChangeLog-3.0.md file. I'll take care of it when I merge the branch into master.

Does it make sense for me to review the branch right now or are you still working on things?

MajorCaiger · 2015-10-19T13:23:53Z

OK, I have removed the ChangeLog changes.

I think it is ready for review. All tests pass and I have run the unit tests with coverage on another project with and without these changes and got the same output.

sebastianbergmann · 2015-10-19T14:01:06Z

I tried the speedup branch with the test suite from https://github.com/sebastianbergmann/money/ and, among other differences, found the following:

master

speedup

The report generated by master is correct and the one generated by speedup is not.

MajorCaiger · 2015-10-19T14:04:51Z

Hmm back to the drawing board then. I will take a look at the config from the money repo and see what I am missing.

Thanks

MajorCaiger · 2015-10-22T14:47:41Z

Hi @sebastianbergmann

I have rebased and squashed the commits as they were getting a little messy.

I have altered the changes now so that all tests still pass, it shouldn't effect the outcomes of any other test suite (but I would be grateful if you could double check this). You now only get the speed improvements if you are running with processUncoveredFilesFromWhitelist turned on, otherwise there is no way of telling which files we can include_once prior to running the tests.

Maks3w · 2015-10-25T08:40:47Z

Shouldn't include a test for scenarios like the money repo?

- Init data by running xdebug with 'unused' and 'dead code' flags on all whitelist files - Run xdebug without flags for each test method - Cache num lines check for each file, to prevent superflous checks - (SB) Fix CS/WS issues

sebastianbergmann · 2015-11-06T08:07:04Z

I just rebased the branch to ease testing of its changes together with changes by @Maks3w.

Maks3w · 2015-11-06T09:17:13Z

src/CodeCoverage/Driver/Xdebug.php

+        if ($determineUnusedAndDead) {
+            xdebug_start_code_coverage(XDEBUG_CC_UNUSED | XDEBUG_CC_DEAD_CODE);
+        } else {
+            xdebug_start_code_coverage();


Path coverage will need all flags enabled.

Anyway path coverage in this point only facilitate ccn so it's ok.

Maks3w · 2015-11-06T09:22:52Z

I guess this is an alternative to sebastianbergmann/phpunit#1878

sebastianbergmann · 2015-11-06T10:01:01Z

Re: #387 (comment)

My plan is to merge the path coverage related changes before the speedup changes. We'll have to see then how the speedup changes need to be adapted.

goetas · 2015-12-21T14:07:33Z

any news on this?

paolomainardi · 2016-01-21T02:51:18Z

Guys please keep us updated

estahn · 2016-03-21T22:20:46Z

Any updates on this feature? Our tests which usually run in total of 3 minutes (without code coverage) run with code coverage in 1:30h.

I guess a more PHPUnit related question: Is there a way to execute code before and after the setUp method (e.g. subscribeEvent('onSetUpStart', ...), subscribeEvent('onSetUpFinish', ...)). Because the following actually reduces our test run time drastically:

    public function setUp()
    {
        xdebug_stop_code_coverage();
        ...
        xdebug_start_code_coverage();
    }

I would like to avoid adding the xdebug function calls everywhere.

sebastianbergmann · 2016-04-21T05:21:33Z

Closing because this has run out-of-sync with master. Please send a new pull request. Thanks!

MPV · 2016-07-26T15:44:42Z

Has any one thought about trying this out again? @MajorCaiger? 😃

The speed improvements mentioned as part of this (in #384) sounded very promising. 👍

MajorCaiger · 2016-07-27T13:26:21Z

Yeah, the changes got merged here :)

#433

MajorCaiger force-pushed the speedup branch from 9f6f381 to 8866c89 Compare September 27, 2015 18:59

sebastianbergmann reviewed Sep 28, 2015
View reviewed changes

MajorCaiger force-pushed the speedup branch from 798fcfa to df9ddc8 Compare September 28, 2015 20:59

MajorCaiger force-pushed the speedup branch from f8ba165 to e426b75 Compare October 2, 2015 13:01

MajorCaiger force-pushed the speedup branch 2 times, most recently from e2a7fe6 to bbd500b Compare October 19, 2015 07:40

MajorCaiger force-pushed the speedup branch from fd22ad8 to 91ace14 Compare October 22, 2015 14:44

Rob Caiger added 2 commits November 6, 2015 09:05

Speed improvements when using xdebug

b28af18

- Init data by running xdebug with 'unused' and 'dead code' flags on all whitelist files - Run xdebug without flags for each test method - Cache num lines check for each file, to prevent superflous checks - (SB) Fix CS/WS issues

Was appending superfluous data during initialize

605738a

sebastianbergmann force-pushed the speedup branch from 8eb6056 to 605738a Compare November 6, 2015 08:06

Maks3w reviewed Nov 6, 2015
View reviewed changes

sebastianbergmann closed this Apr 21, 2016

sebastianbergmann deleted the speedup branch May 4, 2016 05:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Speedup #387

Speedup #387

MajorCaiger commented Sep 27, 2015

sebastianbergmann Sep 28, 2015

MajorCaiger commented Sep 28, 2015

sebastianbergmann commented Oct 6, 2015

MajorCaiger commented Oct 6, 2015

sebastianbergmann commented Oct 6, 2015

MajorCaiger commented Oct 9, 2015

MajorCaiger commented Oct 19, 2015

sebastianbergmann commented Oct 19, 2015

MajorCaiger commented Oct 19, 2015

sebastianbergmann commented Oct 19, 2015

MajorCaiger commented Oct 19, 2015

sebastianbergmann commented Oct 19, 2015

MajorCaiger commented Oct 19, 2015

MajorCaiger commented Oct 22, 2015

Maks3w commented Oct 25, 2015

sebastianbergmann commented Nov 6, 2015

Maks3w Nov 6, 2015

Maks3w Nov 6, 2015

Maks3w commented Nov 6, 2015

sebastianbergmann commented Nov 6, 2015

goetas commented Dec 21, 2015

paolomainardi commented Jan 21, 2016

estahn commented Mar 21, 2016

sebastianbergmann commented Apr 21, 2016

MPV commented Jul 26, 2016 •

edited

Loading

MajorCaiger commented Jul 27, 2016

Speedup #387

Speedup #387

Conversation

MajorCaiger commented Sep 27, 2015

sebastianbergmann Sep 28, 2015

Choose a reason for hiding this comment

MajorCaiger commented Sep 28, 2015

sebastianbergmann commented Oct 6, 2015

MajorCaiger commented Oct 6, 2015

sebastianbergmann commented Oct 6, 2015

MajorCaiger commented Oct 9, 2015

MajorCaiger commented Oct 19, 2015

sebastianbergmann commented Oct 19, 2015

MajorCaiger commented Oct 19, 2015

sebastianbergmann commented Oct 19, 2015

MajorCaiger commented Oct 19, 2015

sebastianbergmann commented Oct 19, 2015

master

speedup

MajorCaiger commented Oct 19, 2015

MajorCaiger commented Oct 22, 2015

Maks3w commented Oct 25, 2015

sebastianbergmann commented Nov 6, 2015

Maks3w Nov 6, 2015

Choose a reason for hiding this comment

Maks3w Nov 6, 2015

Choose a reason for hiding this comment

Maks3w commented Nov 6, 2015

sebastianbergmann commented Nov 6, 2015

goetas commented Dec 21, 2015

paolomainardi commented Jan 21, 2016

estahn commented Mar 21, 2016

sebastianbergmann commented Apr 21, 2016

MPV commented Jul 26, 2016 • edited Loading

MajorCaiger commented Jul 27, 2016

MPV commented Jul 26, 2016 •

edited

Loading