Runtime dependency on scoverage?

[gijsbert802]

When I build artifacts (a jar) using sbt coverage test package, the class files in the artifact have references to scoverage files. I then get runtime exceptions when using the jar.
This didn't happen before version 1.0 I think.

For example, this is the output of a unit test that uses the class DocumentGenerator from my jar:

[info] Exception encountered when attempting to run a suite with class name: nl.zorgdomein.messaging.MyTest *** ABORTED ***
[info]   java.io.FileNotFoundException: /var/lib/jenkins/jobs/my-lib/workspace/target/scala-2.11/scoverage-data/scoverage.measurements.52 (No such file or directory)
[info]   at java.io.FileOutputStream.open(Native Method)
[info]   at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
[info]   at java.io.FileWriter.<init>(FileWriter.java:107)
[info]   at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:45)
[info]   at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:45)
[info]   at scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:190)
[info]   at scala.collection.concurrent.TrieMap.getOrElseUpdate(TrieMap.scala:633)
[info]   at scoverage.Invoker$.invoked(Invoker.scala:45)
[info]   at nl.zorgdomein.mylib.DocumentGenerator$class.$init$(DocumentGenerator.scala:27)
[info]   ...

Output of javap -c DocumentGenerator.class looks like this:

Compiled from "DocumentGenerator.scala"
public abstract class nl.zorgdomein.mylib.DocumentGenerator$class {
  public static java.io.ByteArrayOutputStream toRtf(nl.zorgdomein.mylib.DocumentGenerator, java.lang.String);
    Code:
       0: getstatic     #13                 // Field scoverage/Invoker$.MODULE$:Lscoverage/Invoker$;
       3: iconst_1
       4: ldc           #15                 // String /var/lib/jenkins/jobs/my-lib/workspace/target/scala-2.11/scoverage-data
       6: invokevirtual #19                 // Method scoverage/Invoker$.invoked:(ILjava/lang/String;)V
       9: aload_0
      10: ldc           #21                 // String application/rtf
      12: aload_1
      13: invokestatic  #25                 // Method transform:(Lnl/zorgdomein/mylib/DocumentGenerator;Ljava/lang/String;Ljava/lang/String;)Ljava/io/ByteArrayOutputStream;
      16: areturn

Is this supposed to happen?

[dkarpenko]

@gijsbert802, what output do you get when run sbt test on your project?

Personally I see the same issue both for sbt coverage test and sbt test..

[gijsbert802]

sbt test works fine when the target directory is clean, if I run sbt test after sbt coverage test then the same issue occurs.

[brian-dechoux]

Hi, any solution for this issue ?

[alexflav23]

Hi @buzz2buzz You need to disable coverage with coverageOff before publishing or packaging and everything will be just fine. I've ran into this issue myself and that was the quick fix.

[kevinmeredith]

Hey @alexflav23 - I ran:

>coverageOff
>test

But I still ran into the above issue:

java.io.FileNotFoundException: /var/lib/jenkins/jobs/my-lib/workspace/target/scala-2.11/scoverage-data/scoverage.measurements.52 (No such file or directory)

Could you please let me know of any other fixes?

[hibikir]

@alexflav23 has it right. It's in the README, but I'd argue it's not the clearest:

If you're running the coverage reports from within an sbt console session (as opposed to one command per sbt launch), then the coverage command is sticky. To turn it back off when you're done running reports, use the coverageOff command or reset coverageEnabled with set coverageEnabled := false.

I wonder if the maintainers would appreciate a PR that made this part of the documentation more explicit. Maybe have an example that went sbt clean coverage test coverageOff publish

[kevinmeredith]

@alexflav23, don't the results of this test repo (https://github.com/kevinmeredith/test_scoverage_instrumentation) demonstrate that running ... coverageOff publish still includes the instrumentation in the compiled prod code, i.e. it's not a valid fix?

[captify-akraievoy]

Any updates on this? Apparently the issue is still here, as @kevinmeredith suggested?

@gslowikowski , as your'e the most recent and active contributor to this project, I'll mention you and kindly ask for help

[gslowikowski]

It works with sbt-scoverage plugin version 1.4.0.

This issue should be closed. @captify-akraievoy confirm if it still doesn't work for you.

Additionally, scalac-scoverage-runtime dependency scope has been changed from provided to default (compile) so it will work even if you don't turn scoverage off before publishing (but, of course, you should not publish instrumented code).

[captify-akraievoy]

Thanks, your solution - adding coverageOff to our builds - just did the trick, we are using 1.4.0 right now.
I believe this issue could be closed.

[nmaggioni]

If coverageEnabled := true is set in build.sbt, che coverageOff option seems to have no effect.

I was getting the FileNotFoundException even when I ran sbt coverageOff dist, and setting coverageEnabled to false in the build configuration solved it. I'm running v1.5.0.

I'm leaving this note here for the posterity 🙂

[WayneWang12]

Hi~ I'm using play framework and command activator coverageOff dist to package my application. But it won't run in production environment because of the exception below:

Oops, cannot start the server.
java.io.FileNotFoundException: /home/wayne/IdeaProjects/backend/target/scala-2.11/scoverage-data/scoverage.measurements.1 (No such file or directory)
        at java.io.FileOutputStream.open0(Native Method)
        at java.io.FileOutputStream.open(FileOutputStream.java:270)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
        at java.io.FileWriter.<init>(FileWriter.java:107)
        at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:42)
        at scoverage.Invoker$$anonfun$1.apply(Invoker.scala:42)

It seems that scoverage packaged my local file path to the application and will interfere the execution when the disted zip uploaded to production server.

I have to set coverageEnabled to false to disable it. However, this is really inconvenient because I cannot control the unit test code coverage of our project. Hope this could be fixed soon. Thanks very much in advance.

[bugorz]

@scweang

FYI, I removed coverageEnabled := false in build.sbt and set coverageEnabled:=true in the build (TeamCity build). Not sure if this can help you.

[WayneWang12]

@bugorz Thanks for your reply. I have resoulve the problem by setting as below:

coverageEnabled in(Test, compile) := true,
coverageEnabled in(Compile, compile) := false,

Seems it works fine.

[gslowikowski]

You should not add any coverageEnabled setting to build.sbt.

coverage/coverageOn and coverageOff commands toggle coverageEnabled in ThisBuild setting. It affects the project (every project in multi-module build) and coverage should work properly unless you override it with coverageEnabled added to project settings in build file.

I know, the readme file is not clear in this area.

[holograph]

This is actually quite easily missed, and in my opinion isn't well documented at all. While it's entirely my fault, I rather expected scoverage to work with runtime code injection via a JVM agent (similar to other coverage tools), and consequently coverageEnabled would affect runtime options as opposed to generated code. I submit that this is an easy mistake to make, and even highlighting it in the documentation isn't likely to completely eliminate the problem.

On the other hand, it should be blatantly obvious that coverage-enabled artifacts are generally not intended to be published; how about modifying publish to clean-and-rebuild all artifacts with coverage enabled, unless explicitly told to do otherwise (think coveragePublishArtifacts := false by default)? Would that make sense?

(As an aside, this might have undesirable ramifications on build times, but I believe this can be minimized fairly easily; also, publishing doesn't happen very often, so the actual impact ought to be acceptable).

[todor-kolev]

I am trying to generate coverageReport from a release plugin step. This doesn't work for me as no scoverage files are getting generated. Seems that the coverageOn bit has no effect. Has anybody observed similar behaviour?

commands += Command.command("prepareRelease")((state: State) => {
  println("Preparing release...")
  val extracted = Project extract state
  var st = extracted.append(Seq(releaseProcess := Seq[ReleaseStep](
    runClean,
    checkSnapshotDependencies,
    inquireVersions,
    setReleaseVersion,
    commitReleaseVersion,
    tagRelease,
    releaseStepCommand("coverageOn"),
    runTest,
    releaseStepTask(coverageReport),
    releaseStepCommand("coverageOff"),
    releaseStepTask(dist)
  )), state)
  Command.process("release with-defaults", st)
})

[scottf]

sbt clean coverageOff docker:publishLocal doesn't work meaning it doesn't turn coverage off
1.0.4 sbt
2.12.4 scala
1.5.1 scoverage plugin

[naveencotha-zt]

I have these two lines in the build.sbt of my single module sbt project,

coverageEnabled.in(Test, test) := true
coverageEnabled in(Compile, compile) := false

I still do see this error, when I build the jar file with assembly.

[gslowikowski]

Guys, can you provide simple test projects exposing the issue?

[mpampin]

I had the same problem last week. Tried everything in this thread and nothing seemed to fix the issue, until I realized it was one of our own dependencies in the project that had scoverage and 'coverageEnabled := true' in the build.sbt.

[Bongani]

I know there's a lot of discussion about setting 'coverageEnabled := false' in the build.sbt, but this ends up disabling scoverage in our build pipeline. This is what ended up working for us in the our build.sbt file

Default / coverageEnabled := false
Test / coverageMinimum := 80
Test / coverageFailOnMinimum := true
Test / coverageHighlighting := true

[jie-wu-exa]

Any update?
Now we remove the coverageEnabled from the build.sbt file and use these commands in pipeline:
sbt "set publishToArtifactory := true" publish
sbt clean clean coverage test
sbt coverageOff

But I think the test is running twice. One is for generating coverage file, the other one is for publish. Is there any way to optimize this?

[uccmen]

You should not add any coverageEnabled setting to build.sbt.

coverage/coverageOn and coverageOff commands toggle coverageEnabled in ThisBuild setting. It affects the project (every project in multi-module build) and coverage should work properly unless you override it with coverageEnabled added to project settings in build file.

I know, the readme file is not clear in this area.

This worked for us. Removed coverageEnabled in build.sbt and only call coverage/coverageOn where needed.
Thank you 🙌🏽

[JeroenSchmidt]

We also encountered this problem when we decided to use sbt-scoverage in one of our projects.

What is the recommended pattern for using sbt-scoverage? When should it be included in the build.sbt and when should it simply be called from the sbt shell?

[ckipp01]

Hey @JeroenSchmidt personally I've found that having it in an alias works quite well. For example at work we use it in a testCoverage alias that basically just does a coverage ; test ; coverageAggregate ; coverageOff. Keep in mind that coverage is sticky so it'll be on and instrumenting your code until you run a coverageOff. That's the main reason it's not recommended to just use coverageEnabled in your build as you'll always be instrumenting your code.

[JeroenSchmidt]

@ckipp01 Thanks for the suggestion.

Is there a reason why this should not be included in the SBT file?
coverageEnabled in(Test, compile) := true,
coverageEnabled in(Compile, compile) := false,