Skip to content

feat: switch to using Node's built in coverage #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 10, 2018
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 6 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# c8 - native v8 code-coverage
# c8 - native V8 code-coverage

Code-coverage using [v8's Inspector](https://nodejs.org/dist/latest-v8.x/docs/api/inspector.html)
Code-coverage using [V8's Inspector](https://nodejs.org/dist/latest-v8.x/docs/api/inspector.html)
that's compatible with [Istanbul's reporters](https://istanbul.js.org/docs/advanced/alternative-reporters/).

Like [nyc](https://github.com/istanbuljs/nyc), c8 just magically works:
@@ -10,20 +10,10 @@ npm i c8 -g
c8 node foo.js
```

The above example will collect coverage for `foo.js` using v8's inspector.
The above example will collect coverage for `foo.js` using V8's inspector.

## Disclaimer

c8 uses bleeding edge v8 features (_it's an ongoing experiment, testing
what will eventually be possible in the realm of test coverage in Node.js_).

For the best experience, try running with [a canary build of Node.js](https://github.com/v8/node).

## How it Works

Before running your application c8 creates [an inspector session](https://nodejs.org/api/inspector.html) in v8 and enables v8's
[built in coverage reporting](https://v8project.blogspot.com/2017/12/javascript-code-coverage.html).

Just before your application exits, c8 fetches the coverage information from
v8 and writes it to disk in a format compatible with
[Istanbul's reporters](https://istanbul.js.org/).
c8 uses
[bleeding edge Node.js features](https://github.com/nodejs/node/pull/22527),
make sure you're running Node.js `>= 10.10.0`.
15 changes: 6 additions & 9 deletions bin/c8.js
Original file line number Diff line number Diff line change
@@ -4,31 +4,28 @@
const foreground = require('foreground-child')
const mkdirp = require('mkdirp')
const report = require('../lib/report')
const {resolve} = require('path')
const { resolve } = require('path')
const rimraf = require('rimraf')
const sw = require('spawn-wrap')
const {
hideInstrumenteeArgs,
hideInstrumenterArgs,
yargs
} = require('../lib/parse-args')

const instrumenterArgs = hideInstrumenteeArgs()

const argv = yargs.parse(instrumenterArgs)

const tmpDirctory = resolve(argv.coverageDirectory, './tmp')
rimraf.sync(tmpDirctory)
mkdirp.sync(tmpDirctory)

sw([require.resolve('../lib/wrap')], {
C8_ARGV: JSON.stringify(argv)
})
process.env.NODE_V8_COVERAGE = tmpDirctory

foreground(hideInstrumenterArgs(argv), (out) => {
report({
include: argv.include,
exclude: argv.exclude,
reporter: Array.isArray(argv.reporter) ? argv.reporter : [argv.reporter],
coverageDirectory: argv.coverageDirectory,
watermarks: argv.watermarks
watermarks: argv.watermarks,
resolve: argv.resolve
})
})
6 changes: 5 additions & 1 deletion lib/parse-args.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const Exclude = require('test-exclude')
const findUp = require('find-up')
const {readFileSync} = require('fs')
const { readFileSync } = require('fs')
const yargs = require('yargs')
const parser = require('yargs-parser')

@@ -28,6 +28,10 @@ yargs
default: './coverage',
describe: 'directory to output coverage JSON and reports'
})
.option('resolve', {
default: '',
describe: 'resolve paths to alternate base directory'
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will be required for the Node.js codebase itself.

})
.pkgConf('c8')
.config(config)
.demandCommand(1)
50 changes: 45 additions & 5 deletions lib/report.js
Original file line number Diff line number Diff line change
@@ -1,14 +1,29 @@
const Exclude = require('test-exclude')
const libCoverage = require('istanbul-lib-coverage')
const libReport = require('istanbul-lib-report')
const reports = require('istanbul-reports')
const {readdirSync, readFileSync} = require('fs')
const {resolve} = require('path')
const { readdirSync, readFileSync } = require('fs')
const { resolve } = require('path')
const v8CoverageMerge = require('v8-coverage-merge')
const v8toIstanbul = require('v8-to-istanbul')

class Report {
constructor ({reporter, coverageDirectory, watermarks}) {
constructor ({
exclude,
include,
reporter,
coverageDirectory,
watermarks,
resolve
}) {
this.reporter = reporter
this.coverageDirectory = coverageDirectory
this.watermarks = watermarks
this.resolve = resolve
this.exclude = Exclude({
exclude: exclude,
include: include
})
}
run () {
const map = this._getCoverageMapFromAllCoverageFiles()
@@ -25,9 +40,34 @@ class Report {
}
_getCoverageMapFromAllCoverageFiles () {
const map = libCoverage.createCoverageMap({})
const mergedResults = {}
this._loadReports().forEach((report) => {
report.result.forEach((result) => {
if (this.exclude.shouldInstrument(result.url)) {
if (mergedResults[result.url]) {
mergedResults[result.url] = v8CoverageMerge(
mergedResults[result.url],
result
)
} else {
mergedResults[result.url] = result
}
}
})
})

this._loadReports().forEach(function (report) {
map.merge(report)
Object.keys(mergedResults).forEach((url) => {
const result = mergedResults[url]
// console.info(JSON.stringify(result, null, 2))
try {
const path = resolve(this.resolve, result.url)
const script = v8toIstanbul(path)
script.applyCoverage(result.functions)
map.merge(script.toIstanbul())
} catch (err) {
// most likely this was an internal Node.js library.
if (err.code !== 'ENOENT' && err.code !== 'EISDIR') throw err
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TimothyGu any thoughts about how to easily tell whether the coverage outputted was for one of Node.js' built in libraries, e.g., internal/inspector.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about checking whether the URL was an absolute path or not?

}
})

return map
81 changes: 0 additions & 81 deletions lib/wrap.js

This file was deleted.

6,285 changes: 2,426 additions & 3,859 deletions package-lock.json

Large diffs are not rendered by default.

60 changes: 20 additions & 40 deletions package.json
Original file line number Diff line number Diff line change
@@ -5,16 +5,10 @@
"main": "index.js",
"bin": "./bin/c8.js",
"scripts": {
"bundle": "bundle-dependencies update",
"test": "nyc mocha ./test/*.js",
"test": "./bin/c8.js --reporter=html --reporter=text mocha ./test/*.js",
"posttest": "standard",
"release": "standard-version"
},
"c8": {
"exclude": [
"test/*.js"
]
},
"standard": {
"ignore": [
"test/fixtures"
@@ -28,49 +22,35 @@
"profiler",
"inspector"
],
"engines": {
"node": ">=8.8.1"
},
"author": "Ben Coe <ben@npmjs.com>",
"license": "ISC",
"dependencies": {
"find-up": "^2.1.0",
"find-up": "^3.0.0",
"foreground-child": "^1.5.6",
"istanbul-lib-coverage": "^1.1.1",
"istanbul-lib-report": "^1.1.2",
"istanbul-reports": "^1.1.3",
"istanbul-lib-coverage": "^2.0.1",
"istanbul-lib-report": "^2.0.1",
"istanbul-reports": "^2.0.0",
"mkdirp": "^0.5.1",
"rimraf": "^2.6.2",
"signal-exit": "^3.0.2",
"spawn-wrap": "^1.4.2",
"test-exclude": "^4.1.1",
"uuid": "^3.1.0",
"test-exclude": "^5.0.0",
"uuid": "^3.3.2",
"v8-coverage-merge": "^1.1.2",
"v8-to-istanbul": "^1.2.0",
"yargs": "^10.0.3",
"yargs-parser": "^8.0.0"
"yargs": "^12.0.2",
"yargs-parser": "^10.1.0"
},
"devDependencies": {
"bundle-dependencies": "^1.0.2",
"chai": "^4.1.2",
"mocha": "^4.0.1",
"nyc": "^11.4.1",
"standard": "^10.0.3",
"standard-version": "^4.2.0"
"mocha": "^5.2.0",
"standard": "^12.0.1",
"standard-version": "^4.4.0"
},
"engines": {
"node": ">=10.10.0"
},
"bundledDependencies": [
"find-up",
"foreground-child",
"istanbul-lib-coverage",
"istanbul-lib-report",
"istanbul-reports",
"mkdirp",
"rimraf",
"signal-exit",
"spawn-wrap",
"test-exclude",
"uuid",
"v8-to-istanbul",
"yargs",
"yargs-parser"
"files": [
"lib",
"bin",
"LICENSE"
]
}
2 changes: 0 additions & 2 deletions test/fixtures/timeout.js → test/fixtures/async.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
const a = 'apple' ? 'banana' : 'grape'

function cool () {

}
5 changes: 0 additions & 5 deletions test/fixtures/c.js

This file was deleted.

11 changes: 0 additions & 11 deletions test/fixtures/c.mjs

This file was deleted.

15 changes: 0 additions & 15 deletions test/fixtures/fib.js

This file was deleted.

12 changes: 12 additions & 0 deletions test/fixtures/multiple-spawn.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
const {spawnSync} = require('child_process')
{
const output = spawnSync(process.execPath, ['./test/fixtures/subprocess',
'1'])
console.info(output.stdout.toString('utf8'))
}

{
const output = spawnSync(process.execPath, ['./test/fixtures/subprocess',
'2'])
console.info(output.stdout.toString('utf8'))
}
2 changes: 1 addition & 1 deletion test/fixtures/normal.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
require('./timeout')
require('./async')

console.info('i am a line of code')

15 changes: 15 additions & 0 deletions test/fixtures/subprocess.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
function first () {
console.info('first')
}

function second () {
console.info('second')
}

if (process.argv[2] === '1') {
first()
}

if (process.argv[2] === '2') {
second()
}
40 changes: 27 additions & 13 deletions test/integration.js
Original file line number Diff line number Diff line change
@@ -1,26 +1,40 @@
/* global describe, it */

const {spawnSync} = require('child_process')
const { spawnSync } = require('child_process')
const c8Path = require.resolve('../bin/c8')

require('chai').should()

describe('c8', () => {
it('reports coverage for script that exits normally', () => {
const {output} = spawnSync(c8Path, [
const { output } = spawnSync(c8Path, [
'--exclude="test/*.js"',
process.execPath,
require.resolve('./fixtures/normal')
], {
env: process.env,
cwd: process.cwd()
})
])
output.toString('utf8').should.include(`
------------|----------|----------|----------|----------|----------------|
File | % Stmts | % Branch | % Funcs | % Lines |Uncovered Lines |
------------|----------|----------|----------|----------|----------------|
All files | 91.67 | 100 | 80 | 91.67 | |
normal.js | 85.71 | 100 | 50 | 85.71 | 14,15,16 |
timeout.js | 100 | 100 | 100 | 100 | |
------------|----------|----------|----------|----------|----------------|`)
-----------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
-----------|----------|----------|----------|----------|-------------------|
All files | 91.18 | 88.89 | 0 | 91.18 | |
async.js | 100 | 100 | 100 | 100 | |
normal.js | 85.71 | 75 | 0 | 85.71 | 14,15,16 |
-----------|----------|----------|----------|----------|-------------------|`)
})

it('merges reports from subprocesses together', () => {
const { output } = spawnSync(c8Path, [
'--exclude="test/*.js"',
process.execPath,
require.resolve('./fixtures/multiple-spawn')
])
output.toString('utf8').should.include(`
-------------------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
-------------------|----------|----------|----------|----------|-------------------|
All files | 100 | 77.78 | 100 | 100 | |
multiple-spawn.js | 100 | 100 | 100 | 100 | |
subprocess.js | 100 | 71.43 | 100 | 100 | 9,13 |
-------------------|----------|----------|----------|----------|-------------------|`)
})
})