@@ -30,8 +30,6 @@ install.packages("ggplot2")
30
30
install.packages(" plyr" )
31
31
```
32
32
33
- [ wrk ] : https://github.com/wg/wrk
34
-
35
33
## Running benchmarks
36
34
37
35
### Running individual benchmarks
@@ -43,7 +41,7 @@ conclusions about the performance.
43
41
Individual benchmarks can be executed by simply executing the benchmark script
44
42
with node.
45
43
46
- ```
44
+ ``` console
47
45
$ node benchmark/buffers/buffer-tostring.js
48
46
49
47
buffers/buffer-tostring.js n=10000000 len=0 arg=true: 62710590.393305704
@@ -65,7 +63,7 @@ measured in ops/sec (higher is better).**
65
63
Furthermore you can specify a subset of the configurations, by setting them in
66
64
the process arguments:
67
65
68
- ```
66
+ ``` console
69
67
$ node benchmark/buffers/buffer-tostring.js len=1024
70
68
71
69
buffers/buffer-tostring.js n=10000000 len=1024 arg=true: 3498295.68561504
@@ -78,7 +76,7 @@ Similar to running individual benchmarks, a group of benchmarks can be executed
78
76
by using the ` run.js ` tool. Again this does not provide the statistical
79
77
information to make any conclusions.
80
78
81
- ```
79
+ ``` console
82
80
$ node benchmark/run.js arrays
83
81
84
82
arrays/var-int.js
@@ -98,7 +96,7 @@ arrays/zero-int.js n=25 type=Buffer: 90.49906662339653
98
96
```
99
97
100
98
It is possible to execute more groups by adding extra process arguments.
101
- ```
99
+ ``` console
102
100
$ node benchmark/run.js arrays buffers
103
101
```
104
102
@@ -119,13 +117,13 @@ First build two versions of node, one from the master branch (here called
119
117
120
118
The ` compare.js ` tool will then produce a csv file with the benchmark results.
121
119
122
- ```
120
+ ``` console
123
121
$ node benchmark/compare.js --old ./node-master --new ./node-pr-5134 string_decoder > compare-pr-5134.csv
124
122
```
125
123
126
124
For analysing the benchmark results use the ` compare.R ` tool.
127
125
128
- ```
126
+ ``` console
129
127
$ cat compare-pr-5134.csv | Rscript benchmark/compare.R
130
128
131
129
improvement significant p.value
@@ -159,16 +157,14 @@ _For the statistically minded, the R script performs an [independent/unpaired
159
157
same for both versions. The significant field will show a star if the p-value
160
158
is less than ` 0.05 ` ._
161
159
162
- [ t-test ] : https://en.wikipedia.org/wiki/Student%27s_t-test#Equal_or_unequal_sample_sizes.2C_unequal_variances
163
-
164
160
The ` compare.R ` tool can also produce a box plot by using the ` --plot filename `
165
161
option. In this case there are 48 different benchmark combinations, thus you
166
162
may want to filter the csv file. This can be done while benchmarking using the
167
163
` --set ` parameter (e.g. ` --set encoding=ascii ` ) or by filtering results
168
164
afterwards using tools such as ` sed ` or ` grep ` . In the ` sed ` case be sure to
169
165
keep the first line since that contains the header information.
170
166
171
- ```
167
+ ``` console
172
168
$ cat compare-pr-5134.csv | sed ' 1p;/encoding=ascii/!d' | Rscript benchmark/compare.R --plot compare-plot.png
173
169
174
170
improvement significant p.value
@@ -190,15 +186,15 @@ example to analyze the time complexity.
190
186
To do this use the ` scatter.js ` tool, this will run a benchmark multiple times
191
187
and generate a csv with the results.
192
188
193
- ```
189
+ ``` console
194
190
$ node benchmark/scatter.js benchmark/string_decoder/string-decoder.js > scatter.csv
195
191
```
196
192
197
193
After generating the csv, a comparison table can be created using the
198
194
` scatter.R ` tool. Even more useful it creates an actual scatter plot when using
199
195
the ` --plot filename ` option.
200
196
201
- ```
197
+ ``` console
202
198
$ cat scatter.csv | Rscript benchmark/scatter.R --xaxis chunk --category encoding --plot scatter-plot.png --log
203
199
204
200
aggregating variable: inlen
@@ -229,7 +225,7 @@ can be solved by filtering. This can be done while benchmarking using the
229
225
afterwards using tools such as ` sed ` or ` grep ` . In the ` sed ` case be
230
226
sure to keep the first line since that contains the header information.
231
227
232
- ```
228
+ ``` console
233
229
$ cat scatter.csv | sed -E ' 1p;/([^,]+, ){3}128,/!d' | Rscript benchmark/scatter.R --xaxis chunk --category encoding --plot scatter-plot.png --log
234
230
235
231
chunk encoding mean confidence.interval
@@ -290,3 +286,6 @@ function main(conf) {
290
286
bench .end (conf .n );
291
287
}
292
288
```
289
+
290
+ [ wrk ] : https://github.com/wg/wrk
291
+ [ t-test ] : https://en.wikipedia.org/wiki/Student%27s_t-test#Equal_or_unequal_sample_sizes.2C_unequal_variances
0 commit comments