Performance issues in Buffer.writeUInt* functions?

[hjerabek]

I recently wrote some convenience functions for writing unsigned integers into uint8Arrays, buffers, or other array-like objects. Out of curiosity, I wanted to check how much slower these functions were compared to the writeUint* functions available in Node.js' buffers. Strangely, it seems that the convenience functions were significantly faster (1.5-4 times depending on the type/size of the uint). The code posted here is a plug-and-play benchmark script for the uint32 case. While, the convenience functions improve over time, probably due to TurboFan optimizations kicking in, the performance of Node.js' buffer functions seems to degrade (see output of benchmark script).

Version

v14.15.0

Platform

Linux X 4.15.0-117-generic #118~16.04.1-Ubuntu SMP Sat Sep 5 23:35:06 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

What steps will reproduce the bug?

const N_WARMUP_ITERATIONS=8;
const N_OUTPUT_ITERATIONS=8;
const N_NUMBERS=10e6;

// define custom uint32 writer function
const toUint32=function(number,target,startindex){
	var t=target||new Uint8Array(4);
	var i=startindex||0;
    var n=number>>>0;
    t[i+3]=n%256;
    if (n<256) {
    	t[i+2]=0;
    	t[i+1]=0;
    	t[i]=0;
    	return t;
    }
    n>>>=8;
    t[i+2]=n%256;
    if (n<256) {
	    t[i+1]=0;
    	t[i]=0;
	    return t;
	}
    n>>>=8;
    t[i+1]=n%256;
    if (n<256) {
    	t[i]=0;
	    return t;
	}
    n>>>=8;
    t[i]=n%256;
    return t;
};

// define benchmark functions
var bench0=(nItr,numbers,buffer)=>{
	var i,j,n=numbers.length;
	console.log("\nbuffer.writeUInt32BE(number,index)");
	for (var j=0;j<nItr;j++) {
		for (i=0;i<n;i++) numbers[i]=Math.trunc(Math.random()*nMaxValue);
		console.time("::");
		for (i=0;i<n;i++) buffer.writeUInt32BE(numbers[i],i*4);
		console.timeEnd("::");
	}
}
var bench1=(nItr,numbers,buffer)=>{
	var i,j,n=numbers.length,f=toUint32;
	console.log("\ntoUint32(number,buffer,index)");
	for (var j=0;j<nItr;j++) {
		for (i=0;i<n;i++) numbers[i]=Math.trunc(Math.random()*nMaxValue);
		console.time("::");
		for (i=0;i<n;i++) f(numbers[i],buffer,i*4);
		console.timeEnd("::");
	}
}

// warmup
console.log("warming up ...");
var i,j,b0=Buffer.alloc(4*N_NUMBERS),b1=Buffer.alloc(4*N_NUMBERS),nMaxValue=Math.pow(2,32),numbers=new Array(N_NUMBERS);
for (j=0;j<N_WARMUP_ITERATIONS;j++) {
	for (i=0;i<N_NUMBERS;i++) numbers[i]=Math.trunc(Math.random()*nMaxValue);
	for (i=0;i<N_NUMBERS;i++) b0.writeUInt32BE(numbers[i],i*4);
	for (i=0;i<N_NUMBERS;i++) toUint32(numbers[i],b1,i*4);
}

// test equality of output
console.log("\noutput equality verification:",b0.compare(b1)===0);

// run benchmarks
if (Math.random()>0.5) {
	bench1(N_OUTPUT_ITERATIONS,numbers,b1);
	bench0(N_OUTPUT_ITERATIONS,numbers,b0);
} else {
	bench0(N_OUTPUT_ITERATIONS,numbers,b0);
	bench1(N_OUTPUT_ITERATIONS,numbers,b1);
}

What is the expected behavior?

buffer.writeUInt* functions should at least be as fast as pure JavaScript implementations

What do you see instead?

buffer.writeUInt* functions' performance is lower and degrades over time compared to the submitted pure JavaScript implementations

[bnoordhuis]

I'm not surprised an open-coded version is faster than the equivalent Buffer prototype method because the latter validates the input in several ways.

For new code, I would suggest either open-coding or using DataView.