Hermes does not canonicalize NaNs stored in typed array

[JimWongM]

Bug Description

Hermes return different results in O0 and Og.

Hermes git revision (if applicable):01d96f4198789446b0e14226ec646b38f0ec3fd0 in branch static_h
OS: Linux
Platform: x86_64

Steps To Reproduce

1. Compile hermes in Debug mode
2. Run: ./hermes -O0 test.js
3. Run: ./hermes -Og test.js

code example:

let buffer = new ArrayBuffer(96);
var arr64 = new Float64Array(buffer);
var arr32 = new Float32Array(buffer);
arr32[4] = -NaN;
print(arr64[2]);

Result of Step 2:
1.058925634e-314

Result of Step 3:
2.1199235295e-314

The Expected Behavior

The results of Step 2 and Step 3 should be the same.

[tmikov]

Thank you. A better repro:

let buffer = new ArrayBuffer(4);
var arrInt = new Int32Array(buffer);
var arr32 = new Float32Array(buffer);
var a = NaN;
arr32[0] = -a;
print(arr32[0]);
print(arrInt[0].toString(16));

v8: -400000
sh-hermes: 7fc00000

[tmikov]

Is this actually a bug? These seem to be different representations of NaN.

[tmikov]

According to the spec:

1. 1. If type is float32, then
      a. a. Let rawBytes be a List whose elements are the 4 bytes that are the result of converting value to IEEE 754-2019 binary32 format using roundTiesToEven mode. The bytes are arranged in little endian order. If value is NaN, rawBytes may be set to any implementation chosen IEEE 754-2019 binary32 format Not-a-Number encoding. An implementation must always choose the same encoding for each implementation distinguishable NaN value.

Not 100% sure what it means.

[tmikov]

So far my interpretation of the spec and various discussions is that this is fine. Canonicalizing every NaN on writes to a floating point typed array might get pretty expensive in some scenarios.