[wasm] Enable SIMD (#80797)

* [wasm] Enable SIMD

* Let old V8 use simd

Co-authored-by: Ankit Jain <[email protected]>

* [wasm] ManagedToNativeGenerator: Skip unmanaged dlls

.. instead crashing with an exception like:

```
src/mono/wasm/build/WasmApp.Native.targets(296,5): error MSB4018: (NETCORE_ENGINEERING_TELEMETRY=Build) The "ManagedToNativeGenerator" task failed unexpectedly.
System.BadImageFormatException: This PE image is not a managed executable.
   at System.Reflection.MetadataLoadContext.LoadFromStreamCore(Stream peStream)
   at System.Reflection.MetadataLoadContext.LoadFromAssemblyPath(String assemblyPath)
   at PInvokeTableGenerator.Generate(String[] pinvokeModules, String[] assemblies, String outputPath) in /_/src/tasks/WasmAppBuilder/PInvokeTableGenerator.cs:line 42
   at ManagedToNativeGenerator.ExecuteInternal() in /_/src/tasks/WasmAppBuilder/ManagedToNativeGenerator.cs:line 68
   at ManagedToNativeGenerator.Execute() in /_/src/tasks/WasmAppBuilder/ManagedToNativeGenerator.cs:line 53
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask)
```

The wasm targets currently are not able to differentiate the managed
assemblies from the unmanaged ones. so it needs to be handled here.

* [wasm] WasmApp.Native.targets: do not trigger relinking when WasmEnableSIMD=true

Instead, let the property be effective only for the AOT case.

* Use staging image with newer v8

* [wasm] WasmAppHost: Add support for host arguments

* [wasm] console template: Add --experimental-wasm-simd to node, and remove v8 config as that isn't supported

* [wasm] Pass --experiment-wasm-simd for aot library tests too

* Try to pass full path to v8

Also remove --experimental-wasm-simd option, which shouldn't be needed
for newer v8

* Use docker image for all wasm helix jobs

As we need newer v8

* Set DOTNET_CLI_HOME under workitem payload

The correlation payload is read only

* Revert "Try to pass full path to v8"

This reverts commit a15b3ef.

* Use DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1

Let see whether it will avoid all the writes to correlation payload
path

* Set NeedsEMSDKNode to false

To try if we can use newer node in path

* Revert "Set DOTNET_CLI_HOME under workitem payload"

This reverts commit 5dec1aa.

* Set DOTNET_CLI_HOME under workitem payload

The correlation payload is read only. And even with
`export DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1` we have still many
failures.

* Reverse the test here as we don't relink anymore

* Set _ExtraTrimmerArgs for tests

So that linker substitution files are used

* Put the SIMD/trim related props in separate group

More changes from Ankit

* Add dynamic dependency

* Disable Vector128IsHardwareAcceleratedTest

Til we have the interpreter support in place

* Disable GenericVectorTests.IsHardwareAcceleratedTest

Til we have the interpreter support in place

* Disable check_no_intrinsic_cattr in emit_vector_2_3_4

And add note about opened issue

* [mono] Disable few quaternion intrinsics

For * and / operators as these are not the same as vector operations.
Opened #82408 to implement
correct ones in future.

* Change comparison order

* Use stable images

* WBT: Add --engine-arg=--experimental-wasm-simd for node, and v8

* Do not skip dotnet 1st time experience anymore

---------

Co-authored-by: Ankit Jain <[email protected]>

Author: radekdoulik

eng/pipelines/libraries/helix-queues-setup.yml

@@ -193,7 +193,7 @@ jobs:
 
     # Browser WebAssembly
     - ${{ if eq(parameters.platform, 'browser_wasm') }}:
-      - Ubuntu.1804.Amd64.Open
+      - (Ubuntu.1804.Amd64)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-webassembly
 
     # Browser WebAssembly Firefox
     - ${{ if eq(parameters.platform, 'browser_wasm_firefox') }}:

eng/testing/WasmRunnerAOTTemplate.sh

@@ -35,8 +35,10 @@ if [[ "$XHARNESS_COMMAND" == "test" ]]; then
 	if [[ -z "$JS_ENGINE" ]]; then
 		if [[ "$SCENARIO" == "WasmTestOnNodeJS" || "$SCENARIO" == "wasmtestonnodejs" ]]; then
 			JS_ENGINE="--engine=NodeJS"
+			JS_ENGINE_ARGS="$JS_ENGINE_ARGS --engine-arg=--experimental-wasm-simd"
 		else
 			JS_ENGINE="--engine=V8"
+			JS_ENGINE_ARGS="$JS_ENGINE_ARGS --engine-arg=--experimental-wasm-simd"
 		fi
 	fi
 

eng/testing/tests.wasm.targets

@@ -35,6 +35,21 @@
     <BuildAOTTestsOn Condition="'$(BuildAOTTestsOn)' == ''">local</BuildAOTTestsOn>
   </PropertyGroup>
 
+  <PropertyGroup Condition="'$(BuildAOTTestsOn)' == 'local'">
+    <!--
+      When building for BuildAOTTestsOnHelix=true, the WasmApp*targets are *not* imported, because
+      they get instead used by the AOT proxy project on helix.
+
+      On the build machine only the regular part of the build is run, which includes trimming. But if
+      WasmApp*targets modify any trimming arguments, then those will not get picked up by this build.
+      For example - linker substitution files used with simd builds.
+
+      So, set those parameters explicitly here.
+      -->
+    <_ExtraTrimmerArgs Condition="'$(WasmEnableSIMD)' == 'true'">$(_ExtraTrimmerArgs) --substitutions &quot;$(MonoProjectRoot)\wasm\build\ILLink.Substitutions.WasmIntrinsics.xml&quot;</_ExtraTrimmerArgs>
+    <_ExtraTrimmerArgs Condition="'$(WasmEnableSIMD)' != 'true'">$(_ExtraTrimmerArgs) --substitutions &quot;$(MonoProjectRoot)\wasm\build\ILLink.Substitutions.NoWasmIntrinsics.xml&quot;</_ExtraTrimmerArgs>
+  </PropertyGroup>
+
   <ItemGroup>
     <_AOT_InternalForceInterpretAssemblies Include="@(HighAotMemoryUsageAssembly)" />
   </ItemGroup>

src/libraries/System.Numerics.Vectors/tests/GenericVectorTests.cs

@@ -27,6 +27,7 @@ static GenericVectorTests()
         }
 
         [Fact]
+        [ActiveIssue("https://github.com/dotnet/runtime/issues/81785", TestPlatforms.Browser)]
         public unsafe void IsHardwareAcceleratedTest()
         {
             MethodInfo methodInfo = typeof(Vector).GetMethod("get_IsHardwareAccelerated");

src/libraries/System.Runtime.Intrinsics/tests/Vectors/Vector128Tests.cs

@@ -1,6 +1,7 @@
 // Licensed to the .NET Foundation under one or more agreements.
 // The .NET Foundation licenses this file to you under the MIT license.
 
+using System.Diagnostics.CodeAnalysis;
 using System.Numerics;
 using System.Reflection;
 using System.Runtime.InteropServices;
@@ -11,6 +12,8 @@ namespace System.Runtime.Intrinsics.Tests.Vectors
     public sealed class Vector128Tests
     {
         [Fact]
+        [DynamicDependency(DynamicallyAccessedMemberTypes.PublicProperties, typeof(Vector128))]
+        [ActiveIssue("https://github.com/dotnet/runtime/issues/81785", TestPlatforms.Browser)]
         public unsafe void Vector128IsHardwareAcceleratedTest()
         {
             MethodInfo methodInfo = typeof(Vector128).GetMethod("get_IsHardwareAccelerated");

src/libraries/sendtohelix-wasm.targets

@@ -46,7 +46,7 @@
 
     <NeedsWorkload Condition="'$(Scenario)' == 'BuildWasmApps'">true</NeedsWorkload>
     <NeedsEMSDK Condition="'$(NeedsToBuildWasmAppsOnHelix)' == 'true'">true</NeedsEMSDK>
-    <NeedsEMSDKNode Condition="'$(Scenario)' == 'WasmTestOnNodeJS' or '$(Scenario)' == 'BuildWasmApps'">true</NeedsEMSDKNode>
+    <NeedsEMSDKNode Condition="'$(Scenario)' == 'WasmTestOnNodeJS' or '$(Scenario)' == 'BuildWasmApps'">false</NeedsEMSDKNode>
     <NeedsToRunOnBrowser Condition="'$(Scenario)' == 'WasmTestOnBrowser' or '$(Scenario)' == 'BuildWasmApps'">true</NeedsToRunOnBrowser>
     <NeedsBuiltNugets Condition="'$(Scenario)' == 'BuildWasmApps'">true</NeedsBuiltNugets>
 

src/mono/mono/mini/simd-intrinsics.c

@@ -1954,7 +1954,8 @@ emit_vector_2_3_4 (MonoCompile *cfg, MonoMethod *cmethod, MonoMethodSignature *f
 
 	id = lookup_intrins (vector2_methods, sizeof (vector2_methods), cmethod);
 	if (id == -1) {
-		check_no_intrinsic_cattr (cmethod);
+		// https://github.com/dotnet/runtime/issues/81961
+		// check_no_intrinsic_cattr (cmethod);
 		return NULL;
 	}
 

src/mono/wasm/Wasm.Build.Tests/HostRunner/NodeJSHostRunner.cs

@@ -8,8 +8,8 @@ namespace Wasm.Build.Tests;
 public class NodeJSHostRunner : IHostRunner
 {
     public string GetTestCommand() => "wasm test";
-    public string GetXharnessArgsWindowsOS(XHarnessArgsOptions options) => $"--js-file={options.jsRelativePath} --engine=NodeJS -v trace";
-    public string GetXharnessArgsOtherOS(XHarnessArgsOptions options) => $"--js-file={options.jsRelativePath} --engine=NodeJS -v trace --locale={options.environmentLocale}";
+    public string GetXharnessArgsWindowsOS(XHarnessArgsOptions options) => $"--js-file={options.jsRelativePath} --engine=NodeJS -v trace --engine-arg=--experimental-wasm-simd";
+    public string GetXharnessArgsOtherOS(XHarnessArgsOptions options) => $"--js-file={options.jsRelativePath} --engine=NodeJS -v trace --locale={options.environmentLocale} --engine-arg=--experimental-wasm-simd";
     public bool UseWasmConsoleOutput() => true;
     public bool CanRunWBT() => true;
 }

src/mono/wasm/Wasm.Build.Tests/HostRunner/V8HostRunner.cs

@@ -9,7 +9,7 @@ namespace Wasm.Build.Tests;
 
 public class V8HostRunner : IHostRunner
 {
-    private string GetXharnessArgs(string jsRelativePath) => $"--js-file={jsRelativePath} --engine=V8 -v trace";
+    private string GetXharnessArgs(string jsRelativePath) => $"--js-file={jsRelativePath} --engine=V8 -v trace --engine-arg=--experimental-wasm-simd";
 
     public string GetTestCommand() => "wasm test";
     public string GetXharnessArgsWindowsOS(XHarnessArgsOptions options) => GetXharnessArgs(options.jsRelativePath);

src/mono/wasm/Wasm.Build.Tests/WasmSIMDTests.cs

@@ -18,7 +18,7 @@ public WasmSIMDTests(ITestOutputHelper output, SharedBuildPerTestClassFixture bu
 
         [Theory]
         [MemberData(nameof(MainMethodTestData), parameters: new object[] { /*aot*/ false, RunHost.All })]
-        public void BuildWithSIMD_NoAOT_ShouldRelink(BuildArgs buildArgs, RunHost host, string id)
+        public void BuildWithSIMD_NoAOT_ShouldNotRelink(BuildArgs buildArgs, RunHost host, string id)
         {
             string projectName = $"sim_with_workload_no_aot";
             buildArgs = buildArgs with { ProjectName = projectName };
@@ -29,7 +29,7 @@ public void BuildWithSIMD_NoAOT_ShouldRelink(BuildArgs buildArgs, RunHost host,
                                     new BuildProjectOptions(
                                         InitProject: () => File.WriteAllText(Path.Combine(_projectDir!, "Program.cs"), s_simdProgramText),
                                         Publish: false,
-                                        DotnetWasmFromRuntimePack: false));
+                                        DotnetWasmFromRuntimePack: true));
 
             if (!_buildContext.TryGetBuildFor(buildArgs, out _))
             {

src/mono/wasm/build/WasmApp.Native.targets

@@ -113,8 +113,6 @@
       <!-- build AOT, only if explicitly requested -->
       <WasmBuildNative Condition="'$(RunAOTCompilation)' == 'true' and '$(RunAOTCompilationAfterBuild)' == 'true'">true</WasmBuildNative>
 
-      <WasmBuildNative Condition="'$(WasmBuildNative)' == '' and '$(WasmEnableSIMD)' == 'true'">true</WasmBuildNative>
-
       <WasmBuildNative Condition="'$(WasmBuildNative)' == '' and @(NativeFileReference->Count()) > 0" >true</WasmBuildNative>
 
       <WasmBuildNative Condition="'$(WasmBuildNative)' == ''">false</WasmBuildNative>
@@ -125,7 +123,6 @@
       <!-- AOT==true overrides WasmBuildNative -->
       <WasmBuildNative Condition="'$(RunAOTCompilation)' == 'true'">true</WasmBuildNative>
       <WasmBuildNative Condition="'$(WasmBuildNative)' == '' and @(NativeFileReference->Count()) > 0" >true</WasmBuildNative>
-      <WasmBuildNative Condition="'$(WasmBuildNative)' == '' and '$(WasmEnableSIMD)' == 'true'">true</WasmBuildNative>
 
       <!-- not aot, not trimmed app, no reason to relink -->
       <WasmBuildNative Condition="'$(WasmBuildNative)' == '' and '$(PublishTrimmed)' != 'true'">false</WasmBuildNative>
@@ -203,7 +200,7 @@
       <_EmccCommonFlags Include="-v"                                Condition="'$(EmccVerbose)' != 'false'" />
       <_EmccCommonFlags Include="-s DISABLE_EXCEPTION_CATCHING=0"   Condition="'$(WasmEnableExceptionHandling)' == 'false'" />
       <_EmccCommonFlags Include="-fwasm-exceptions"                 Condition="'$(WasmEnableExceptionHandling)' == 'true'" />
-      <_EmccCommonFlags Include="-msimd128"                         Condition="'$(WasmEnableSIMD)' == 'true'" />
+      <_EmccCommonFlags Include="-msimd128"                         Condition="'$(WasmEnableSIMD)' == 'true' and '$(_WasmShouldAOT)' == 'true'" />
 
       <_EmccIncludePaths Include="$(_WasmIntermediateOutputPath.TrimEnd('\/'))" />
       <_EmccIncludePaths Include="$(_WasmRuntimePackIncludeDir)mono-2.0" />

src/mono/wasm/build/WasmApp.targets

@@ -93,7 +93,7 @@
   <PropertyGroup>
     <WasmDedup Condition="'$(WasmDedup)' == ''">true</WasmDedup>
     <WasmEnableExceptionHandling Condition="'$(WasmEnableExceptionHandling)' == ''">false</WasmEnableExceptionHandling>
-    <WasmEnableSIMD Condition="'$(WasmEnableSIMD)' == ''">false</WasmEnableSIMD>
+    <WasmEnableSIMD Condition="'$(WasmEnableSIMD)' == ''">true</WasmEnableSIMD>
 
     <!--<WasmStripAOTAssemblies Condition="'$(AOTMode)' == 'LLVMOnlyInterp'">false</WasmStripAOTAssemblies>-->
     <!--<WasmStripAOTAssemblies Condition="'$(WasmStripAOTAssemblies)' == ''">$(RunAOTCompilation)</WasmStripAOTAssemblies>-->

src/mono/wasm/templates/templates/console/runtimeconfig.template.json

@@ -4,12 +4,10 @@
             {
                 "name": "node",
                 "js-path": "main.mjs",
-                "Host": "nodejs"
-            },
-            {
-                "name": "v8",
-                "js-path": "main.mjs",
-                "Host": "v8"
+                "Host": "nodejs",
+                "host-args": [
+                    "--experimental-wasm-simd"
+                ]
             }
         ]
     }