[gcb] gulp-core-build-sass sometimes invokes node-gyp

[poke]

I’m currently trying to use the gulp-core-build-sass package as part of the default SharePoint SPFx Yeoman template. Unfortunately, the dependencies of this particular package make this very painful.

In my particular situation, I am having troubles installing node-sass 4.9.3, which is failing at the native compilation step with node-gyp for me, probably because I am running Visual Studio 2019 and that is not perfectly supported yet with Node and npm.

However, I am able to install node-sass 4.12.0 which was released two months ago. But because of the way the dependencies are specified, I cannot use that. Instead, gulp-core-build-sass will always attempt to install node-sass 4.9.3 explicitly, preventing me from installing this package and breaking the SPFx experience for me completely.

Is there an actual reason why the dependencies are fixed at the patch-level? Doing this completely ignores SemVer compatibility and prevents any future fixes from being applied automatically. It’s likely also a source of the 2k vulnerabilities npm currently reports when you install the SharePoint template.

[iclanton]

Is there an actual reason why the dependencies are fixed at the patch-level?

In the past we've had issues with packages breaking with patch updates, so to ensure the experience is consistent long after SPFx has been released, we locked down dependency versions.

This is the first time I've heard of this issue happening. Which version of Node are you running and which version of SPFx are you installing? In my experience, node-sass uses prebuilt bindings instead of trying to build them itself. A long time ago, we eliminated the need for extra compiler packages from SPFx, so it shouldn't need to run node-gyp at all.

[octogonz]

Some simple repro steps would help this get resolved faster.

[poke]

I’m using Node 12.6.0 with npm 6.10.0, Yeoman 3.1.0, and @microsoft/generator-sharepoint 1.8.2. I execute this on a Windows 8.1 with Visual Studio 2019 Enterprise:

PS> yo @microsoft/sharepoint

     _-----_
    |       |    .--------------------------.
    |--(o)--|    |      Welcome to the      |
   `---------´   |  SharePoint Client-side  |
    ( _´U`_ )    |    Solution Generator    |
    /___A___\    '--------------------------'
     |  ~  |
   __'.___.'__
 ´   `  |° ´ Y `

Let's create a new SharePoint solution.
? What is your solution name? Example
? Which baseline packages do you want to target for your component(s)? SharePoint Online only (latest)
? Where do you want to place the files? Use the current folder
Found npm version 6.10.0
? Do you want to allow the tenant admin the choice of being able to deploy the solution to all sites immediately without
  running any feature deployment or adding apps in sites? Yes
? Will the components in the solution require permissions to access web APIs that are unique and not shared with other
  components in the tenant? No
? Which type of client-side component to create? Extension
? Which type of client-side extension to create? Application Customizer
Add new Application Customizer to solution example.
? What is your Application Customizer name? HelloWorld
? What is your Application Customizer description? HelloWorld description

At some point, the npm install then outputs this:

> [email protected] postinstall C:\Users\poke\Desktop\spfx-example\node_modules\node-sass
> node scripts/build.js

Building: C:\Program Files\nodejs\node.exe C:\Users\poke\Desktop\spfx-example\\node_modules\node-gyp\bin\node-gyp.js rebuild --verbose --libsass_ext= --libsass_cflags= --libsass_ldflags= --libsass_library=

Then at some point, it tries to do this:

gyp verb using MSBuild: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\15.0\Bin\MSBuild.exe
gyp info spawn C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\15.0\Bin\MSBuild.exe

This fails because that is not the correct location for MSBuild in VS 2019. That appears to be fixed in a later release of node-gyp according to this issue but that isn’t bundled with npm yet. I tried the workaround there but couldn’t get it to work, though I wouldn’t be surprised if this is because of some left-over stuff on my computer.

In the end, I can reproduce this easily on my machine by doing npm install [email protected]. That one fails because it tries to compile the underlying libsass. Installing the current 4.12.0 works fine for me without having to compile anything.

Yes, I am aware that according to the node-sass project readme, I need to use node-sass 4.12 in order to run it on Node 12, so that is likely the source of my problems above (although I do get a compile step in earlier node-sass versions when running on Node 10). However, my problem is not with node-sass at this point, my problem is rather that I cannot use a newer version of that dependency with the gulp-core-build-sass package but it instead forces me to use an older incompatible version.

[octogonz]

my problem is not with node-sass at this point, my problem is rather that I cannot use a newer version of that dependency with the gulp-core-build-sass package but it instead forces me to use an older incompatible version.

The node-sass version is intentionally locked to a specific number in order to avoid regressions. After we test a particular installation plan, that testing would be invalidated if the customer later installs a newer version. It's too expensive to offer support for version combinations that were never even tested. We unfortunately can't lock /indirect/ dependencies (it's impractical due to how NPM shrinkwrap files work), but locking these direct dependencies did improve things noticeably.

Returning to your issue, the real problem isn't that node-gyp can't find MSBuild. The problem is that node-gyp is being invoked at all. The whole concept of node-gyp was to empower package maintainers to distribute prebuilt binaries: they do the hard work, so life is easy for their consumers. When package maintainers instead invoke node-gyp inside an npm install hook, it's lazy and creates a poor experience for consumers, since now each person needs the right versions of C++ and Python on their machine, which is tricky as you have seen. Also, now every single npm install operation has to wait for an expensive compiler operation, which is bad for the ecosystem.

When this problem first came up, we did work to banish the node-gyp dependencies from our toolchain. If it has somehow gotten reintroduced, we should try to fix that.

As a workaround, on Windows you can install the windows-build-tools package, which is a shortcut for getting a working Python/C++ setup for node-gyp. (I haven't needed that for years, though. We really need to figure out what happened with node-sass.)

@patmill @VesaJuvonen FYI

[octogonz]

I just ran npm install @microsoft/gulp-core-build-sass, and it does NOT invoke node-gyp:

> [email protected] install D:\qq\node_modules\node-sass
> node scripts/install.js

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.9.3/win32-x64-64_binding.node
Download complete .] - :
Binary saved to D:\qq\node_modules\node-sass\vendor\win32-x64-64\binding.node
Caching binary to C:\Users\Owner\AppData\Roaming\npm-cache\node-sass\4.9.3\win32-x64-64_binding.node

> [email protected] postinstall D:\qq\node_modules\core-js
> node scripts/postinstall || echo "ignore"

I'm using Windows 10 Pro with npm -v = 6.4.1 and node -v = v10.13.0.

So this seems to be configuration specific.

[octogonz]

I need to use node-sass 4.12 in order to run it on Node 12

Hrmmm... Node 12 is not LTS. Unstable NodeJS versions are notoriously buggy. We don't make any attempt to offer support for them.

[elliot-nelson]

Closing - please open a new issue if you experience issues on stable node12 with gulp-core-build-sass.