tree-shake bundle step

[jasonkuhrt]

Use tree shaking and aggressive optimizations to create Nexus apps that are as small as possible. This means NOT shipping Nexus server side apps with the traditional source+node_modules but an optimized combination of these. Reasons this is desirable:

• smaller docker images
• constrained serverless environments
• faster serverless cold boot times
• ...

Baselines

Here are the current baselines for the hello-world and plugin-prisma examples

TODOs

• Try webpack

• Try Rollup

• Get Rollup producing a functioning build
  see here: https://github.com/graphql-nexus/examples/tree/spike/tree-shaking/hello-world

• Rollup Questions

  • Find way to bundle lodash in friendly way. Currently we have to used namedExoprts feature of rollup. Imagine user brought the dep. Forgot tree-shaking for a moment, we just want their app to be bundle-able. And its not right now.

  • When node module, like prisma plugin is marked as external dep, how does the user go to production with the resulting bundle? If the external dep is a Node builtin, its fine, since presumably wherever they deploy to will have Node and thus the builtin, but what about third party node modules? Example where we just install and use the prisma plugin (which doesn't have esm builds yet):

     'nexus-plugin-prisma' is imported by node_modules/.build/app.js, but could not be resolved – treating it as an external dependency
    

• prisma plugin

• nexus build --bundle to integration
  - graceful bail?
  - where to put bundle?
  - keep ts build around too?
  - or, no ts emit, instead rollup ts plugin approach?

• tool for Nexus plugin authors to build their plugin with esm build out of the box --> Create Nexus tooling graphql-nexus/nexus#862

• nexus start given that we now have a bundle step, making output less predictable? no

• resolve circ dep warnings coming from Nexus (run examples branch bundle script to see) --> Remove circular deps from codebase #853

• resolve unused dep warnings (that give us no context about where...) (SOQ) Allow promises to be returned in a field in the resolver of connection graphql-nexus/nexus#853

• improve how Nexus uses eval https://rollupjs.org/guide/en/#avoiding-eval (may not matter if all the locations are tree-shaked anyways, however, its a nice touch anyways) --> Improve eval usage #856

[jasonkuhrt]

CC @timsuchanek

[jasonkuhrt]

The tutorial Heroku app with Prisma came to 80mb in size today.

[chrisdrackett]

our Heroku app is 314MB 😳

[jasonkuhrt]

@chrisdrackett You are using the Prisma plugin, correct? Does that number include the node_modules?

[chrisdrackett]

not sure, its just what Heroku tells me:

remote: -----> Discovering process types
remote: Procfile declares types -> web
remote:
remote: -----> Compressing...
remote: Done: 314.2M
remote: -----> Launching...
remote: ! Warning: Your slug size (314 MB) exceeds our soft limit (300 MB) which may affect boot time.

not super worried about it yet, still trying to get it to even work :D

[jasonkuhrt]

@chrisdrackett ok, and with Prisma plugin right?

[chrisdrackett]

yep!

[Weakky]

Baselines

Here are the current baselines for the hello-world and plugin-prisma examples

[jasonkuhrt]

Rollup tree-shaking on @nexus/schema after/before esm builds (87% savings!):

[jasonkuhrt]

@Weakky and I both got nexus apps to be bundled and work after by rollup tonight.

At 10mb, it represents a 9.7x/970% decrease!

[jasonkuhrt]

Actually we'll do a lot better soon. Once we figure out why TypeScript isn't getting shaken out, and fix that, we'll be around 2mb, bringing the size savings to around a whopping 48x.

[jasonkuhrt]

We were able to tree-shake TypeScript, bundle at 2.4mb now, size savings around 4000% now!

[jasonkuhrt]

Update

• The commonjs rollup plugin proved too limited for a general bundler solution for Nexus for now. Something as simple as adding lodash will cause rollup to fail

  We are still interested in rollup in the long-term, it feels like a great tool, but so far it couldn't get it to work in such a way that wouldn't require real world projects, in most cases (almost all?), to augment the rollup configuration with e.g. custom configuration for package ignores, namedExport, and so on.

• webpack-asset-relocator and by extension ncc fail to work with the Nexus plugin system.

  The issue is that they treat require.resolve paths as demarcating an opaque asset which will no longer be traversed as a module. This means that nexus plugins would have their worktime.ts/testtime.ts/runtime.ts modules treated as assets, because they are pointed to with static strings usually using require.resolve. Finally, the consequence is that all imports within these three modules will fail because they will not be in the bundle. And this behaviour is not configurable.

• @Weakky had great success with node-file-trace.

  While it doesn't tree-shake as well as rollup it provides good "module-shaking" which for our case is still extremely helpful. We get prisma hello world apps to 20mb (from hundreds of mb).

  Its behaviour is similar to webpack-asset-relocator in the sense of its traversing robustness, however it doesn't stop at what it considers "assets". Instead every path is followed and traversed if it can be (maybe via checking path .ext or something). Therefore, whereas ncc failed with Nexus' plugin system, node-file-trace works!

  There were some wrinkles that came out of this that we designed through:

    - In order to make node-file-trace "module-shake" worktime/testtime modules we will use the [ignore option of node-file-trace](https://github.com/zeit/node-file-trace#ignore). The Way that we populate this is from data from the step in which we read the used plugins manifests.
    - BUT... A side-effect of this solution is that files targeted by `require.resolve` would no longer exist in the bundle. At runtime, when a plugin is used `use(prisma())`, `require.resolve` would throw a `MODULE_NOT_FOUND` error. To solve this, we decided to drop `ignore` field in favour of `readFile` field, and then, stub out the modules with empty strings. This having the benefit of effectively ignoring but also solving this side-effect.
  
    - Once we bring in plugin manifest optionality node-file-trace would break. To fix it we'll use the `readFile` hook again, but this time to intercept processing of plugin entrypoints (where the manifest is) itself. Here, we will inline `require`s that should make node-file-trace continue the traversal. This is TBD.
  

• We realized that even if we aren't going with a formal bundler right now (node-file-trace is not that right?) we'll probably want to in the future. It'll improve our internal architecture to have a spine for running transformations and tree shaking and etc. over the users code. It will also give users a way to easily plug into the system that is not locked into Nexus system. E.g. They could use @rollup/plugin-alias to support TypeScript path alias feature. Or in that case we might actually bundle that by default!

• We become aware of some others tools that we might want to review in the future via @tim

  • https://github.com/evanw/esbuild/
  • https://swc-project.github.io/