Lazy app initialization code

[jasonkuhrt]

Current Workaround

Please see this graphql-nexus/nexus#732 (comment)

Perceived Problem

• "nexus build" broken when app.ts contains running async code #732
• Nexus Reflection system means users should not have module-level side-effects
• For a refresher on what reflection is see https://www.nexusjs.org/#/architecture
• Because we are API over configuration/convention we have to run reflection over the whole app, rather than being able to only run reflection over the e.g. schema modules.

Ideas / Proposed Solution(s)

Raw sketches (with @Weakky)

import { log, settings, state, use } from 'nexus';
import { prisma } from 'nexus-plugin-prisma';

lifecycle.on('dev|build|prod', () => {})
lifecycle.is('dev|build|prod')
env

interface State {

}

const app = App.create({
  isDev: process.env.NEXUS_STAGE === 'dev',
  isReflectionTime: process.env.NEXUS_IS_CONFIGURATION_TIME
})

if (!state.isReflectionTime) {
  // heavy db setup
}

if (state.isDev && !state.isReflectionTime) {
  // add schema middleware for dev only
}

// process.env.NODE_ENV === 'production'

// env.stage === 'reflection'

(env.stage === 'build' || env.stage === 'dev') !== state.reflectionTime
(env.stage === 'build') === state.reflectionTime




// concepts
// deployment stage: development / prodution (/ qa / testing / ...)
// execution phase:  run / reflection
// if (state.executionPhase === 'run|reflection') {}
// if (state.deploymentStage === 'development|production|qa|testing') {}

// isReflectionTime -> isRun
// run | reflection
// run

// if (state.stage === 'development|production|qa|testing|reflection') {

// }

if (state.lifecycle === 'start')

let state = {
  prismaClient: PrismaClient
}

// making the logic lazy
on.start(() => {
  prismaClient = new prismaClient()
})

// making the logic lazy
on.start(() => {
  state.prismaClient = new prismaClient()
})

export { state };

// nexus.config.ts -> plugins and settings
// schema.ts

// schema.ts -> only importing the schema types -> ts-node file

state.prismaClient

in('dev', () => {
  on('start', (e) => {


  })
})

// deployment stage   =>    development | qa | production | testing
//                                        _________________________  production / not development
// build stage        =>    development | production


const __dev__ = NODE_ENV.env === 'development'

if (__dev__) {
  
}

// why not just import { stage } from 'nexus'

on('start', (event, stage) => {
  if (stage === 'development') {

  }
})

on.boot(() => {

})

on('start').in('development').run(() => {

})

if (state.stage === '')



lifecycle.start = (() => {

})

if (state.executionPhase === 'run') {
  require('./on.start.ts')
}

on('run', () => {

})

// on('start', (runtime) => {
//   const heavy
//   runtime.use.prisma() // runtime
// })



use(prisma({
  client: {
    instance: prismaClient
  }
))

settings.change({
  logger: {
    level: 'trace',
  },
  schema: {
    connections: {
      foobar: {},
      toto: {},
    },
  },
  server: {
    startMessage: info => {
      settings.original.server.startMessage(info)
      log.warn('piggy back message!')
    },
  },
})



const schema = schema.make()

await server.start({ schema })

• We identified that the execution phase is not the same thing as the deployment stage (Deployment stage conditional code #535)

• The deployment stage is a union of e.g. dev|prod|qa|test

• The execution phase is a union of run|reflect

• We think most use-cases will be for

   if (phase === 'run') { ... }

  not:

   if (phase === 'reflect') { ... }

  For one thing reflection isn't meant to be user-facing concept, but an internal engine one that just impacts the user with constraints.

  For another, we just don't see use-cases for it right now.

• We think this code would be far from ideal:

   import { stage } from 'nexus'
   if (state.phase === 'run' && state.stage === 'prod') { ... }

• We think an event concept would be easiest to understand

   import { on } from 'nexus'
   
   on.boot(() => {})

• Lifecycle wise we need an event that is:

  • before schema is created
  • before server starts

• We think that event can be called boot.

References

• precedent for this kind of lazy approach can be found here https://fastapi.tiangolo.com/advanced/settings/#settings-in-a-dependency
• Dependency Injection seems related too (but its motivations are different than ours)

Instances

• "nexus build" broken when app.ts contains running async code #732
• New issue with makeSchema (getTypeMap of undefined) graphql-nexus/nexus#523 (comment)
• feat: Better graphql-js interop, add mergeSchema to consume external schemas graphql-nexus/nexus#983
• https://github.com/graphql-nexus/nexus/issues/1201

[jasonkuhrt]

Another use-case came up recently: graphql-nexus/nexus#523 (comment)

The reason we can consider it a use-case is that we can tap into the lifecycle to solve it, for example:

on.before.start((data) => {
  server.express.get('*', () => {
    // ...
  })
})

[jasonkuhrt]

New idea for an API.

Also, we now have better definition of the runtime hooks.

• .<getters> that will check they are called in the right order
• .<getters> that will short-circuit (via throw) if in reflection, e.g. [1] [2]
• hierarchical, await parent or lower levels
• two styles, tailored to taste/need.

  • The on.<before|after>.<hook> approach can feel a bit more focused and simple, no forced async, no forced indirection to get hooks access, etc.

  • The single-function-body of awaitable hooks makes it very nice to share state between hooks, where otherwise module-level let bindings would have had to be used 🤢 .

     let foo
     
     on.before.a(({ something }) => { foo = lala(something) /* or whatever */ })
     on.after.g(() => { foo() /* or whatever */ })

    Compare to the following:

     on(({ before, after }) => {
       const { something } = await before.a
       const foo = lala(something) // or whatever
       await after.g
       foo() // or whatever
     })

    This is also much better for type safety, basically just works with zero effort.

// style 1

on(async ({ before, after }) => {
  let data

  data = await before.assembly
  data = await before.assembly.loadPlugins
  data = await before.assembly.makeSchema
  data = await before.assembly.makeServer 
  data = await before.assembly.checks
  data = await before.start
  data = await before.start.serverListening

  data = await after.assembly
  data = await after.assembly.loadPlugins // [1] reflect on used plugins short circuit here
  data = await after.assembly.makeSchema // [2] reflect on schema short circuit here
  data = await after.assembly.makeServer 
  data = await after.assembly.checks
  data = await after.start
  data = await after.start.serverListening
})

// style 2

on.before.start((data) => {})
...

// Example style 1 v 2

on(async ({ before }) => {
  const data = await before.start

  server.express.get('*', () => {
    // ...
  })
})

on.before.start((data) => {
  server.express.get('*', () => {
    // ...
  })
})

Style 1 may pose too challenging to implement but I'm interested in trying. It will require some trickery but so far doesn't look impossible.

Also, quite interesting to think about DX of top-level await that we could explore in the future:

let data

data = await on.before.assembly
data = await on.before.assembly.loadPlugins
data = await on.before.assembly.makeSchema
data = await on.before.assembly.makeServer 
data = await on.before.assembly.checks
data = await on.before.start
data = await on.before.start.serverListening

data = await on.after.assembly
data = await on.after.assembly.loadPlugins // [1] reflect on used plugins short circuit here
data = await on.after.assembly.makeSchema // [2] reflect on schema short circuit here
data = await on.after.assembly.makeServer 
data = await on.after.assembly.checks
data = await on.after.start
data = await on.after.start.serverListening

[jasonkuhrt]

We could also entertain worktime hooks, but probably in a new issue, and later.

on.worktime.build(async ({ before, after }) => {
  // e.g.
  await before.reflection
  await after.typescript.checker.pass
  await before.typescript.emit
})

[jasonkuhrt]

Discussion with @Weakky the other day.

• With callbacks we don't need before/after because that precision is available in the callback itself naturally via await:

   on.before.start(() => {
   	foo()
   })
   
   on.after.start((data) => {
   	bar(data)
   })

   on.start((after) => {
   	foo()
   	const data = await after
   	bar(data)
   })

• Another way to enforce ordering is callbacks that receive data from the previous callbacks of previous events and returns data to be passed to callbacks of future events. Reflection would be used to model the types correctly.

   on.before.a((data, ctx) => {
   	foo = lala(data.something) // or whatever
   	return {
   		foo
   	}
   })
   
   on.after.g((data, ctx) => {
   	ctx.foo() // or whatever
   })

  vs previous options

   let foo
    
   on.before.a(({ something }) => {
   	foo = lala(something) // or whatever
   })
   
   on.after.g(() => {
   	foo() // or whatever
   })

    on(({ before, after }) => {
      const { something } = await before.a
      const foo = lala(something) // or whatever
      await after.g
      foo() // or whatever
    })

[jasonkuhrt]

Another instance graphql-nexus/nexus#983

[gustawdaniel]

Wow! Listening on events in booting process is grat feature.

What to you think about global event emitter with events:

before.assembly
after.assembly.checks

Accessible in:

app.emmiter

Node js Api:
https://nodejs.org/api/events.html

[jasonkuhrt]

@gustawdaniel

What to you think about global event emitter with events:

I think that API is more than we need and would degrade the simplicity of the Nexus API. Open to hearing specific pros about it though.

[jasonkuhrt]

Another instance #1201

[jasonkuhrt]

Users should probably be able to tap into reflection as well. Doing so will require removing the code at build time, so that bundling doesn't drag in work time dependencies. This would make it easy for users to, for example, generate a genql client. While in that case Nexus might integrate it (genql) the point is any tool.

Some of the events we have:

reflection
reflection.loadPlugins
reflection.generateArtifacts
reflection.generateArtifacts.generateTypegen
reflection.generateArtifacts.generateSDL

[jasonkuhrt]

Another idea. Path and module name convention. I’m not convinced we should do this. But I want to document it.

[<path>]/on/<eventName>.ts

Examples:

on/start.ts
on/reflectionDone.ts

What’s nice about this is:

• users write a module like normal, no new API to learn
• nexus doesn’t need to to strip reflection hooks from AST at build time

What’s not good about this:

• nexus doesn’t have many file system conventions. Surprising.
• cannot receive or return data so limited to side-effects only.
• slower to interact with. Creating the file is more manual than auto-importing “on” and accessing the API.
• importing this anywhere manually into the app would be a mistake. Nexus would catch this case and warn but still.
• API gives us a place to write jsdoc
• if user makes typo in folder name they get no feedback; if they make typo on API TS tells them right away

[jasonkuhrt]

I would like to unify a few things: runtime events, worktime events, stage "events" (?). Stage is about what "environment" the app is running in: dev, preview, prod, etc.

Example, rewriting use-case of https://github.com/graphql-nexus/nexus/issues/1201:

import { on, schema, settings } from 'nexus'
import Redis from 'redis';

on.run.before(() => {
  schema.addToContext({
    redis: Redis.createClient(settings.current.redis)
  })
})

Note, this requires allowing objects to be passed to addToContext. In my mind @Weakky this use-case, lazy initialization, puts that debate to rest. We need it.

Example, generate genql:

import { generate } from 'genql-cli'
import { on } from 'nexus'

on.reflect.after(async (project) => {    // <-- this codeblock stripped at build time <3
  await generate({
    schema: project.data.schema,
    output: project.path('client'),
    scalarTypes: {
      MongoID: 'string',
    },
  })
})

Access to booleans:

if (on.stage.now === 'production') {

}

if (on.reflect.now) {

}

if (on.run.now) {

}

[jasonkuhrt]

Addressing another use-case https://prisma.slack.com/archives/CM2LEN7JL/p1594329877332200

Example, access built graphql schema

import { createServer } from 'http';
import { SubscriptionServer } from 'subscriptions-transport-ws';
import { execute, subscribe } from 'graphql';
import { schema } from './my-schema';

on.run.graphql.after(({ schema }) => {
  const websocketServer = createServer((request, response) => {
    response.writeHead(404)
    response.end()
  })

  const subscriptionServer = SubscriptionServer.create(
    {
      schema,
      execute,
      subscribe,
    },
    {
      server: websocketServer,
      path: '/graphql',
    },
  )
  
  return new Promise(res => { websocketServer.listen(5000, res) }).then(() => {
    console.log(
      `Websocket Server is now running on http://localhost:${WS_PORT}`
    )
  })
})