[WIP] Add pipeline script for node-inspect

[jkrems]

Highly WIP but I guess it's better to put it here before I lose my local docker container...

• Figure out if there's a maintainable way to do the multi-arch builds
• Support build against nightlies (quick hack: just support ORG_MIRROR via param?)
• Support setting the commit/PR status (optional)

See:

• jenkins: add http-parser.jenkinsfile #922
• Migrate Jenkins job configuration to pipeline files #838

[jkrems]

I'm really tempted to walk back on the declarative syntax here given the amount of repetition once all possible architectures are added. Unless we'd move that logic to an inline script {} section and not model it as parallel stages. Would have to see how that would be rendered in the UI.

[refack]

This is what came out https://ci.nodejs.org/job/node-inspect/2/flowGraphTable/
(first steps are checking out the pipeline script itself from @jkrems's fork)

[jkrems]

Can someone give nodejs/node-inspect#53 a quick look to get the latest tap into master?

[jkrems]

For reference, here's how an example run renders in blue ocean when I was testing this script locally:

That view makes the job output a lot easier to consume. Unfortunately the out-of-the-box experience of looking at parallel stages in Jenkins is pretty bad (by which I mean pretty much unusable). :(

[refack]

That view makes the job output a lot easier to consume. Unfortunately the out-of-the-box experience of looking at parallel stages in Jenkins is pretty bad (by which I mean pretty much unusable). :(

It's not that bad, it's just not trivial to find:

• each step is avalible from the flowGraphTable
  for example the first failing step:
  https://ci.nodejs.org/job/node-inspect/2/execution/node/43/log/
• An overview of the history of the job can be found on the "Full Stage View"
  
  --
  

[jkrems]

Yes, I know. But with non-trivial workflows (e.g. parallel + anything happening afterwards) the flattened stage view becomes super confusing [especially if any stage name isn't globally unique]. This isn't a problem yet here (since it has just two fake "architectures" right now and nothing happening afterwards) but it won't get any better the more complex the workflow gets. Though maybe it'll be fine for node's builds. I might be scared by our internal deployment pipeline builds (with multiple validation stages consisting of parallelized things).

[refack]

@jkrems the log visibility issue was brought up by @joaocgreis and AFAIK is the last hindrance for full adoption.
But as I see it we can continue to use MultiJob. With the current setup the fan-out is the last step of each sub-job and any other steps after that are in a new sub-job. So the flattened log is not that bad, just requires learning where to look for the failures.

On the other hand the step tree:

Allows for a not log-reading-dependant way to quickly figure out what failed.

---

tl;dr IMHO we should move away from log-based-groking to using parsing plugins (like tap or tap2junit), while also look-for/develop a better log parser.

[sam-github]

Stale pipeline PR related to the pipeline experiment.