Configure Bot to inform PR owner about failed CI

[Flarna]

Is your feature request related to a problem? Please describe.
I happened twice to me that I submitted a PR, automatic CI failed but I forgot to take a look resulting in PR lying around longer then needed.

Describe the solution you'd like
Bot should inform PR owner about failed CI.

[Trott]

The upside to this is clear. The downside is that it makes our notifications even noisier and a lot of us already find the notifications for the repository to be overwhelming. (Not saying we can't or won't do it. Just trying to identify all relevant considerations.)

@nodejs/github-bot @nodejs/automation

[JasonEtco]

I built https://github.com/JasonEtco/ci-reporter for this kind of use case. I think that if managed well, it prevents an overwhelming amount of notifications - however, I don't think that ci-reporter manages notifications well, so this isn't an advertisement so much as a cautionary tale.

This is a thing you can do, if it shares more information than just "CI failed!" then cool, but information overload is a thing. My $0.02!

[joyeecheung]

To implement this using existing code I believe one just needs to send a PR to https://github.com/nodejs/github-bot that takes advantage of https://github.com/nodejs/node-core-utils/blob/master/docs/ncu-ci.md (or just the Jenkins API - ncu-ci just has a bunch of pattern-matching code on top of the Jenkins API to guess the reasons non-js/cctest failures that are very specific to our CI). See nodejs/reliability#17 on an example output of ncu-ci that aggregated the results of multiple CI runs and formatted them as markdown.

But I tend to worry about the downside as mentioned by @Trott ..

[joyeecheung]

Another factor is that our CI is not always green and there tend to be irrelevant failures (flakes). This is the result produced by ncu-ci rate pr just now:

UTC Time	RUNNING	SUCCESS	UNSTABLE	ABORTED	FAILURE	Green Rate
2018-11-19 10:32	0	48	8	5	39	50.53%

[Flarna]

Just to be clear, I talk about the "small" CI triggered on commit/PR creation, not the full CI triggered manually.

[refack]

Just to be clear, I talk about the "small" CI triggered on commit/PR creation, not the full CI triggered manually.

For the Travis job you can configure Travis to notify you.
For the "lite" job IMHO the optimal semantics would be to post a message in the thread (possibly with details of what failed).
For the manual job there is a plan (backlogged) for opt-in email notification - nodejs/build#1502 & nodejs/build#746

[Flarna]

Thanks!