Run internet tests with the CI

[BridgeAR]

Due to a comment from @cjihrig nodejs/node#21559 (comment) it came to my attention that we have quite some tests that are not run with the regular CI, since they are prone to network issues. I would like to test how bad that is by running these tests with the regular CI but by just flagging them as flaky as long as they are not stable. Is that an option?

[refack]

This was brought up recently and the internet suite was added to run as part of the daily master test: https://ci.nodejs.org/job/node-daily-master/
If that is not enough, we could consider adding them as a special job, similar to the worker suite?

[joaocgreis]

Ref: nodejs/node#13061
Last daily run: https://ci.nodejs.org/job/node-test-commit-custom-suites/656/

I believe the reason for not running them in regular CI was to not overload 3rd party servers with requests (I can't find a reference for this, probably a comment in some unrelated thread).

[refack]

I believe the reason for not running them in regular CI was to not overload 3rd party servers with requests (I can't find a reference for this, probably a comment in some unrelated thread).

I think it's also that some of our worker are behind firewalls.

[BridgeAR]

The dailies seem to be green runs. That is at least a indication that they are not as flaky as first anticipated.

I also just checked the servers that the internet tests contact and I would not bother about our requests against those.

The list currently contains these entries:

• 8.8.8.8 // Google Public DNS IPv4 address
• 2001:4860:4860::8888 // Google Public DNS IPv6
• something.invalid // invalid host that cannot be resolved
• nodejs.org
• blog.nodejs.org
• _jabber._tcp.google.com // A host with SRV records registered
• 8.8.8.8.in-addr.arpa // A host with PTR records registered
• sip2sip.info // A host with NAPTR records registered

So I would be in favor of running these tests with the regular CI. That way their coverage should also be included in the overall coverage report, right?

[refack]

The dailies seem to be green runs. That is at least a indication that they are not as flaky as first anticipated.

The daily internet job runs on a single platform, on worker that are known to be reliable and not behind a firewall. We could consider adding internet to the coverage job, as it also runs on a known reliable server (https://ci.nodejs.org/job/node-test-commit-linux-coverage/nodes=benchmark/)

[BridgeAR]

We could consider adding internet to the coverage job, as it also runs on a known reliable server

That would probably already be enough in this case :-).

[mhdawson]

As context, a number of tests were moved to internet specifically to avoid overload/loading external sites as mentioned.

[refack]

Resolved in CI:
https://ci.nodejs.org/job/node-test-commit-linux-coverage/jobConfigHistory/showDiffFiles?timestamp1=2018-09-08_19-05-38&timestamp2=2018-09-09_23-00-55