compiletest: Re-land using the new non-libtest executor by default

[Zalathar]

This PR re-lands #139998, which had the misfortune of triggering download-rustc in its CI jobs, so we didn't get proper test metrics for comparison with the old implementation. So that was PR was reverted in #140233, with the intention of re-landing it alongside a dummy compiler change to inhibit download-rustc.

---

Original PR description for #139998:

The new executor was implemented in #139660, but required a manual opt-in. This PR activates the new executor by default, but leaves the old libtest-based executor in place (temporarily) to make reverting easier if something unexpectedly goes horribly wrong.

Currently the new executor can be explicitly disabled by passing the -N flag to compiletest (e.g. ./x test ui -- -N), but eventually that flag will be removed, alongside the removal of the libtest dependency. The flag is mostly there to make manual comparative testing easier if something does go wrong.

As before, there should be no user-visible difference between the old executor and the new executor.

---

r? jieyouxu

[rustbot]

Some changes occurred in src/tools/compiletest

cc @jieyouxu

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

[Zalathar]

@bors rollup=never

[jieyouxu]

Indeed, thanks

[jieyouxu]

@bors r+

[bors]

📌 Commit 1670de4 has been approved by jieyouxu

It is now in the queue for this repository.

[bors]

⌛ Testing commit 1670de4 with merge 43e62a7...

[bors]

☀️ Test successful - checks-actions
Approved by: jieyouxu
Pushing 43e62a7 to master...

[github-actions]

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 10fa3c4 (parent) -> 43e62a7 (this PR)

Test differences

No test diffs found

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 43e62a789c772642f79086f2cceef171cff30e63 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

1. dist-aarch64-linux: 5215.3s -> 8154.4s (56.4%)
2. dist-x86_64-apple: 7512.3s -> 9320.8s (24.1%)
3. x86_64-apple-2: 5502.6s -> 4297.1s (-21.9%)
4. dist-apple-various: 7821.8s -> 6120.8s (-21.7%)
5. x86_64-gnu-stable: 7498.7s -> 6693.7s (-10.7%)
6. aarch64-apple: 4081.6s -> 3747.4s (-8.2%)
7. dist-i586-gnu-i586-i686-musl: 5211.7s -> 4936.9s (-5.3%)
8. i686-gnu-2: 6633.6s -> 6287.8s (-5.2%)
9. dist-armhf-linux: 5315.4s -> 5044.6s (-5.1%)
10. x86_64-apple-1: 7218.4s -> 6861.6s (-4.9%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

[rust-timer]

Finished benchmarking commit (43e62a7): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.5%	[-0.5%, -0.5%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (primary -0.4%, secondary -2.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.4%, 0.6%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.1%	[-2.1%, -2.1%]	1
Improvements ✅ (secondary)	-2.5%	[-2.5%, -2.5%]	1
All ❌✅ (primary)	-0.4%	[-2.1%, 0.6%]	3

Cycles

Results (primary 0.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.5%, 0.5%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.5%	[0.5%, 0.5%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 763.667s -> 764.554s (0.12%)
Artifact size: 365.13 MiB -> 365.12 MiB (-0.00%)