[stdlib] [benchmarks] [NFC] Fix benchmarking for public repo

[martinvuyk]

Fix benchmarking for public repo. This uses a dirty trick of replacing magic's lib/mojo/stdlib.mojopkg because otherwise setting MODULAR_MOJO_NIGHTLY_IMPORT_PATH to the build directory doesn't actually work.

I tried using mojo run --disable-builtins -I ${BUILD_DIR} to make it import stdlib from there but it doesn't work.

Just running the script (lit.cfg.py setting MODULAR_MOJO_NIGHTLY_IMPORT_PATH) used to work before the switch to magic.

[JoeLoser]

@Ahajha is this PR still needed? You recently removed these nightly variables.

[Ahajha]

My guess is no, but I'd like to understand the issue. I assume at time of writing, magic run benchmarks would use the packaged stdlib instead of the in-tree one?

A while back (December-ish) the nightly config sections were merged into the normal config sections (so no more distinction between mojo-max-nightly and mojo-max), and these environment variables are there to override the values. So essentially these env vars were just going into the void since that change. Though you also mentioned that the errors have existed longer than that.

[martinvuyk]

I assume at time of writing, magic run benchmarks would use the packaged stdlib instead of the in-tree one?

@Ahajha exactly

A while back (December-ish) the nightly config sections were merged into the normal config sections (so no more distinction between mojo-max-nightly and mojo-max), and these environment variables are there to override the values. So essentially these env vars were just going into the void since that change. Though you also mentioned that the errors have existed longer than that.

Maybe it was around that date that I noticed it not working anymore. I've been using it mostly for PR #3528 and some few others. But since I don't use it that often I can't remember when exactly it broke. But it did work for a while executing it with magic when I added the code in PR #3697 and when PR #3825 was merged

[Ahajha]

I merged a change last week that moved this variable to the non-NIGHTLY variant, could you try rebasing on main and see if it works again?

[martinvuyk]

@Ahajha still getting the same error in PR #3528 because I changed the return type of the function in-tree

/mojo/stdlib/benchmarks/collections/bench_string.mojo:117:30: error: cannot implicitly convert 'List[String]' value to 'List[StringSlice[items]]'
            res = items.split(sequence.value())
                  ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~

[Ahajha]

@martinvuyk Thanks for checking, I'll take a look then.

[Ahajha]

Haven't forgotten about this - I am going to try to get to it this week.

[martinvuyk]

#4149