cpython 3.13 installed with UV slow and not compiled with --enable-experimental-jit=yes-off`
#535
Comments
|
Thanks for the report. Do you know if Conda builds with any particular performance flags? |
|
I did an actual performance run (Linux x86-64) and it looks like there's a significant difference here I consider this fairly high priority, but I don't know what the source of the difference is. |
|
Looking at https://github.com/conda-forge/python-feedstock/blob/main/recipe/build_base.sh and not seeing anything obvious. |
|
@paugier What platform and architecture did you run your benchmarks on? |
|
It looks like they might use -O3 instead of -O2 edit: This is already the default in CPython |
|
Our v3 builds are a bit better, but that's not the bulk of it (geometric mean: 1.20x slower) |
This was referenced Feb 25, 2025
Closed
|
FWIW, I don't see any slowdown on conda-forge 3.13 vs uv 3.14.0a5 on my machine (AMD64 Linux) on this benchmark: In fact, the conda forge is significantly slower |
|
3.14 <-> 3.13 doesn't feel like a fair comparison since we're using the tail calling interpreter. |
|
Oh wow, I do see a significant slowdown on 3.13 (compare the previous comment) |
zanieb
added a commit
that referenced
this issue
Feb 27, 2025
See #535 This builds the JIT, but disables it by default. Users can opt-in to enable it at runtime. 3.14 and macOS support will follow, there are some hiccups there.
linux-x86_64-gnu. Sorry for this late answer. I was offline skying with the family. |
|
Should be fixed in the next release |
|
And using hyperfine |
|
That's awesome but could you also tell us what has been done to get this result? |
|
Yeah, it was the LLVM 20 upgrade with the backport for the LLVM 19 regression (conda-forge is on LLVM 18 on macOS and gcc on Linux) llvm/llvm-project#114990. This was merged here in #553 This is briefly discussed in https://github.com/astral-sh/python-build-standalone/releases/tag/20250311 |
|
Note that on Linux, I still get a notable difference : Python 3.13 conda-forge compiled with GGC is approximately 35% faster than Python 3.13 UV - PBS compiled with Clang 20. Details of the results here and code here. Good news: one can now enable the JIT with PBS Python 3.13 and 3.14! But the effect is tiny, much less than with conda-forge Python 3.13. |
|
I did a comprehensive benchmark and I'm seeing about a 6% difference This seems far more inline with what I'd expect. The statically linked libpython probably accounts for the additional difference. |
|
I can reproduce some of the difference you're reporting for that microbenchmark (but I also expect this to be related to libpython) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I tried a very simple pure Python benchmark (see https://gricad-gitlab.univ-grenoble-alpes.fr/augierpi/augierpi.gricad-pages.univ-grenoble-alpes.fr/-/tree/branch/default/content/docs/2025/about-py-jit) and figured out that CPython 3.13 installed with UV is slower than CPython 3.13 installed from conda-forge.
The benchmark is very simple (the goal was to be able to observe an effect of the new JIT in CPython 3.13):
def short_calcul(n): result = 0 for i in range(1, n+1): result += i return result def long_calcul(num): result = 0 for i in range(num): result += short_calcul(i) - short_calcul(i) return resultThe results:
This is bad:
--enable-experimental-jit=yes-offThe text was updated successfully, but these errors were encountered: