Publish wheels to pypi

[tobiasdiez]

For easier use, it would be nice if cypari2 would publish wheels to pypi.

[videlec]

The main problem is that Python packages do not support C libraries dependencies. PyPI is a Python package manager and is far too limited. The only reasonable wheel that could be provided would be a cypari2 bundled with gmp and pari inside. We already intensively discussed the matter some years ago in #19.

What is your use case?

[tobiasdiez]

cypari2 bundled with gmp and pari inside

That would be helpful, indeed! My use case is https://trac.sagemath.org/ticket/30371, where pipenv is used to install all python dependencies of sage. Right now one needs a system install of pari and gmp etc for this to work. It is working, but using pre-built wheels for cypari would make things easier.

[videlec]

cypari2 bundled with gmp and pari inside

That would be helpful, indeed! My use case is https://trac.sagemath.org/ticket/30371, where pipenv is used to install all python dependencies of sage. Right now one needs a system install of pari and gmp etc for this to work. It is working, but using pre-built wheels for cypari would make things easier.

Using a cypari2 bundled with gmp and pari in SageMath would be a terrible design choice. SageMath has a lot of interoperating libraries. You don't want to duplicate them (see for the example of the recent effort made to use system libraries). My point is still the same : pip is not aimed to manage C libraries. To my mind, the problem is the usage of pip itself as a dependency tracker for SageMath the distribution. This is fine as far as SageMath source code is concerned because it consists of Python/Cython code. But not for SageMath the distribution. Of course if SageMath had only Python dependencies that would be simpler. But it is not the case.

[tobiasdiez]

The aim is not to change the way sage is distributed, but only how developer can work with it locally. For this, it might be helpful (depending a bit on the system) if wheels are used for all python dependencies and one only needs to build C-libraries that are directly used by sage.

Apart from this use case, wheels would make it easier to use cypari. This is a bit similar to how much easier it is use use the wheels of scipy/numpy instead of compiling them from source.

[videlec]

I made a difference between SageMath the distribution and SageMath the source code to make clear what I talked about. It had nothing to do with actually distributing the software.

• SageMath the distribution : bunch of libraries and softwares that interact with each others
• SageMath source code: the Python code in $SAGE_ROOT/src/sage

If your goal is to allow developers to modify locally the SageMath source code, it is not helping them to provide wheels. For example I might have a customized PARI/GP on my system that I compiled myself and I would like to use in SageMath. That can be for the very concrete purpose of upgrading the version of PARI/GP we are using in SageMath. I think I understood the purpose of ticket 30371. What you seem to propose very restrictive regarding the level of interactions of this code with the rest of the libraries in SageMath the distribution. Please tell me if I misunderstood.

[tobiasdiez]

Yes, if you work on the parts of sage that involve changes to the required libraries (such as updates), then my ticket is not of much use. What is facilitated by it is the more day-to-day work of writing/changing the python code.

Another use cases of having a wheel, would be the recent progress towards modularization of sage. Once sage publishes wheels to pypi (for parts of its interface) and all other python libraries involved also do this, then one can use sage without the need to install any of the c++ libraries.

[videlec]

Another use cases of having a wheel, would be the recent progress towards modularization of sage. Once sage publishes wheels to pypi (for parts of its interface) and all other python libraries involved also do this, then one can use sage without the need to install any of the c++ libraries.

What is the point of this? SageMath does depend on C, C++, fortran and lisp softwares and libraries. Even its core part uses gmp, mpfr, flint and arb for arithmetic. How can you propose not to install them? Hiding the C/C++ libraries inside the Python site-packages is a terrible solution. Just admit that pip is broken for the purpose of installing SageMath the distribution. Why do you want to use pip for that purpose?

[tobiasdiez]

For the same reason that numpy/scipy and a many other libraries provide wheels: it makes using the libraries more easy.

And it's for Sage the library, i.e. if people decide to use (part) of sage in their own projects (which is one of the goals of the modularization). Then it would be nice if people could just install the wheel instead of fiddling around with different libraries on different OS.

But I do agree, the whole ecosystem around distributing python libraries relying on c++ libraries has a lot of room for improvement.

[videlec]

For the numpy/scipy situation, please have a look at their documentation : https://www.scipy.org/install.html. They advice strongly to use a reasonable package manager, namely conda. Moreover in their case, providing wheels on PyPI sort of make sense since they have a single dependency on libatlas.

Going back to SageMath, providing wheels with bundled C libraries will simply not work. gmpy2 and cypari2 depends on libgmp but are completely independent Python libraries. SageMath the distribution depends on both of them. If there would exist wheels for both of them that bundles libgmp then how will it work?

[tobiasdiez]

If there would exist wheels for both of them that bundles libgmp then how will it work?

Both of them providing and using their own copy of libgmp using the correct r-path? Anyway, the point was not to discuss alternatives of how sagemath should be distributed.

[culler]

Tobias, Please run "pip install cypari" and see if that does the trick for you. - Marc

…

On Thu, Feb 11, 2021 at 8:40 AM Tobias Diez ***@***.***> wrote: If there would exist wheels for both of them that bundles libgmp then how will it work? Both of them providing and using their own copy of libgmp using the correct r-path? Anyway, the point was not to discuss alternatives of how sagemath should be distributed. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#105 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJ6CPZJPAO66OVK7ECYPBDS6PT4DANCNFSM4XMWOWUA> .

[mkoeppe]

See https://trac.sagemath.org/ticket/31251 on several pointers regarding distribution of packages with native library dependencies as wheels

[dimpase]

why is this a problem, lack of C support?
This is what cibuildwheel would take care of.

[dimpase]

e.g. see my https://gitlab.com/videlec/pplpy/-/merge_requests/19 for a draft of such a config for pplpy