Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add notes for package maintainers #25912

Merged
merged 14 commits into from
Feb 27, 2018
Merged

Add notes for package maintainers #25912

Changes from 13 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
5be21a9
Add notes for package maintainers
simonbyrne Feb 6, 2018
a5c1d79
typo and link
simonbyrne Feb 6, 2018
bc026a0
more typos
simonbyrne Feb 6, 2018
ea24f56
rephrase sentence
simonbyrne Feb 6, 2018
7b0d032
gentoo link
simonbyrne Feb 6, 2018
501bff0
Add comment re: newer LLVM versions
simonbyrne Feb 6, 2018
0e7526d
Add mention of LLVM makefile
simonbyrne Feb 6, 2018
9534d93
Add link to awk
simonbyrne Feb 6, 2018
3e10b43
Update based on @staticfloat's comments
simonbyrne Feb 6, 2018
a3f39c6
Tweak text
simonbyrne Feb 6, 2018
665035a
Update based on @eli-schwartz's comments
simonbyrne Feb 7, 2018
0d1846e
Add notes on libuv
simonbyrne Feb 7, 2018
5fa9587
minor tweak to text
simonbyrne Feb 7, 2018
59c26e7
link to packages.html
simonbyrne Feb 10, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 39 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -294,15 +294,15 @@ Building Julia requires that the following software be installed:

Julia uses the following external libraries, which are automatically downloaded (or in a few cases, included in the Julia source repository) and then compiled from source the first time you run `make`:

- **[LLVM]** (3.9) — compiler infrastructure.
- **[LLVM]** (3.9 + patches) — compiler infrastructure (see [note below](#llvm)).
- **[FemtoLisp]** — packaged with Julia source, and used to implement the compiler front-end.
- **[libuv]** — portable, high-performance event-based I/O library.
- **[libuv]** (custom fork) — portable, high-performance event-based I/O library.
- **[OpenLibm]** — portable libm library containing elementary math functions.
- **[DSFMT]** — fast Mersenne Twister pseudorandom number generator library.
- **[OpenBLAS]** — fast, open, and maintained [basic linear algebra subprograms (BLAS)](https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms) library, based on [Kazushige Goto's](https://en.wikipedia.org/wiki/Kazushige_Goto) famous [GotoBLAS](https://www.tacc.utexas.edu/research-development/tacc-software/gotoblas2).
- **[OpenBLAS]** — fast, open, and maintained [basic linear algebra subprograms (BLAS)](https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms) library, based on [Kazushige Goto's](https://en.wikipedia.org/wiki/Kazushige_Goto) famous [GotoBLAS](https://www.tacc.utexas.edu/research-development/tacc-software/gotoblas2) (see [note below](#blas-and-lapack)).
- **[LAPACK]** (>= 3.5) — library of linear algebra routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems.
- **[MKL]** (optional) – OpenBLAS and LAPACK may be replaced by Intel's MKL library.
- **[SuiteSparse]** (>= 4.1) — library of linear algebra routines for sparse matrices.
- **[SuiteSparse]** (>= 4.1) — library of linear algebra routines for sparse matrices (see [note below](#suitesparse)).
- **[ARPACK]** — collection of subroutines designed to solve large, sparse eigenvalue problems.
- **[PCRE]** (>= 10.00) — Perl-compatible regular expressions library.
- **[GMP]** (>= 5.0) — GNU multiple precision arithmetic library, needed for `BigInt` support.
Expand All @@ -318,6 +318,7 @@ Julia uses the following external libraries, which are automatically downloaded
[patch]: http://www.gnu.org/software/patch
[wget]: http://www.gnu.org/software/wget
[m4]: http://www.gnu.org/software/m4
[awk]: http://www.gnu.org/software/gawk
[gcc]: http://gcc.gnu.org
[clang]: http://clang.llvm.org
[python]: https://www.python.org/
Expand Down Expand Up @@ -347,12 +348,46 @@ Julia uses the following external libraries, which are automatically downloaded
[mbedtls]: https://tls.mbed.org/
[pkg-config]: https://www.freedesktop.org/wiki/Software/pkg-config/

### Notes for distribution package maintainers

Package maintainers will typically want to make use of system libraries where possible. Please refer to the above version requirements and additional notes below.

Currently community maintained packages are:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these were all removed as unsupported in #23484 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see.

My argument for having this is so that maintainers can look to see what others have done. Perhaps a permalink to somewhere julialang.org or a discourse page would be more useful.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about describing them as no-longer-maintained and still mentioning them here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are maintained, just not by us. OTC a different bullet below mentions packages which are unmaintained.

We could move instructions for packagers to a separate file so that casual users don't find them. But honestly we're talking about README.md, not the main download page, so I'm not too concerned. Anyway people are going to use these packages, so refusing to mention them is a bit silly IMHO. It's even useful to distinguish actively maintained packages from totally outdated ones to prevent surprises.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the stance of the upstream project, that people should be using them or that how they're built can't be vouched for and may introduce problems that are strictly due to packaging? Maintaining a list of what's maintained is bound to get out of date and doesn't sound so useful. https://julialang.org/downloads/platform.html says "It is strongly recommended that the official generic linux binaries from the downloads page be used to install Julia" and "The following distribution-specific packages are community contributed. They may not use the right versions of Julia dependencies or include important patches that the official binaries ship with. In general, bug reports will only be accepted if they are reproducible on the generic linux binaries on the downloads page." so why should the readme have redundant or contradictory statements about them?

Copy link
Contributor Author

@simonbyrne simonbyrne Feb 8, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree we should probably move them, but for different reasons: the README is a snapshot: it isn't that useful to know which packages were maintained with 0.6 when looking at the 0.7 README. I think better to move them to a maintainable place, such as a discourse wiki page, or even a subpage of https://julialang.org/downloads/, and mention that in the README.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which of these would people prefer?

Copy link
Contributor Author

@simonbyrne simonbyrne Feb 8, 2018 •
edited by nalimilan
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I didn't see https://julialang.org/downloads/platform.html, we can just use that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, let's put that information on that page, with all warnings we may want to add, especially regarding the fact that distribution packages generally lag behind upstream since they only provide the lastest version which was available when the distro was released. Better acknowledge that these packages exist and inform users than ignore them and leave users on their own discovering the potential issues.

- Fedora: [official](https://src.fedoraproject.org/rpms/julia), [Copr (for backports)](https://copr.fedorainfracloud.org/coprs/nalimilan/julia/)
- [Arch](https://www.archlinux.org/packages/community/x86_64/julia/)
- [Gentoo](https://packages.gentoo.org/packages/dev-lang/julia)
- [Raspbian](https://github.com/JuliaBerry/julia-raspbian)

Additionally, the following unmaintained packages may still be useful:
- [Homebrew](https://github.com/staticfloat/homebrew-julia)
- [Ubuntu PPA](https://launchpad.net/~staticfloat/+archive/ubuntu/juliareleases) (0.5)
- [Debian](https://packages.debian.org/sid/julia) (0.4)

### System Provided Libraries

If you already have one or more of these packages installed on your system, you can prevent Julia from compiling duplicates of these libraries by passing `USE_SYSTEM_...=1` to `make` or adding the line to `Make.user`. The complete list of possible flags can be found in `Make.inc`.

Please be aware that this procedure is not officially supported, as it introduces additional variability into the installation and versioning of the dependencies, and is recommended only for system package maintainers. Unexpected compile errors may result, as the build system will do no further checking to ensure the proper packages are installed.

### LLVM

The most complicated dependency is LLVM, for which we require version 3.9 with some additional patches from upstream (LLVM is not backward compatible). For packaging Julia, we recommend either:
- bundling a Julia-only LLVM library inside the Julia package, or
- adding the patches to the LLVM 3.9 package of the distribution.
* A complete list of patches is available in `deps/llvm.mk`, and the patches themselves are in `deps/patches/`.
* The only Julia-specific patch is the lib renaming (`llvm-symver-jlprefix.patch`), which should _not_ be applied to a system LLVM.
* The remaining patches are all upstream bug fixes, and have been contributed into upstream LLVM.

Using an unpatched or different version of LLVM will result in errors and/or poor performance. Though Julia can be built with newer LLVM versions, support for this should be regarded as experimental and not suitable for packaging.

### libuv

Julia uses a custom fork of libuv. It is a small dependency, and can be safely bundled in the same package as Julia, and will not conflict with the system library. Julia builds should _not_ try to use the system libuv.

### BLAS and LAPACK

As a high-performance numerical language, Julia should be linked to a multi-threaded BLAS and LAPACK, such as OpenBLAS or ATLAS, which will provide much better performance than the reference `libblas` implementations which may be default on some systems.

### SuiteSparse

SuiteSparse is a special case, since it is typically only installed as a static library, while `USE_SYSTEM_SUITESPARSE=1` requires that it is a shared library. Running the script `contrib/repackage_system_suitesparse4.make` will copy your static system SuiteSparse installation into the shared library format required by Julia. `make USE_SYSTEM_SUITESPARSE=1` will then use the SuiteSparse that has been copied into Julia's directory, but will not build a new SuiteSparse library from scratch.
Expand Down