Missing baseman from requirements.txt.

[GrahamDumpleton]

Required by 04.13-Geographic-Data-With-Basemap.

This was is annoying because it can't be installed from PyPi using pip. You instead need to list the URL of where it is located on SourceForge.

• https://downloads.sourceforge.net/project/matplotlib/matplotlib-toolkits/basemap-1.0.7/basemap-1.0.7.tar.gz

[GrahamDumpleton]

But that also presents a problem as you must have the GEOS library installed on the system.

$ pip install https://downloads.sourceforge.net/project/matplotlib/matplotlib-toolkits/basemap-1.0.7/basemap-1.0.7.tar.gz           
Collecting https://downloads.sourceforge.net/project/matplotlib/matplotlib-toolkits/basemap-1.0.7/basemap-1.0.7.tar.gz              
  Downloading https://downloads.sourceforge.net/project/matplotlib/matplotlib-toolkits/basemap-1.0.7/basemap-1.0.7.tar.gz (129.7MB) 
    100% |████████████████████████████████| 129.7MB 8.8kB/s                                                                         
    Complete output from command python setup.py egg_info:                                                                          
    checking for GEOS lib in /opt/app-root ....                                                                                     
    checking for GEOS lib in /usr ....                                                                                              
    checking for GEOS lib in /usr/local ....                                                                                        
    checking for GEOS lib in /sw ....                                                                                               
    checking for GEOS lib in /opt ....                                                                                              
    checking for GEOS lib in /opt/local ....                                                                                        

    Can't find geos library . Please set the                                                                                        
    environment variable GEOS_DIR to point to the location                                                                          
    where geos is installed (for example, if geos_c.h                                                                               
    is in /usr/local/include, and libgeos_c is in /usr/local/lib,                                                                   
    set GEOS_DIR to /usr/local), or edit the setup.py script                                                                        
    manually and set the variable GEOS_dir (right after the line                                                                    
    that says "set GEOS_dir manually here".  

[GrahamDumpleton]

Just to be even more annoying, basemap package doesn't define an install_requires for numpy. Because it is an explicit URL, pip decides to go and install it before other listed modules in the requirements.txt file for some reason. The end result is that it fails to install because numpy isn't yet installed.

Collecting https://downloads.sourceforge.net/project/matplotlib/matplotlib-toolkits/basemap-1.0.7/basemap-1.0.7.tar.gz (from -r requirements.txt (line 11))
Downloading https://downloads.sourceforge.net/project/matplotlib/matplotlib-toolkits/basemap-1.0.7/basemap-1.0.7.tar.gz (129.7MB)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-qvf1vE-build/setup.py", line 1, in <module>
import sys, glob, os, numpy, subprocess
ImportError: No module named numpy

----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-qvf1vE-build/

This makes it impossible to add it to the requirements.txt file as pip will fail.

[GrahamDumpleton]

Only apparent solution is to install baseman direct from the GitHub repository using latest. There has been no release of basemap since the last one on SourceForge which fixes the missing install_requires in the setup.py file. Thus need to use:

• https://github.com/matplotlib/basemap/archive/master.zip

in the requirements.txt file if want it to be possible to use pip with a requirements.txt file and have it install all modules in one go.

[jakevdp]

I'll recommend using conda, in which case these issues don't come up 😄

Added to requirements... thanks!

[GrahamDumpleton]

Unfortunate. Means I need to provide a fork of your repo just to change the requirements.txt file so can provide a means for people to easily spin up all your notebooks so they can play with them as the system using doesn't use Anaconda, but normal Python distribution. Wish the basemap people could sort out getting updated package on PyPi. Apparently the limits on file size could be lifted for their specific project if necessary as noted in matplotlib/basemap#198.

[GrahamDumpleton]

Investigating whether can use platform.python_implementation environment marker in requirements.txt so can have it work differently for Anaconda Python vs normal Python.

• https://www.python.org/dev/peps/pep-0345/#environment-markers

[GrahamDumpleton]

No environment marker which is distinct for Anaconda Python unfortunately. :-(

[jakevdp]

If pip install basemap doesn't work, then I consider it a fault of pip and/or basemap. For what it's worth, to me this is just the latest in a long string of examples showing why pip is unfit for use with the data science stack, which is one of the reasons I recommend conda in the text.

If you think it's important to directly support pip, though, we could add a requirements-pip.txt or something similar to support those users. I'd be happy to accept a pull request if you find a configuration that works.

[GrahamDumpleton]

The reason is because basemap package is not on PyPi itself. The pip program no longer supports off site packages linked from PyPi. They couldn't upload some versions of basemap to PyPi because of a size limitation (must be for binary packages). The restriction on size on PyPi could be lifted for their project but it appears they haven't gone done that path. There are various issues about it against basemap GitHub issue tracker. See matplotlib/basemap#198

[jakevdp]

Ah, got it. Fortunately, until that is fixed, conda still works perfectly well 😄

[bsipocz]

@GrahamDumpleton - What about installing it directly from github? This works for master, but tags/releases can be done, too.
pip install https://github.com/matplotlib/basemap/archive/master.zip

[jakevdp]

Can that be listed in a requirements file?

[bsipocz]

Probably, at least we use it in the rtd-pip-requirements file. And we install dev versions with this inside ci-helpers.

[GrahamDumpleton]

@bsipocz Yes, installing direct from master of GitHub works and having the URL for that listed explicitly in the requirements.txt file was solution I used as mentioned above. I felt that Jake might perhaps prefer not want to use explicit GitHub URL, especially to master since code could change and suddenly break, so I didn't push it. If we can get a tagged released of basemap in GitHub so a specific version can be installed that would be better.

Jake might still want to have a requirements-conda.txt and requirements-pip.txt where only latter uses URL. The requirements.txt file could by default include -r requirements-conda.txt to delegate to conda version. Only suggest multiple files so clearer that for different install methods.

I don't know too much about conda, but not sure if other option is to have a requirements.yml file for conda using its format for package dependencies which can use when running conda. So you tell people to either use requirements.yml if using conda and requirements.txt if using pip. Just the requirements.txt would use the URL to GitHub.