Monday, October 13, 2014

When using python pip: read operation timeouts and distributions not found

What is Python pip?
Python pip is a package management system used in installing and managing python packages [1].

Why did I use pip?
In my Python docker cartridge for Apache Stratos project, I have created some python web applications and hosted them in github [suhan at github].
I installed the following dependencies prior using pip in my ubuntu 12.04.
apt-get install -y build-essential python-dev python-pip python-virtualenv libpq-dev

Prior running the python web applications, I have to install their python dependencies. Usually I put them inside a requirements.txt file as follows,
flask
cherrypy

Then I try to install the requirements via pip as follows,
pip install -r requirements.txt

Errors
Somewhere in the middle during installation I get these errors.
---- Installing git python web application project requirements ----
Downloading/unpacking flask
  Running setup.py egg_info for package flask
    
    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
    no previously-included directories found matching 'docs/_themes/.git'
Downloading/unpacking cherrypy
  Running setup.py egg_info for package cherrypy
    
Downloading/unpacking Werkzeug>=0.7 (from flask)
  Running setup.py egg_info for package Werkzeug
    
    warning: no files found matching '*' under directory 'werkzeug/debug/templates'
    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
Downloading/unpacking Jinja2>=2.4 (from flask)
  Running setup.py egg_info for package Jinja2
    
    warning: no files found matching '*' under directory 'custom_fixers'
    warning: no previously-included files matching '*' found under directory 'docs/_build'
    warning: no previously-included files matching '*.pyc' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'jinja2'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
Downloading/unpacking itsdangerous>=0.21 (from flask)
  Could not find any downloads that satisfy the requirement itsdangerous>=0.21 (from flask)
No distributions at all found for itsdangerous>=0.21 (from flask)
Storing complete log in /root/.pip/pip.log

CherryPy server fails to start since dependencies are not properly installed.

---- Starting CherryPy Server ----
Traceback (most recent call last):
 File "/var/www/www/server.py", line 5, in <module>
   from app import app
 File "/var/www/www/app.py", line 1, in <module>
   from flask import Flask

ImportError: No module named flask

Sometimes I get the following error.

Exception:
Traceback (most recent call last):
  File "/vagrant/venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/vagrant/venv/local/lib/python2.7/site-packages/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/vagrant/venv/local/lib/python2.7/site-packages/pip/req.py", line 1092, in prepare_files
    self.unpack_url(url, location, self.is_download)
  File "/vagrant/venv/local/lib/python2.7/site-packages/pip/req.py", line 1238, in unpack_url
    retval = unpack_http_url(link, location, self.download_cache, self.download_dir)
  File "/vagrant/venv/local/lib/python2.7/site-packages/pip/download.py", line 622, in unpack_http_url
    download_hash = _download_url(resp, link, temp_location)
  File "/vagrant/venv/local/lib/python2.7/site-packages/pip/download.py", line 495, in _download_url
    chunk = resp.read(4096)
  File "/usr/lib/python2.7/socket.py", line 380, in read
    data = self._sock.recv(left)
  File "/usr/lib/python2.7/httplib.py", line 561, in read
    s = self.fp.read(amt)
  File "/usr/lib/python2.7/socket.py", line 380, in read
    data = self._sock.recv(left)
  File "/usr/lib/python2.7/ssl.py", line 241, in recv
    return self.read(buflen)
  File "/usr/lib/python2.7/ssl.py", line 160, in read
    return self._sslobj.read(len)
SSLError: The read operation timed out

These errors are due to internet connectivity problems, slow internet speed, etc...
Since I'm using a VM to build my docker images these issues/errors are further amplified.


As a temporary solution I did retry several times and finally it was successfully installed.
retry: pip install -r requirements.txt

Then my CherryPy server started successfully.
---- Starting CherryPy Server ----
ENGINE Bus STARTING
ENGINE Started monitor thread 'Autoreloader'.
ENGINE Started monitor thread '_TimeoutMonitor'.
ENGINE Serving on http://0.0.0.0
ENGINE Bus STARTED

However according to stackoverflow [2] if you can use the following command the problem will be resolved.
pip --default-timeout=100 install django

Reference:
[1]http://en.wikipedia.org/wiki/Pip_(package_manager)
[2]http://stackoverflow.com/questions/18958508/sslerror-the-read-operation-timed-out-when-using-pip/19334551#19334551