OSError: [Errno 13] Permission denied when installing awscli in Travis build environment

[MikeTheCanuck]

Error during Travis build

Build 16 from this repo coughed up the following error and halted:

...
Installing collected packages: colorama, pyasn1, rsa, futures, docutils, jmespath, six, python-dateutil, botocore, s3transfer, PyYAML, awscli
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 345, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/local/lib/python2.7/dist-packages/pip/wheel.py", line 316, in clobber
    ensure_dir(destdir)
  File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 83, in ensure_dir
    os.makedirs(path)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/colorama'
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
You are using pip version 9.0.1, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command "pip install --upgrade awscli" failed and exited with 2 during .

[MikeTheCanuck]

This stackoverflow article indicates that we just need to be running a version of Python greater than the default one that's used in the Travis build environment.

I suspect that's why most of our updated Dockerfile, since this was originally built, all include an initial directive such as:
FROM python:3.5

or
FROM python:3.6.5-stretch

According to this discussion in the 2018 backend exemplar repo, the "stretch" variant of Python was used to enable development scenarios, and Ed makes a reasonable case for consistency over minimalization, so I'll try the stretch variant here too.

According to Docker Hub the latest stretch variant is 3.7.0-stretch, so that's what we'll use from now on.

[MikeTheCanuck]

However, since we're already deriving this container FROM nginx, and because using multiple FROM statements is a bug not feature we'll have to be careful here.

Later in that stackoverflow article they describe how to use multiple FROM statements - one for the "build" environment, another for the "run" environment - if you're on Docker "moby" (v17.05-ce) or later.

However, I later realized that the awscli install is there only in/for Travis, and that it's in the .travis.yml that this problem should be resolved, not in the container (whose build occurs afterwards).

So I'll try adding this to the .travis.yml:

python:
  - "3.7"

[MikeTheCanuck]

Well crap, that didn't help at all. Which tells me I was fixing the wrong problem - and sure enough I was able to find other repos whose Travis builds got past this step fine, even with the same InsecurePlatformWarning.

Looking a little closer there's an OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/colorama', so better to dig into that.

Let's try this stackoverflow article's advice first - install awscli as --user.

[MikeTheCanuck]

Yup, that fixed the Travis build error.

Fixed by 69dc9ac.

[znmeb]

I'm a little late to the party but I strongly recommend against upgrading from Python 3.6 to 3.7. We should update whenever the third digit in 3.6.x updates if it's a security fix, otherwise not.