Mount should be chowned

[txomon]

I have been running into trouble when running the docker image with the file backend.

The line I have been using is docker run -p 8080:8080 -v (pwd)/config.ini:/etc/pypicloud/config.ini:ro -v (pwd)/pypicloud:/var/lib/pypicloud stevearc/pypicloud

According to docker/compose#3270 (comment) we have to add a chown on the folder to have a proper mount

[I] javier@sam ~/p/k/h/pypiserver (master) [2]> docker run -p 8080:8080 -v (pwd)/config.ini:/etc/pypicloud/config.ini:ro -v (pwd)/pypicloud:/var/lib/pypicloud stevearc/pypicloud        
*** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...                                                                                                                                       
*** Running /etc/my_init.d/10_syslog-ng.init...                                                                                                                                               
Aug  9 12:27:39 0d54f0559a32 syslog-ng[12]: syslog-ng starting up; version='3.5.6'                                                                                                            
Aug  9 12:27:40 0d54f0559a32 syslog-ng[12]: EOF on control channel, closing connection;                                                                                                       
*** Running /etc/my_init.d/pypicloud-uwsgi.sh...                                                                                                                                              
[uWSGI] getting INI configuration from /etc/pypicloud/config.ini                                                                                                                              
*** Starting uWSGI 2.0.17 (64bit) on [Thu Aug  9 12:27:40 2018] ***                                                                                                                           
compiled with version: 5.4.0 20160609 on 12 June 2018 17:04:40                                                                                                                                
os: Linux-4.17.12-arch1-1-ARCH #1 SMP PREEMPT Fri Aug 3 07:16:41 UTC 2018                                                                                                                     
nodename: 0d54f0559a32                                                                                                                                                                        
machine: x86_64                                                                                                                                                                               
clock source: unix
detected number of CPU cores: 4
current working directory: /
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on 0.0.0.0:8080 fd 4
uwsgi socket 0 bound to TCP address 127.0.0.1:43723 (port auto-assigned) fd 3
Python version: 3.5.2 (default, Nov 23 2017, 16:37:01)  [GCC 5.4.0 20160609]
Python main interpreter initialized at 0x19f98f0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 15 seconds
mapped 1531320 bytes (1495 KB) for 20 cores
*** Operational MODE: preforking ***
Loading paste environment: config:/etc/pypicloud/config.ini
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/local/lib/python3.5/dist-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
    return context.create()
  File "/usr/local/lib/python3.5/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/usr/local/lib/python3.5/dist-packages/paste/deploy/loadwsgi.py", line 146, in invoke
    return fix_call(context.object, context.global_conf, **context.local_conf)
  File "/usr/local/lib/python3.5/dist-packages/paste/deploy/util.py", line 55, in fix_call
    val = callable(*args, **kw)
  File "/usr/local/lib/python3.5/dist-packages/pypicloud/__init__.py", line 146, in main
    config.include('pypicloud')
  File "/usr/local/lib/python3.5/dist-packages/pyramid/config/__init__.py", line 839, in include                                                                                             
    c(configurator)
  File "/usr/local/lib/python3.5/dist-packages/pypicloud/__init__.py", line 76, in includeme
    config.include('pypicloud.cache')
  File "/usr/local/lib/python3.5/dist-packages/pyramid/config/__init__.py", line 839, in include                                                                                             
    c(configurator)
  File "/usr/local/lib/python3.5/dist-packages/pypicloud/cache/__init__.py", line 27, in includeme                                                                                           
    kwargs = cache_impl.configure(settings)
  File "/usr/local/lib/python3.5/dist-packages/pypicloud/cache/sql.py", line 141, in configure
    kwargs = super(SQLCache, cls).configure(settings)
  File "/usr/local/lib/python3.5/dist-packages/pypicloud/cache/base.py", line 43, in configure
    'storage': get_storage_impl(settings),
  File "/usr/local/lib/python3.5/dist-packages/pypicloud/storage/__init__.py", line 22, in get_storage_impl                                                                                  
    kwargs = storage_impl.configure(settings)
  File "/usr/local/lib/python3.5/dist-packages/pypicloud/storage/files.py", line 27, in configure                                                                                            
    os.makedirs(directory)
  File "/usr/lib/python3.5/os.py", line 241, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/var/lib/pypicloud/packages'
*** /etc/my_init.d/pypicloud-uwsgi.sh failed with status 22

*** Killing all processes...
Aug  9 12:27:41 0d54f0559a32 syslog-ng[12]: syslog-ng shutting down; version='3.5.6'

config.ini for easy reproduction of the problem:

[app:main]
# Defaults for stuff
use = egg:pypicloud
pyramid.reload_templates = False
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en
# Not relevant, it's used for cache
db.url = sqlite:////var/lib/pypicloud/db.sqlite


# Permissions packets
pypi.default_read =
    authenticated
pypi.default_write =
    authenticated
# Local fs
pypi.storage = file
storage.dir = /var/lib/pypicloud/packages
# If missing package, do 404
pypi.fallback = none
# TODO: We want to show the proper url to use in clients
# pypi.download_url = 

# Users, groups and pass
auth.admins =
  admin
user.admin = $6$rounds=535000$qtKonMlaWPPAtHCY$PPcMJghdnEK.UN396yrkPfQMmg3dwKrLeN01l36YyL2nXfFK3tVT.yQwuxlUwvZXHw64L7nlD14/gxMb5Rvu6/

# For some random stuff, secret!
session.encrypt_key = Y1bLqCYfRFQ6fxF9ssVaeVUAxnxuxFr5VNy35F5+NN8=
session.validate_key = T88Xl4IGUinT9ZAy9ykACbLOuP6pgkLyo7FGRRQOyFs=
session.secure = False
session.invalidate_corrupt = true

###
# wsgi server configuration
###

[uwsgi]
paste = config:%p
paste-logger = %p
master = true
processes = 20
reload-mercy = 15
worker-reload-mercy = 15
max-requests = 1000
enable-threads = true
http = 0.0.0.0:8080
uid = pypicloud
gid = pypicloud

###
# logging configuration
# http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/logging.html
###

[loggers]
keys = root, botocore, pypicloud

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = INFO
handlers = console

[logger_pypicloud]
level = DEBUG
qualname = pypicloud
handlers =

[logger_botocore]
level = WARN
qualname = botocore
handlers =

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)s %(asctime)s [%(name)s] %(message)s

docker/compose#3270 (comment)

[stevearc]

I think that comment is only for use with docker-compose. If you're mounting a local filesystem on a host machine, I don't see how docker could safely perform a chown without a massive security breach. Plus, the folder does already belong to the correct user:

$ docker run --rm -it stevearc/pypicloud /bin/sh -c "ls -ld /var/lib/pypicloud"
drwxr-xr-x 2 pypicloud pypicloud 4096 Aug 14 15:24 /var/lib/pypicloud

If you want to use it this way you'll have to either make sure that the mounted directory is very open (chmod 777) or you could chown it to the pypicloud user before running the docker command. Since that user doesn't exist on your host machine, you'll have to use the raw UID/GID:

$ docker run --rm -it stevearc/pypicloud /bin/sh -c "id pypicloud"
uid=999(pypicloud) gid=999(pypicloud) groups=999(pypicloud)
$ chown 999:999 pypicloud

[txomon]

Just in case, I solved this by running as root. Would you be interested on a pypicloud helm chart?

I have developed it for the use of local storage (PVC backed) + local (config.ini secret) auth, and I would be happy to share.

[stevearc]

Sure! I've not played with Kubernetes at all; it would be neat to see. Could be useful to include in the repo as an example like I've done with the compose files

[iorbitearth]

I'm also looking to store the packages in a volume mounted locally. None of the three approaches seem to work for me (run as root, open permissions, changing ownership of local directory to pypicloud).

I figured file storage backing mounted as a local volume would be a fairly general use case. Is this not recommended?

[stevearc]

I believe the vast majority of users prefer to store packages in S3 or GCS, as it's pretty simple and way more durable than the disk of an individual machine.