Relative Paths with Docker and Portainer

I run docker workloads at home through Portainer. Some time ago I wanted to standardize on Docker Compose to manage container definitions via code, rather than configuring containers by hand in Portainer. Luckily, Portainer has the concept of "stacks", just their way of handling Docker Compose. Stacks can be used a few ways in Portainer, configured directly in Portainer or referenced from a Git repository, such as Gitlab. This is an attractive feature, even for a homelab, as it allows built in version control of your Docker Compose file, with a built in backup (sort of)!

Something that I have been trying to do for a while is store a container configuration file along side its docker-compose.yml file, this is supported in Docker, but I wasn't able to get this working via Portainer stacks. I was finding that I would either need to mount a network attached volume, an NFS or SMB share, where the config file for my container was or I would need to actually build my own Dockerfile to be able to copy in the config file, not something I wanted to do any time I needed to update the container config. The issue seemed to be with finding the config file in the repo, where the docker-compose.yml file lives, and getting it to the correct location in the container itself at run time.

This was a bit odd as environment files seem to work just fine with Portainer Git deployed stacks. One project I run is Firefly III, a personal finance software deployed via Docker. In my Gitlab repo for Firefly I have the environmental files living right next to the docker-compose.yml file, as can be seen below, with the following docker-compose.yml definition and it works just fine!

firefly_importer:
    image: fireflyiii/data-importer:latest
    hostname: importer
    restart: always
    ports:
      - '9124:8080'
    depends_on:
      - firefly_app
    env_file: .importer.env

Searching online returned a number of confusing results as what I was trying to do did indeed seem to be supported with Docker Compose natively, it seemed the issue was specifically with how Portainer stacks copied the config file into the container, if it copied it at all. For my core issue, I was trying to spin up a metrics related stack with Prometheus, Grafana, and assorted helper containers. I was storing the prometheus.yml file on a SMB share on my network, and this as working just fine, but I wanted to remove that dependency.

In my metrics repo I have the following folder structure:

docker-compose.yml
prometheus/
    prometheus.yml

Nothing I tried in my docker-compose.yml file seemed to work, in the volumes section of my config for Prometheus, the below is a sample of the things I tried, none of which worked.

- ./prometheus/prometheus.yml:/etc/prometheus
- ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
- ./prometheus/:/etc/prometheus
- ./prometheus/:/etc/prometheus/
- ./prometheus/:/etc/prometheus
- prometheus/:/etc/prometheus
- /prometheus:/etc/prometheus

I felt quite stuck, partially due to knowing that this was possible via Docker Compose (- ./prometheus/:/etc/prometheusshould work in Docker Compose from the command line) and failing to have the correct words to search for what I was trying to do. I finally came across one post that called out that Portainer Business supported relative paths for Docker Compose files, that sounded like what I might want to do, but did not have a Portainer Business license, I was using the Portainer CE version. A few more searches led me to the following information.

• Portainer Business has a program called Take 5where your first 5 nodes in Portainer are free, this includes homelabs!
• Relative paths for Portainer stacks managed by Git is exactly what I was looking for!

Warning

Portainer recently reduced the amount of hosts that you can use with their free Business Edition license from 5 hosts to 3 hosts. If you have an existing 5 node license you are able to keep it, but for any new requests the limit is 3 nodes.

After upgrading to the Portainer Business license, when creating a new stack you should see the "Enable Relative Path Volume" option, it looks like the following image. After turning that on you will be prompted to enter a path on your Docker host that will be used, in my case I used /mnt

After deploying the stack via Git the following path was created on my Docker host /mnt/portainer-compose-unpacker/stacks/metrics/metricand in that folder I had the full Gitlab repo accessible! This means that the prometheus/prometheus.yml file I was looking for was now located at /mnt/portainer-compose-unpacker/stacks/metrics/metric/prometheus/prometheus.yml and could be used from my docker-compose.yml file volumes definition with the following line.

- /mnt/portainer-compose-unpacker/stacks/metrics/metrics/prometheus:/etc/prometheus

This works! Since the relative path feature gets the config file from our repo onto our Docker host in a reliable and accessible location, we can use this reference to specify the config file for our container. This process took far too long to figure out, and while I'm sort of hoping that some random internet person will tell me a different/non-feature gated way to do this, this sure works for me for now!