this post was submitted on 14 Jul 2023
12 points (83.3% liked)

Selfhosted

40769 readers
714 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
ruud
Loki
CannaVet
devve
HybridSarcasm
[email protected]
12
Templating engine for docker compose (lemm.ee)
submitted 1 year ago* (last edited 1 year ago) by [email protected] to c/selfhosted
18 comments fedilink hide all child comments
 

Maybe this is a little bit off-topic. I would like to ask how you manage your dockerfile.

I have a git repo hosting my configurations (docker-compose, traefik, etc). Then, I have a python script that reads data from JSON, renders the placeholder inside these files (the {{replace_me}}) by an actual value and outputs them to another directory. Finally, I cd to that directory and run docker-compose up -f .... (This approach takes inspiration from the terraform templatefile)

That JSON file is generated by some terraform code, along with terraform code for other stuffs (storage bucket, vps, dns, etc).

It works well for me so far. Especially for:

  • templating traefik toml configuration (I like it a lot more than the label approach).
  • secret in the docker env file (so my docker.env file has the form of secrect={{secret}}.

I know most templating docker part can be replaced by directly interpolating with environment variables but I don't really like it because it seems environment variables are not persistent.

Do you have any suggestions for my workflow ? I am always feel a litte bit off about this approach.

Edit: Thank you for your suggestions. I will try k8s for edge computing and if it does not work really well, I will stick with my current approach.

top 16 comments
sorted by: hot top controversial new old
[–] [email protected] 5 points 1 year ago (1 children)

Man you're so close to Helm at this point, for me if I'm getting into templating I'd rather go on full Kubernetes and Helm

[–] [email protected] 3 points 1 year ago (1 children)

I know about Helm but I don't need k8s features and my VPS is just too weak for k8s I think :/

[–] [email protected] 3 points 1 year ago

Honestly look into k3s or other "distros" of k8s. There are some versions of the orchestrator made for edge computing that are quite slim in their size and gives you all the perks of the k8s api layer.

[–] [email protected] 2 points 1 year ago (1 children)

I was recently helping someone working on a mini-project to do a bit of parsing of docker compose files, when I discovered that the docker compose spec is published as JSON Schema here.

I converted that into TypeScript types using JSON Schema to TypeScript. So I can create docker compose config in code and then just export it as yaml - I have a build/deploy script that does this at the end.

But now the great thing is that I can export/import that config, share it between projects, extend configs, mix-in, and so on. I've just started doing it and it's been really nice so far, when I get a chance and it's stabilised a bit I'm going to tidy it up and share it. But there's not much I've added beyond the above at the moment (just some bits to mix-in arrays, which was what set me off on this whole thing!)

[–] [email protected] 1 points 1 year ago

so you are also generating the docker-compose from code. I think I am looking for something that aware of both templating and docker-compose deployment because right now, at the end of the day, I am still have to run docker-compose up -f ... while helm can do both templating and deployment.

[–] itsamenathan 2 points 1 year ago* (last edited 1 year ago) (1 children)

Here is an ansible role that will create a compose file using variables passed in. ironicbadger/ansible-role-docker-compose-generator

I tried this but ended up using ansible directly to manage my containers.

[–] [email protected] 1 points 1 year ago (1 children)

I dont think Ansible is a general templating engine ? I do templating for all most everything and not only docker compose, for example this for traefik configuration.

[–] ryapric 1 points 1 year ago (1 children)

fwiw Ansible uses Jinja under the hood, which itself is a templating engine.

[–] [email protected] 1 points 1 year ago

That sounds like what I need. Thank you very much !

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago) (2 children)

So far I just hand roll my docker-compose (at home, anyway). However, docker-compose does also support overrides via yaml merging, maybe that’s worth looking into?

My idea with that is to have a base compose that configures also my services and then to have a few override yamls with environment specific stuff (like prod, local, …)

This is similar to Kustomize from kubernetes land which I’ve worked with in the past

[–] [email protected] 3 points 1 year ago (1 children)

I actually need more than merging. For example, right now my traefik.template will look like this:

[http.routers.{{ router_name }}]
  rule = "{{ router_rule }}"
  service = "{{ service_name }}"
  middlewares = [{{ middlewares | map("tojson") | join(", ") }}]
{% block router %}{% endblock %}
  [http.routers.{{ router_name }}.tls]
    certResolver = "leresolver_http"

{% if service_host is defined %}
[[http.services.{{ service_name }}.loadBalancer.servers]]
  url = "{{ 'https' if service_use_https is defined else 'http' }}://{{ service_host }}{{ ':' ~ service_port if service_port is defined else '' }}"
{% endif %}

and then one of my traefik.toml could look like this

{% extends "template/traefik.jinja" %}

{% set router_name = "dozzle" %}
{% set router_rule = "Host(`dozzle.example.com`)" %}
{% set service_name = "dozzle" %}
{% set service_host = "dozzle" %}
{% set service_port = 8080 %}
[–] [email protected] 2 points 1 year ago

This is definitely a job for templating, seems you’ve got the right tool to me!

[–] [email protected] 2 points 1 year ago

I was using file merging, but one issue I found was that arrays don't get merged - and since switching to use Traefik (which is great) there are a lot of arrays in the config! And I've since started using labels for my own tooling too.

[–] [email protected] 1 points 1 year ago (1 children)

nix is pretty nice

[–] [email protected] 2 points 1 year ago (1 children)

you mean nix the package manager ?

[–] [email protected] 1 points 1 year ago

Yeah, maybe it can be used for that somehow.

load more comments
view more: next ›