this post was submitted on 03 Jul 2023
203 points (96.8% liked)

Selfhosted

40324 readers
661 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
ruud
Loki
CannaVet
devve
HybridSarcasm
[email protected]
203
New house, new server - step 1: update diagram (lemmy.world)
submitted 1 year ago by mauns to c/selfhosted
49 comments fedilink hide all child comments
 

Hi all, new to Lemmy but this seems to be the best community for this that is decently active. Apologies if not!

I got into home servers in my first house a couple years ago, but our stay in that house was unexpectedly brief and everything got put back into boxes. It's time to setup at the new place, and I have many improvements in mind from the first implementation - so while I wait for server parts to arrive, I decided to update the diagram for planning.

In no order, here's a list of lessons I learned from V1:

  • The blade form factor doesn't work for me. I enjoyed getting one and learning about them, but my use cases are small (&quiet) enough that a tower and a small network rack works better.
  • In the quest for automatic home lighting, I shouldn't have gone all-in on smart bulbs rather than switches. There get to be too many in the house, and when a couple start inevitably failing, expensive bulbs and misplaced warranty info are a gigantic pain. So now the bulbs are just for special things like ceiling fans and floor lamps.
  • I need to put more attention on storage. That's what gets used the most, by multiple users, so I will use TrueNAS Scale as my host instead of ESXi. I was not enough of a power user for that to be important to me. The rest of it is mostly for play and doesn't need to be perfect.
  • My media streaming needs are very simple, so I think I may like Jellyfin better than Plex.
  • I need to be 'a little' more lax about security. I don't think my server is realistically likely to be heavily attacked, and when I tried to go all out on best practices, more often than not I just broke things and upset my family users. My server will not have an outside access except via VPN, and my IOT devices will not speak unless spoken to - I think that will be enough.

In particular, I tried so hard last time to have a tagged management VLAN in UniFi and always just broke connectivity between something that required a hard reset. I'm planning to skip that this time but if someone has a pointer to a good setup guide, I could try that again.

Thanks for reading/looking, all comments or suggestions are welcome! I also still need to find more applications I can selfhost so I will be keeping an eye on the community for ideas.

top 40 comments
sorted by: hot top controversial new old
[–] sturlabragason 18 points 1 year ago (2 children)

I hope you're generating your diagram from code and not editing it by hand like a caveman? If not I suggest https://kroki.io/

[–] mauns 10 points 1 year ago

unga bunga

I enjoy making diagrams, and I spent as much time on this as I did because my internet has been out for 2 days and counting and it was an offline tool I already had. Nevertheless, thanks for the link to a new tool to look at

[–] Pulsar 3 points 1 year ago

I have a project on mind and I was planning on using nomnoml directly. https://kroki.io/ seems to a better way to interact with it. Thank you for sharing.

[–] [email protected] 14 points 1 year ago

Bro, tag that shit NSFW. It's straight up porn.

[–] [email protected] 11 points 1 year ago* (last edited 1 year ago) (1 children)

My Home Automation and Home Assistant brain is asking "Why two concurrent Zigbee networks?"

Phillips Hue is based on Zigbee and should work with your Conbee setup. Running them separately weakens both of them. Especially since the line of sight that light bulbs have makes them some of the best Zigbee routers in your household.

[–] mauns 7 points 1 year ago (1 children)

You're definitely right, I'm going to remove the Hue hub and connect everything to the ConBee. They were separated before because Hue came first, and I didn't want to rebuild my lighting setup at the time. HA automations were set up to use Hue-side groups, rather than HA-side groups. The thought hadn't occurred to me yet that I can unify them now - but I definitely will.

I did think I remembered that Hue bulbs don't work properly as ZigBee routers, but could be wrong there

[–] [email protected] 3 points 1 year ago

The only bulbs I know of that don't are SengLED bulbs because it was a deliberate (and misguided) choice by the manufacturer. I have some Hue bulbs mixed in with my network and they work the same as everything else.

[–] Octavius 11 points 1 year ago (2 children)

Hey. That looks great. How did you mange to make it that pretty? Just one Question: is your Modem really part of VLAN 1? I am running a pretty similar setup and my modem just runs into my USG and terminates my internet connection there (so my public IP is at my USG).

[–] [email protected] 8 points 1 year ago (1 children)

I wonder the same, how is it that pretty? Op what software did you use?

[–] mauns 5 points 1 year ago (1 children)

Thank you! I'm using draw.io aka diagrams.net

[–] [email protected] 1 points 1 year ago

I use selfhosted draw.io as well; yours looks way better.

It looks like the network is based off the ISP router instead of the firewall. Should the firewall be the only thing on the router and everything else hang off that?

[–] mauns 1 points 1 year ago* (last edited 1 year ago)

Thanks! Too much spare time on the weekend, haha

Good point, mine might be the same. I was linking things up from memory since it's been >1year since I last had the server interface up, and I can't remember how the modem was represented now that you mention it - I'll have to fix that up

[–] [email protected] 5 points 1 year ago (2 children)

That's a really nice setup! I run most of my things on a docker swarm (the docker hosts are VMs running on Proxmox hosts), though that was an overkill in retrospect, and causes more problems with no practical advantages.

The range of services I run is similar to yours, but I also have a bunch of services for personal finance (beancont/fava, as well as automatic importers and such), a more extensive media setup (with qBitTorrent and *arr apps), a gitea server, and a vaultwarden instance.

[–] [email protected] 3 points 1 year ago

I'm curious which part you think is overkill and how you would redo this? I have a proxmox cluster and run docker amongst other things, but haven't set up any sort of high availability.

I don't need live migrations, but something that could help with load balancing and reducing any potential downtime if a host fails would be great.

[–] mauns 2 points 1 year ago

Thanks! My last setup was quite similar to you, just with ESXi - definitely wanted to cut out as many problem points as possible this time.

Personal finance has come up twice now, I hadn't thought of it but will go check out the services now, I bet I could find a use as well.

[–] [email protected] 5 points 1 year ago (2 children)

Wow that's a cool setup, I'll definitely steal some ideas.

I'm used to slinging lots of data around and one of the more helpful tools for general purpose automation has been n8n. Though it might have limited use if you're not trying to glue all kinds of services together. I also host actualbudget to keep track of finances. Both are running comfortably in their own little docker containers.

I'm currently looking into setting up Nextcloud and experimenting some more with presence detection for Home Assistant. I'm considering CO2 sensors, which will either tell me my home is ventilated properly, or which rooms are occupied.

[–] mauns 3 points 1 year ago

Thanks for the ideas! I hadn't thought about budget apps but that could be a good one.

Not sure if I have use cases for n8n or not, but I'll go learn about it and find out!

[–] [email protected] 3 points 1 year ago

That awesomebudget looks nice! I'm more of a beancont/fava guy, and too invested in my setup to try something vastly different - but it sure looks like a cool option for people starting out.

[–] [email protected] 5 points 1 year ago (2 children)

Moving to Caseta for lighting from the random mix of bulbs which never quite work was amazing. It's also much cheaper to put in one controllable switch than replace the 6 bulbs in the light fittings connected to the wall switch. Those bulbs always fail in weird and non-debuggable ways.

I use Crafty Controller (https://craftycontrol.com/) to manage the minecraft servers. It runs in a docker instance and gives you a nice web UI to manage each minecraft server. I use it to delegate control to my kids to create and manage servers as necessary.

Finally, if you're not using a config mgmt tool, I'd start looking, so you can make everything easily re-doable. Personally I'm using Ansible, but puppet, chef, salt, etc all work too. Ansible is easiest given it does need it's own infra. I like it so if something dies I can redeploy everything onto a different server.

[–] [email protected] 3 points 1 year ago (1 children)

I personally use Pterodactyl for my Minecraft servers because it's versatile enough that it can host any game server, not just Minecraft. It's pretty much guaranteed that any game you'll want to host will already have an install script someone has made for it.

[–] [email protected] 3 points 1 year ago

That’s cool. I’ve used crafty for years, but mainly because I outgrew my scripts to manage each server instance and my kids need instance responses to restarting servers. So I went looking for something Minecraft specific to give them restart and reset commands.

[–] mauns 3 points 1 year ago

This is awesome, I didn't know there were game server managers. I definitely see the point as I did some ugly file hacking a few times. I'll check out both Crafty Controller and Pterodactyl!

I've used Ansible for other stuff but never thought about it for server config, definitely going to figure that out too, been bitten there before as well.

Thank you!

[–] [email protected] 5 points 1 year ago (1 children)

For media streaming, what mattered most for me was client software. When I first started with Jellyfin, the Roku client wasn't as feature rich as the Android and web. I stayed with Plex until the Roku Jellyfin clients had what I needed.

If you're just on Android and the web, you should be fine with Jellyfin. If you've got a smart TV with it's own app store, you may look at what's there and what features are available.

[–] mauns 5 points 1 year ago

Thanks for the hint, it's a good point. I need to check that my TV app store has it, I could see that being the catch.

That being said, I have been learning that my TV's software is complete garbage and I may just go back to Android casting for TV control in general...

[–] [email protected] 4 points 1 year ago* (last edited 1 year ago) (1 children)

You don't have a logging stack? You should set one up, it helps greatly to see your servers long term snd short term status at a glance, especially if you hook it up with grafana to visualize. You can even use grafana with Home Assistant, and do some pretty crazy things with notifications, but i must say, it is a rabbit hole, especially grafana, and quite a large one at that.

I'm currently using grafana with loki and looking to set up notifications with ntfy but if you have the resources you could set up some other logging stacks like elk, or greylog but they're a bit resource hungry and you dont seem to have too many services set up to warrant a larger stack.

Anyway you have a really nice setup, good job!

Edit: typo

[–] mauns 2 points 1 year ago (1 children)

Thank you, and good suggestion! I installed grafana once last time and failed to get anything useful as I didn't invest enough time. Notifications sound great so I may look into the same combo you mentioned and see if I can wrangle it this time. Did you follow any particular guide when you started out with that? Otherwise I'll just dive into the google

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago) (1 children)

Sorry for the late reply. I followed a great tutorial on youtube by Techno Tim, it explains everything pretty well. It's a bit long, but thorough.

I used this docker-compose file as a base because it connects loki automatically, but you have to add the volumes manually and its not too hard to connect it manually. You can just use the one that Techno Tim uses in the video if you want less complexity in your compose files.

My docker compose file for reference:

version: "3"
services:
  loki:
    container_name: 'loki'
    image: grafana/loki:2.8.0
    ports:
      - "20110:3100"
    command: -config.file=/etc/loki/loki-config.yaml
    volumes: 
      - ./loki:/etc/loki
    networks:
      - loki

  promtail:
    image: grafana/promtail:2.8.0
    volumes:
      - /var/log:/var/log
      - ./promtail:/etc/promtail
    command: -config.file=/etc/promtail/promtail-config.yaml
    networks:
      - loki

  grafana:
    container_name: 'grafana'
    image: grafana/grafana-oss:latest
    environment:
      GF_PATHS_PROVISIONING: /etc/grafana/provisioning

      GF_AUTH_ANONYMOUS_ENABLED: false
#      GF_LOG_MODE: "console file" #optional, used if you want a log file
 #     GF_SERVER_ROOT_URL: "(Full url here eg. https://grafana.example.com") #optional, used for redirects
    entrypoint:
      - sh
      - -euc
      - |
        mkdir -p /etc/grafana/provisioning/datasources
        cat <<EOF > /etc/grafana/provisioning/datasources/ds.yaml
        apiVersion: 1
        datasources:
        - name: Loki
          type: loki
          access: proxy 
          orgId: 1
          url: http://loki:3100
          basicAuth: false
          isDefault: true
          version: 1
          editable: false
        EOF
        /run.sh
    volumes:
      - ./grafana/data:/var/lib/grafana
#      - ./grafana/logs:/var/log/grafana #log file
    ports:
      - "20100:3000"
    networks:
      - loki

networks:
  loki:
    name: loki
  frontend:
    external: true

Sorry if the formatting looks bad, since I'm on mobile. I use frontend as a network that includes containers that connect to nginx proxy manager.

If you have ARM then you might have problems, more info in the comment below

Edit: Better wording Edit2:typo, edited refrence for clarity and added arm warning

[–] mauns 2 points 1 year ago (1 children)

Not a problem, thanks!! I'm sure this will be a big help

[–] [email protected] 2 points 1 year ago

Oh and one last thing (i promise) is that if you are using or will use ARM for loki, you have to build the docker driver for loki from source.

A quick tutorial is here for anyone that needs it: Install go with sudo apt install go Then clone the github repo with git clone https://github.com/grafana/loki.git Then cd into it with cd loki Once you're inside the directory, do GOOS=linux/windows/whatever GOARCH=arm-version(eg. armv7,arm64) go build ./clients/cmd/docker-driver and wait for it to finish. The resulting file should (if i remember correclty) be called either loki or docker-driver. It's reccomended to do this on another machine then import it into your arm machine.

[–] Colin1224 4 points 1 year ago (1 children)

Should the wifi range extenders be represented as an extension of the APs?

[–] mauns 2 points 1 year ago

Good point, possibly - I haven't set one up before (new house has a wider range to cover) so I threw it in the diagram and didn't think further. I'll fix it up :)

[–] [email protected] 4 points 1 year ago (1 children)

Do you use home assistant?

https://home-assistant.io

It integrates with everything smart, gives you a central management dashboard and automation. You won't even need a Phillips hue bridge if you get a USB ZigBee stick.

[–] mauns 3 points 1 year ago

I do! I am hosting HAOS as one of my VMs, I love it. I'm planning an upgrade to have a wall mounted tablet with the dashboard this time.

That's a good point - I had Hue first, then got a ZigBee stick to add later devices, but left Hue alone because 'not broken/don't fix' - but this time, I would like to ditch the Hue hub and set up everything on the proper ZigBee network. Thanks for reminding me!

[–] [email protected] 3 points 1 year ago (1 children)

What tool did you use for that Diagram? It looks awesome!

[–] mauns 4 points 1 year ago

Thank you! It's draw.io

[–] [email protected] 3 points 1 year ago (1 children)

I see you have a container for a Space Engineers server, would you mind telling me what you are using? I would love to have a container for it instead of running torch on the desktop.

[–] mauns 2 points 1 year ago

Howdy! I have used this one in the past: mmmaxwwwell

[–] KevinKraft 2 points 1 year ago (1 children)

Looks awesome. Would you ever consider replacing the server with a cloud hosted solution? I know it kinda defeats the point of hosting it yourself, but wouldn't it be easier/cheaper/more flexible? Of course it could also be slower to transfer data.

[–] mauns 3 points 1 year ago

Fast data is priority 1, and learning how to host and administrate is priority 2, so I wouldn't consider it - but it's a fair question!

[–] [email protected] -5 points 1 year ago

Thanks, now I know exactly how to hack you.

load more comments
view more: next ›