Sysadmin

Went to do a test restore of one of my databases and I noticed the dump files over the last few months were all 0kb. Glad I caught it this way and not because I needed to restore. Put it on your calendar, schedule a test restore of your critical stuff a couple times a year. I know y'all are busy but it is worth the time and effort. A backup you can't actually restore isn't a backup at all.

cross-posted from: https://gregtech.eu/post/5084911

Essentially, I'd like to have pictrs delete all of the images that aren't uploaded by my users, because my storage usage was going through the roof, so I just disabled the proxying of images. Here is my config:

x-logging: &default-logging
  driver: "json-file"
  options:
    max-size: "50m"
    max-file: "4"

services:
  proxy:
    image: docker.io/library/nginx
    volumes:
      - ./nginx_internal.conf:/etc/nginx/nginx.conf:ro,Z
      - ./proxy_params:/etc/nginx/proxy_params:ro,Z
    restart: always
    logging: *default-logging
    depends_on:
      - pictrs
      - lemmy-ui
    labels:
      - traefik.enable=true
      - traefik.http.routers.http-lemmy.entryPoints=http
      - traefik.http.routers.http-lemmy.rule=Host(`gregtech.eu`)
      - traefik.http.middlewares.https_redirect.redirectscheme.scheme=https
      - traefik.http.middlewares.https_redirect.redirectscheme.permanent=true
      - traefik.http.routers.http-lemmy.middlewares=https_redirect
      - traefik.http.routers.https-lemmy.entryPoints=https
      - traefik.http.routers.https-lemmy.rule=Host(`gregtech.eu`)
      - traefik.http.routers.https-lemmy.service=lemmy
      - traefik.http.routers.https-lemmy.tls=true
      - traefik.http.services.lemmy.loadbalancer.server.port=8536
      - traefik.http.routers.https-lemmy.tls.certResolver=le-ssl


  lemmy:
    image: dessalines/lemmy:0.19.8
    hostname: lemmy
    restart: always
    logging: *default-logging
    volumes:
      - ./lemmy.hjson:/config/config.hjson:Z
    depends_on:
      - postgres
      - pictrs
    networks:
      - default
      - database

  lemmy-ui:
    image: ghcr.io/xyphyn/photon:latest
    restart: always
    logging: *default-logging
    environment:
      - PUBLIC_INSTANCE_URL=gregtech.eu
      - PUBLIC_MIGRATE_COOKIE=true
#      - PUBLIC_SSR_ENABLED=true
      - PUBLIC_DEFAULT_FEED=All
      - PUBLIC_DEFAULT_FEED_SORT=Hot
      - PUBLIC_DEFAULT_COMMENT_SORT=Top
      - PUBLIC_LOCK_TO_INSTANCE=false



  pictrs:
    image: docker.io/asonix/pictrs:0.5
    # this needs to match the pictrs url in lemmy.hjson
    hostname: pictrs
    # we can set options to pictrs like this, here we set max. image size and forced format for conversion
    # entrypoint: /sbin/tini -- /usr/local/bin/pict-rs -p /mnt -m 4 --image-format webp
    #entrypoint: /sbin/tini -- /usr/local/bin/pict-rs run  --max-file-count 10  --media-max-file-size 500 --media-retention-proxy 10d --media-retention-variants 10d  filesystem sled -p /mnt
    user: 991:991
    environment:
      - PICTRS__STORE__TYPE=object_storage
      - PICTRS__STORE__ENDPOINT=https://s3.eu-central-003.backblazeb2.com/
      - PICTRS__STORE__BUCKET_NAME=gregtech-lemmy
      - PICTRS__STORE__REGION=eu-central
      - PICTRS__STORE__USE_PATH_STYLE=false
      - PICTRS__STORE__ACCESS_KEY=redacted
      - PICTRS__STORE__SECRET_KEY=redacted
      - PICTRS__MEDIA__RETENTION__VARIANTS=0d
      - PICTRS__MEDIA__RETENTION__PROXY=0d
      - PICTRS__SERVER__API_KEY=redacted_api_key
      #- PICTRS__MEDIA__IMAGE__FORMAT=webp
      #- PICTRS__MEDIA__IMAGE__QUALITY__WEBP=50
      #- PICTRS__MEDIA__ANIMATION__QUALITY=50
    volumes:
      - ./volumes/pictrs:/mnt:Z
    restart: always
    logging: *default-logging

  postgres:
    image: docker.io/postgres:16-alpine
    hostname: postgres
    volumes:
      - ./volumes/postgres:/var/lib/postgresql/data:Z
      #- ./customPostgresql.conf:/etc/postgresql.conf:Z
    restart: always
    #command: postgres -c config_file=/etc/postgresql.conf
    shm_size: 256M
    logging: *default-logging
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_USER=lemmy
      - POSTGRES_DB=lemmy
    networks:
      - database
  postfix:
    image: docker.io/mwader/postfix-relay
    restart: "always"
    logging: *default-logging

networks:
  default:
    name: traefik_access
    external: true
  database:

The storage usage is at 340GB currently, which is a lot and it's rapidly increasing. I use Backblaze B2 for my storage. Here is my docker compose file: x-logging: &default-logging driver: "json-file" options: max-size: "50m" max-file: "4"

services:
  proxy:
    image: docker.io/library/nginx
    volumes:
      - ./nginx_internal.conf:/etc/nginx/nginx.conf:ro,Z
      - ./proxy_params:/etc/nginx/proxy_params:ro,Z
    restart: always
    logging: *default-logging
    depends_on:
      - pictrs
      - lemmy-ui
    labels:
      - traefik.enable=true
      - traefik.http.routers.http-lemmy.entryPoints=http
      - traefik.http.routers.http-lemmy.rule=Host(`gregtech.eu`)
      - traefik.http.middlewares.https_redirect.redirectscheme.scheme=https
      - traefik.http.middlewares.https_redirect.redirectscheme.permanent=true
      - traefik.http.routers.http-lemmy.middlewares=https_redirect
      - traefik.http.routers.https-lemmy.entryPoints=https
      - traefik.http.routers.https-lemmy.rule=Host(`gregtech.eu`)
      - traefik.http.routers.https-lemmy.service=lemmy
      - traefik.http.routers.https-lemmy.tls=true
      - traefik.http.services.lemmy.loadbalancer.server.port=8536
      - traefik.http.routers.https-lemmy.tls.certResolver=le-ssl


  lemmy:
    image: dessalines/lemmy:0.19.8
    hostname: lemmy
    restart: always
    logging: *default-logging
    volumes:
      - ./lemmy.hjson:/config/config.hjson:Z
    depends_on:
      - postgres
      - pictrs
    networks:
      - default
      - database

  lemmy-ui:
    image: dessalines/lemmy-ui:0.19.8
    volumes:
      - ./volumes/lemmy-ui/extra_themes:/app/extra_themes:Z
    depends_on:
      - lemmy
    restart: always
    logging: *default-logging
    environment:
      - LEMMY_UI_LEMMY_INTERNAL_HOST=lemmy:8536
      - LEMMY_UI_LEMMY_EXTERNAL_HOST=gregtech.eu
      - LEMMY_UI_HTTPS=true

  pictrs:
    image: docker.io/asonix/pictrs:0.5
    # this needs to match the pictrs url in lemmy.hjson
    hostname: pictrs
    # we can set options to pictrs like this, here we set max. image size and forced format for conversion
    # entrypoint: /sbin/tini -- /usr/local/bin/pict-rs -p /mnt -m 4 --image-format webp
    #entrypoint: /sbin/tini -- /usr/local/bin/pict-rs run  --max-file-count 10  --media-max-file-size 500 --media-retention-proxy 10d --media-retention-variants 10d  filesystem sled -p /mnt
    user: 991:991
    environment:
      - PICTRS__STORE__TYPE=object_storage
      - PICTRS__STORE__ENDPOINT=https://s3.eu-central-003.backblazeb2.com/
      - PICTRS__STORE__BUCKET_NAME=gregtech-lemmy
      - PICTRS__STORE__REGION=eu-central
      - PICTRS__STORE__USE_PATH_STYLE=false
      - PICTRS__STORE__ACCESS_KEY=redacted
      - PICTRS__STORE__SECRET_KEY=redacted
      - MEDIA__RETENTION__VARIANTS=4d
      - MEDIA__RETENTION__PROXY=4d
      #- PICTRS__MEDIA__IMAGE__FORMAT=webp
      #- PICTRS__MEDIA__IMAGE__QUALITY__WEBP=50
      #- PICTRS__MEDIA__ANIMATION__QUALITY=50
    volumes:
      - ./volumes/pictrs:/mnt:Z
    restart: always
    logging: *default-logging

  postgres:
    image: docker.io/postgres:16-alpine
    hostname: postgres
    volumes:
      - ./volumes/postgres:/var/lib/postgresql/data:Z
      #- ./customPostgresql.conf:/etc/postgresql.conf:Z
    restart: always
    #command: postgres -c config_file=/etc/postgresql.conf
    shm_size: 256M
    logging: *default-logging
    environment:
      - POSTGRES_PASSWORD=password
      - POSTGRES_USER=lemmy
      - POSTGRES_DB=lemmy
    networks:
      - database
  postfix:
    image: docker.io/mwader/postfix-relay
    restart: "always"
    logging: *default-logging

  #pictrs-safety:
  #  image: ghcr.io/db0/pictrs-safety:v1.2.2
  #  hostname: pictrs-safety
  #  environment:
  #  ports:
  #    - "14051:14051"
  #  user: 991:991
  #  restart: always
  #  logging: *default-logging
  #  depends_on:
  #    - pictrs
networks:
  default:
    name: traefik_access
    external: true
  database:

cross-posted from: https://lemmy.world/post/22872422

Screenshot of a Twitter post by user JonErlichman

Average cost for 1 gigabyte of storage:

45 years ago: $438,000 40 years ago: $238,000 35 years ago: $48,720 30 years ago: $5,152 25 years ago: $455 20 years ago: $5 15 years ago: $0.55 10 years ago: $0.05 5 years ago: $0.03 Today: $0.01

Windows Server 2022 creates one recovery partition just on the right of the C: partition. So, when it is required to expand the C: partition it is impossible due to this recovery partition. I realised of this problem because our IT department provides Windows Server virtual machines users are unable to expand.

I would like to know how are you dealing with this problem. Do you remove the recovery partition? Do you keep the recovery partition? how?

Found here.

If you're in the U.S. anyway.

Recently Microsoft released the link 365 which is basically a thin client for Azure. You can't run anything locally nor is there any local files. It literally just connects you to a desktop elsewhere.

Do you think this is what Windows 12 might look like? I feel like this idea is not practical for average consumers. Maybe they will make something that's like Chrome OS?

It's just a fad. It'll pass.

Spent the last 3 months getting requirements for computer upgrades. After that picked out some decent laptops. (Thinkpad L and T series)

Nothing fancy, but I'm just tired of diagnosing problems with previous sysadmin purchased Vostro laptops.

After getting quotes from multiple vendors, finally got everything and sent the CEO to confirm. Guess fucking what... It got fucking denied.

"Look for cheaper laptops and replace only whats critical"

Employees are rocking 7 year old laptops with 128G SSDs! The bloody things can't even run Win 11! The whole upgrade costs less than their single "teambuilding"! I hate this these cheapskates so fucking much...

I was literally told to set up this new service as quickly as possible and it didn't need to be correct or best practice because this was just a proof of concept.

Here we are 6 months later and I'm still cleaning up my own mess.

Let me tell you about the hell that is Roku. They create so much wireless interference and I can't just get rid of them.

The problem stems from WiFi direct. They automatically scan for the busiest channel and then broadcast full strength on that channel. I don't know why they do this but it creates a crazy amount of interference.

And before you ask, no you can't turn WiFi direct off. Also the remote uses WiFi direct for some reason.

WHY, ROKU WHY!

What are you guys working on? Anything new in the world

Doing a PhD in humanities and enjoy it. I’ve recently really started to enjoy Linux, self hosting, and messing around with various lab stuff.

[email protected]

This is a Chinese attack that targets Azure

Tick tick

Anyone here have any experience with a Datto Backup Appliance?

I have just been told that they've never run a full restoration in the six years that it's been in service, deployed for the backup of four mission critical virtual Windows Servers, four Windows Workstation and a (physical?) Linux PABX server.

The actual appliance is apparently a "Datto S3-2000 BCDR"

Edit: The anal retentive in me is going WTF in a tight loop. The industry professional with 40 years experience in the field is going, different day, same old...

I realised that I didn't actually ask the pertinent question, the hamster wheel was running full tilt, but is this normal, or is this WTF, or somewhere in-between?

As you all might be aware VMware is hiking prices again. (Surprise to no one)

Right now Hyper-V seems to be the most popular choice and Proxmox appears to be the runner up. Hyper-V is probably the best for Windows shops but my concern is that it will just become Azure tied at some point. I could be wrong but somehow I don't trust Microsoft to not screw everyone over. They already deprecated WSUS which is a pretty popular tool for Windows environments.

Proxmox seems to be a great alternative that many people are jumping on. It is still missing some bigger features but things like the data center manager are in the pipeline. However, I think many people (especially VMware admins) are fundamentally misunderstanding it.

Proxmox is not that unique and is built on Foss. You could probably put together a Proxmox like system without completely being over your head. It is just KVM libvirt/qemu and corosync along with some other stuff like ZFS.

What Proxmox does provide is convenience and reliability. It takes time to make a system and you are responsible when things go wrong. Doing the DIY method is a good exercise but not something you want to run in prod unless you have the proper staff and skillset.

And there is where the problem lies. There are companies are coming from a Windows/point in click background who don't have staff that understand Linux. Proxmox is just Debian under the hood so it is vulnerable to all the same issues. You can install updates with the GUI but if you don't understand how Linux packaging works you may end up with a situation where you blow off your own foot. Same goes for networking and filesystems. To effectively maintain a Proxmox environment you need expertise. Proxmox makes it very easy to switch to cowboy mode and break the system. It is very flexible but you must be very wary of making changes to the hypervisor as that's the foundation for everything else.

I personally wish Proxmox would serious consider a immutable architecture. TrueNAS already does this and it would be nice to have a solid update system. They would do a stand alone OS image or they could use something based on OStree. Maybe even build in a update manager that can update each node and check the health.

Just my thoughts

What is this thing?

That's sounds strange to say but hear me out. Before ransomeware there was no economic incentive for companies to worry about security. There was a strong "why would you hack us" vibe that made it hard to talk management into doing anything basic like locking down ports.

Nowadays everyone and there mom is worried about getting compromised. I've seen companies who historically didn't care at all about IT suddenly invest heavily in security. We are now much more secure than we were previously as everyone has suddenly realized that the internet had a huge risk. I doubt we will see any of the old style worms we had back in the day that would infect millions of machines.