this post was submitted on 14 Jun 2024
34 points (94.7% liked)

Selfhosted

40372 readers
415 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
ruud
Loki
CannaVet
devve
HybridSarcasm
[email protected]
34
Tools and ideas for backup of many files (feddit.nl)
submitted 5 months ago* (last edited 5 months ago) by [email protected] to c/selfhosted
17 comments fedilink hide all child comments
 

Hello everyone,

In a day or two, I am getting a motherboard with an N100 integrated CPU as a replacement to the Raspberry Pi 4 (2 GB Model). I want to run Jellyfin, the *arr stack and Immich on it. However, I have a lot of photos(for Immich) and movies(for Jellyfin) (in total about 400 GB) that I want to back up, just in case something happens. I have two 1TB drives, one will have the original files, and the second will be my boot drive and have the backup files.

How can I do that? Just copy the files? Do I need to compress them first? What tools do I need to use, and how would you do it?

Thanks in advance.

EDIT: I forgot to mention that I would prefer the backups to be local.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 6 points 5 months ago* (last edited 5 months ago) (1 children)

For local backups I use this command

$ rsync --update -ahr --no-i-r --info=progress2 /source /dest

You could first compress them, but since I have the space for the important stuff, this is the only command I need.

Recently I also made a migration similar to yours.

I've read jellyfin is hard to migrate, so I just reinstalled it and manually recreated the libraries, I didn't mind about the watch history and other stuff.
IIRC there's a post or github repo with a script to try to migrate jellyfin.

For immich you just have to copy this database files with the same command above and that's it (of course with the stack down, you don't want to copy db files while the database is running).
For the library I already had it in an external drive with a symlink, so I just had to mount it in the new machine and create a simlar symlink.

I don't run any *arr so I don't know how they'd be handled.
But I did do the migrarion of syncthing and duplicati.
For syncthing I just had to find the config path and I copied it with the same command above.
(You might need to run chown in the new machine).

For duplicati it was easier since it provides a way to export and import the configurations.

So depending on how the *arr programs handle their files it can be as easy as find their root directory and rsync it.
Maybe this could also be done for jellyfin.
Of course be sure to look for all config folders they need, some programs might split them into their working directory, into ~/.config, or ./.local, or /etc, or any other custom path.

EDIT: for jellyfin data, evaluate how hard to find is, it might be difficult, but if it's possible it doesn't require the same level of backups as your immich data, because immich normally holds data you created and can't be found anywhere else.

Most series I have them in just the main jellyfin drive.
But immich is backedup with 3-2-1, 3 copies of the data (I actually have 4), in at least 2 types of media (HDD and SSD), with 1 being offsite (rclone encrypted into e2 drive)

[–] [email protected] 1 points 5 months ago (1 children)

Thanks for responding. I actually don't have Immich yet on the Raspberry Pi, so it's the first time I will be installing it and then importing the photos. I don't actually care a lot about the migration, since I can just reconfigure the services. I want to ensure that if a drive fails, I can restore the data. I would try RAID, but I read that "RAID is not backup". Or I could just run the command you provided in a cronjob.

[–] [email protected] 2 points 5 months ago

In that case I'd recommen you use immich-go to upload them and still backup only immich instead of your original folder, since if something happens to your immich library you'd have to manually recreate it because immich doesn't update its db from the file system.
There was a discussion in github about worries of data being compressed in immich, but it was clarified the uploaded files are saved as they are and only copies are modified, so you can safely backup its library.

I'm not familiar with RAID, but yeah, I've also read its mostly about up time.

I'd also recommend you look at restic and duplocati.
Both are backup tools, restic is a CLI and duplocati is a service with an ui.
So if you want to create the crons go for restic.
Tho if you want to be able to read your backups manually maybe check how the data is stored, because I'm using duplicati and it saves it in files that need to be read by duplicati, I'm not sure if I could go and easily open them unlike the data copied with rsync.