this post was submitted on 14 Jul 2023
5 points (77.8% liked)

Selfhosted

40763 readers
968 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

I'd like to swap my spinning disks with SSD drives. I have the new disks and they're just larger than the old ones. My configuration is a RAID-5 with 3 disks (and one hot spare). Can I hot swap a single disk (HDD to SSD), wait for the new disk to rebuild, then repeat?

I'm thinking that I'd mark down the hot spare, replace it with an SSD, mark the SSD as hot spare, mark HDD 1 as "bad" causing the hot spare to activate, then repeat for the other 2 HDDs. I don't have a lot of experience with RAID, but did perform a single disk swap once with success.

If this is a bad idea, why? What's the best way to upgrade?

I'm not sure if this is the right community for this question. If not, please guide me to the right one.

all 16 comments
sorted by: hot top controversial new old
[–] xebix 6 points 1 year ago* (last edited 1 year ago)

First thing, back up your data in case something goes south.

With that out of the way, I have done this exact thing many times with both zfs and an array managed by mdadm. Depending on the filesystem on top of your raid array, there may be some additional commands you'll need to run to extend your partition to use all of the available space in your larger array. You do have to wait between each disk swap to wait for the array to rebuild so it might be quicker to just make a backup, remove the old drives, and build a new array with your new drives and then copy the data back over. That being said, it is fun to hot swap all of those disks and not lose any data or have downtime assuming everything goes smoothly.

[–] SGG 5 points 1 year ago

Yes, it's a bad idea to do it this way. The most likely time a RAID array will fail is during a rebuild as that is a whole bunch of drive activity over a sustained timeframe.

Better to perform a backup or copy, power down, remove all the old drives, install the new ones, power back up, configure a new array (most people recommend to use RAID 6 at a minimum, no hot spare, so you have two drive redundancy) then restore or copy back the data.

This way you can also keep the old drives as a cold backup of sorts, potentially reimporting the configuration if needed.

[–] dktr 3 points 1 year ago

You might also want to check that your controller really supports hot-swapping. Not all do.

[–] [email protected] 3 points 1 year ago

The safest way is to make a backup and restore it to the new array.

Your way sounds pretty fun though. I'm sure the firmware will complain, but once you have a solid backup you can go nuts!

Also consider a solution such as Bachefs - RAID is cool and all, but this will get you most of the way there too. (Decide on the data safety for yourself but I'm running a 90tb pool with 24tb of metadata / cache on SSD)

[–] [email protected] 3 points 1 year ago (1 children)

Do you really need the RAID online all the time? Because if you can afford to shut it down for a few hours, it is way less work to do a backup, and then build a new RAID with your SSDs.

I'm not sure if the RAID controller will like two different kind of drives. I'd check the docs if it says something.

[–] Anonymouse 1 points 1 year ago

I don't really need it online all the time, but I don't expect that I'll find time to do it all at once and I thought swapping may be a way to break up the job into interruptible segments.

[–] [email protected] 2 points 1 year ago

The "Right answer" is copy all the content to another device, upgrade disks, copy content back to new array that can take advantage of the larger disks. Or even safer, set up a completely new device in parallel and then copy data over, decommission the old device.

Some RAID systems will let you do what you say, but will not let you expand the usable storage space. Some will have particular disks they'll even take. It really depends on the RAID system.

[–] [email protected] 2 points 1 year ago

You've got the right community IMHO. This is something that I've never tackled but I could imagine that it would work. Just make certain you have solid backups just in case the worst should happen.

[–] Ghostbusterinthemach 2 points 1 year ago

The answer to this is going to depend on your raid controller. Frequently the answer is yes, but some raid controllers are more particular about exactly what drives are in the array. Software raid controllers are typically more forgiving of this type of change.

[–] RunningInRVA 2 points 1 year ago (1 children)

Consider moving to RAID-6. Single redundancy is not cool anymore.

[–] Anonymouse 1 points 1 year ago (1 children)

This is just a homelab, running on old hardware. I don't think the raid controller supports RAID-6 and I only got 3 new SSDs, and I read that RAID-6 needs 4 drives.

[–] RunningInRVA 3 points 1 year ago

Correct. Get a 4th drive. You will be thankful one day down the road when you are rebuilding the array and you lose a drive during the rebuild.

[–] [email protected] 2 points 1 year ago (1 children)

I have a feeling you'd end up with a bunch of big drives with small volumes on them if it did work.

Warning you, I've had issues with RAID combining SSD and HDD. Basically I was on an older dell server and I wanted to do mirroring and the bios straight up refused to do it because it didn't want to mix ssds and hdds.

[–] Anonymouse 1 points 1 year ago (1 children)

I suspect I'll have an issue after reading many of the comments. This is also an older Dell server. My only real advantage is that it's currently hosting 2 VMs and one is just a test server, so I don't mind losing the data.

[–] [email protected] 2 points 1 year ago

Imaging software is a godsend for that sort of thing. I ended up using BartPE for something like that, and it worked great -- it has a free imaging program on it. You only need a removable drive large enough for all your files since it'll compress everything.