this post was submitted on 21 Sep 2023
25 points (96.3% liked)

Selfhosted

40774 readers
1283 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
ruud
Loki
CannaVet
devve
HybridSarcasm
[email protected]
25
Qestions about eSATA vs USB3.0 for a drive enclosure connected to Proxmox (self.selfhosted)
submitted 1 year ago by AlphaAutist to c/selfhosted
11 comments fedilink hide all child comments
 

I am trying to figure out the optimal way to connect an 8 bay drive enclosure to a Dell Optiplex 7040 Micro. The end goal is to have the drives made available to a Proxmox cluster and kubernetes cluster. This is all for learning experience as well as to run services for personal use.

The cluster will be made up of 2x Optiplex 7040 and 2x Optiplex 3040. All have i7-6700t CPUs, the 3040s have 16GB DDR3 and 1TB SATA SSD each, and the 7040s each have 32GB DDR4 and 2TB NVMe drive with an additional empty SATA port on the motherboard. The enclosure is a MediaSonic ProBox with USB3.0 and eSATA interfaces available

I have heard that you shouldn’t use USB to connect to storage so I have been trying to figure out a way to use eSATA even though the Optiplex does not have an eSATA port. I found some SATA to eSATA cables on eBay, would that enable me to connect the enclosure directly to the free SATA port on the Optiplex?

Would this setup work? Is it worth it to sacrifice the additional SATA port on one of the 7040s in order to avoid using USB? I would like to maximize stability and speed.

I have not yet decided how I want to configure the drives but was planning to look into either a ZFS pool or ceph. All drives in the enclosure will be for media storage (movies/tv/music, was planning to keep pictures and documents elsewhere) and passed to LXCs and a kubernetes cluster I plan to run on Proxmox.

Any guidance on the connection setup, storage configuration, or my plans in general would be appreciated. Thanks in advance!

top 11 comments
sorted by: hot top controversial new old
[–] [email protected] 8 points 1 year ago (1 children)

8 drives over USB 3.0 won't perform well, though a single SATA connection isn't without issues either. That cable will be the I/O limiter in an uncommon fashion. I'd go for the eSATA converter option but neither is ideal. A big question is how will the drives be seen by Proxmox? If it's as one big drive that means you're SOL for safe storage options, if so I wouldn't store anything you don't want to lose on there and just make a ZFS storage pool of it in Proxmox.

As for plans I would down the line look at Ceph because it does really well with cheap hardware. No need for a competent RAID controller etc.

[–] AlphaAutist 2 points 1 year ago (1 children)

Ya I realize this isn’t a great way to go about storage but I already have the enclosure so I might as well use it for now. At some point down the line I will build something that will work better.

If I connect it using USB I am able to see each drive individually in Proxmox. I am unsure if it will be the same if I use eSATA. In the manual it says that the eSATA interface card needs to support Port Multiplier which I fear means the eSATA to SATA option may not work but I was hoping someone here may know more about that.

If I have to go the USB route and I am able to use each drive individually, would you recommend going with a ZFS pool or ceph?

[–] [email protected] 2 points 1 year ago (1 children)

I'd go with ZFS because balancing between the included disk in a Ceph pool will consume I/O and especially if one disk fails, that rebuild would take days if not weeks. If you want to try Ceph then don't include all the 8 drives initially and see how the performance is with the minimum 3 drives.

[–] AlphaAutist 1 points 1 year ago

Thanks, I may hold off on ceph for now in that case

[–] SheeEttin 4 points 1 year ago (1 children)

Hmm. I would probably use some kind of SATA to eSATA adapter for the least amount of purchasing.

But if you want to have small form factor compute nodes, I'd suggest replacing the dumb enclosure with a smarter (and faster) NAS or SAN. This way, you wouldn't be relying so much on janky hacks.

[–] AlphaAutist 2 points 1 year ago

Ok thanks and ya I plan to upgrade to something better suited for the job at some point. I just want to get started and use what I have as efficiently as I can.

[–] [email protected] 3 points 1 year ago* (last edited 1 year ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
NAS Network-Attached Storage
PCIe Peripheral Component Interconnect Express
RAID Redundant Array of Independent Disks for mass storage
SAN Storage Area Network
SATA Serial AT Attachment interface for mass storage

5 acronyms in this thread; the most compressed thread commented on today has 13 acronyms.

[Thread #154 for this sub, first seen 21st Sep 2023, 17:35] [FAQ] [Full list] [Contact] [Source code]

[–] doitforthelolz 3 points 1 year ago (1 children)

Honestly if you want to optimize for stability and speed I would ditch the MediaSonic Box and try to move the drives into the dell servers themselves. If the MediaSonic drives are 2.5 in it looks like the 7040s have an internal bay. For the 3040s it sounds like you are only using 1 of the 2 5.25in drive bays. I would say it's better to have 4 drives hooked up via pcie then 8 via eSATA. (got the drive data via dells website, might be different)

[–] AlphaAutist 2 points 1 year ago (1 children)

Unfortunately the drives in the enclosure are 3.5. I do have a spare SATA spot in each of the 7040s but you can only fit 1 SATA drive in the 3040s and no m2 drives. That’s why I am trying to decide whether it would be better to sacrifice a SATA port on one of the 7040s for (hopefully) better speeds and stability or use USB and put an extra drive in each of the 7040s

[–] doitforthelolz 3 points 1 year ago

I would put an extra drive in each of the 7040s regardless of whether you go with the eSATA adapter cable or USB. As you said this is for a learning experience I would try the adapter, worse case scenario you are out 5 bucks and have to use USB until you get a more permanent solution.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago)

it says on that mediasonic link

Important Note: • For eSata connection: Make sure your eSata port Support port multiplier. Most onboard eSata and some eSata PCI-E card only Support up to 5 drives. To see all 8 hard drives in eSata you need a eSata PCIe card that supports 8 drives.

I'm assuming the enclosure doesn't do any of the raid/array configuration, it just passes data through.

as far as I know only USB and eSata can do port multiplying. I think if you want to get access to all the drives you'll have to get a pcie card to handle the eSata or just use USB3. eSata (6gb/s) is faster than USB3 (5gb/s) and you might actually manage to saturate the connection trying to read or write to 8 drives though one cable.

in your use case both options are less than desirable but esata (if done correctly) could be faster. USB3 will probably be fine unless you really need that extra gb/s of speed

Edit: It looks like sata port multiplying can exist but its not really supported by manufacturers nor required by the standard so hit and miss as to whether a board can handle it.