this post was submitted on 18 Jan 2024
12 points (87.5% liked)

Selfhosted

40021 readers
913 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

Issue Description: I have been having this issue with my raspberry pi running dietPi where it seems to lock up and I cannot SSH / access any of the services on it. The interesting part is that the interface seems to be up and I can still ping it on the local network but shows no video output. usually I get about 3-6 days before the issue appears again.

I am not really sure where to start to diagnose this issue. Any help would be appreciated!

Things Tried:

Reduced operating temp by getting a fan. I want to say this improved the length in between this issue appearing but don't really have any hard evidence.

uninstalled unused services

Limited active torrents in QbitTorrent

EDIT: Small thing to mention is that the CPU load is usually really high - like not uncommon for the load to be between 8-10 but I have seen it as high as 24.

Temporary fix:

Power cycle - everything comes up again in less than a minute.

Raspberry Pi 3B v2

OS: DietPi

Services:

Lidarr

Radarr

Sonarr

Prowlarr

Qbittorrent

Mullvad VPN - WireGaurd

SSH

all 22 comments
sorted by: hot top controversial new old
[–] [email protected] 10 points 9 months ago (2 children)

@AverageGoob I have this issue with one of my hosts as well. It appears to be a problem with the micro SD card. Same card, different pi = same problem. I'm currently working around it with a watchdog but will need to replace the card soon.

Are you running your OS from USB or from a micro SD card?

[–] a_fancy_kiwi 4 points 9 months ago (3 children)

I’d bet $1 it’s the SD card. My 3B+ used to have the same problem. Been running pis off some sort of SSD ever since, no issues.

[–] [email protected] 3 points 9 months ago

Pi 3B has dedicated bus for SD card but ethernet and usb share bandwidth. Enable zram, disable all swap and keep using sd card.

[–] [email protected] 2 points 9 months ago* (last edited 9 months ago)

@a_fancy_kiwi I agree, same here. This is the last pi that's running off an SD card with services that do "significant" disk I/O. I have a few zeros that only really write to the card for OS updates. Their job is to collect data and send it via the network. I haven't had issues with that kind of workload using micro SD cards.

Edit: For Pis with write workloads I'm using basic USB3 SSDs. Didn't have good results with USB sticks though.

[–] AverageGoob 1 points 9 months ago (1 children)

Id be willing to try this. How do you have it connected? Just using an external USB attached one?

[–] a_fancy_kiwi 1 points 9 months ago* (last edited 9 months ago) (1 children)

I upgraded to the Pi4 but I use this case. It has a daughter board that lets me use an m.2 SATA SSD over USB. But any USB to SATA adapter should work fine

[–] [email protected] 1 points 9 months ago

You should get a scsi enabled adapter though, otherwise you may have to disable it in the kernel boot settings. And if you forget that it will run at like kbit/s.

[–] AverageGoob 1 points 9 months ago (1 children)

I am running it from an SD card. Did setting up the watchdog ultimately work for you? I did come across a watchdog as a possible workaround.

[–] [email protected] 2 points 9 months ago

@AverageGoob The watchdog saves me from rebooting the host manually, but at the risk of data loss (though not more than a locked up SD card). I configured a custom script that writes to a file, when the card has problems, the watchdog kicks in. To keep the script from stressing the card even more, the script only writes to the file every few minutes.
As you said it's only a workaround. I'll move the stuff on the problematic host to a VM with SSD shortly.

[–] [email protected] 3 points 9 months ago (1 children)

Not sure if the rpi3 can use the 64bit version, or if it's possible for it use an SSD like the the 4 can?

[–] [email protected] 1 points 9 months ago

Pi 3b+ can you just need to enable it, look it up. Tried and tested.

[–] [email protected] 3 points 9 months ago (1 children)

Yeah, I had the same issue. Sometimes it was the SD card, sometimes the network interface (not your case obviously), sometimes things connected to USB, sometimes it was running hot... I gave up and now I just run everything on an older Slimbook Zero. Yes, power consumption is higher (still pretty low) but so is stability.

[–] AverageGoob 1 points 9 months ago

Yeah it seems a lot of people are saying the SD card is the issue which wouldn't surprise me. I do have some spare space on my proxmox server but it would just be a huge pain in the dick to move everything... But it's looking like I may need to sadly.

[–] v81 2 points 9 months ago

Have had this issue myself asking with other DD card related issues.

I can't understand why the pi foundation persist with using SD as the only physically practical storage option.

They're looking post the point of needing a way to snap on reliable EMMC storage, as a default, in a way that doesn't leave a cable or something permanently plugged into a USB port.

Sure, USB is a fine option, but I hate that it's only an option and not a designed default.

Most of us only need 8GB or so for the OS, 8GB or good quality durable EMMC should hardly cost anything.

Other tiny computers and even economy notebooks and Chromebooks already use this.

[–] [email protected] 2 points 9 months ago (1 children)

@[email protected] Consider running something on it to monitor temps/memory/etc over time and push them to a separate device as it does so - that way you can measure things leading up to a crash, and know when it crashes (due to it suddenly stopping its reporting)

[–] AverageGoob 1 points 9 months ago

That is a good idea! Do you have any recommendations for remote logging?

[–] [email protected] 2 points 9 months ago (1 children)

This is likely an issue with your SD card...Pi's eat them up, even the good ones that supposedly can handle it get chewed up pretty quickly. It really sucks that they haven't transitioned to emmc but insist on using SD cards.

[–] AverageGoob 1 points 9 months ago

Yeah seems like a weird choice to have a default storage type that is known in the community to be unreliable.

[–] [email protected] 1 points 9 months ago* (last edited 9 months ago) (1 children)

I've had this happen when I had too many USB devices plugged into it. It was having power underrun, and acting unresponsive while trying to compensate. I solved it with a powered USB hub.

Edit: I've had pairing it with an off brand power brick cause the same problem, too. Apparently the 3 and later Pi really want better power quality regulation, and some of the cheapo bricks I had lying around - while providing the right Volts and Amps, didn't control the variation well enough for the modern Pi computer.

[–] AverageGoob 2 points 9 months ago

That's the weird part is that I don't have any USB devices attached. I have Ethernet, power cable, and the fan on the case has pins going to some headers.

The case did come with another power supply so maybe I'll try that and see if anything changes.

[–] [email protected] 1 points 9 months ago

Consider using a USB3 SSD as your boot drive if you want long term usage from your pi. The SD card is prone to failure relatively quickly on Raspian and is even worse on OSes that aren't optimized for the PI directly.