I have made a few tweaks and turned federation back on.
Images will be down for a few days while I keep working on this
this post was submitted on 19 Jun 2023
2 points (100.0% liked)
Weekly Explosions Blog
3 readers
1 users here now
Weekly Explosions Blog
This is where we blog about the articles and discussions that have been active this week.
founded 2 years ago
MODERATORS
I did notice the website going in and out all night.
Also the number of federated posts just jumped big time. We were at something like Ninety thousand total just a couple weeks ago and now we're at over 125,000 new OPs.
Yeah I have been up working on it since 2am this morning.
Not much to show for my efforts yet sadly.
Onwards and upwards we go.
My posting had slowed down because of it but hey it's a marathon, not a sprint, and it seems to be back on track now.
The server is running a lot smoother today compared to yesterday. I have been doing a lot of small things tonight.
I do truly appreciate all of this hard work considering how difficult the situation must be with the sudden influx in the Fediverse and all the moving parts that must go with this.
Good chance it's just increased usage from all the new federated users. Sounds like the lemmy devs have a plan to optimize things. There may be some low hanging fruit optimization we can look into doing with the postgres config. I'll try to find some time into reading the current default configs.
"You can use the PGTune tool to tune your postgres to meet your server memory and CPU."
from https://github.com/LemmyNet/lemmy-ansible
It was configured to a much larger machine. So I plugged in the specs for my linode and updated it.
I also upgraded to V17.4 even though that breaks images for some reason.
I figured better to be up and running smoothly with no images, then up and down all the time.
I will watch the server today and see if it performs better and let you know.
Fingers crossed.
Good news, these tweaks have fixed the cpu spikes and the Disk i/o spikes.
You should be seeing better performance.
Still researching what is causing the images not to display.
awww, hope things get better soon.
Me too.
V16 was very very stable for us.
V17 has been a disaster. This could in part be caused by active Lemmyverse users increasing by a factor of 20 in the last month. It could also be some of the new instances we federated with trying some funky stuff.
In any case I know the devs are working very hard. I have never seen do many different people contributing to the development effort.
So despite these scaling problems, the future looks bright.
Bring on V18
Thanks for the update