this post was submitted on 28 Jun 2023
1867 points (99.1% liked)

Lemmy.World Announcements

28940 readers
5 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages πŸ”₯

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to [email protected] e-mail.

Report contact

Donations πŸ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 1 year ago
MODERATORS
 

We've upgraded lemmy.world to 0.18.1-rc.1 and rolled back that upgrade because of issues.

(If you had posted anything in those 10 minutes between upgrade and rollback, that post is gone. Sorry!)

The main issue we saw is that users can't login anymore. Existing sessions still worked, but new logins failed (from macos, ios and android. From linux and windows it worked)

Also new account creation didn't work.

I'll create an issue for the devs and retry once it's fixed.

Edit Contacted the devs, they tell me to try again with lemmy-ui at version 0.18.0. Will try again, brace for some downtime!

Edit 2 So we upgraded again, and it seemed to work nicely! But then it slowed down so much it was unuseable. There were many locks in the database. People reported many JSON errors. Sorry, we won't be on 0.18.1 any time soon I'm afraid..

top 50 comments
sorted by: hot top controversial new old
[–] MrPear 439 points 1 year ago (1 children)

Thank you for the transparancy!

[–] GatoB 48 points 1 year ago

I love post like this, another reason of why this instance is so good

[–] morelikepinniped 218 points 1 year ago

Thanks for all your hard work @[email protected]. It's not easy!

[–] [email protected] 104 points 1 year ago (2 children)

Both Dessalines and Nutomic have been working their butts off to get 0.18.x ready for the Reddit API changes. Huge hopes they can pull through!

Dessalines:

Nutomic:

[–] [email protected] 55 points 1 year ago (2 children)

Yeah they've been hard at work all month. But it's also okay if things aren't ready in time. Most of the people who matter are already here.

Maybe we will blow up soon, maybe later, but the quality of content here is sufficient to drive growth regardless of whether or not we get the prophesized huge migrations from reddit

load more comments (2 replies)
load more comments (1 replies)
[–] ulu_mulu 100 points 1 year ago (1 children)

is that users can’t login anymore

Ouch, that's bad :(

Thank you for your work, as always :)

[–] LazaroFilm 39 points 1 year ago

I mean that’s a Tuesday on Reddit lmao.

[–] rrobin 73 points 1 year ago (2 children)

As any engineer who does ops can tell you - you did the right thing - the solution is always to roll back, never force a roll forward, ever.

We should totally do pre and post update parties though. Even if the update fails we can have an excuse for drinks and a fun thread.

load more comments (2 replies)
[–] slimerancher 63 points 1 year ago

Oh. Good luck with the next try!

[–] itadakimasu 39 points 1 year ago

Thanks for the hard work /u/ruud, we'll get there

[–] Shadywack 35 points 1 year ago

Running one of the largest and most vibrant Lemmy instances certainly makes this deployment a great test case for the devs and the system overall. @[email protected] this community really appreciates you!

[–] ekZepp 33 points 1 year ago (2 children)
load more comments (2 replies)
[–] BlueN1te 32 points 1 year ago (3 children)

pretty brutal bug ngl. was hoping I'd be using jerboa again soon

load more comments (3 replies)
[–] Nintendo 28 points 1 year ago (1 children)

appreciate the transparency. how are things looking in the back in lemmy.world (server wise)? will we get to a point where it wont require complete rollbacks on the state when a botched update gets rolled out?

load more comments (1 replies)
[–] ilex 26 points 1 year ago (4 children)

I want to upvote, but I get this instead.

[–] antik 7 points 1 year ago* (last edited 1 year ago)

Can you clear your browser cache? Should be ok after

[–] kurwa 18 points 1 year ago

Oh man, for a sec there you made me thing I was getting errors πŸ˜…

load more comments (2 replies)
[–] Guy_Fieris_Hair 25 points 1 year ago (14 children)

Suddenly my Jerboa won't change from local and I can't change the sort. Not sure if this is related, but it was working fine yesterday after I updated the app. But no longer this morning.

load more comments (14 replies)
[–] TeaHands 23 points 1 year ago (4 children)

Was worth a try! But yeah that was pretty bad 😬

load more comments (4 replies)
[–] Sterben 23 points 1 year ago* (last edited 1 year ago) (1 children)

I am glad I didn't write my last post during the upgrade process.

Maybe next time, give a warning, or maintenance notice.

Thanks for trying though.

load more comments (1 replies)
[–] pampoon 22 points 1 year ago (1 children)

I’ve found from hopping around some other instances that have upgraded to 0.18 that it is still pretty buggy. It does seem to be giving more information about the errors, instead of just failing like in 0.17, but spend any time browsing on those instances and you’re bound to be inundated with JSON and query errors. It also seems to get worse the longer you browse.

The UI changes are nice, and I do appreciate not having my feed auto-updating constantly, but I think you’d be making the right choice to hold off on upgrading until they can iron 0.18.1 out all the way. I’m not super knowledgeable about TS and Rust, but as a user it seems that switching from WebSocket created/shined a light on Lemmy’s issues with caching in general.

load more comments (1 replies)
[–] [email protected] 20 points 1 year ago* (last edited 1 year ago) (4 children)

Im already in lemmy 0.19.0

this comment is from the future

load more comments (4 replies)
[–] naneek 19 points 1 year ago* (last edited 1 year ago) (2 children)

Thanks for the transparency. Maybe it’s a good idea to have a test instance and some test cases/validation done there before updating the main instance. This is a regular process in any software/tech company/stack.

Testing should never be done directly on the prod instance.

load more comments (2 replies)
[–] ren 19 points 1 year ago (1 children)

Good luck! May the software gods bless you with a functioning website with minimal bugs!

load more comments (1 replies)
[–] Buffalox 18 points 1 year ago

Linux/Firefox After upgrade many things didn't work, but they continued not to work after rollback.

Turns out I needed to clear the cache, then everything worked fine again.

[–] [email protected] 17 points 1 year ago

Dude I posted the funniest comment in those 10 minutes but you all must have missed it. Oh well πŸ˜…

[–] SeaJaye 16 points 1 year ago (26 children)

I'm so new, I'm a baby, I have no idea what I'm doing. I did just create my account, I tried using the app but it wouldn't let me. Saying server version too low. I'm sorry for my ignorance, but is this what that means? And this means we can't use the app yet right?

Also, thank you for all of this. I will be here for it all, I'm staying forever! I already love this so much more than anything else πŸ₯°

load more comments (26 replies)
[–] snargledorf 16 points 1 year ago* (last edited 1 year ago) (3 children)

~~RC2~~ RC4 was just released, here's hoping it solves the issues you were experiencing with performance of RC1!

load more comments (3 replies)
[–] DRx 16 points 1 year ago

Damn, Im glad you tried, and thanks for the update! I just really want the front page fixed without changing to page 0 lol. That and the 3/6/12 hour filter option.

Keep us updated. Im bouncing back and forth between LW, KBIN, and .ml for now though

any recommendations on other instances I should try outside the lemmy-verse and kbin-verse?

[–] MarsAgainstVenus 16 points 1 year ago

It seems that my Hot feed has been stuck for the last day or two.

load more comments
view more: next β€Ί