Due to a corrupted database following the upgrade we’ve needed to roll back to a point before the upgrade happened.
I’m really sorry about this, but we’ve had to take this step so we can continue to properly run the site.
Anything added to the site in the last 12 hours will have been wiped, i.e. posts, communities. We’re looking at ways to make sure this doesn’t happen in future.
Happy to answer any questions below.
All good, thanks for your efforts
FYI the discussion under the upgrade announcement included comments about a minor bug affecting Voyager web app and Memmy users where we could comment using logged in accounts but trying to access our account details resulted in a ‘user doesn’t exist’ type error
No errors again after the rollback
Thank you - appreciate the patience too.
I think that was probably related to the db getting corrupted. Something went seriously wrong with the migration but it seems unique to this instance, and replicating it without using live data is almost impossible unfortunately.
deleted by creator
Soon, once we know what the issue is and how to fix it.
Shout-out to backups
deleted by creator
Which community? Your above comment has federated to lemmy.world for example, so it appears to be working at the moment.
Edit: its quite possible that it does cause issues btw, yes, which is why I dont like to do it unless there are no other options
No worries. This kind of stuff is bound to happen while building the foundations for something this intensive. Take your time and get things right. Anything that can be done to prevent loss of data like this should be the biggest priority, imo.
Yeah I agree, we’re having discussions on changing infrastructure so we can spin up clones of the site to do testing on before pushing to live, but this would be too big/intensive to host on the VPS we’re on so looking at alternative dedicated servers which would probably be better value than spinning up two VPS instances, which is what we have to do at the moment.
No worries!
Btw, is it intentional that you are using the lemmy icon with the ears and whiskers in front instead of the fixed one? :D
Haha I just thought it looked cooler to me, makes it stand out more.
The federation propagation seems bugged now, posts and comments from here aren’t properly propagating onto other instances! I can’t see my two recent polls about favorite Pokémon on the other forums. How long is it until this is fixed?
I can see a post from you if i go to https://lemmy.ml/c/pokemon from about an hour ago - is this the one?
I can currently see it on lemmy.world
The posts seem to be propagating now…
There is definitely an issue, we can see it in the logs, but I think its slowing down federation.
It seems to be catching up… the missing Pokémon posts I made are starting to appear on Lemmy.world.