Something bad happened over with SWGEMU today recently. They were quick to update to the forums and Facebook to let members know what was going on. No excuses, no twisting of the facts but just a simply notification of what was happening and how they intended to fix it. As many members of the community have said, a roll back is better than vendor loss. They worked hard and quickly to try to correct the issue. Here’s the update in case you missed it.
From SWGEMU forums:
Well something got messed up pretty bad. Unfortunately bunch of vendors, shuttleports and starports went poof. While we are investigating the issue, we are making backups and trying to find the underlying cause. At the moment the worst case scenario would be a rollback. If that happens, and we are hoping it wont have to come to that, all characters created in the last few hours, all items looted or crafted, all XP gained, all skill boxes trained, pretty much everything you’ve done after the last backup, will no longer exist in the database, and for all intents and purposes will be gone.
Rest assured that we are trying to resolve this as painlessly as possible, and we will do everything in our power to avoid item or character loss.
Backing up the server takes approximately 90 minutes and a reboot will take a few hours on top of that, so we are looking at 4 hours downtime at least. Of course, in situations like this its very difficult to predict how long it might take and how many restarts might be needed.
Thank you for your patience and support.
~The SWGEmu Team
They did try their best to fix it without having to do a rollback but in the end, the rollback was needed. Vlada kept everyone regularly updated on the forums and they tried a little at a time, to avoid going back too far with the rollback.
Vlada explained the issues with going back too far:
Well, some vendors are still missing, so we are going to attempt one final rollback. We are going to push the server back as far as we can. Pushing it back beyond this point would start causing more problems instead of solving them. We are aware that some vendors might still missing even after this rollback, but doing another rollback after this one just isn’t a risk we are willing to take.
We apologize for this inconvenience and we hope we will still have your support after this unfortunate incident.
And then TheAnswer came in yesterday to explain why the problem was happening and what they were still trying to do about it:
The vendors and items are not deleted from the database, they are just failing to load at server startup. The problem is the info of exactly where the vendors were is probably lost, but the items are not.
Ill be analyzing the data more in depth for the next couple of days and come up with a solution to recover them, most likely spawn all the missing vendors in a common place where the owners can go and retrieve them.
EDIT: That is if everything goes as planned and there are no other corruptions in the data…
All in all, the situation was handled as well as can be expected. TheAnswer is still updating the thread as to what they are doing to try to correct it and how you can possibly help. It’s an ongoing issue at this time but most of the community who tool the time to respond were really friendly, reminding staff that it’s the holiday season and not to stress over it. It’s really nice to see that in a community. Way to go SWGEMU staff and community. Here’s hoping you get all those vendors back!
