Originally Posted by hdev_swc
The physical server, which was set up as a hypervisor for multiple servers, had a hard fault that caused the machine to be taken down without us being notified. The root cause was said to be a failed drive, though the hardware should have been fine to have a drive hot swapped in and that did not happen. Our host replaced the drive while the machine was down, which took much longer than expected. The event logs support the theory that the drives could have been imaged during this time. The timing of the outage was also very suspect.
Unlike the password fiasco from a couple years ago, this wasn't a break in attempt to steal funds; it appears to have been an attempt to get information about the operator of the server. There is still the possibility that it was a freak timing and hardware coincidence but given everything that was happening we couldn't seriously write off the events as coincidence. The reality is that it was becoming clear that no matter where your server is hosted, it is going to be under constant attack. If not from people trying to break in to steal, then by people trying to extort money from you, and with the NGCB actions, from governments.
We always put the security of player balances first and with everything going on around us, it was getting to the point that I didn't know if we could honestly live up to that.
far fucking out man. dat timing. its certainly suspicious.
thoughts:
- if the narrative being presented is 'oh the drive failed so we gotta hot swap', whether or not they mirrored the original would not have impacted that timeline... is there forensic evidence in like, dmesg indicating that a third drive was introduced to the system at some point that went away after a measure of time attributable to a full-drive dd?
- an obvious point perhaps but this is why we use raid + striping; grab a drive, you get nada.
- did the .ro provider have root on the hypervisor OS? if so, the keys to your kingdom are in those memory images and those get wiped when the machine loses power.. suggesting this wasnt an attempt to grab data or that the belligerent parties were fantastically incompetent. i mean, if youre going to compromise a clients host for profit, ok sure you might have to boot into single user mode, but why bother mirroring a drive when you can install a thin backdoor, wait a few hours for RAM to gather significant passwords, then log in via backdoor and bobs your uncle.
- wow
- gecmis olsun my friend. what a huge pain in the ass this must have been. hopefully your troubles are behind you.
Reply With Quote
