Broken Forum vanished

Never mind - I see a maintenance message now.

Apparently been hacked, but I don’t know any more than that.

No, it wasn’t me.

OH NO! MY LIKE RATIO!!

Maybe the hacker wasn’t getting enough likes, and decided to go all Kobayashi Maru on the problem.

Dean said on Twitter that Bahimiron said that Lum said it was Russian hackers rooting the server. Probably just for fun and training…

lol, that was actually my first thought too. sure enough, it’s back up and instead of being almost exactly even, I’m 100 likes down! the horror!

According to Lum’s server message, he had to restore a backup from last year. That’s before my account even existed!

Alas, all those moments lost, like…teardrops in the rain…

They’re living up to their name!

Seriously? Damn. The LP people are gonna be bummed.

That sucks. Nothing worse than data loss. Hope Lum can recover.

Now it’s just a basic not found. I hope it’s just because he pulled the server for a reset including finding a decent backup. Regardless, good luck Lum.

Anything other than the root address returns a 404.

The root directs to this message at the moment:

[B]Frequently Asked Questions About Why You Can’t Load brokenforum.com

Why can’t I load brokenforum.com?[/B]

Because during a minor forum upgrade I discovered that the server had been compromised and used to send spam email.

How can you be sure of that?

The presence of an email server I didn’t install from a user on the web server I didn’t add was a good clue.

How did that even happen? WHY DO YOU NOT SECURITY?

I normally do a pretty good job of keeping up with such things, but the operating system the server runs on (Linux/Ubuntu) hadn’t been updated since 2010. While that’s not normally THAT bad, it was approaching end-of-life for support.

Should I change my password then?

Probably. Generally you should have a seperate password for every site you use, since the cold fact is that any site, no matter how secure, can be compromised if the attacker puts enough time, resources and knowledge into it. I recommend an automatic password tracker like 1Password or LastPass to make it possible to keep track of hundreds of unique passwords.

So bring up a backup, duh!

I tried, and it failed repeatedly; the incremental backups I keep daily have been corrupted. The last full snapshot was from last year. (You may have seen it online earlier today)

NO REALLY BRING UP A BACKUP I WANT IT BACK NOOOOOOOOOOW

I’m trying! I was able to recover enough of the incremental backup database to I THINK bring the forum back up to where it was on Friday.

Well. That’s all you had to say. Now, why can’t I load brokenforum.com?

Because I’m updating the server to an OS from this decade and loading the backups back on to it. There are about 14 gigs of data to upload back up. Most of it I suspect is animated gifs or possibly likes. It will take a few hours to upload as I am not blessed with Google Fiber.

Oh. Can I have a cookie, then?

Sure!

Man that sucks. Hackers ruin everything, except when they do something good by exposing really horrible people… But it appears Russian, Chinese, Syrian,and Indian hackers have no care for social justice unlike several western hacker types.

Not that we hear off that is certain. Maybe the ‘police state’ apparatus is just too good at catching them out before they can get word out? Which is sort of where we were heading before Wikileaks and Snowdon.

Actually, Syrian website defacers (wouldn’t call them hackers, mostly) are politically motivated. Chinese hackers are primarily government employees.

Russians and eastern Europeans are exclusively commercial. The hacker didn’t take down brokenforum; he wanted it up to send spam or mine bitcoins or whatever. Lum took it down in trying to fix the penetration.

I’m sure there are Indian hackers, as there are over a billion people living in India and they’re technologically adept, but they keep under the radar.

It’s back up.

Never more has this thread title been more apt.

Recap here.
I'm going to hold off on upgrading the forum to the latest version (what caused all this mess) while I let things settle down for a bit. Forum may run slow for a bit (or it may not) until I re-enable nginx caching (which, on this version of the server, should be a lot easier to do). But hey, it's back!

We’re saved!

I used to get soooooooooo mad when I found my sites hack, and get all paranoid, etc. Then I realized that 99.999% of all hacks are completely automated and created by very unskilled scripters looking for servers and machines to use for spam, etc. I got so tired of fighting them and patching holes, etc. that I vowed to never run a web site on my own hardware ever again. It’s just too much trouble.

edit; Removed, because Tortilla prefers censorship. He wins, etc.