PDA

View Full Version : Server down time last night and this morning



ddavison
June 29th, 2013, 17:33
Our server was down unexpectedly for roughly 15 hours as it was exploited and used as part of DDoS attack. As a precaution, our host (1&1) had removed it from the network and the only way we could access the server was through a serial connection. We worked with our host to resolve the issue and got the site back online.

Note that while the server was down, no games were able to use the alias feature or check the validation on an Ultimate license.

In the event of any future server down instances, you can continue to connect with each other by using your external IP address if your users have Full, Lite or Ultimate licenses. You can find it on the screen before you load your campaign. Give this out to your players and have them connect to it instead of to your normal alias.

We apologize for the inconvenience. We are going to continue monitoring and tweaking things on the server to prevent or limit any future attacks.

jimshu
June 29th, 2013, 19:49
You realize that's not really a solution for Ultimate licenses. That makes Ultimate licenses no better than Full licenses but they cost 110 dollars more. I realize this doesnt happen all the time, but my group who is just starting out on Fantasy Grounds now have this negative impression right out of the gate. This group was four players and a GM, so we paid even more for the ultimate license than we would have needed to for four lites and full.

While I was frustrated last night when this happened, I can understand that outages occur. Your workaround is just not really fair to your customers. Perhaps a license/key pairing that is generated off the machine information the application is installed on that is used when it cant access your validation system?

ddavison
June 29th, 2013, 19:54
Jimshu, you are correct. We do realize this and haven't found a solution for it yet. There have been very few instances of down-time in the last five years or so, so it hasn't really been an issue so far. The recent issues are making us re-evaluate the situation.

Mistindantacles
June 29th, 2013, 20:13
Thanks for communicating what happened. I would categorize this as an "uncommon situation", as how often is your server compromised.

Yeah, my regular Saturday morning group missed a week. It's life. It happens.

Just keep up the great work, Devs!

Mellock
June 29th, 2013, 20:56
I hope the little hooligans enjoyed disprupting my Saturday game. I look forward to game-time all week.

Off to buy Savage Worlds out of sympathy. Hope they leave the server alone now.

Willot
June 29th, 2013, 23:19
Jimshu, you are correct. We do realize this and haven't found a solution for it yet. There have been very few instances of down-time in the last five years or so, so it hasn't really been an issue so far. The recent issues are making us re-evaluate the situation.

Yeah ive had FG for a number of years and this is the first time Ive had any serious issues with it.

Trenloe
July 1st, 2013, 00:54
No one can upload any attachments to forum posts. As part of the lockdown over the last couple of days has the ability to upload attachments been suspended/removed?

damned
July 1st, 2013, 01:46
keep your pr0n to yourself Trenloe!

ddavison
July 1st, 2013, 03:03
We are planning to update the forum software next week. Stay tuned for more info.

Gadreun
July 1st, 2013, 08:57
thanks for the info. Glad to hear it is rare.

Willot
July 1st, 2013, 10:34
We are planning to update the forum software next week. Stay tuned for more info.
Probaby about time the forums had a face lift. It's had the same format for a number of years now

Doswelk
July 1st, 2013, 18:21
Probaby about time the forums had a face lift. It's had the same format for a number of years now
I'll miss it though...

Mellock
July 1st, 2013, 19:01
Perhaps it's just the innards that are being operated on, not an actual facelift? Perhaps we'll get to earn cool badges to put by our name depending on which games we play or enjoy :p

damned
July 2nd, 2013, 03:10
most likely a patch/upgrade to latest version. most website hacks these days go thru application vulnerabilities...

ddavison
July 4th, 2013, 20:57
Correct, we will be upgrading the forum to the latest stable version. There are newer versions than that available but they have a bug list a mile long. We shouldn't lose anything -- but you never know. In addition, we'll probably be pushing this out to a new server that we install fresh over the next few days and go live on it sometime before Monday.

---

For now, I just made several firewall changes. Please let me know if anything stops working. I created a brief blip earlier today that shut down some things we actually needed open, but as far as I know everything that should be accessible is still accessible again now.

The new firewall should have some really cool anti-DoS and reactive address blocking to make it even more secure and it links up with some well known ip address list maintainers to block known hacker addresses. Hopefully that is all done now. From the logs, it doesn't appear that we were ever targeted specifically, but being on an international host with lots of servers means that we have to always be on alert for blanket attacks.

Trenloe
July 5th, 2013, 06:58
For now, I just made several firewall changes. Please let me know if anything stops working. I created a brief blip earlier today that shut down some things we actually needed open, but as far as I know everything that should be accessible is still accessible again now.
Hi Doug - is the ability to upload attachments still being blocked? I, and others, can't upload attachments to the forums.

ddavison
July 6th, 2013, 06:21
It is until we finish updating the forum software. We'll be doing this along with a server move at the same time and the server move is under way.