Last Tuesday, an attack on the two servers that host one of the largest flight simulation communities online, avsim.com, has, according to the site’s founder, “effectively destroyed” the site with little chance of recovery. This grim reality of a volunteer driven site has opened some eyes for community members and outsiders who heard about a tragic loss of a popular flight enthusiast destination.
Attack on avsim.com results in total loss. (IMG: R. Kruijer)
Avsim.com was almost completely community driven and thousands of members strong. Launched in 1996, the site focused on every aspect of the flight simulation, and held one of the largest repositories of user created content for the flight simulation community. The attack on Tuesday left some in a state of shock, others in a state of rage, but as is common with hardcore communities, everyone is willing to chip in and help. The problem is that those with rage and smug superiority complexes are starting to rock the boat by beating a single aspect of the aftermath to the attack with large overused stick.
According to Tom Allensworth, who founded avsim.com, in both a forum statement and comments to the BBC, the first sign of problems was loss of functionality to one of the servers hosting the site. “That progressed to not being able to access the web server via FTP or SSH. Finally, the web/forum server tripped offline,” Allensworth said.
“Next the library/email server started to exhibit issues. We went in to reboot it, and that is when it failed entirely. The partitions on both of the servers had been removed. We shut down both servers until we can get a technical member up to our Network Operations Center (NOC), and do a comprehensive inspection of the damage.”
What started after those public announcements was a mix of support outpouring and baseless accusations. Almost instantly, when Allensworth said that with support from the community the ultimate goal was to rebuild, said support started flooding in. There were 60,000 forum members alone, and of those, most of them were considered active in the community. After that, the talk turned to the missing data, which includes untold amounts of articles, images, game mods, and of course forum content.
The big picture was that avsim.com made a huge error when placing their eggs in one basket by using a second server to backup data from the first as a recovery plan. One thing cited as proof by those pointing out the failure is that because both servers were attacked at once, the backup system was rendered useless; something not considered when the recovery plan was first created.
However, this shortcoming, while admittedly an error according to statements made by Allensworth, was one that was going to happen when you consider the community model. Everyone who started kicking the dead horse, including a series of obvious commercial pitches from one member, offering ideas and backup solutions both to prevent the loss of the site and content before the attack and after, are forgetting two things.
The fist thing forgotten is that this is a community driven site. It operated out of the pockets of the administrators and donations. Like most community driven sites, there is no way to scale the development of infrastructure to the levels expected by some of those who criticized operations after the attack. Avsim.com was not an IT operation, so some of the solutions, including tape drive backups, Network Attached Storage solutions, hosted backups, or backup servers in a cluster on a separate subnet, are too expensive and thus out of the scope of available options.
The second thing forgotten, related to the first, is that future recovery plans cannot revolve around IT operations. While there is sure to be a new design to data recovery and security, this is still a volunteer driven community. Donations might help with the addition of three or four servers, two of which would be used for content and backup, but that does not mean it will prevent future attacks, nor does it ensure that they will work. Who would oversee the data management? Who would volunteer their time to do so without pay, and would do it at the levels needed?
Other options, including hosting of the site on a new provider, mentioned on a few forums related to other flight simulation sites, are moot. If you trust a third party to host all of your backups, and never create and manage any of your own, then you are asking for problems. It will not matter who your host is, if you cannot create and manage your own backups, you’re toast should you have to recover lost data.
This is because often hosting providers will backup data as it changes, so if the backups are instant, and a file is deleted, some hosts will manage that file removal in your backups for you, by deleting it. In short, if you rely on your host to manage backups, make good and sure you understand exactly what they are backing up, how they manage backups, and you need to test them on a regular basis.
As today is the first business day since his last comment on the attack, Allensworth will be learning exactly what was lost in the attack. It could turn out that the NOC could recover the data. In forum postings, there have been mentions that some of the repository of mods and other files and articles might actually be salvageable. The safe bet is to prepare for the event that they are a loss as well. However, the community will ultimately be the cause AVSIM’s recovery. Many have archives of their own, and in the event they are needed, DVDs are being prepped so they can be mailed out to whoever needs them.
A temporary forum for AVSIM is here .
If you want to donate to them, there are links on the forum including PayPal and a postal address. From the collective of geeks here at The Tech Herald, we hope things get back to proper running order soon at AVSIM, such a stong community deserves to continue.
Want regular updates from The Tech Herald? Follow us on Twitter .Join our Facebook Group .
Comment on this Story