Update about our server crash on Friday by Walt Thiessen
(libertarian)
Sunday, February 28, 2010
We experienced a severe server crash on Friday afternoon that stemmed from insufficient memory (RAM, otherwise known as Random Access Memory) on the server. Unfortunately, since our site is currently located on a virtual dedicated server, we don't have an option to increase the amount of available memory. So, the plan is to move to a larger, 100% dedicated server.
The trigger for all this was a tremendous increase in traffic to the site in recent days, including (according to messages I've received from users) a reference to the site's ten question survey by Glenn Beck of Fox News on the air. This led people to visit the site in droves, even larger numbers than we'd already received. As a result, roughly 40,000 visitors tried to take the survey within a 24 hour period, in addition to exploring other pages on the website. We normally get roughly 60,000 to 80,000 unique visitors per month, viewing typically about 200,000 total articles per month. We received half that amount yesterday alone! While the server valiantly handled the increase as well as it could for a number of days, eventually the load became too much to handle due to the Beck announcement, thus leading to the crash and the subsequent poor response times since then.
As you can see from the above graph taken from our Google Analytics account, we've experienced a tremendous burst in traffic in the past few days. The graph measures total traffic over the last 30 days. The relatively consistent line you see leading up to the spike is typical of what the site normally receives.
Needless to say, the outage affected thousands of users attempting to reach the website. We spent the past day-and-a-half attempting to identify the problem. Once our server's administrators found the source of the problem, we redirected traffic away from the server to a temporary error page. Once the memory errors were all cleared, we adjusted some of the automated background jobs that update headlines and statistics on the site to run less often. This will (hopefully) enable us to get through the next few days while we prepare to move to the new server.
The good news is that once the move is completed, the server will be much more able to handle bursts in traffic to the site. This will be very important as we approach the mid-term elections in November 2010 and then the next presidential election in 2012. We experienced our peak traffic levels in January 2008 at the height of interest in the Ron Paul campaign, so we can be reasonably sure that we will experience similar visitor levels as the next major elections approach. This upgrade should help us make sure we can meet the anticipated demand.
Please bear with us while we attempt to iron out the situation, and thank you for your support and understanding during the transition period.
Did you like this article? If you did, Thumb It! 51
thumbs so far
The views expressed
in this article are those of Walt Thiessen only and
do not represent the views of Nolan Chart, LLC or its affiliates.
Walt Thiessen is solely responsible for the contents
of this article and is not an employee or otherwise affiliated
with Nolan Chart, LLC in his/her role as a columnist.
Walt, will Glenn Beck's quiz be reposted? There are undoubtably MANY of us who would still like to take it to determine just where we really stand according to the quiz and to see if our self image matches the quiz reality.
Posted By: Walt Thiessen
Date: 2010-02-28 12:51:50
You can find the survey by clicking the Survey link at the top of any page on the website. Once we finish the changeover to the new server in a few days, the survey should work without problems.
Want to comment on
this article? Leave your comment
here. Your email address is required to track your
comment. However, we will neither publish your email
address nor distribute it to other organizations or
persons. The only reason we might use it would be if
we needed to contact you regarding your comment. All
comments are subject to our
terms of use policy.
