Today has been pretty interesting. Early in the morning my server basically over loaded until all of the services crashed, including ssh. This is crazy as hell, but I'm not kidding when I say that 800+ Yahoo web crawlers like ( lj602050.inktomisearch.com ip 72.30.132.224 ) pounded on my server until all services died : (
I had to reboot my server and I had over 140 days up. My hosting clients can attest to that. On a positive note, I think that my technology site is getting pretty popular with the search engines. I just can't wait for more traffic. This big index or deep indexing was the first time that a search engine has done a crawl of that magnitude on my site, but I guess you'd have to do that to index millions of pages = )~
It was pretty obvious to me that I'm gonna have to step up to a dual xeon server. Preferably with hyper-threading technology. That's what I use for my production work systems.
No comments:
Post a Comment