Live By The Google, Die By The Google

The Googlebot came by my blog on Friday to spider it. I’m not sure the exact time it showed up but according to Google Webmaster Tools, 193 URLs didnโ€™t get indexed because of a “Network unreachable” error. That normally means my blog was down when the spider came. Looking at my logs, I concluded that the Googlebot must have came around the time when the crazy Comment Relish plugin did the MySQL DOS attack on me and took the blog down for almost two hours.

The next day, I find my ranking had gone down for many of the keywords I ranked for, including make money online. With 193 unreachable URLs, I’m not surprised that search traffic is down. This should be an easy fix. Now that the site is running at full speed again, all I need to do is invite the Googlebot to come back and try again. That should restore everything.

I want to thank everyone for sending me an “OMG! You’re not #1 anymore!” email. This kind of situation always highlight the saying, “Live by the Google, die by the Google.” For many websites, their only source of Internet traffic is Google. Without it, their traffic is nothing. That is a very scary situation to be in because if Google wants to, they can remove you from their index at the snap of a finger and there’s nothing you can do about it.

Yes, you should work on getting Google traffic, but you shouldn’t do it the exclusion of everything else. If the bulk of your traffic comes from Google, then you need to work on getting other traffic sources. Just as I advise that Google AdSense shouldnโ€™t be your sole online income source, Google search shouldn’t be your sole traffic source.

76 thoughts on “Live By The Google, Die By The Google”

  1. Andrew P says:

    Darn, I am sorry to hear about your loss in rank. Good post reminding people to not just concentrate on Google traffic. I want to be John Chow when I grow up :D.

    Oh yeah, and I forgot to say…
    Omg! You’re not #1!

    ๐Ÿ˜‰ ๐Ÿ˜‰

    1. Hi John – it happened to me as well. I have 21,000 pages in my sitemap and Google at one point showed me that I had over 800 pages with errors. I fixed a few pages and asked Google to came back and it’s now down to under 90. It’s had a great impact on rankings since it’s gone back up.

    2. That darn plugin.. and I was about to implement it on my site.

  2. Mike Zak says:

    To invite Googlebot you can use QUIT:

    This tool is amazing when it comes to making Googlebot come to you ASAP, and it does nothing illegal.

    1. How does it work Mike?

  3. Very true. Is there a schedule googlebot has or is it random for you?

  4. Travis says:


    1. adam says:

      I sent an email :mrgreen:

    2. Jack Book says:

      Hey you guys!
      how could you post your comment that quick? i thought i was the first to read this post. since i always let my rss reader active.

      1. Andrew P says:

        I know! I thought I had the first comment too…
        Are they manually approved or something? This is my third comment for this post and none of them have been published!

      2. Kumiko says:

        You gotta be quicker than quick on this site!

        1. Marc says:

          That’s true. I spend my whole day staring at John Chow’s site doing noting but refreshing my browser. Even still I don’t get first comment that often ๐Ÿ˜ฏ

      3. John Chow says:

        RSS isn’t real time. The post you see in the reader may not be the latest post.

        1. Brendan says:

          Ive noticed this quite a few times.

  5. Nomar says:

    I agree with you John, When google bans you from adsense or their search engine, and you have enought back ups, it won’t hurt you that bad. But when you only have google as your main income and main traffic source, your screwed.

    1. Google-dependence is a hard habit to break, but I agree with you and John — you gotta spread those wings and utilize other means of traffic and income.

    2. It’s hard to really have backups when Google is such a dominant player. If you are working AdSense you don’t get nearly as much from other countries as you do from the US because of Geotargeting and Google is #1 in the US.

  6. Ankur says:

    It cant bother you much john, With a blog having a dedicated (and addicted ๐Ÿ˜ˆ ) reader base like yours, screw google!

  7. Shaun Carter says:

    Yeah Google has most webmasters by the you know what. The percentage of traffic I get from Yahoo, MSN and others combined is less than 5% of total traffic.

    This whole Google thing is a large part of Terry Semel stepping down at Yahoo. He just couldn’t make Yahoo competitive enough to keep up with Google’s blistering pace of rising profits and acquisitions.

    1. Marc says:

      I’ve got similar numbers as you Shaun. The thing is that every time I make positive changes to my website, I rank better in Google with no change to the others… They really are trying to match their algo to what’s best for users. They don’t always do that, but they’re much closer than the other two horses, far off in the distance.

      1. I have over 130,000 visitors in the past month to my site. 87% of them came from search engines and 94% of those came from Google. I rank in the other search engines – I just don’t think people use them as much as the almighty Google.

  8. Andrew P says:

    You don’t have to worry about Google at all with your RSS feed. No, but hopefully you take back that number 1 spot next update :).

    1. SEO Blog says:

      It shouldn’t even take that long. I’d expect the rankings to be back within a few days. If they aren’t then John might have some deeper issues but as I explain below, I’m pretty sure John’s right on the cause of this one.

  9. ouchs says:

    yeah i noticed you were on the sixth page….

  10. Let’s see how fast you can get your traffic back.

  11. Google has me on the first page for “‘personal finance’ blog”


    1. Brilliances says:

      I’m first for brilliances but I’ve not seen anyone search for that yet ๐Ÿ˜‰

  12. Marc says:

    I also emailed you, i’m glad I wasn’t alone ๐Ÿ™‚

  13. John – I’d love to see what your top 5 traffic sources were when Google went away. Any chance of a couple of referrer stats shots?

    1. Brendan says:

      I want to see where his biggest dot is on the analytics maps.

  14. Joel Badinas says:

    How I wish I can be just half of your traffic, john. (^_^)

  15. I’m sure with those 5000+ RSS subscribers you’d still do alright but I think you’ll make it back to top position anyway!

  16. Wallace says:

    it is true,
    we can not live without Google now,
    just like you will die without water.
    you can see Google will lead the internet in the next 5-10 years.

    1. SEO Blog says:

      That’s not what John was saying at all! He’s saying you can and should be able to live without Google. Google is fickle and if you’re dependent upon them, the foundation of your site and your traffic is built on sand. Read up on “defensible traffic”. It will open your eyes.

  17. wu zhiyong says:

    Thank you, I sent this article translated into Chinese

  18. Phil Bader says:

    Yes John, This kind of thing happens to me a lot of times when my site gets down and Googlebot tries to crawl it,nevertheless you should always look for other traffic sources and I believe the best one is the word of mouth and popularity which comes by your content quality

  19. Set backs like these are temporary — if you decide to seek other opportunities as well.

  20. Linky Love says:

    My WordPress server was downn thanks to an attack as well. My webhost provider told me to upgrade my WordPress, so I did that.

    Didn’t know plugins could harm me as well, so thanks for that tip John.

    1. Brendan says:

      What plugin allowed your site to be attacked?

      1. Linky Love says:

        The latest plugin I installed was:
        Aizatto’s Related Posts, but it would be wrong to say that this is the cause.

        I cannot access my error log to find out what really happened.

        Am gonna ask my webhostprovider to explain in detail.

  21. LOL, sure John like u really need Google to make your day ๐Ÿ˜† folks like me are just happy to even show up on google period!!

  22. Tim says:

    Pitty most search bots copy from each other aswell…

  23. Tom says:

    with your loyal base of thousand of daily recurring visitors, a hit by google is to be seen as an opportunity to analyze the traffic and click patterns of those that come to you anyway.

  24. now, i know why your website drop down in ranking. ๐Ÿ˜€

  25. Andrew P says:

    Dang… A Google Ad just told me “John Chow is Old News”. ๐Ÿ™

  26. VORN says:

    You are now 58th result on Google.
    Very bad.

  27. Brilliances says:

    omg. you’re not even first for ‘johnchow’ or ‘john chow’

    hope getting back to the top is as easy as you say it is…

    1. SEO Blog says:

      That’s actually a good sign for John. Dropping your rankings for just a term or two suggests there’s an issue with your site for that term or maybe you’re being penalized for something. Dropping all your ranks for even your site name suggest you’ve got a technical issue, such as a bad .htaccess or robots.txt or in this case, site down time. I’d expect to see John back at the top within the week.

      1. SEO Blog says:

        I should clarify, if your site ends up getting penalized you can often drop rankings across the board as well. I was talking about some sort of “over optimization” penalty which is usually term specific rather than site wide.

  28. Lance says:

    Congrat! I’ve just check,you are on the top again on google search John. ๐Ÿ˜Ž

  29. Chuck says:

    Or…maybe Google just finally got word of the true depths of depravity which inhabit the heart of “The Royally Evil One”. :mrgreen:

  30. shaun says:

    Hopefully you can get back that would be a disaster.

  31. Pingback: Marketing Pilgrim
  32. Antabax says:

    I guess maybe your blog is one of the favourites among the google staffs. once they realize you are not number 1, they quickly restore your blog back to number 1. hehehe.

  33. Andy says:

    How annoying must that be for that to happen.

  34. Casey says:

    “If the bulk of your traffic comes from Google, then you need to work on getting other traffic sources. Just as I advise that Google AdSense shouldnโ€™t be your sole online income source, Google search shouldnโ€™t be your sole traffic source.”

    Those are wise words. It’s like everyone says, “Don’t put all your eggs in one basket.”

  35. John,

    Just because your site was offline when the spider came to eat does not mean google dropped your rankings. Network glitches (whether your problem, your ISP, your ISP’s carrier etc) happen all the time and google is smart enough to account for it.

    Likely, you are bobbing around in the rankings for other reasons. Some of which are: the rate that you build your links, using the same anchor text in your links and the rate at which your links are found/not found by google.

    Typically, until your links are ‘aged’ – most feel that ‘aged’ is one year old – your position will continue to yo-yo around.


    1. SEO Blog says:

      Secure, “bobbing around” in the rankings is not going from number 1 to the 60’s. John just recently had an issue with his robots.txt disallowing the bots from a good chunk of his site. If 190+ urls were not found in their latest crawl, that could definitely cause the big G to take notice. While outages do happen, they can and do affect rankings, especially when you combine this with his earlier problems.

      The rate at which this site builds links is not an issue as it’s already well established and it has a LARGE link profile already. It’s true that as sites and links age they tend to become more valuable, however, John’s been at the top of this search consistently for quite a while now. As I said the last time he plummeted in the rankings, a drop of this magnitude suggests a technical issue, not some sort of optimization issue.

      1. Nah, I stick by my bobbing comment.

        Whether this blog is already ranked for other terms does not matter because this blog has recently tried to rank for a new term only a few months ago. i.e. John changed the focus of his blog to ‘make money online’, changed the title, dropped the phrase into posts all over the place, got a blast of inbound links from untrusted sites etc. None of these activities are ‘organic’ so google knows there is game here. Many feel that google keeps a closer eye on the kind of links and frequency that John is doing – and the fact that all the links are 2-way is not helping either. In other words, google knows he’s trying to game the term and they are not giving him the full link juice for it because the links are all coming from other blogs, mostly which are not ranked well or ranked at all — AND are not aged either. i.e. the links John is getting cannot be trusted, so google bobs him around while they figure out if they can trust his links for ‘make money online’ by looking at the older links, the newer ones, all the same anchor text, coming from blogs that looks just like his etc.

        John, what you need to do to pin the tail on this donkey is get some .edu links and one-way links from ‘trusted’ sites within your niche and that will help google see that they should trust you for ‘make money online’. Or you can wait a year for your current links to age and you’ll get the juice then as well. ๐Ÿ™„


        1. SEO Blog says:

          @Secure, sticking by it doesn’t make it right. You think Google ranks sites they don’t trust at #1 for competitive terms? John has been consistently ranking for Make Money Online and that’s not a phrase that is light on competition. Using your theory, how do you explain the fact that this site now ranks only 57th for the term “John Chow”? That’s not a new term he’s tried to target…

          John’s review exchange does end up creating 2 way links but that doesn’t necessarily mean they aren’t valued links. Link exchanges went the way of the dodo bird but interlinking between related sites (many of the people reviewing this site have related blogs) often happens naturally and it usually takes John a while to reciprocate the link. When you also consider the fact that the site has been linked to from a LOT of established sites ( you’ll be hard pressed to justify your position.

  36. mvandemar says:


    Ok, real good way to test your theory here. Currently on a Datacenter that has you ranking very low (including low rankings for your own name, by the way, which is a bit more harsh than just dropping for a money term), it is showing you as having 3,180 pages indexed:


    According to your theory, as soon as Google re-indexes the missing pages your rankings should be fine. Now, considering that since your homepage is showing spidering date of today, we have to assume that many pages have already been re-spidered since the downtime. So… how many pages do you think it will take to bring back your rankings?


    1. John Chow says:

      I don’t think it’ll take very long. As some comments above have already reported, I’m back at number one.

      1. mvandemar says:

        Maybe, but they might also be looking at a cached version of the page. I just checked a multi-dc rank checker and you weren’t ranking on any the the datacenters it checked. Do you know which dc’s you might have gotten rankings back on? Maybe we can compare the caches on the various dc’s and see if anything looks suspicious…

        Also, did you know that you have 2 meta description tags on each page…?


  37. Tom says:

    I did notice this but I knew you’d be told by loads of other people anyway.

    I agree Google is a little too powerful for my liking. They can decide if you survive or fail should they choose too.

  38. zaki says:

    Yeah…I’m feeling that now…Thanks to google for introducing and catalysing some virtual income source for me. I hope it can be a full time income someday

  39. Asela says:

    Hey john I think you should install Live Plugin

    “Live is a free WordPress plug-in that lets you watch your website activity in real time. You can watch as people are visiting your pages, leaving comments, and grabbing your feeds.”

    so it will help you to know when your traffic is low & Play with the WordPress at right time . . . ๐Ÿ˜Ž

  40. It must be nice to have a ton of people checking your SEO rank daily. ๐Ÿ™‚

  41. BTW, is there a way to find out where your site ranks for a specific search term instead of scrolling through all the pages and looking for it?

  42. John Cow says:

    Google me this, Google me that. Wonder when the next PR update is

Comments are closed.