Google’s indexing scheme
When I first created the buy-ebay-stuff-online site, google indexed me right away. They indexed me before I was done designing stuff so they indexed a whole bunch of pages that done exist anymore. I ended up having to change my urls after google had already indexed them so for the last couple of weeks, site:www.buy-ebay-stuff-online.com returned 33 results all for pages which didn’t exist.
This morning the same site: search returns 6 results. They’re interesting. One of them is the homepage, which I would expect. Google tends to hit home pages every couple of days and updates them in it’s index. They don’t however, seem to follow past the homepage when doing this except for links which are to external pages. I know this because the other 5 pages the site: search returns now are pages I’ve linked to from www.jonasblog.com and from www.jonasfam.com. I put links to some pages there to see if google would follow them and start indexing some of my other stuff. Sure enough, they did.
Also, they’ve gotten rid of the pages that don’t exist anymore. Some how google must go back and check on pages it has already indexed periodically. If it doesn’t exist after a certain period of time, it removes them from it’s index.
So, the result is that every couple of days google indexes homepages. It also follows external links on the homepage and indexes those pages, but probably doesn’t really follow any links. Once a month, google goes through and does a large index of sites where they’ll follow most links (not all, I’m not sure how they do this) and index as much as they can.
Filed under Search Engines by