|
|
|
7 Essential Steps to Web Site Promotion
Web Site promotion should be the major part of your web site marketing plan. It's not enough just to design a beautiful web site and put in on the Net. Promoting your web site has to be done constantly if you want to get a continuous stream of...
An Introduction To Accepting Credit Cards On The Internet
If you have a website and you don't accept credit cards then you're losing massive amounts of business. Even if you do accept credit cards it may be worth considering an alternative that will either offer you better rates or save you time. ...
DIY SEO prt 2 Tags n' things
DIY SEO Part 2 Tips 4 tags Meta tags are the descriptive tags used by website designers to explain to the search engines exactly what your website is all about. They look like this, Put title of your page here, include important keywords ...
Search Engine Ranking: The Real Truth
There are perhaps more myths and urban legends about how to get your website ranked well on the search engines than any other subject on the Internet. In this article, I'm going to tell you how you can tell the fact from the fiction. After reading...
Why You Should Use (Not Abuse) Forums to Increase Your Traffic
There are dozens of reasons why you should look up the forums that are related to your market and post to them often. Here are 3 to get you started.
1- Get to Know Your Market as both an Associate and An Expert
The research alone is a good...
|
|
| |
|
|
|
|
|
|
Search Engine Robots - How They Work, What They Do (Part I)
Automated search engine robots, sometimes called "spiders" or "crawlers", are
the seekers of web pages. How do they work? What is it they really do? Why are
they important?
You'd think with all the fuss about indexing web pages to add to search engine
databases, that robots would be great and powerful beings. Wrong. Search engine
robots have only basic functionality like that of early browsers in terms of
what they can understand in a web page. Like early browsers, robots just can't
do certain things. Robots don't understand frames, Flash movies, images or
JavaScript. They can't enter password protected areas and they can't click all
those buttons you have on your website. They can be stopped cold while indexing
a dynamically generated URL and slowed to a stop with JavaScript navigation.
How Do Search Engine Robots Work?
Think of search engine robots as automated data retrieval programs, traveling
the web to find information and links.
When you submit a web page to a search engine at the "Submit a URL" page, the
new URL is added to the robot's queue of websites to visit on its next foray out
onto the web. Even if you don't directly submit a page, many robots will find
your site because of links from other sites that point back to yours. This is
one of the reasons why it is important to build your link popularity and to get
links from other topical sites back to yours.
When arriving at your website, the automated robots first check to see if you
have a robots.txt file. This file is used to tell robots which areas of your
site are off-limits to them. Typically these may be directories containing only
binaries or other files the robot doesn't need to concern itself with.
Robots collect links from each page they visit, and later follow those links
through to other pages. In this way, they essentially follow the links from one
page to another. The entire World Wide Web is made up of links, the original
idea being that you could follow links from one place to another. This is how
robots get around.
The "smarts" about indexing pages online comes from the search engine engineers,
who devise the methods used to evaluate the information the search engine robots
retrieve. When introduced into the search engine database, the information is
available for searchers querying the search engine. When a search engine user
enters their query into the search engine, there are a number of quick
calculations done to make sure that the search engine presents just the right
set of results to give their visitor the most relevant response to their query.
You can see which pages on your site the search engine robots have visited by
looking at your server logs or the results from your log statistics program.
Identifying the robots will show you when they visited your website, which pages
they visited and how often they visit. Some robots are readily identifiable by
their user agent names, like Google's "Googlebot"; others are bit more obscure,
like Inktomi's "Slurp". Still other robots may be listed in your logs that you
cannot readily identify; some of them may even appear to be human-powered
browsers.
Along with identifying individual robots and counting the number of their
visits, the statistics can also show you aggressive bandwidth-grabbing robots or
robots you may not want visiting your website. In the resources section of the
end of this article, you will find sites that list names and IP addresses of
search engine robots to help you identify them.
How Do They Read The Pages On Your Website?
When the search engine robot visits your page, it looks at the visible text on
the page, the content of the various tags in your page's source code (title tag,
meta tags, etc.), and the hyperlinks on your page. From the words and the links
that the robot finds, the search engine decides what your page is about. There
are many factors used to figure out what "matters" and each search engine has
its own algorithm in order to evaluate and process the information. Depending on
how the robot is set up through the search engine, the information is indexed
and then delivered to the search engine's database.
The information delivered to the databases then becomes part of the search
engine and directory ranking process. When the search engine visitor submits
their query, the search engine digs through its database to give the final
listing that is displayed on the results page.
The search engine databases update at varying times. Once you are in the search
engine databases, the robots keep visiting you periodically, to pick up any
changes to your pages, and to make sure they have the latest info. The number of
times you are visited depends on how the search engine sets up its visits, which
can vary per search engine.
Sometimes visiting robots are unable to access the website they are visiting. If
your site is down, or you are experiencing huge amounts of traffic, the robot
may not be able to access your site. When this happens, the website may not be
re-indexed, depending on the frequency of the robot visits to your website. In
most cases, robots that cannot access your pages will try again later, hoping
that your site will be accessible then.
Resources
SpiderSpotting - Search Engine Watch
http://searchenginewatch.com/webmasters/spiders.html
Robotstxt.org
List of robots and protocols for setting up a robots.txt file.
http://www.robotstxt.org/
Spider-Food
Tutorials, forums and articles about Search Engine spiders and Search Engine
Marketing.
http://spider-food.net/
Spiderhunter.com
Articles and resources about tracking Search Engine spiders.
http://www.spiderhunter.com/
Sim Spider Search Engine Robot Simulator
Search Engine World has a spider that simulates what the Search Engine robots
read from your website.
http://www.searchengineworld.com/cgi-bin/sim_spider.cgi
About the Author
Daria Goetsch is the founder and Search Engine Marketing Consultant for Search Innovation Marketing (www.searchinnovation.com), a Search Engine Promotion company serving small businesses. She has specialized in search engine optimization since 1998, including three years as the Search Engine Specialist for O'Reilly & Associates, a technical book publishing company.
|
|
|
|
|
| Gentoo Forums :: Index |
| Discussion forums for topics around the distribution, Linux applications, installation, administration, portage, frequently asked questions. |
| forums.gentoo.org |
|
| Readers' Opinions - New York Times |
| Books Reading Group. Readers are discussing “All Aunt Hagar’s Children,” Edward P. Jones’s new short story collection. More Books Forums ... |
| www.nytimes.com |
|
| Ubuntu Forums |
| This is a discussion help and support forum for Ubuntu Linux. |
| ubuntuforums.org |
|
| Joomla! Community Forum - Index |
| Joomla! Forum - The Joomla Community. Support, discussion and interaction with the Core Team. |
| forum.joomla.org |
|
| Internet forum - Wikipedia, the free encyclopedia |
| A typical Internet forum discussion, with common elements such as emoticons, ... Like other forms of online communication, Internet forums are home to many ... |
| en.wikipedia.org |
|
| Java Technology Forums |
| Online discussion forums offering help for users in solving issues. Includes FAQs, and links to other Sun technical databases. |
| supportforum.sun.com |
|
| Adobe - Forums |
| Share questions, suggestions, and information about Adobe graphics products. |
| www.adobe.com |
|
| WoW Forums -> Forum Index |
| Read our Introduction to the Forums. This includes information about getting ... blizzard provides the World of Warcraft community forums for its players to ... |
| forums.worldofwarcraft.com |
|
| MySQL AB :: MySQL Forums |
| Forum to discuss the sample database included with MySQL ... The main discussion forum for MySQL. Please use specialized forums for specific topics. ... |
| forums.mysql.com |
|
| SitePoint Forums: Resources, Design, HTML, CSS, PHP, ASP, MySQL ... |
| Webmaster Forums for Design, CSS, XHTML, PHP, Ruby, JavaScript, Affiliate Marketing, ASP.NET, Promotion. |
| www.sitepoint.com |
|
| Delphi Forums |
| Offers free and paid forum hosting. The paid offer is ad-free. |
| www.delphiforums.com |
|
| Home - MozillaZine Forums |
| Search the forums for others who may be having the same issues you are. You can also browse the sticky'd posts at the top of the forum for the product ... |
| forums.mozillazine.org |
|
| Web Hosting, Free Web Hosting and Web Tools from Bravenet.com. |
| Maintain a direct mailing list. Message Forum Host your own online discussions. Vote Caster Perform a multi-question poll. Web Poll Perform a quick poll. ... |
| www.bravenet.com |
|
| Forums | NewsTalk |
| The Detroit News Online does not tolerate offensive language in its forums. The News will kill offensive posts. Participants can and will be denied access ... |
| info.detnews.com |
|
| osCommerce Community Support Forums |
| Official community support forums for the osCommerce project. |
| forums.oscommerce.com |
|
| The GardenWeb Forums - GardenWeb |
| The GardenWeb Forums comprise the largest community of gardeners on the Internet. ... If this is your first visit to the forums, you might want to visit our ... |
| forums.gardenweb.com |
|
| AnandTech |
| Is server configuration a pain? DSL or Cable Modem giving you fits? Get your questions answered and your topics discussed in the AnandTech Networking Forum. ... |
| forums.anandtech.com |
|
| Chronicle Forums - Index |
| Chronicle Forums - Index. ... Talk online about graduate-school life in our new forum. ... Forum Stats, 220919 Posts in 17109 Topics by 3306 Members. ... |
| chronicle.com |
|
| Forums | drupal.org |
| Try searching the forums first or a specific project's bug reports. Pre installation questions. Is Drupal a viable solution for my website? ... |
| drupal.org |
|
| Dev Shed Forums - Open Source web development |
| Dev Shed Forums - over 195246 members on our Dev Shed forums. |
| forums.devshed.com |
|
|