I recently got to know Greg Hartnett from the web directory: Best of the Web… or BOTW for short. BOTW is the internet’s oldest web directory. Greg has been in the search industry since the beginning. As a matter of fact, BOTW is the first reference in Larry Page and Sergey Brin’s college thesis paper that started Google (I ask about that during the interview). In my opinion, there is no better person to interview about web directories. Enjoy!
Q: What is a Web Directory, and how is it different from a Search Engine, or Internet Yellow Pages?
A: Typically a directory is categorized and sorted by humans â€“ actual people review each site, describe and title it, and categorize it accordingly. Each site is reviewed to ensure editorial guidelines are met, helping to keep spam at bay. The search results from a directory are limited to the pool of sites that are listed within the directory. The Internet Yellow Pages would be an example of a niche directory, as they tend to deal with commercial sites exclusively.
By contrast, search engines tend to be driven by spiders attempting to crawl all of the content online, and sorting the results according to proprietary algorithms. Search engines draw their results from a much wider set of data, and use filters to combat spam and irrelevant results.
Q: What is the benefit of getting your site listed in a Web Directory?
A: Getting listed in a quality directory can prove very beneficial to a site owner. Through proper categorization, a site owner should expect relevant traffic from the users of the directory.
Additionally, most search engines tend to value listings in a directory as a trusted link. Due to the editorial control of a directory listing, search engines view a listing in an authoritative directory as a sign of confidence that your site is a quality resource.
Q: Are web directories just acceptable Link Farms?
A: That would really depend upon the type of directory you are referring to.
A link farm is no more than a collection of links, typically uncategorized, and displayed in a haphazard way. Link farms are designed for the sole purpose of manipulating search results, with no benefit for a user, and little, if any benefit to the site owner.
A web directory, while also a collection of links, is categorized, and all listings undergo editorial review. The directory should be built in a manner that is intuitive and provides users with relevant resources. A directory is built for the user, not for the search engines.
So a quality directory would have no resemblance to a link farm, while a directory built solely as a means for webmasters to pay their way to higher search listings would have a lot in common with a link farm.
Q: Are there any web directories, or “Spam Directories” we should stay away from?
A: Absolutely. As is the case in any industry, there are some businesses you should work with, and some you should steer clear of. A site owner should do some homework, and make sure that they have confidence in the quality of the directory. Some things that may signal a sub-par directory:
– Run of site links
– Listings of poor quality
– Overuse of keywords in titles and descriptions
– Directories less than a year old
– Directories with overpopulated top level categories, and empty sub categories
– Directories powered by an off-the-shelf script
Basically, go with your gut. If you come across a directory that doesnâ€™t feel right to you, or if you have doubts as to the quality of the directory, then pass on submitting. There are plenty of places online for you to find quality â€“ donâ€™t settle for less.
Q: Which directories are considered to be the most trustworthy, and why?
A: I think that Yahoo and DMOZ would inarguably rate as the top directories. Their comprehension, user base, commitment to quality, and age help to set them apart. Of course, Iâ€™d like to think that BOTW is regarded as a trustworthy and authoritative directory, but my opinion might be considered a little biased. I also think that Business.com and Microsoftâ€™s bCentral do a good job with their directories.
Q: What reasons would BOTW decline a site submission?
A: We have a pretty straightforward set of guidelines. If the site contains a substantial amount of unique content, relevant to the category, it should get in. It also helps if the site is designed well, and is easy to navigate. Most sites are rejected because of poor content.
Q: Why do some directories have free submission, and others have a fee? What’s the difference?
A: With few exceptions, directories are run as businesses. As such, the owners would like to make a profit. So, most directories worth their salt will carry some type of fee. I think that directories not charging any type of review fee, or not monetizing their submissions, are short sighted, and will have trouble with scalability.
Additionally, many directory owners have a submission fee to simply discourage spammers. A site owner is going to think twice about submitting 25 subdomains of his craptastic, made-for-Adsense site if there is a fee associated with submitting each one.
Q: How is BOTW’s submission fee determined?
A: Running a successful directory is a resource-intense operation. It takes a lot of human capital, and quite a bit of technology to make everything run smoothly. We try to price our review fee to allow for directory growth, while striking the right balance of value for the site owner. More than anything though, the review fee is priced upon supply and demand, and I think we have found a nice price point that enables us to grow, while still providing the site owner with a value.
Q: Is it more important to get a link on a relevant niche directory, than a comprehensive directory like BOTW?
A: It is more important to get a link in a quality directory â€“ period. Niche or general is not really important â€“ itâ€™s the quality that counts.
Q: Is it important for online marketers to use the smaller web directories, or is submission to BOTW good enough?
A: Submission to BOTW is definitely not good enough. While a good start, there is much more for a webmaster to do than simply submit to BOTW. I would definitely look at smaller web directories, bearing in mind what I said previously about quality. I would rather submit my site to just a handful of quality directories than have a link in every site from some directory list.
Q: Google said it will devalue links that are bought for the sole intention of increasing PageRank. BOTW costs money, and many people buy it for the PR. Are you worried BOTW’s links will be devalued?
A: I try not to concern myself with those matters very much. We are fully committed to building a quality directory â€“ a resource for users to find relevant information. We focus on doing our thing, and let the search engines do their thing. The chips fall where they may.
It is important to note that webmasters canâ€™t buy their way into BOTW. We do charge a fee for an expedited review, and all commercial sites are required to pay for review. Site owners pay for the review â€“ which by no means guarantees a listing. People pay and get rejected frequently.
Also, we do accept free submissions for non-commercial sites. Though there is no guarantee on review time, we do get around to looking at the free submissions too. Moreover, for every site that we add via the submission process, our editors add more than a hundred other sites on their own. The directory build is a colossal undertaking â€“ if we relied solely on submissions, the project would be a shadow of what it is today.
Q: For people concerned about Google PageRank, can people rely on web directories as a valuable link in their link portfolio?
A: A listing in a directory is a valuable link in the portfolio of all webmasters â€“those concerned with PageRank or not. The editorial control of the directory lets humans and spiders know that the listing is relevant to the category, and can be trusted to provide quality content.
Q: Should site owners expect click-thru traffic from a web directory? Or, are web directories really just for link popularity?
A: Yes, you should definitely expect click-through traffic. Of course, traffic will vary depending upon the directory, and upon the category in which you are listed.
Q: What are the top 3 most popular categories, and their top sub-category?
A: Our most popular categories are: Adult, Business, and Society. Over the course of the last six months, we have noticed a marked increase of interest in the Regional branch â€“ part of a growing trend towards local search.
Q: What if someone submits their site, and after inclusion in the directory they decide to radically change the site? Is there some kind of detection process in place for deception?
A: Sites are reviewed by editors on a continual basis. At a minimum, all sites are reviewed once a year to ensure that they are still relevant to the category. That is one of the reasons for our annually recurring fee.
Q: Are there any enhancements, or new features we can expect from BOTW?
A: Absolutely. Weâ€™re never content resting on our laurels, and we are continuously developing new products and services.
Just recently, we enhanced our search functionality on the Best of the Web blog directory, and going forward we will pursue additional opportunities. The social networking aspect of the web really fascinates me, and Iâ€™d love to see us make more or a splash in that arena.
Q: Why would someone use BOTW’s Blog Directory instead of Google’s Blog Search or Technorati? What’s different about BOTW?
A: Quality over quantity. BOTW offers users splog-free results (no spam blogs).
Q: BOTW is the internet’s oldest web directory. What changes/adaptations do the web directories today need to make to survive?
A: Directory owners need to stay focused on the long view. Too many directories, in an effort to get rich quick, sacrifice on the quality of the listings, or go for the big advertising money. Directory owners need to be prepared for a slow grow, and would be wise to plan on not making very much money for a long time.
Q: I read the official announcement of why the Zeal Directory shutdown. Do you think there is another, untold reason why they shutdown?
A: Zeal was a drain on an already struggling company. Looksmart never had a focus, or a model around Zeal â€“ they simply threw money at it, hoping it would provide some synergies down the road. Unfortunately, that day did not seem to arrive.
Q: What was BOTW’s involvement with the thesis “Backrub” by Sergey Brin and Larry Page?
A: In the paper â€œThe Anatomy of a Large-Scale Hypertextual Web Search Engineï¿½?, Page and Brin cite BOTW in regards to quality of search. So, we had pretty much no involvement, save the role of grateful recipient of the citation ï¿½??
Q: What is your biggest challenge?
A: My biggest challenge is managing our resources effectively. We run a pretty tight ship, with only a handful of full time employees and a freelance crew of approximately 100. We have so much that we try to get done, and only so many people to handle the load.
Another major challenge for us is striking the right balance between comprehension and quality. While we strive to build a comprehensive directory of all information online, we also need to maintain the high standards that people expect when they visit BOTW.
Q: What’s the best part of your job?
A: Two things in particular. First, I am thrilled to be involved with the creation of a quality resource. I love the fact that we are building something with legitimate value â€“ something that will endure.
Secondly, and probably more important, I am lucky to be involved with a great group of people â€“ each of whom share my passion about the directory. After years of laboring in corporate America, I am proud to help run the anti-corporation. Being in the presence of intelligent and dedicated people really helps push me to excel.
Speaking for myself, and all the readers, we really appreciate the wealth of knowledge you’ve shared today.
Kudos to you and the entire team at BOTW