Shimon Sandler

Web Directories – Interview with Greg Hartnett

I recently got to know Greg Hartnett from the web directory: Best of the Web… or BOTW for short. BOTW is the internet’s oldest web directory. Greg has been in the search industry since the beginning. As a matter of fact, BOTW is the first reference in Larry Page and Sergey Brin’s college thesis paper that started Google (I ask about that during the interview). In my opinion, there is no better person to interview about web directories. Enjoy!

Q: What is a Web Directory, and how is it different from a Search Engine, or Internet Yellow Pages?

A: Typically a directory is categorized and sorted by humans – actual people review each site, describe and title it, and categorize it accordingly. Each site is reviewed to ensure editorial guidelines are met, helping to keep spam at bay. The search results from a directory are limited to the pool of sites that are listed within the directory. The Internet Yellow Pages would be an example of a niche directory, as they tend to deal with commercial sites exclusively.

By contrast, search engines tend to be driven by spiders attempting to crawl all of the content online, and sorting the results according to proprietary algorithms. Search engines draw their results from a much wider set of data, and use filters to combat spam and irrelevant results.

Q: What is the benefit of getting your site listed in a Web Directory?

A: Getting listed in a quality directory can prove very beneficial to a site owner. Through proper categorization, a site owner should expect relevant traffic from the users of the directory.

Additionally, most search engines tend to value listings in a directory as a trusted link. Due to the editorial control of a directory listing, search engines view a listing in an authoritative directory as a sign of confidence that your site is a quality resource.

Q: Are web directories just acceptable Link Farms?

A: That would really depend upon the type of directory you are referring to.

A link farm is no more than a collection of links, typically uncategorized, and displayed in a haphazard way. Link farms are designed for the sole purpose of manipulating search results, with no benefit for a user, and little, if any benefit to the site owner.

A web directory, while also a collection of links, is categorized, and all listings undergo editorial review. The directory should be built in a manner that is intuitive and provides users with relevant resources. A directory is built for the user, not for the search engines.

So a quality directory would have no resemblance to a link farm, while a directory built solely as a means for webmasters to pay their way to higher search listings would have a lot in common with a link farm.

Q: Are there any web directories, or “Spam Directories” we should stay away from?

A: Absolutely. As is the case in any industry, there are some businesses you should work with, and some you should steer clear of. A site owner should do some homework, and make sure that they have confidence in the quality of the directory. Some things that may signal a sub-par directory:

– Run of site links
– Listings of poor quality
– Javascript links to sites
– Overuse of keywords in titles and descriptions
– Directories less than a year old
– Directories with overpopulated top level categories, and empty sub categories
– Directories powered by an off-the-shelf script

Basically, go with your gut. If you come across a directory that doesn’t feel right to you, or if you have doubts as to the quality of the directory, then pass on submitting. There are plenty of places online for you to find quality – don’t settle for less.

Q: Which directories are considered to be the most trustworthy, and why?

A: I think that Yahoo and DMOZ would inarguably rate as the top directories. Their comprehension, user base, commitment to quality, and age help to set them apart. Of course, I’d like to think that BOTW is regarded as a trustworthy and authoritative directory, but my opinion might be considered a little biased. I also think that and Microsoft’s bCentral do a good job with their directories.

Q: What reasons would BOTW decline a site submission?

A: We have a pretty straightforward set of guidelines. If the site contains a substantial amount of unique content, relevant to the category, it should get in. It also helps if the site is designed well, and is easy to navigate. Most sites are rejected because of poor content.

Q: Why do some directories have free submission, and others have a fee? What’s the difference?

A: With few exceptions, directories are run as businesses. As such, the owners would like to make a profit. So, most directories worth their salt will carry some type of fee. I think that directories not charging any type of review fee, or not monetizing their submissions, are short sighted, and will have trouble with scalability.

Additionally, many directory owners have a submission fee to simply discourage spammers. A site owner is going to think twice about submitting 25 subdomains of his craptastic, made-for-Adsense site if there is a fee associated with submitting each one.

Q: How is BOTW’s submission fee determined?

A: Running a successful directory is a resource-intense operation. It takes a lot of human capital, and quite a bit of technology to make everything run smoothly. We try to price our review fee to allow for directory growth, while striking the right balance of value for the site owner. More than anything though, the review fee is priced upon supply and demand, and I think we have found a nice price point that enables us to grow, while still providing the site owner with a value.

Q: Is it more important to get a link on a relevant niche directory, than a comprehensive directory like BOTW?

A: It is more important to get a link in a quality directory – period. Niche or general is not really important – it’s the quality that counts.

Q: Is it important for online marketers to use the smaller web directories, or is submission to BOTW good enough?

A: Submission to BOTW is definitely not good enough. While a good start, there is much more for a webmaster to do than simply submit to BOTW. I would definitely look at smaller web directories, bearing in mind what I said previously about quality. I would rather submit my site to just a handful of quality directories than have a link in every site from some directory list.

Q: Google said it will devalue links that are bought for the sole intention of increasing PageRank. BOTW costs money, and many people buy it for the PR. Are you worried BOTW’s links will be devalued?

A: I try not to concern myself with those matters very much. We are fully committed to building a quality directory – a resource for users to find relevant information. We focus on doing our thing, and let the search engines do their thing. The chips fall where they may.

It is important to note that webmasters can’t buy their way into BOTW. We do charge a fee for an expedited review, and all commercial sites are required to pay for review. Site owners pay for the review – which by no means guarantees a listing. People pay and get rejected frequently.

Also, we do accept free submissions for non-commercial sites. Though there is no guarantee on review time, we do get around to looking at the free submissions too. Moreover, for every site that we add via the submission process, our editors add more than a hundred other sites on their own. The directory build is a colossal undertaking – if we relied solely on submissions, the project would be a shadow of what it is today.

Q: For people concerned about Google PageRank, can people rely on web directories as a valuable link in their link portfolio?

A: A listing in a directory is a valuable link in the portfolio of all webmasters –those concerned with PageRank or not. The editorial control of the directory lets humans and spiders know that the listing is relevant to the category, and can be trusted to provide quality content.

Q: Should site owners expect click-thru traffic from a web directory? Or, are web directories really just for link popularity?

A: Yes, you should definitely expect click-through traffic. Of course, traffic will vary depending upon the directory, and upon the category in which you are listed.

Q: What are the top 3 most popular categories, and their top sub-category?

A: Our most popular categories are: Adult, Business, and Society. Over the course of the last six months, we have noticed a marked increase of interest in the Regional branch – part of a growing trend towards local search.

Q: What if someone submits their site, and after inclusion in the directory they decide to radically change the site? Is there some kind of detection process in place for deception?

A: Sites are reviewed by editors on a continual basis. At a minimum, all sites are reviewed once a year to ensure that they are still relevant to the category. That is one of the reasons for our annually recurring fee.

Q: Are there any enhancements, or new features we can expect from BOTW?

A: Absolutely. We’re never content resting on our laurels, and we are continuously developing new products and services.

Just recently, we enhanced our search functionality on the Best of the Web blog directory, and going forward we will pursue additional opportunities. The social networking aspect of the web really fascinates me, and I’d love to see us make more or a splash in that arena.

Q: Why would someone use BOTW’s Blog Directory instead of Google’s Blog Search or Technorati? What’s different about BOTW?

A: Quality over quantity. BOTW offers users splog-free results (no spam blogs).

Q: BOTW is the internet’s oldest web directory. What changes/adaptations do the web directories today need to make to survive?

A: Directory owners need to stay focused on the long view. Too many directories, in an effort to get rich quick, sacrifice on the quality of the listings, or go for the big advertising money. Directory owners need to be prepared for a slow grow, and would be wise to plan on not making very much money for a long time.

Q: I read the official announcement of why the Zeal Directory shutdown. Do you think there is another, untold reason why they shutdown?

A: Zeal was a drain on an already struggling company. Looksmart never had a focus, or a model around Zeal – they simply threw money at it, hoping it would provide some synergies down the road. Unfortunately, that day did not seem to arrive.

Q: What was BOTW’s involvement with the thesis “Backrub” by Sergey Brin and Larry Page?

A: In the paper “The Anatomy of a Large-Scale Hypertextual Web Search Engine�?, Page and Brin cite BOTW in regards to quality of search. So, we had pretty much no involvement, save the role of grateful recipient of the citation �??

Q: What is your biggest challenge?

A: My biggest challenge is managing our resources effectively. We run a pretty tight ship, with only a handful of full time employees and a freelance crew of approximately 100. We have so much that we try to get done, and only so many people to handle the load.

Another major challenge for us is striking the right balance between comprehension and quality. While we strive to build a comprehensive directory of all information online, we also need to maintain the high standards that people expect when they visit BOTW.

Q: What’s the best part of your job?

A: Two things in particular. First, I am thrilled to be involved with the creation of a quality resource. I love the fact that we are building something with legitimate value – something that will endure.

Secondly, and probably more important, I am lucky to be involved with a great group of people – each of whom share my passion about the directory. After years of laboring in corporate America, I am proud to help run the anti-corporation. Being in the presence of intelligent and dedicated people really helps push me to excel.


Thanks Greg.
Speaking for myself, and all the readers, we really appreciate the wealth of knowledge you’ve shared today.
Kudos to you and the entire team at BOTW🙂

Be Sociable, Share!


  1. Ernster says:

    Awesome stuff, I love directory owner interviews:)

  2. “We have a pretty straightforward set of guidelines. If the site contains a substantial amount of unique content, relevant to the category, it should get in.”

    Then why is your company owned site listed in your blog directory since it just scraps content from other blogs and wraps ads around it?

  3. good question ross.

    as you are aware, since you formerly wrote for that blog, it was not always an aggregator site. thanks for pointing it out though.

    i am sure that there more than a handful of sites in the blog directory that need some looking at. if you come across any others, feel free to let us know.

  4. Kind of backtracking my way from an old blog in, and came here. It seems running a directory is a hobby to everyone and their dogs. Which I am guilty of also.

    What is said in the interview about having to seek out and add quality sites to your directory by yourself besides depending on submissions to trickle in is very true. Otherwise a decent directory can never be a great directory if everything was automatic. Thanks for the good insights.

  5. Well i would have to say, by submitting your website to a quality directory, you can immediately get some quality points from google. But Please remember these quality directories weren’t always quality and popular. So whose to say what and how will be a quality directory in the future. I would submit monitoring sites that seem to do a lot of promoting. It will cost you less to submit to them now, than to wait until they get popular.

  6. Thank you Greg and Simon for some great points. Talking about quality control versus payment for listing, payment for review. Would not webmasters or owners of most quality resources pass on a listing for payment, because they have already established both reputation and numbers of incoming links. Would not it be better to be able to invite owners of quality recourses to be listed for free, just to keep the quality of your directory on the level, i definitely advocate this approach with my directories.

  7. What is up with the link farm question? A web directory can be created as a business directory and charge fees to get listed. If people submit to gain position in the SERPs, that is their business.

  8. I completely agree with Simon in relation to a “slow grow” approach, directory owners need to look at quality over quantity and to avoid a get rich quick scheme. There are no shortcuts to creating a quality directory.

  9. Good article and useful information.

  10. I was looking at some strong Directories for link-buys and I’ve narrowed my search to three that I’ve decided to buy links on:
    Best of the Web Directory
    Yahoo Directory
    Aviva Directory

    Any other ones I should target?


  1. […] Shimon Sandler, blogger and search engine marketer, interviewed me this morning about my experience with web directories. I really enjoy these types of interviews, as they get me thinking about the business at a macro level. Finding out what your users have questions about really helps to refine your product or service. […]

  2. […] The last week has felt like a trip in Bob Massa’s Magical SEO Time Machine as Directory listings, directories and seo relevance and trust building links have taken center stage in many a search engine discussion. One of the latest SEO Excellent Adventures is taking place on Shimon Sandler’s blog with an Interview with Best of the Web’s Greg Hartnett. […]

  3. […] Simon Sandler posted a informative interview of Greg Hartnett the owner of BOTW directory on the subject of Web Directories. Greg answered questions on topics from the Zeal directory closing down to the qualities of a Spam Directory. […]

  4. […] Web Directories – Interview with Greg Hartnett Some thoughtful questions and answers with the owner of the Best of the Web directory. […]

  5. […] For an inside look at the operations of top-tier directories, check out the interviews with Greg Hartnett of Best Of The Web and Shawn Walters of Uncover the Net. […]

  6. […] Shimon Sandler interviewed Greg Hartnett the other day. I didn’t know that Best of The Web is the oldest web directory did you? Here is an interesting snippet: Q: What was BOTW’s involvement with the thesis “Backrubâ€? by Sergey Brin and Larry Page? […]

  7. […] Greg Hartnett, the President of Best of the Web, otherwise known as BOTW (pronounced BotWah). My 1st interview with Greg was back in April 2006. So much has changed in the Search Industry and Social landscape since my […]

Speak Your Mind