Dmoz.org or ODP (the Open Directory Project) is a web directory which was originally created in 1998 as Gnuhoo by Rich Skrenta and Bob Truel while working at Sun Microsystems. The first controversy started soon Soon after the launch the directory and lead to renaming it to NewHoo after wide spread accusation that the name of the directory implied open source yet the directory was a closed source which negates the spirit of free software of GNU projects which the directory was originally named after.
The directory was sold to Netscape which is now part of AOL/Time warner and currently boasts of having 5,229,762 sites, 71,337 editors, in over 590,000 categories. The importance of the directory increased and its status and the status of websites listed in it also increased despite the Internet trends of using search engines to locate websites faster than navigating through a multitude of categories in a web directory to find what you’re looking for. The popularity and importance of Dmoz to webmasters followed the increased importance and relevance big search engines like Google, Yahoo, and MSN placed on the directory and on sites listed in it. This can be seen clearly in search results with sites listed in Dmoz listed on top of most search results for most search term. Another value that has been closely associated to Dmoz was Pagerank, which is a value Google places on web pages, one of the most important factor that drives Pagerank up when Google does its calculation of Pagerank is the number of back links, and the Pagerank value of those linking web pages.
Dmoz is important not only because it is a great web directory, but because many search engines adopted it as its de facto directory because of the ease of importing the the whole directory data and reusing it as its own. Google’s own web directory is based on that of Dmoz. The estimated number of web sites that use Dmoz’s data as a base for their own directories, and the number of mirror sites of Dmoz are not known exactly but it can be argued to be in the thousands. This number translates to a huge number of back links to websites listed in the directory. This alone makes Dmoz very important to any serious webmaster in their website’s marketing strategy, contrary to what Dmoz editors on their public web forums try to convince webmasters otherwise.
Having a very valuable Directory at their disposal the editors of Dmoz have a great deal of power at their finger-tips and they know it. I say they know it because of the over the top attitudes of take it or leave it, and its our way or the highway attitudes expressed at the resource-zone forums, the hangout of the egotistic editors. Many webmasters depend on their websites for a living and hence they try to market their websites with everything available to them in order to ensure a steady stream of income. When there is a perception that there is a great resource that can help make or break a website by the mere fact of being listed or not listed in Dmoz makes it hard for webmasters to swallow. This is not a bad thing nor should it not be controlled by competent editors. yet the stories of unethical editorial behavior, kickbacks, and revenge and overwhelming feeling of corruption and egotistic attitudes towards webmasters makes a person cringe at the mere mention of Dmoz submission.
There are many other stories and complaints about Dmoz, about its editors, about the way they run the directory, about the refusal to listen to suggestions for improvements, about their lack of sensitivity to webmasters, about the length of time it takes to review a website though they have thousands of editors, about their inability to get with the time and roll out better measures to control and run the directory while minimizing abuse and setting in place tools that help in enforcing a fair and equatable checks and balances approach when including websites.
There has been many suggestions to the directory and its editors, most have been dismissed as unworkable, or too much of a burden on the directory and its army of editors to handle, yet they forget that they have a lot of responsibilities to the general web users and webmasters. A constant theme with the editors is that the directory is here to serve the public i.e. the surfer/visitor public and not the webmasters, yet the webmasters are the ones who made the whole Internet what it is right now, the webmasters work is what the public seeks. Measures to accommodate webmasters and release that they DO count and that the directory does need them is required and DMOZ does need to reach out and work with webmasters to help make the directory better and more reflective of the reality of the Internet.
When looking at Dmoz as a project there are a multitude of features that are badly lacking and in need of improvement, some of the needed improvements are critical to the well being of the directory and to its survival, others are nice to have and will help make the directory better and help give the project the respect that it needs and is currently lacking.
One of the most important improvements is a system that will make submission and review faster than the current system which can take up to 3 years for a website to be reviewed and added to the directory. This is not accepted today as it was not accepted in the first days of the Internet. In todays world of constant change, it is very hard to even find a decent website that does not make changes weekly or daily let alone yearly. A site submitted a year ago may not be the same a year form now and maybe rejected on that bases alone.
A system that will decrease the SPAM the directory receives is the first step to improving responsiveness to site submission and review process. Editors claim that over 95% of websites that they receive can be classified as SPAM or BOT submissions. Captcha (a way to control web forms, which distinguishes between human and automated robotic entries and submission) or a similar measure should be put in place ASAP. If this measure is implemented to weed out 90% of web site submission the total wait time will improve accordingly. A web site that might take 6 months to be reviewed will be reviewed in 18 days (6*30*0.10=18). That alone would be a great improvement and it would elevate the heavy load on editors.
The second critical improvement to the directory and to its image would be in implementing a system of better control over editors and their power in accepting and rejecting websites, especially those that directly compete with their own websites or websites that they are affiliated with. A good start would be by implementing a system that marks websites added to the directory with an existing affiliation to an editor. Currently Dmoz require editors to declare their association, yet the public does not have access to base any complaints that is supported by this claim. Dmoz itself does not have to publish this association in the public directory but should make sites associated with editors clearly stated in their public profile, so when someone has a complaint about how a directory branch is edited they can find any relevant information in the editor profile where they can base their investigation of any unfair editorial claims.
Another benefit of the above would be helpful for the directory mirror sites when editing listing and deciding on a relevancy of websites to be included in their implementation of the directory data. I know first hand of some of the major implementation of web directory based on Dmoz data that administrators would like to know which websites exactly belong to editors which may need to re-reviewed by a third party before inclusion in their directory but the volume of the links in the directory makes it hard to weed out any sites that are implanted unfairly by corrupt editors and are being spread unintentionally throughout the Internet. This is in no way meant to punish or remove sites belonging to editors, but after all the accusations and controversy this may be needed and website using the Dmoz data can decide on their own what to do with those websites.
There are many more things that can be done to improve the directory but the above two suggestions would make a huge difference in restore the respect, the image and the spirit that the directory was created under.