Search data shows that .directory is winning in terms of total indexed pages.
No, this article isn’t about whether .xyz or .club should be considered the most registered new domain name so far.
Instead, it’s a different look at how to measure the most successful new top level domain names.
Matt Bentley, who you may recall from back in the day at Sedo, now runs a search engine optimization service called CanIRank. He collated some data on how many unique URLs on these domain names are indexed in Google. Here’s a snapshot:
This chart represents total indexed pages, not unique domain names.
You might expect .directory to come out on top given that directory sites have, by definition, a lot of pages.
It also makes sense that .photos and .pics are ranked above .photography despite having a lot fewer registrations. Photos (pics) are more numerous that photographers; a photo site has a lot more pages than that of the typical photographer.
Some domain names are dominated by a handful of sites. Fotos.directory takes up basically positions 6-25 in the results for site:.directory.
Obviously, this way of measuring relative “success” is not perfect, but it’s one more way to rank domain name success so far.
This entire chart is a rounding error compared to some existing new TLDs, but it’s not fair to compare to domain names that have been out for a decade.
Perhaps more interesting was data Matt shared about how the indexed domain names are being used:
There’s a whole lot of forwarding going on.
35% of indexed URLs for new TLDs are forwarding to another site. (I noticed that a number of .link domains are being used for URL shorteners and these are getting indexed.) 60% are resolving to the matching website.
So who’s #1? .Directory, by at least one measure.
lennard says
yippie! I own 1 dwi.directory however no content on it yet
ChuckWagen says
guru top with a ninja bottom
George Kirikos says
I’m not sure how they came up with their numbers, but if they simply did a Google search for:
site:directory
(in the case of .directory) and reported the hit count from the first page, that’s *not* a sound method. As most folks should be aware, the Google hit count is just an estimate, and the estimate actually *changes* once you go deeper into the search results.
e.g. I view 30 pages of results per page, and by the time I hit page 3, the hit count has change from 6.47 million, to just *87*, with the statement:
“In order to show you the most relevant results, we have omitted some entries very similar to the 87 already displayed.”
For .clothing, the first page estimated hit count is 854,000, but eventually drops down to just 398 (not 398,000, but 398, LOL).
Jothan says
Not surprising to see so many redirected. I think that a breakdown of those might be more challenging to identify, but it would be worth looking at the re-directions with slightly more granularity… A couple deliberate and reasonable theories on why a portions of the % of re-directions being temporary are:
Deliberate Re-direction: Registration under a specific and purposeful TLD might be intended to be a quick-shortcut to a sub-page of a primary website.
.UNO highlited this as a method for Spanish speaking consumers to more rapidly access Spanish language content that might otherwise be many links (and perhaps not clearly found) deep inside an English-language version of a website.
Temporary Re-Direction: Transitioning from a domain to a new one is a nuanced process if one hopes to maintain their ‘Search Engine Juice’. Google’s Matt Cutts and other Search Engine Marketing thought leaders reccomend a gradual and measured transition over a duration of time. This ensures ranking benefits of an existing name are not lost.
Matt says
Good points Jothan! Of the 35% redirecting, about 2/3 are 301 permanent redirects, and 1/3 are 302 temporary redirects. Of course, that doesn’t tell us much because so many people just don’t set up their servers correctly 🙂
I thought a lot of the redirects might be thinking something like: “hey, I know I’ll want this domain for something down the road, but I’m just not sure what I want to do with it yet, or I am but it will take time to build, so let me just point it at my main site in the meantime.”
For example, one of the .club sites is http://www.ironman.club, and while they’re just redirecting it to ironman.com right now, I bet somebody there is thinking “you know, this would be awesome to have if we ever want an official Ironman Club at some point…:”
John McCormac says
Actually the redirects can be quite useful when parsed, Matt,
The standard practice is to break them down into Internal Site Redirects (Index page being redirected to a lower page on the same site), Holding Page Redirects (Some redirects are pointed to a generic holding page when a domain is not developed or suspended.), PPC lander redirects (Some are pointed to a generic PPC page by the registrant or registrar while the site is being developed), Exact Match Redirect to a site in another TLD (The domain matches the site in another TLD.), Other TLD redirect (The redirect is to a site in another TLD. Very common with keyword domains in a new TLD being directed to the registrant’s site in another TLD, In Zone Redirect (Directing to another site in the same TLD. Again common with keyword/generic domains.)
George Kirikos says
site:guru — 4.96 million hit count on page 1 (seems to match the chart), but when you go through to the next pages, it eventually ends up with a hit count of just 193 (not 193,000, but 193!)
site:photos — 1.85 million hit count on page 1 of Google (similar to the chart), when you through to the next pages, it eventually ends up with a hit count of merely 240 (not 240,000, but 240, LOL)
I think given these results, the authors of the report should redo things, or explain their actual methodology.
Matt says
Hi George! I collected the data. You’re absolutely right, there are many limitations to this data. It’s just intended to provide another point of comparison amongst TLDs, inspired by .Club’s recent blog post defining success as # of active websites (which is a much harder metric to measure than # pages indexed). The reason for the drop-off is not because Google’s estimates are off by several orders of magnitude, but because each of these TLDs have a lot of pages in Google’s supplemental indices, which is understandable if you consider that most of these are brand new sites and therefore not very important in the eyes of a search engine (which is what determines which sites end up in the supplemental index, and which end up in the main index). If you click on the “repeat the search with the omitted results include” link, you can see all of the results in the supplemental index — even though Google doesn’t consider them important yet, they still are legitimate pages on legitimate websites (at least, to the extent that any of them are!), which is why I thought the total indexed pages number was a more valid comparison than the # of indexed pages in the main index only.
George Kirikos says
Those results are pretty meaningless, because a few database-driven sites with automation can produce millions of pages. It doesn’t mean that they’re ending up visible on any competitive search queries by users.
As a counter-example, Alexa isn’t perfect but they do produce a list of their top 1 million sites, which should at least pickup some sites getting real visitors (as opposed to bots). Here are the totals for a few new gTLDs:
.directory – 23 (yes, 23 out of the top 1 million)
.guru — 44
.today — 35
.photos — 4
.pics — 4
.center — 10
.support — 6
(.com sites represent around 577,000 of the top 1 million)
Matt says
Great data point! I did look at a few other metrics for approximating # of active websites (which, as you point out, doesn’t say anything about traffic), but everything else seemed even more limited than Google’s # of indexed pages.
Bill Kara says
I agree with you full George.
Acro says
XYZ FTW! 😀
John McCormac says
Ironically .XYZ may become the first new gTLD to break 100K. Now how many of those registrations ever become websites is another question entirely.
George Kirikos says
The fact that Google hit counts are weak estimates has been known for a long, long time. See, for example:
http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/google-result-counts-are-a-meaningless-metric.html
“To demonstrate for yourself that these estimates are meaningless
numbers, take a few searches and click on the “Next” button to bring
up further pages of results until you reach the last page. You’ll see
that the actual number of results, known once you reach the last page,
will almost always be nothing like the estimated number of results
that appeared on all of the prior pages.”
http://en.wikipedia.org/wiki/Wikipedia:Search_engine_test#Google_distinct_page_count_issues
“Note also, that the number of search string matches reported by
search engines is only an estimate. For example, Google will only
calculate the actual number of matches once the user navigates through
all result pages, to the last one, and even then it places
restrictions on the figure. At times, the “match” count estimate can
be significantly different (by one or more orders of
magnitude) to the total count of results shown on the last results page.”
Mike says
hello people
search.cab
search.ventures
search.photography
are you interested?
Kindly
Mike
Matt says
The really interesting thing will be to see whether or not these numbers have any correlation with renewal rates once those roll around.
I know it hasn’t historically been the role of the registry, but wouldn’t it be awesome if registries who are the “good guys” and invest in their customer’s success with advice, tools, co-promotion, awareness-boosting, etc. end up getting rewarded financially?
Richard says
Anyone interested in these two?
Freelance.Directory
Traffic.Training
offers to [email protected]
John says
How much are you willing to pay me to take it off your hands ????
Tommy says
Exactly your just going to pay renewals, and eventually drop them. Begging people on a domain discussion forum to buy your domains is the writing on the wall. Good luck your gonna need it
Richard says
You two seem to have forgot the internet is still VERY YOUNG. It’s a new-born-baby in the grand scheme of things and within OUR lifetime, the .com will 100% become a distant memory. They will still have value but only as the internet’s first antiques. It really amazes me that people don’t seem able to comprehend how insignificant .com’s will become for the internet. These new gTLD’s will 100% change the way people use the web. It will become a more structured and logical network. These changes have been planned for a VERY long time. They tested the concept almost 15 years ago with the introduction 7 gTLDs including .info.
The web will ALWAYS have skeptics for everything that’s different but mark my words, I guarantee these gTLD domains will not fail. It’s actually common sense when you think about the web DNS. The internet NEEDS these gTLD’s
Bob says
@richard
That was the most erroneous posting I have ever read. The gtld’s will clearly fail and soon too. To think otherwise is delusional.
Joseph Peterson says
It’s funny how people think the same thoughts at the same time.
Just Saturday, I had set myself up to grab exactly the same numbers using the Bing API.
Beat me to the punch! Thanks for saving me time, Matt.
Matt says
Great minds think alike? 🙂
Happy to share the full data with you if you’d like Joseph; just email me: matt at canirank.com
Joseph Peterson says
Late minds think like to think so.
I’d be interested in looking at your data. Sending you an email. Thanks, Matt
John McCormac says
The Google site: operator is incredibly flakey and a extremely poor way of measuring development. Google can’t even solve a simple webspam issue (“natural” and “unnatural” links) so there will probably be duplication and false attribution in those figures.
The classifications do not look like those of a real web usage and classification survey result. (I appreciate that it is not meant to be one.) That’s a highly specialised thing because it is the precursor to building a search engine index. You’ve got to be a search engine developer to understand all the wonderful ways that people can break even the simplest of HTML. There is no breakdown on usage or PPC so even a simple holding page might qualify as an “active” website. I run a survey of approximately 350K domains each month and the parser with the largest number of REGEXs is the one for parsing holding/coming soon pages. The holding page parser used for the larger (>1M domains) surveys is even larger. Some registries get worried when they see the results of real surveys especially when they had been relying on dodgy methodology ones that classify redirects to Godaddy’s PPC lander for undeveloped domain names as a simple redirect (COInternet and its initial iffy usage “figures” which were more press release fodder than real development and usage analysis.) Others such as Eurid engage in limited surveys (5K samples) with a bunch of academics and students trying to classify usage and extrapolate that to the entire TLD – statistically sound but technologically iffy because of poor classification and misunderstandings about the nature of the Web. Analysis has to be done at HTML code level because it is possible to have what appears to be a site in one TLD that’s really a site in another because of incorrect DNS settings. So while a user thinks they are looking at a fully developed new gTLD website, they are really looking at a .COM or .ccTLD website.
Right now, there seems to be a lot of scrambling for information about the new gTLDs and people are trying to convince themselves that their investment is will bear fruit in the future. Web development and usage surveys are how mainstream TLDs are distinguished from non-core TLDs.
It is probably a bit too early to run surveys for some new gTLDs as they are still in Landrush phase. Domains tend to have lifecycles and in the Landrush phase, development and usage is not as high as that in a more mature TLD. Some domains can go for months or years without being developed while others can be up and running with full websites (or cookie-cutter Wordpress installations) from the day they were registered.
Individually, the new gTLDs are small enough to survey. It might be interesting to run a few surveys on the top new gTLDs later this week. Some of the results might not be what people expect.
Lance Wolak (@LanceWolak) says
Given that Google is the viewport for many people seeing and getting to websites, it is an interesting number to watch – the number of indexed pages for a specific TLD. Best to understand this as simply the TLD’s footprint in Google, and look at it as a progress check for just that.
timmayeur says
Well I own quite a few new gTLDs and .DIRECTORY is outperforming them all in terms of traffic. I generate around 12 visits a day from MALL.DIRECTORY.