Top-level domain considerations

Published on September 21, 2020 by

We want to give some insight into top-level domains (TLDs) supported by Intelligence X. Even though there are a total of 1,508 available TLDs (full list by IANA), we choose to only support certain TLDs in order to prevent spam, false positives, and maintain a high quality data set.

Validation of top-level domains appears in these places:

  1. Search term input: Detect when the user enters a valid domain (as well as email address, URL)
  2. Indexing of search results: Detect when a valid domain name appears in text

False Positives

“ZIP” is an example TLD that we intentionally do not support. It has only 2 domains (according to DomainTools), but supporting it would mean millions (if not billions) of false positives, due to its common use as file extension for ZIP files. “test.zip” as filename is on its own indistinguishable from “test.zip” as domain name.

If you try to search for a domain with an unsupported TLD, you will get this message:

Count of supported TLDs

2 years ago we last updated the list of supported TLDs and included 362 domains. Today, we added 29 that meet our criteria for quality. Interestingly, there are some TLDs that met the criteria 2 years ago, but now do not.

Crawling

Intelligence X operates public web crawlers. At the time of writing, 8 TLDs are currently being crawled and more are in preparation.

Related articles

Newsletter 2020-10-19

Published on October 19, 2020 by

October 2020: Hunter Biden, Maltego Transform v4, Decentralized TLDs 🥳 It has been 2 years since the launch of Intelligence X! At this point we would like to thank our users & customers for their trust and we look forward to the future. -Intelligence X Team Make sure to follow us on Twitter for the


An OSINT investigation into the alleged Hunter Biden email

Published on October 14, 2020 by

Disclaimer: Intelligence X is politically neutral. This investigation is based on public information (open source intelligence). tl;dr: It appears that the email addresses appearing in the alleged email are accurate. This does not validate nor refute the content of said email. The New York Post published today the two articles “Hunter Biden emails show leveraging


Newsletter 2020-08-27

Published on August 27, 2020 by

August 2020: Latest News & Statistics Public API keys are retired We have retired public API keys. All 3rd party tools and integrations must use per-user API keys. Note: This also affects SpiderFoot users. We have updated our SDK to reflect that change. You can find your personal API key here: https://intelx.io/account?tab=developer In related news,


Search the blog: