Newsletter 2020-08-27

Published on August 27, 2020 by

August 2020: Latest News & Statistics

Public API keys are retired

We have retired public API keys. All 3rd party tools and integrations must use per-user API keys. Note: This also affects SpiderFoot users. We have updated our SDK to reflect that change.

You can find your personal API key here: https://intelx.io/account?tab=developer

In related news, the tool Intel Owl v1.5.0, which “analyze files, domains, IPs in multiple ways from a single API at scale”, was just released and supports our Phonebook API for observables: https://github.com/intelowlproject/IntelOwl/releases/tag/v1.5.0.

New selector type: Bitmessage address

We have added support for Bitmessage addresses. Bitmessage is a “decentralized, encrypted, P2P protocol to send encrypted messages”. Bitmessage addresses are now being indexed across all of our data sources and made searchable.

Example: https://intelx.io/?s=BM-NBshYub7dbBDdzv73kZnru3hoQmjTfTC

Statistics and Data Growth

Sci-Hub update

In March, we started indexing public documents from Sci-Hub. We are now finished! In total, we indexed 84,374,229 documents which equals to 74 TB of data. Here is a breakdown of the selectors:

Total selectors: 237,111,135. Unique:
🔹 52,996,669 URLs
🔹 14,070,158 email addresses
🔹 4,678,421 domains
🔹 476 Bitcoin addresses
🔹 24 IPFS hashes

Tor hidden services

Current statistics of Tor hidden services (.onion domains):

🔹 51,498 active
🔹 3,566,621 inactive
🔹 239,867 spam/deleted/blacklisted

Active means alive within the last 7 days.

An interesting observation is that there are more spam domains than active ones. One recent example of removed spam was 29,630 hidden services having “owncloud” in its subdomain name with 375,814 associated website copies from our index using 1,267 MB of storage.

The road to 100 billion records

As we cross the mark of 25 billion records with 100+ TB of storage, it is time to set sail for 100 billion records. Read more in this blog post.

Hidden feature: Similarity hash

There is a hidden feature of Intelligence X to find phishing copies of darknet websites, even if they are slightly modified – similarity hashes. You can find the similarity hash of websites when you click on a search result and click on the History tab.

Here is an example search: https://intelx.io/?s=8822b95eb6ab37be


Kleissner Investments s.r.o., Na Strzi 1702/65, 14000 Prague, Czech Republic

If you don’t wish to receive this newsletter anymore, please click here to unsubscribe.

Related articles

The road to 100 billion records

Published on August 23, 2020 by

At Intelligence X, we value quality over quantity. Our goal is continuous improvement, sustainability, and stability. As we cross the mark of 25 billion records with 100+ TB of storage, it is time to set sail for 100 billion records. Counting records A single record is an extracted selector (search term) like “test.com”. A search


Newsletter 2020-08-03

Published on August 3, 2020 by

August 2020: Recap of 3rd-party OSINT tools and integrations We are listing all approved 3rd-party integrations here: https://intelx.io/integrations h8mail: “an email OSINT and breach hunting tool using different breach and reconnaissance services” Maltego Transform subfinder: “subdomain discovery tool that discovers valid subdomains for websites by using passive online sources” theHarvester: “The tool gathers emails, names,


Why we are going to block Tor IPs

Published on July 5, 2020 by

tl;dr: Nothing ever good comes out of Tor. We are going to block Tor IP addresses from signup, login, and search starting on July 5, 2020. Our reasons are rooted in: Tor Hidden Services Are a Failed Technology, Harming Children, Dissidents and Journalists. Read this blog post by lawfare. According to our own investigation, Tor


Search the blog: