Newsletter 2019-11-12

November 2019: Tools and Public Web

🕵🏻 OSINT Tools: New Tabs & Updates

We have updated our free OSINT tools: intelx.io/tools

✅ New YouTube DataViewer: Shows metadata and thumbnails
✅ New Vehicle Identification Number (VIN) tab
✅ Added 4 sites to Domain search, including crt.sh, Robtex, and Hurricane Electric
✅ Added 3 sites to IP search

The YouTube DataViewer is forked from Amnesty International’s version. You can access it here: intelx.io/tools?tab=youtube

🌍 Public Web

As announced in our last newsletter from October, we are now indexing the public web. We started to crawl domains from certain TLDs (top-level domains). There are new search categories for the public web, including “Web: Germany”, which covers DE, AT, CH, and LU and “Web: Russia”, which covers RU, KZ, and UA.

More TLDs are to follow – we are optimizing our algorithms and storage handling before scaling up. Even though we’re crawling only a handful of TLDs, and currently have set a crawling depth of just 10 sub-pages per domain, it takes up 2 TB of space already.

This new data category contains already more than 824 million selectors. Click here for an example of a search result, that shows historical copies of spiegel.de: intelx.io/?s=spiegel.de&m=30

Open Sourcing our “fileconversion” Library

We open-sourced our Go “fileconversion” library: github.com/IntelligenceX/fileconversion

It supports the conversion of many files to plaintext and provides other file format related functions. It was tested on 184+ million files and is used by our search engine. We are happy to give back to the open source community.

✉️ Connect with us