At Intelligence X we categorize data sources into buckets. Buckets can be used as filters and to broadly identify the source of individual search results. For example, the bucket “Darknet Tor” indicates the result origins from some a Tor hidden service (.onion domain) and was collected by our Tor crawler.
Buckets have human readable names and machine readable names, in the above example, it is “darknet.tor”. Another one would be “web.public.com” for data coming from the public web .com domains.
Note that access to some buckets is restricted to paid licenses, while some may be available as partial preview, and others may be available fully redacted, depending on your license.
You can use buckets as filters to only search specific buckets. Specifying “web.public.com” in the buckets filter (for the API endpoint /intelligent/search) will only search in that bucket.
Below is a non-exhaustive list of the most important buckets available. Buckets may be added and remove at any time.
| Bucket | Info |
|---|---|
| darknet.i2p | I2P Darknet, .i2p sites |
| darknet.tor | Tor Darknet, .onion sites |
| documents.public.scihub | Documents Public: Sci-Hub |
| dumpster | Dumpster: Everything that does not fit anywhere else |
| dumpster.web.1 | Dumpster: Interesting other websites with high-value information |
| dumpster.web.ssn | Dumpster: SSN related websites |
| leaks.logs | Bot logs: Data collected by and leaked from viruses |
| leaks.private.general | Restricted Leaks |
| leaks.public.general | Public Leaks |
| leaks.public.wikileaks | WikiLeaks: Published leaked emails and documents, Snowden Documents, Cryptome |
| pastes | Pastes |
| patent.us | Patents: US |
| usenet | Usenet |
| web.gov.ru | Russia Governmental Public Sites |
| web.public.af | Public Web |
| web.public.ams | Public Web |
| web.public.aq | Public Web |
| web.public.business | Public Web |
| web.public.cee | Public Web |
| web.public.cn | Public Web |
| web.public.com | Public Web: .COM |
| web.public.de | Public Web: Germany. AT, DE, LU, CH |
| web.public.eu | Public Web: EU |
| web.public.gov | Public Web: Government US |
| web.public.info | Public Web: .INFO |
| web.public.kp | North Korea: Public Government Sites |
| web.public.mea | Public Web |
| web.public.misc | Public Web |
| web.public.net | Public Web: .NET |
| web.public.nord | Public Web |
| web.public.oc | Public Web |
| web.public.org | Public Web: .ORG |
| web.public.peer | Public Web: Decentralized blockchain based TLDs, BIT, BAZAR |
| web.public.ru | Public Web: Russia. UA, KZ, RU |
| web.public.social | Public Web |
| web.public.tech | Public Web |
| web.public.we | Public Web |
| whois | Whois data |
