Prerequisites
Last updated
Last updated
API keys of Passive DNS sources
100% accurate open public DNS resolvers
A VPS(Virtual Private Server)
Whenever a domain is alive on the internet, to access it, a DNS query needs to be made to the DNS resolver. With special probes activated on the DNS resolver, it is possible to record these queries and store them into a database. This doesn't record which client made the request but, just the fact that at some point a domain has been associated with a specific DNS record.
Hence, we can know, what subdomains of a particular root domain existed with the help of these DNS record database. These subdomains in the present time may be alive or dead. (we need further find out which are the valid ones). There exists various services/companies that are doing this work for past several years. Along with this, various companies have their internet crawlers which continuously keep on crawling the whole internet and discover new domains.
There are a number of services/sources(SecurityTrails, Censys, Shodan, VirusTotal, WhoisXMLAPI, DNSDB) that provide such historical DNS data & crawled data. These services provide their API keys so that we can query and retrieve subdomains of our choice.
There are 2 types of passive DNS services:-
1) Allow querying their Datasets freely(partially) A number of sources allow users to freely query their DNS datasets. Check out which sources allow to freely query their dataset here. (we don't need to care about these sources as our subdomain enumeration tools like amass, subfinder, assetfinder will query them and do the work for us )
2) Need to generate API keys to query Datasets Also, a number of sources require you to signup on to their platform and generate a unique API key for yourself so that you are authorized to query and retrieve results from their historical DNS datasets.
Good passive sources provide API keys for a limited period. (7 days/20 days)
They provide a limited amount of API query quota. (50 per day/1000 per month )
Limited query results (2 pages of data)
Yes, absolutely, given below is the comparison between running Subfinder with API keys configured and without.
You can clearly see the difference that using API keys gave me a whopping 198,669 more💥 subdomains.
Further, this passive data would be used to generate permutation/alterations which eventually would give us more subdomains.
There are in total 19 services on which you can sign up and obtain API keys.
I have created a detailed excel sheet about which sources to signup on for, what's the validity of API key, their API key quota, rate limits, etc.
Depending on your consumption of API queries and the validity of API keys, you need to make keep making new accounts at a regular interval of time in order to get the maximum results.
More the time you invest in signing up with passive sources, More the subdomains you get ✨
A DNS(Domain Name System) resolver is a service that manages "name to IP address" translations. The process of DNS resolution involves converting a hostname (such as www.example.com) into a computer-friendly IP address (such as 192.168.1.1). In short, if we need to know whether a domain/host is alive or not; we would need to perform a DNS query.
During various subdomain enumeration techniques like DNS bruteforcing or DNS resolution of a large number of domains, we use a base tool called MassDNS. MassDNS is a simple high-performance tool that is used to check whether a given domain is valid or not. For this purpose, MassDNS needs to be provided with a list of public DNS resolvers. These public resolvers perform DNS queries on our behalf and returns the result. Hence, more the number of public resolvers we provided, the more concurrent DNS queries can be made and thus quicker the output.
Dnsvalidator is a tool that helps us to verify/generate a valid list of open public DNS resolvers. The https://public-dns.info is a website that includes a list of around 28.7K open public DNS resolvers. But some of these resolvers wouldn't be working for us. So, dnsvalidator helps us to verify and return only the valid open public DNS resolvers. Dnsvalidator takes this list of 28.7K public resolvers and queries for its resolution using trusted resolvers like Google DNS(8.8.8.8), Cloudflare DNS(1.1.1.1), Quad9(9.9.9.9).
The above method of using dnsvalidator to populate a list of valid public DNS resolver is too much time and resource consuming. Hence, we can depend upon open-source contributions by other researcher for populating the list of DNS resolvers. Various security researchers/companies have created automations that run dnsvalidator periodically(every 24hrs). One can benefit from such contributions. Below is the periodically verified list of DNS resolvers by Trickest.
VPS(Virtual Private Server) can be called as your own dedicated virtual machine in the cloud.
VPS tends to have higher bandwidth and better DNS resolution capabilities as compared to your local home router system which uses ISP's DNS resolver(slow).
A VPS helps while performing various bandwidth-intensive tasks such as DNS resolution & brute-forcing. Alongside when performing such tasks on a local system generally blows up your wifi making it unusable for other users connected on the same network.
Also, with VPS you can perform tasks 24/7; 365 days, unlike your local system.
There are various cloud providers that provide free 200$ credits like Digital Ocean, Linode, Vultr when you signup for the first time. (CreditCard with international transac enabled required) Referral Links:
Check out the excel sheet here
