> For the complete documentation index, see [llms.txt](https://methodological-notes.gitbook.io/methodology/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://methodological-notes.gitbook.io/methodology/recon-strategies-by-other-hackers/recon-with-me.md).

# Recon With Me

Hi Everyone

In this blog, I will cover automating the enumeration part of reconnaissance and finding bugs using the following set of tools.

* [Subfinder](https://github.com/projectdiscovery/subfinder)
* [Chaos](https://github.com/projectdiscovery/chaos-client)
* [Nuclei](https://github.com/projectdiscovery/nuclei)
* [Httpx](https://github.com/projectdiscovery/httpx)
* [Notify](https://github.com/projectdiscovery/notify)
* [Anew](https://github.com/tomnomnom/anew)

![](https://dhiyaneshgeek.github.io/images/recon/logo.png)

**Subfinder**

![](https://dhiyaneshgeek.github.io/images/recon/sub1.png)

![](https://dhiyaneshgeek.github.io/images/recon/description1.png)

* [Download](https://github.com/projectdiscovery/subfinder) and install subfinder using the following command

`GO111MODULE=on go get -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder`

Note: Make sure go1.14+ is installed in your system.

* To run the tool on any of the Target website , use the following command

`subfinder -d google.com`

![](https://dhiyaneshgeek.github.io/images/recon/sub2.png)

* It was able to fetch more than 50246+ subdomains in a short amount of time.
* Then i noticed that we can use `-v` flag to see the verbose output of the sources that are used to enumerate the subdomains.

`subfinder -d google.com -v`

![](https://dhiyaneshgeek.github.io/images/recon/sub3.png)

* Sources like chaos,threatminer,sublist3r,alienvault ,etc,. are being used to enumerate the subdomains of the target.
* In the Subfinder Github Repository it was mentioned that some of the services will not work until you set it up.

![](https://dhiyaneshgeek.github.io/images/recon/sub4.png)

* So i started looking into it to set-up the **config-file** with the API Keys that are mentioned to see what is the **major difference in the results of subdomain**
* Navigate to the following directory

`cd .config/subfinder/`

![](https://dhiyaneshgeek.github.io/images/recon/sub5.png)

`cat config.yaml` to see the config file

![](https://dhiyaneshgeek.github.io/images/recon/sub6.png)

* We can see many of the API Key services are **Empty** , so now are going to fill the necessary API Keys as source for Subdomain Enumeration.

**Note** The below following API Keys are **Free Of Cost** and has a Limited number of request in it.

* binaryedge
* censys
* certspotter
* chaos
* dnsdb
* github
* intelx
* passivetotal
* robtex
* securitytrails
* shodan
* spyse
* urlscan
* virustotal
* zoomeye

**Binaryedge**

**1 :** [Sign up](https://app.binaryedge.io/sign-up) for a free account, verify the account.

**2 :** Login into the account and Navigate to this URL <https://app.binaryedge.io/account/api> and give a name to the **TOKEN** and Click on Generate Token.

![](https://dhiyaneshgeek.github.io/images/recon/sub7.png)

**Censys**

**1 :** [Sign up](https://censys.io/register) for a free account, verify the account.

**2 :** Login into the account and Navigate to this URL <https://censys.io/account/api> and you will be able to get **API ID** and **Secret**

![](https://dhiyaneshgeek.github.io/images/recon/sub8.png)

**Certspotter**

**1 :** [Sign up](https://sslmate.com/signup?for=certspotter_api) for a free account.

**2 :** Login into the account and Navigate to this URL <https://sslmate.com/account/api_credentials> and you will be able to get the **API Key**

Note : 100 queries an hour is **free**.

![](https://dhiyaneshgeek.github.io/images/recon/sub9.png)

**Chaos**

**1 :** Navigate to this URL <https://chaos.projectdiscovery.io/#/>

**2 :** Early access is provided basis on signup and queue and Invite are send out Weekly basis.

**3 :** Contributor access is Provided on the basis of **PR** that is done under `github.com/projectdiscovery/*`.

![](https://dhiyaneshgeek.github.io/images/recon/sub10.png)

**DNSdb**

**1 :** [Sign up](https://www.farsightsecurity.com/dnsdb-community-edition/) for a free community account.

**2 :** It will ask for Company Email , use [Temp Email](https://temp-mail.org/).

**3 :** Create an account and verify the email and get the **API Key**.

Note : It has 30-day renewal (with valid email confirmation)

![](https://dhiyaneshgeek.github.io/images/recon/sub11.png)

**Github**

**1 :** [Sign up](https://github.com/join) for a free account, verify the account.

**2 :** Navigate to this URL <https://github.com/settings/tokens> and generate a **Personal access tokens**.

![](https://dhiyaneshgeek.github.io/images/recon/sub12.png)

**Intelx**

**1 :** [Sign up](https://intelx.io/signup) for a free account, verify the account.

**2 :** Navigate to this URL <https://intelx.io/account?tab=developer> and you will get the **API details**.

Note: Trial 1 week for Free

![](https://dhiyaneshgeek.github.io/images/recon/sub13.png)

**Passivetotal**

**1 :** [Sign up](https://community.riskiq.com/home) for a free account, verify the account.

**2 :** Login into the account and Navigate to this URL <https://community.riskiq.com/settings> and you will be able to get **KEY** and **Secret** .

![](https://dhiyaneshgeek.github.io/images/recon/sub14.png)

**Robtex**

**1 :** Sign in using the google **Gmail Account**

**2 :** Navigate to this URL <https://www.robtex.com/dashboard/> , you will get the **API-Key** details.

![](https://dhiyaneshgeek.github.io/images/recon/sub15.png)

**Security Trails**

**1 :** [Sign up](https://securitytrails.com/app/signup) for a free account, verify the account.

**2 :** Login into the account and Navigate to this URL <https://securitytrails.com/app/account/credentials> and you will be able to get **API Key** .

Note : Monthly Quoto is 50 API Requests.

![](https://dhiyaneshgeek.github.io/images/recon/sub16.png)

**Shodan**

**1 :** [Register](https://account.shodan.io/login) for a shodan account.

**2 :** Login into the account and navigate to this URL <https://account.shodan.io/> , you will get the **API Key** details.

![](https://dhiyaneshgeek.github.io/images/recon/sub17.png)

**Spyse**

**1 :** [Register](https://spyse.com/user/registration) for a Spyse account and verify it.

**2 :** Login into the account and navigate to this URL <https://spyse.com/user> , you will get the **API Token** details.

Note : It has 100 API Token valid for 5 days during the Trail Period.

![](https://dhiyaneshgeek.github.io/images/recon/sub18.png)

**UrlScan**

**1 :** [Sign up](https://urlscan.io/user/signup) for a free account, verify the account.

**2 :** Login into the account and Navigate to this URL <https://urlscan.io/user/profile/> and click on Create new **API Key**.

![](https://dhiyaneshgeek.github.io/images/recon/sub19.png)

**Virustotal**

**1 :** [Register](https://www.virustotal.com/gui/join-us) for a Virustotal account and verify it.

**2 :** Login into the account and navigate to this URL <https://www.virustotal.com/gui/user/username/apikey> , you will get the **API Key** details.

![](https://dhiyaneshgeek.github.io/images/recon/sub20.png)

**Zoom Eye**

**1 :** [Register](https://sso.telnet404.com/accounts/register/) for a ZoomEye account and verify it.

**2 :** Login into the account and navigate to this URL <https://www.zoomeye.org/profile> , you will get the **API Key** details.

![](https://dhiyaneshgeek.github.io/images/recon/sub21.png)

**Now the Final Config File Looks Full!!!**

![](https://dhiyaneshgeek.github.io/images/recon/sub22.png)

Now Let us compare the Results **Before** and **After** Adding API Keys.

**Before API Key**

![](https://dhiyaneshgeek.github.io/images/recon/sub24.png)

**After API Key**

![](https://dhiyaneshgeek.github.io/images/recon/sub23.png)

* From this we can see we got **114817** Addtional subdomains after using **Free** API Keys.
* Only difference is the duration that is taken during the API scan is **slow** than the normal one.

**Httpx**

![](https://dhiyaneshgeek.github.io/images/recon/httpx1.png)

![](https://dhiyaneshgeek.github.io/images/recon/description2.png)

* [Download](https://github.com/projectdiscovery/httpx) and install httpx using the following command

`GO111MODULE=on go get -v github.com/projectdiscovery/httpx/cmd/httpx`

Note: Make sure go1.14+ is installed in your system.

* To run this tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver, use the following command

`cat hosts.txt | httpx`

![](https://dhiyaneshgeek.github.io/images/recon/httpx2.png)

**Nuclei**

![](https://dhiyaneshgeek.github.io/images/recon/nuclei1.png)

![](https://dhiyaneshgeek.github.io/images/recon/description3.png)

* [Download](https://github.com/projectdiscovery/nuclei) and install nuclei using the following command

`GO111MODULE=on go get -v github.com/projectdiscovery/nuclei/v2/cmd/nuclei`

Note: Make sure go1.14+ is installed in your system.

* Scanning for misconfiguration on given list of URLs.

`nuclei -l target_urls.txt -t misconfigured-docker.yaml`

![](https://dhiyaneshgeek.github.io/images/recon/nuclei2.png)

More **Nuclei Templates** can be found here <https://github.com/projectdiscovery/nuclei-templates>.

**Notify**

![](https://dhiyaneshgeek.github.io/images/recon/notify1.png)

![](https://dhiyaneshgeek.github.io/images/recon/description4.png)

* [Download](https://github.com/projectdiscovery/notify) and install notify using the following command

`GO111MODULE=on go get -v github.com/projectdiscovery/notify/cmd/notify`

Note: Make sure go1.14+ is installed in your system.

* Notify also supports piping output of any tool and send it over discord/slack channel as notification.
* Now we need to configure **Slack Webhook URL** under the `notify` config file.
* Navigate to the following directory.

`cd .config/notify/`

cat `notify.conf` to see the config file.

![](https://dhiyaneshgeek.github.io/images/recon/notify3.png)

* From the above image , we can see **slack\_webhookurl,slack\_username,slack\_channel,slack** are madatory to set of a slack notification using **notify**

**Note** : Follow the below steps to create a Slack account and Generate WebhookURL.

**Step 1:** [Sign Up](https://slack.com/get-started#/create) for a slack account and verify the account.

**Step 2:** During the process , it will ask you to setup the **Team Name , Channel Name , Invites**.

![](https://dhiyaneshgeek.github.io/images/recon/notify4.png)

**Step 3:** Navigate to this URL <https://api.slack.com/apps> and click on create app.

![](https://dhiyaneshgeek.github.io/images/recon/notify5.png)

**Step 4:** Enter the **App Name** and **Slack Workspace** as shown below.

![](https://dhiyaneshgeek.github.io/images/recon/notify6.png)

**Step 5:** Choose **Incoming Webhooks** under the App Name and set the Activate Incoming WebhookURL to **ON**.

![](https://dhiyaneshgeek.github.io/images/recon/notify7.png)

**Step 6:** In the bottom of the page you will be able to see **Webhook URLs for Your Workspace**, click on **Add New Webhook to Workspace**.

![](https://dhiyaneshgeek.github.io/images/recon/notify8.png)

**Step 7:** It will ask you for the permission to access slack workspace and the **channel name** and click on **allow**.

![](https://dhiyaneshgeek.github.io/images/recon/notify9.png)

**Step 8:** Now your slack webhookurl is generated as shown below.

![](https://dhiyaneshgeek.github.io/images/recon/notify10.png)

* We have got the Slack WebhookURL , Slack Username and Channel name.

![](https://dhiyaneshgeek.github.io/images/recon/notify11.png)

* Add these into the **notify.conf** , To edit the file use the following command.

`vi .config/notify/notify.conf`

* Uncomment the following fields `slack_webhook_url, slack_username, slack_channel, slack` and add the necessary details.

![](https://dhiyaneshgeek.github.io/images/recon/notify12.png)

![](https://dhiyaneshgeek.github.io/images/recon/notify13.png)

* Sample command to check the configuration works fine.

`curl -X POST -H 'Content-type: application/json' --data '{"text":"Hello, World!"}' https://hooks.slack.com/services/**********************************`

![](https://dhiyaneshgeek.github.io/images/recon/notify14.png)

* Scan for subdomains, check alive domains and send slack notifications using **notify** use the following command.

`subfinder -d hackerone.com | httpx | notify`

![](https://dhiyaneshgeek.github.io/images/recon/notify2.png)

**Anew**

* [Download](https://github.com/tomnomnom/anew) and install this Tool using the following command

`go get -u github.com/tomnomnom/anew`

* This tool is used to compare the output with old file and give us what is newly added in the file

For example `domain.txt` contains a set of sub-domains and now `subs.txt` same set of subdomains but included few additional extra new subdomains.

* Using the following command we can compare both the files and get what is newly added.

`cat subs.txt | anew domain.txt`

![](https://dhiyaneshgeek.github.io/images/recon/anew.png)

**Security Through Intelligent Automation**

* Now by using the above all tools we are going to create a **One Linear**. **subfinder** to find new subdomains , check wheather it is alive or not using **httpx** then scan it with **nuclei** and send us a slack notification of the output using **notify**.

**First Run**

`subfinder -silent -dL domains.txt | anew subs.txt`

Note : `-dL` - File containing list of domains to enumerate.

**Second Run**

`while true; do subfinder -dL domains.txt -all | anew subs.txt | httpx | nuclei -t nuclei-templates/ | notify ; sleep 3600; done`

Note : `while true; do` - Keeps the script alive. `sleep 3600; done` - Run after exactly every one hour.

* The **First Run** is madatory for this process so that it will do subdomain enumeration for the list of domains that you give and save it in a `.txt` file.
* The **Second Run** is used to find new subdomains , check alive status , scan it with nuclei templates and notify us the output.

Note : I highly suggest to run these one linear on VPS system like Digital Ocean or AWS instance.

![](https://dhiyaneshgeek.github.io/images/recon/automation.png)

**Why Automation is needed in reconnaissance ?**

* The chances of getting **duplicates** is **very less** and you will be focusing more on **NEW subdomains**

**For Example**

**Real Scenario**

* I got an alert at 3:54 PM Feb 05, 2021

![](https://dhiyaneshgeek.github.io/images/recon/slack1.png)

* I reported the vulnerability to the program at 3:59 PM Feb 05, 2021

![](https://dhiyaneshgeek.github.io/images/recon/slack2.png)

* Reported got Triaged and Valid !!!


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://methodological-notes.gitbook.io/methodology/recon-strategies-by-other-hackers/recon-with-me.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.