An Introduction to OWASP Amass 4 - Part 7 - In Depth Subdomain Enumeration and Network Mapping
The OWASP Amass project is an open-source, actively developed security tool with extensive community support that focuses on information gathering and reconnaissance. It helps security researchers and penetration testers discover and map the attack surface of their target networks by using a variety of data sources. Whether you are a penetration tester, an auditor, a security researcher or the CISO/IT manager, you have several valid reasons for mapping out the external attack surface of an organisation. This process is also referred to as reconnaissance or information gathering.
Version 4 is a major revision of Amass. If you are familiar with earlier versions then you will need to change your approach to understand how it is organized and how this "framework" works.
In this instalment in our series on OWASP Amass version 4 we take a closer look at subdomain enumeration with the enum feature. This is part 7 of the series. Part 1, is an introduction to the Amass GitHub, Part 2 discusses the data model and the approach to configuration in your workflow, and Part 3 explains a Postgres database setup, and Part 4 explains installation of the CLI tool. In Part 5 we introduced configuration and had our first run of the amass enum command. Part 6 looked at some issues with data sources.
Overview
If you have been following along you should now understand that the Amass ecosystem is a collection of independent components that work together. This should enable independent and collaborative evolution of the ecosystem. This post focuses on the Amass Command Line Interface (CLI) tool and specifically its subdomain enumeration feature.
Enumeration in General
As specified in the user guide, enum performs DNS enumeration and network mapping of systems exposed to the Internet. The enum subcommand populates the configured database with the asset data generated. The enum command is one of two subcommands in the Amass CLI tool along with intel. Version 3 users will recognize that db subscommand is missing from Amass CLI tool (however it remains in the documentation). Each subcommand shares the following general parameters.
-h/help. Provide additional help and program usage.
-config. Identifies the path to the yaml configuration file. See Part 2 for discussion on the config file.
-dir. Path to the directory containing the output files. More on this later since we are using a Postgres database its unclear what this command would do.
-nocolor. Monochrome output. For those who like watching Casablanca.
-silent. Disable output during execution.
So the most basic command, with a scope, database, and datasources file defined in our configuration file, is:
amass enum -config <path to config.yaml file>
with our configuration file being:
scope:
domains: # domain names to be in scope
- owasp.org
ports: # ports to be used when actively reaching a service
- 80
- 443
options:
datasources: "./datasources.yaml"
database: "postgres://<dbuser>:<dbpasswd>@127.0.0.1:5432/assetdb?testing=works"Remember to replace <dbuser> and <dbpasswd> with your actually database user and password credentials.
Enumeration Modes
The enum subcommand has three modes know as normal, active, and passive.
Those familiar in recon will know that Passive Recon refers to performing recon without actually directly accessing the targets assets. Other sources are used who have done that work for us. Think of it as looking at the digital shadow cast by your target. Passive reconnaissance, in general, provide the ability of learning about your target without letting the target know you are looking. Passive mode will only take data from datasources used by Amass.
Active Recon on the other had refers to directly accessing the assets of your target. Active recon can expose you, your system, and its location on the internet.
Normal Mode is the default mode of the enum subcommand. Normal mode provides asset data from sources in the datasource file and refers to DNS to validate findings and further investigate the scope define in configuration file. Based on this definition I would consider Normal Mode a form of passive recon.
amass enum -config <path to config.yaml file>
Active mode will use Normal mode and then perform operations directly on the target defined in the configured scope. The command below directs the Amass CLI tool to perform an active recon and will use the ports defined in the configuration file. For example, in the example configuration file above, ports 80 and 443 will be checked.
amass enum -active -config <path to config.yaml file>
Passive mode will only take data from datasources and not verify the results.
amass enum -passive -config <path to config.yaml file>
The Rest
The Amass CLI tool user guide for enum will list a number of parameters many of which can be controlled from the configuration file. Using a configuration file means you do not need to worry about remembering every command or parameter that you used in the past if you wish to reproduce results. Regardless, we will discuss some of them here and identify their configuration file counterparts.
Scope Parameters
The -addr, -asn, -cidr, and -d parameters all control the scope of your target. These values would be listed under scope: in the configuration file.
The addr parameter allows you to provide IPs and ranges (192.168.1.1-254) separated by commas. This is represented by ips in the configuration file.
amass enum -addr 192.0.2.1,192.0.2.2,192.168.0.3-8,192.168.0.10-192.168.0.20
or
scope:
ips: # IP addresses to be in scope
- 192.0.2.1
- 192.0.2.2
- 192.168.0.3-8
- 192.168.0.10-192.168.0.20The asn parameter represents the Autonomous System Number of the target. An Autonomous System Number (ASN) is a unique identifier assigned to an autonomous system (AS), which is a collection of IP networks and routers under the control of a single organization that presents a common routing policy to the Internet. ASNs are used in routing protocols to determine the best paths for data to travel across interconnected networks. The command line and configuration file accepts ASNs separated by commas (can be used multiple times). Note that not every target has an ASN.
Your Delicate Internet Connection
OWASP Amass is a beast. Particularly in the DNS realm. If you notice that your internet connection goes down when you are trying out amass enum (on a target that you are legally permitted to scan or using -passive) and your children are complaining they cannot access their online game then we need to modify Amass’s behaviour. There are a few ways to do this.
First, ideally, consider running this on a Virtual Private Server (VPS). Everyone’s OPSEC risk is different and you should know yours. That discussion is outside the scope of this blog post.
Secondly, know what DNS servers you are wasting time on. Using verified DNS resolvers in OWASP Amass offers several benefits, particularly in terms of accuracy, reliability, and performance. Here we can define specific DNS Resolvers using the -r flag:"
amass enum -r 8.8.8.8,1.1.1.1
or specify a file of resolvers using -rf:
amass enum -r ./25resolvers.txt
Check out the section below on creating a resolvers file. I have found using a regularly updated resolvers file to have improved results and performance.
Other parameters you may wish to investigate include:
-max-dns-queries: Limit the number of concurrent DNS queries. This helps in reducing the load on your internet connection.-rate-limit: Set a rate limit for DNS queries per second.-timeout: Set the timeout for DNS queries.-max-depth: Limit the depth of recursion during subdomain enumeration.-trqps: Maximum number of DNS queries per second for each trusted resolver
Creating a Resolvers File
I want to have an updated file of known, verified, DNS Resolvers. My bulk source of this information will come from https://public-dns.info/. And specifically we will start with the https://public-dns.info/nameservers.txt file.
Next, we will install a tool called dnsvalidator. Dnsvalidator helps you maintain a list of IPv4 DNS servers by verifying them against baseline servers, and ensuring accurate responses.
┌──(user㉿kanga)-[~/.config/amass]
└─$ cd /opt
┌──(user㉿kanga)-[/opt]
└─$ git clone https://github.com/vortexau/dnsvalidator.git
Cloning into 'dnsvalidator'...
remote: Enumerating objects: 377, done.
remote: Counting objects: 100% (92/92), done.
remote: Compressing objects: 100% (59/59), done.
remote: Total 377 (delta 45), reused 66 (delta 32), pack-reused 285
Receiving objects: 100% (377/377), 98.62 KiB | 1.90 MiB/s, done.
Resolving deltas: 100% (183/183), done.
┌──(user㉿kanga)-[/opt]
└─$ cd dnsvalidator
┌──(user㉿kanga)-[/opt/dnsvalidator]
└─$ pip install .
dnsvalidator git:(master) pip install .
Processing /opt/tmp/dnsvalidator
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
-----------------------8<---------
Successfully installed DNSValidator-0.1Next, we will run dnsvalidator against our public dns list and generate a large list of verified DNS resolvers. I am going to let it run for a few minutes to generate a considerable number of results.
┌──(user㉿kanga)-[~/.config/amass]
└─$ dnsvalidator -tL https://public-dns.info/nameservers.txt -threads 20 -o resolvers.txt
=======================================================
dnsvalidator v0.1 by James McLean (@vortexau)
& Michael Skelton (@codingo_)
=======================================================
[11:33:57] [INFO] [1.1.1.1] resolving baseline
[11:33:57] [INFO] [8.8.8.8] resolving baseline
[11:33:57] [INFO] [103.190.51.106] Checking...
[11:33:57] [INFO] [96.7.137.31] Checking...
[11:33:57] [INFO] [1.226.49.4] Checking...
[11:33:57] [INFO] [71.202.17.98] Checking...
[11:33:57] [INFO] [188.68.41.11] Checking...
[11:33:57] [INFO] [109.73.42.171] Checking...
--------------------8<--------------------------Now that we have a large list of resolvers lets sort and generate a list of 50 we can use with Amass.
sort -R resolvers.txt | tail -n 50 > 50resolvers.txtWe can now use the 50resolvers.txt file to improve our results and the efficiency of Amass:
amass enum -config <config file> -rf ./50resolvers.txt
Alternatively, the resolvers file can be specified in your configuration yaml file using the resolvers section as below:
options:
resolvers:
- <path to resolvers file>Beast to Brute
It should come as no surprise that the Amass CLI tool has a brute force mode. Yes, it can find things from the internet and datasources that will identify subdomains. But sometimes trial and error based on common wordlists will uncover surprises that the internet has yet to spider. For these cases Amass supports a brute force options with a world list.
The -brute flag tells Amass to use brute force, and a wordlist (-w) to brute force subdomains by generating possible subdomains and checking if they exist. Subdomain wordlists can be accessed from several repositories that maintain lists of common subdomain names including SecLists.
amass enum -config <path to config file> -brute -w <path to wordlist>
Brute force will use recursion by default. This can be modified with -min-for-recursive. This parameter helps control the depth and breadth of recursive brute force attacks by setting a threshold for the number of initial subdomain discoveries before Amass starts recursively brute-forcing subdomains of the discovered subdomains. The -max-depth parameter sets the maximum number of subdomain labels for brute forcing. Or we could turn off recursion completely with -norecursive. All these parameters can be defined in the configuration file.
Output
You will notice that the amass enum CLI tool outputs information to standard out. This can be sent to a file instead using the -o command.
amass enum -config <path to config file> -o <output file>
The Full Monty
Of course, for all the commands, check out the user guide.
Wrap Up
In this installment we covered parameters to refine enumeration using DNS resolvers and brute force parameters. Perhaps there will be a future blog post to view results. We can dream.
