An Introduction to OWASP Amass 4 - Part 2 - The Data

The OWASP Amass project is an open-source, actively developed security tool with extensive community support that focuses on information gathering and reconnaissance. It helps security researchers and penetration testers discover and map the attack surface of their target networks by using a variety of data sources. Whether you are a penetration tester, an auditor, a security researcher or the CISO/IT manager, you have several valid reasons for mapping out the external attack surface of an organisation. This process is also referred to as reconnaissance or information gathering.

Version 4 is a major revision of Amass. If you are familiar with earlier versions then you will need to change your approach to understand how it is organized and how this "framework" works.

In this instalment in our series on OWASP Amass version 4 architecture, we provide a brief installation overview, introduce the Open Asset Model and asset-db, and discuss configuration. This is part 2 of the series. Part 1, an introduction, is here.

The Amass Ecosystem

As mentioned in part 1, Amass was a standalone command line tool prior to version 4. The project team felt that for the project to continue to serve the needs of the community it needed to become extensible, it needed to be an “ecosystem” of components. Central to this ecosystem of asset detection and collection is a data model.

The Open Asset Model (OAM)

The OAM is the central data model for the entire Amass “ecosystem”. The OAM resides as a separate repository within the Amass GitHub Project. Within the OAM repository lies the definitions that describe both physical and digital assets that belong to both organizations and individuals along with their interconnected relationships. As with any open source project it is community driven.

At a basic level, a security researching using Amass does not require intimate knowledge of OAMs details. But understanding its existence and role are important to understanding what and how Amass manages asset discovery data.

The Amass Asset Database (Asset-DB)

Amass needs to store what it finds. Prior to version 4, Amass stored its data in a SQLite database. Amass 4 still supports SQLite, but has been expanded to support a Postgres database. The decision to use SQLite versus Postgres will depend on your workflow and project management. If you have several different targets and you wish some form of compartmentalization without managing separate SQLite files then Postgres is a viable option.

As another component in the Amass 4 ecosystem, the database interaction code, resides in the asset-db repository.

Creating, managing, and configuring the database, and Amass to use it has several steps which we will cover in a dedicated blog post.

Amass Configuration

Prior to version 4, Amass configuration was a single key-value pair file. This made it harder to manage if you wanted to compartmentalize information between different targets and engagements. With version 4, data sources (a configuration file containing API key to different OSInt resources) and target configuration has been decoupled providing greater flexibility across engagements. This means you can maintain and evolve a configuration of data sources separate from each project.

Project Configuration

Different projects can have a different configuration file with a different target configured. Here, we can use the word “project” and “engagement” interchangeably. The project configuration file references the data sources configuration file, and this line will probably never change for most users. The project file is in YAML format:

scope:
  domains:
    - example.org
options:
  datasources: "/home/someuser/.config/amass/datasources.yaml"
  database: "postgres://<db-user>:<db-password>@127.0.0.1:5432/assetdb" 

The simple example above defines the scope (or target) of the project, here defined by a domain of “example.org”. The options section identifies my defined data sources file (also in YAML format) as well as the postgres database I created.

What the example project configuration should demonstrate is how I can define different data sources, asset databases, and targets based on my workflow. One workflow may keep the data sources configuration file the same across many projects. However the target scope may differ for each database implying my database server has a separate database for each project. This would he compartmentalize project data and reduce the risk of exposing another client’s data to the wrong client.

When I run the Amass command line tool to enumerate a target I then specify my project configuration file.

amass enum -config ./target-config.yaml

A more complicated example of project configuration can include resolvers file (a list of DNS resolvers we know work) and target ports:

scope:
  domains: # domain names to be in scope
    - example.org
  ports: # ports to be used when actively reaching a service
    - 80
    - 443
options:
  resolvers: 
    - "/home/username/.config/amass/25resolvers.txt" 
  datasources: "/home/username/.config/amass/datasources.yaml" 
  database: "postgres://dbuser:dbpasswd@127.0.0.1:5432/assetdb?testing=works" 

Amass Installation

Some linux distributions already have the main Amass command line tool installed. Kali is one example, however it has the Amass command line tool but not a database or oam-tools. In other cases, you will have to install the Amass command line tool yourself. Here, the Amass Install Guide will help you either with a docker container install or using the package manager for your system, or building from source (requires updated Go install on the target system).

For the remaining components, this will vary based on what your are installing.

Wrap Up

This blog instalment of the OWASP Amass 4 ecosystem covered the basics of the data model and database. It also covered the new way Amass 4 handles configuration of projects and data sources.

The next instalment of this blog series will cover installation and configuration of the asset-db repo.