[#3][ComptiaPentest+] Information Gathering phase

"Information is power", as the saying goes. And in most scenarios it's true: having critical information, at the right time, and especially knowing how to use it, can be a great source of power.

Footprinting and Enumeration

The first step in many penetration tests is to gather information about the organization via passive intelligence gathering methods. Passive methods are those that do not actively engage the target organization's systems, technology, defenses, people, or locations. The information gathered through this process is often called OSINT, or open-source intelligence. Among other data that can be gathered, OSINT is often used to determine the organization’s footprint: a listing of all of the systems, networks, and other technology that an organization has. Of course, if you are conducting a white box test, you may already have all of this information in the documentation provided by the target organization.

OSINT

OSINT includes data from publicly available sources, such as DNS registrars, web searches, security-centric search engines like Shodan and Censys, and a myriad of other information sources. It also includes information beyond technology-centric organizational information. Social media, corporate tax filings, public information, and even the information found on an organization’s website can be part of open-source intelligence gathering.

The goal of an OSINT gathering process is to obtain the information needed to perform an effective penetration test. Since the tests will vary in scope and resources, a list of desired information is built for each engagement. That doesn’t mean you can’t work from a standardized list, but it does mean you need to consider the type of engagement, the information you have available, and the information you need to effectively understand your target. OSINT gathering may continue throughout an engagement as you discover additional information that you want to acquire or if you find additional in-scope items that require you to perform more research.

MITRE

The MITRE corporation is a US not-for-profit corporation that performs federally funded research and development. Among the tools it has developed or maintains are a number of classification schemes useful to penetration testers:

- The Common Attack Pattern Enumeration and Classification (CAPEC) list is a resource intended to help identify and document attacks and attack patterns. It allows users to search attacks by their mechanism or domain and then breaks down each attack by various attributes and prerequisites. It also suggests solutions and mitigations, which means it can be useful for identifying controls when writing a penetration test report. (https://capec.mitre.org)

- The Common Vulnerabilities and Exposures (CVE) list identifies vulnerabilities by name, number, and description. This makes the job of a penetration tester easier, as vendors, exploit developers, and others can use a common scheme to refer to vulnerabilities. A CVE listing will be in the format CVE-[YEAR]-[NUMBER](https://www.cve.mitre.org)

- The Common Weakness Enumeration (CWE) is another community-developed list. CWE tackles a broad range of software weaknesses and breaks them down by research concepts, development concepts, and architectural concepts. Like CAPEC, it describes each weakness and how it can be introduced to code, what platforms it applies to, and what happens when something goes wrong (https://cwe.mitre.org)

Location and Orgization Data

While penetration testers may be tempted to simply look at the networks and systems that an organization uses as targets, some penetration tests require on-site testing. That may take the form of social engineering engagements or in-person security control testing, wireless or wired network penetration, or even dumpster diving to see what type of paper records and other information the tester can recover. Each of those activities means that a tester may need to know more about the physical locations and defenses that a target has in place.

At this point in the information-gathering process, it isn't uncommon to find out that the organization has other locations, subsidiaries, or remote sites. This will help you to identify some of the organization's structure, but you will usually need to search for more information to really understand how the target is logically structured.

Electronic Documents

Electronic documents can often help you understand how an organization is structured. They can also provide a wealth of other information, ranging from technologies used to staff names and email addresses, as well as internal practices and procedures. In addition to the information that is contained in the documents, many penetration testers will also carefully review the document metadata to identify additional useful information. Tools like ExifTool are designed to allow you to quickly and easily view document metadata, as picture below

 

In addition to tools like ExifTool that excel at exposing metadata for individual files, metadata scanning tools like Fingerprinting Organizations with Collected Archives (FOCA) can be used to find metadata. FOCA scans using a search engine—either Google, Bing, or DuckDuckGo—and then compiles metadata information from files like Microsoft Office documents, PDF files, and other file types like SVG and InDesign files.

It is important to remember that the electronic documents that are currently accessible are not the only documents that you can recover for an organization. Web archives like the Internet Archive (https://archive.org) provide point-in-time snapshots of websites and other data. Even when organizations think that they have removed information from the Web, copies may exist in the Internet Archive or elsewhere, including search engine caches and other locations.

Employees

Finding out who is employed by an organization can sometimes be as simple as using an online directory or checking its posted organizational charts. In most cases, identifying employees will take more work. Common techniques include leveraging social media like LinkedIn and Facebook, as well as reviewing corporate email addresses, publications, and public records. Social engineering techniques can also be useful, particularly when searching for information on a specific individual or group.

Infrastucture And Networks

Information about the infrastructure, technologies, and networks that an organization uses is often one of the first things that a penetration tester will gather in a passive information search. Once you have a strong understanding of the target, you can design the next phase of your penetration test.

External footprinting is part of most passive reconnaissance and is aimed at gathering information about the target from external sources. That means gathering information about domains, IP ranges, and routes for the organization.

Domains

Domain names are managed by domain name registrars. Domain registrars are accredited by generic top-level domain (gTLD) registries and/or country code top-level domain (ccTLD) registries. This means that registrars work with the domain name registries to provide registration services—the ability to acquire and use domain names. Registrars provide the interface between customers and the domain registries and handle purchase, billing, and day-to-day domain maintenance, including renewals for domain registrations.

The Domain Name System is often one of the first stops when gathering information about an organization. Not only is DNS information publicly available, it is often easily connected to the organization by simply checking for WHOIS information about its website. With that information available, you can find other websites and hosts to add to your organizational footprint.

WHOIS

Domain ownership and registration is maintained by registrars, with each registrar covering a specific portion of the world. The central authority is the Internet Assigned Numbers Authority, or IANA. IANA manages the DNS root zone and thus is a good starting place for searches at https://www.iana.org. Once you know which regional authority you should query, you can select the appropriate site to visit:

AFRINIC (Africa): http://www.afrinic.net

APNIC (Asia/Pacific): http://www.apnic.net

ARIN (North America, parts of the Caribbean, and North Atlantic islands):  http://ws.arin.net

LACNIC (Latin America and the Caribbean): http://www.lacnic.net

RIPE (Europe, Russia, the Middle East, and parts of central Asia): http://www.ripe.net

Each of the regional NICs provides a WHOIS service. WHOIS allows you to search databases of registered users of domains and IP address blocks and can provide useful information about an organization or individual based on their registration information. In the sample WHOIS query for Google shown in Figure 3.4, you can see that information about Google, like the company’s headquarters location, contact information, and its primary name servers, is all returned by the WHOIS query

IP Ranges

Once you know the IP address that a system is using, you can look up information about the IP range it resides in. That can provide information about the company or about the hosting services it uses. The IP address or hostname can also be used to gather information about the network topology around the system or device that has a given IP address. One of the first stops once you have an IP address is to look up who owns the IP range.

Using trace-route (or tracert on Windows systems), you can see the path packets take to the host. Since the Internet is designed to allow traffic to take the best path, you may see multiple different paths on the way to the system, but you will typically find that the last few responses stay the same. These are often the local routers and other network devices in an organization’s network, and knowing how traffic gets to a system can give you insight into their internal network topology.

Active Reconnaissance and Enumeration

Building a list of all of the resources or potential targets of a specific type is important in this state of a penetration test. Once sufficient open-source intelligence has been gathered, testers typically move on to an active reconnaissance stage with the goal of first building, then narrowing down the list of hosts, networks, or other targets. Techniques for each of these vary, so you will need to be familiar with each of the following methods.

HOSTS

Enumerating hosts on a network may be the first task that most penetration testers think of when they prepare to assess a target. Active scans can identify many hosts, and it can be tempting to just rely on port scanners to identify hosts, but there are quite a few other ways to identify hosts on a network, and combining multiple methods can help to ensure that you didn’t miss systems. A couple of other ways to identify systems to keep in mind are as follows:

In a black box test, you typically won’t be able to get this type of information until later in the test, if you can capture it at all. That doesn’t mean you should ignore it, but it does mean that port scanning remains the first technique that many penetration testers will attempt early in an engagement.

SERVICES

Service identification is one of the most common tasks that a penetration tester will perform while conducting active reconnaissance. Identifying services provides a list of potential targets, including vulnerable services and those you can test using credentials you have available, or even just to gather further information from. Service identification is often done using a port scanner.

Port scanning tools are designed to send traffic to remote systems and then gather responses that provide information about the systems and the services they provide. Therefore, port scans are often one of the first steps in a penetration test of an organization.

An important part of port scanning is an understanding of common ports and services. While ports 0–1023 are known as "well-known ports" or "system ports", there are quite a few higher ports that are commonly of interest when conducting port scanning. Ports ranging from 1024 to 49151 are registered ports and are assigned by IANA when requested. Many are also used arbitrarily for services. Because ports can be manually assigned, simply assuming that a service running on a given port matches the common usage isn’t always a good idea. In particular, many SSH and HTTP/HTTPS servers are run on alternate ports, either to allow multiple web services to have unique ports or to avoid port scanning that only targets their normal port.

Service and Version Identification

The ability to identify a service can provide useful information about potential vulnerabilities as well as verifying that the service that is responding on a given port matches the service that typically uses that port. Service identification is usually done in one of two ways: either by connecting and grabbing the banner or connection information provided by the service or by comparing its responses to the signatures of known services.

Operating System Fingerprinting

The ability to identify an operating system based on the network traffic that it sends is known as operating system fingerprinting, and it can provide useful information when performing reconnaissance. This is typically done using TCP/IP stack fingerprinting techniques that focus on comparing responses to TCP and UDP packets sent to remote hosts. Differences in how operating systems and even operating system versions respond, what TCP options they support,  the order in which they send packets, and a host of other details can often provide a good guess at what OS the remote system is running. 

HAPPY PENTESTING!