Scanning your external perimeter

Being in the blue corner of the ring is always tough. Red and Black teams are usually less restricted, more bold and better equipped.

This guide is for the compact security teams that do not have fancy commercial scanners and are reliant on the open source tools in their daily struggle to protect the company #infoassets. What follows below is just simple attempt to audit the perimeter and do basic footprinting with some level of fingerprinting of what is exposed on the perimeter. This guide is about network scanning, and not going into Web or any other application level.

You have to check what happens on your perimeter regularly. This helps verify that firewall rules, NAT and publicly accessible resources are accounted for by your team. You would be surprised how often your perimeter scan shows unidentified resources that you know nothing about.

The following text is just for your educational purposes and SHOULD NOT, i cannot stress this enough, should not be followed without taking precautions:

1. Take a management approval and target all the public IP addresses and all the domains related to your organization only.
2. Inform corporate IT and Network teams about your exercise and plan it with them.
3. If some of your domain names are pointed to the IPs of the third parties, you must get their approval and work out a plan to run the scan against those IPs.

The general guideline is to have proper authorization in written form. Failing to do this will probably land you in troubles and getting fired may be the least of them.

To scan your perimeter one needs server/computer outside the company network. I would recommend renting a small VPS with public IP, and yet again read the VPS provider’s policy regarding scanning activities. Some of them may ban you from using their resources for running the scan from their IPs without proper prior notification.

If your range of public IPs is about 20-30 than nmap will suffice and you can skip the next part with masscan. If you have more than 100 IPs, I would recommend using masscan for initial footprinting and its output for the later nmap scanning.

To know more about masscan I recommend reading this README of the author and this cheatsheet. These two links deal with the tool explanation, installation guidelines and other interesting stuff.

Because scanning can take a long time, I usually fire up tmux:

	tmux new-session -s scanning

This command will create a tmux session with the name scanning and give me possibility to close the ssh session and return to the scanning process whenever I want.

Rename the tmux window to main.

To start the scanning with masscan lets use:

	masscan XXX.XXX.XXX.0/XX -p0-65535 -oG masscan --rate 100000

If you would like to throw in UDP ports use:

	masscan XXX.XXX.XXX.0/XX -p0-65535,U:0-65535 -oG [file_name] --rate 100000		

This command will initiate a scan of the specified network range for all the ports that are there with 100K packets per second and output the results to the file with name defined by the -oG argument in greppable format. Benefit of masscan is that it can scan 1024 hosts for all the TCP and UDP ports range in under 5 hours. That is bloody fast, but noisy as hell.

Once the scan is finished you need to dissect the output. I usually transfer file to my PC and use something like Notepad++ or Sublime Text to create couple of files that I will use for further analysis.

Modify the masscan output file to CSV so that you have only 3 columns: IP, Port Number and TCP/UDP. Open this CSV file in Excel and create a pivot table with IP column as ROWS, Port Number as COLUMNS and Count of Port Number as VALUES, save the file as XLSX workbook with any name you would like.

Next I usually create several sheets in the excel workbook, one with original data from the CSV file, another is the pivot table for analysis. Just make sure that you name your sheets to something meaningful.

Afterwards, I take the pivot table results and copy it to the third sheet, called “Output” just as values. This data will be used to create presentation table for other teams.

A nice trick I have learned is to change the font to Wingdings for the table part that shows number of ports per IP address and replace numbers with lowercase letter “n”. It will appear as small black square and save you some time otherwise wasted for inserting symbols manually. It looks something like this

Anyway, lets get back to the analysis and scanning. After your masscan is finished and output is analyzed in pivot table, we are ready to proceed to targeted nmap scanning.

Create hosts.lst file from the IP column of the pivot table. I use it because pivot table will naturally de-duplicate the repeated IP addresses. Row with port numbers will be used to supply the arguments for nmap.

Transfer hosts.lst file to your VPS.

My nmap command looks like this:

	nmap -sS -sU -sC -iL hosts.lst -p[list of comma-separated ports from pivot table row] -O -A  -oA [file_name]

If you want, you can throw in additional -v argument for verbosity.

After nmap finishes scan, you will have three new files with the name supplied after -oA argument.

Transfer XML file to your computer from VPS and replace string

	<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?> 

with the

	<?xml-stylesheet href="http://nmap.org/svn/docs/nmap.xsl" type="text/xsl"?>

to open it in the IE browser.

You are now finished with the technical part of the scanning. Information you have so far will be enough to continue with the high level analysis and presentation for your teams.

If this is your first scanning, you can simply create the presentation for your team that is administering firewalls, schedule a meeting and ask them to confirm what service is mapped to each public IP address and port that you have discovered as well as details of the service owner. Do not forget to refer to the nmap results. Based on their reply you can schedule meetings with the service owners to confirm the validity of the services as well as ports. Take appropriate risk based measures based on their reply and work with firewall team to close unnecessary ports and services, delete the old NATings and so on.

It is a good housekeeping practice to create your own register of the public services and map them to the external and internal IPs. You can either keep it in a separate file or in the same excel workbook that you have created for the analysis of the scan.

For this post I’ll assume that services register is stored in the same workbook as your analysis of the scan under the “Service Names” sheet. It looks something like this:

Next I have another sheet with the results of the previous scan, called “Last Scan”, where I have just two columns without headers:

This sheet is similar to the CSV that you created initially to feed the pivot table, except missing TCP/UDP column.

Now we are ready to return to our “Output” sheet and enhance the information that we already have.

Add two Columns to the end of the table, “New IP / Existing IP” and “Service Name”.

The first columns is to show whether the IP address identified is the new or existing one, compared to the previous scan. The first column will be populated with the help of the formula:

	=IF(ISERROR(VLOOKUP(A4,'Last Scan'!A$1:A$100, 1, FALSE)),"New","Exist")

To add a bit of readability I like to apply conditional formating so that any cell with “New” value has a Red background and “Exist” has a green background.

This formula takes the value from the leftmost cell in the row where you are placing it into and checks if it exists in the specified column A within the “Last Scan” sheet. If the value exists the formula shows “Exist”, otherwise it shows “New”. Copy this formula to all the cells in the New IP / Existing IP column.

Another column is to resolve Service name corresponding to the IP. It will use the following formula:

	=VLOOKUP(A4,'Service Names'!A2:C100,3,FALSE)

It again takes the leftmost cell value and checks if it exists in the columns A to C. If it does, than it provides the value from cell from the Column C, otherwise it just shows empty cell. You would need to copy this formula to all the cells in the row “Service Name” within “Output” sheet. The final result would look like this:

That is it. All what is left is to prepare the presentation and repeat this exercise routinely to audit your perimeter for discrepancies with your service register.