PE Info Python Snippet

Trying to dump the PE header info for an unknown Windows PE binary on a linux box?

Solution:

$ sudo aptitude install python-pefile
$ python
>>> import pefile
>>> pe = pefile.PE('/path/to/pefile.exe')
>>> print pe.dump_info()

If you just want to see the imports:

>>> for entry in pe.DIRECTORY_ENTRY_IMPORT:
... print entry.dll
... for imp in entry.imports:
... print '\t', hex(imp.address), imp.name

Javascript decoding using Python Interpreter

I keep forgetting the module to decode % encoded javascript obfuscation using python, so here it is:

$ python

>>> import urllib
>>> s = '%33%76%31%4C%20%6A%73%63%72%69%70%74%21'
>>> urllib.unquote(s)
'3v1L jscript!'

Remove html tags: two sed one liners

1. Quick and easy one liner to remove any html tags from file blah:

cat blah | sed 's/<[^>]*>//g'

2. Remove html tags and format jscript just enough to read file blah:

cat blah | sed 's/<[^>]*>//g' | sed 's/;[^\n]/;\n/g' | sed 's/{/{\n/g'| sed 's/{\s*[^\n]/\n{/g'

note that these only work if the html tag is not broken onto two lines.

John the Ripper Tutorial

Passwords are used to authenticate an account owner to a given system. The characters that are typed in by the user are encrypted with a one-way form of encryption called hashing. It is considered one-way as it is mathematically “impossible” to reverse the encrypted ciphertext hash back into the original plaintext password.

If hash algorithms are one-way, how am I supposed to “crack” a hash encrypted password? Astute readers must be rocking in their chairs at this point. The common means to accomplish password cracking is to guess every single possible plaintext input, run it through the hashing encryption algorithm, and then compare the result to the hashes collected from the target. Yes, your poor computer will be breaking a sweat to accomplish this as fast as it can.

This quick guide will focus on password cracking using John the Ripper, an open source password cracking tool. There is actually a paid version which provides precompiled and optimized binaries for Linux or OS X, quite convenient. We will be patching and compiling the free version of john ourselves.

Shopping List

1. A linux system (physical or virtual) - Ubuntu is a good place to start.


2. john-1.7.3.4.gz


3. The Jumbo patch for 1.7.3.4, revision 3


4. pwdump6 or fgdump - Use either to dump hashes from the SAM database on the target Windows box.

Unpack and make
Start by opening a terminal on your linux box that you downloaded the john files to. You will notice they end in a .gz extension which means these files are compressed using GNU zip (or gzip for short.) To uncompress john, run these commands:

tar xvzf john-1.7.3.4.tar.gz
cd john-1.7.3.4

Let's get this started...
If you are performing a password audit on systems that still have LanMan (LM) hashing enabled, for the win! Your job is nearly done as john supports this format with no patches and can guess them at blazing speed! My 2.6Ghz Core 2 Duo MacBook Pro running john 1.7.3.4 with no optimizations easily hits 22 million hash generations per second! Since LM is case insensitive and splits passwords into two 7 character strings and then hashes them, here's the default john.conf setting for LM hashes:

[Incremental:LanMan]
File = $JOHN/lanman.chr
MinLen = 0
MaxLen = 7
CharCount = 69



So, what if LanMan hashing has been disabled on your target or you need to crack a different hash algorithm? First of all, you will need to patch john to support the desired hash format (we'll assume md4) by following this guide to coaxing john to do your dirty work.

Windows Password Hash Extraction
Windows stores user passwords in the SAM (Security Account Manager) database. This file is locked by the operating system to prevent a user from extracting it’s contents. Yet, it still needs a working copy placed in memory somewhere, so we will exploit this design by copying the contents of the “locked” file directly from memory. I recommend two utilities for the job: pwdump and fgdump.

pwdump has been around for quite some time and has become the defacto choice. Unfortunately, as new features are added into Windows, so much the tool. The current version is pwdmp6 which adds support for 64bit systems and  remote hash collection (with appropriate administrator credentials.)

fgdump is a fork of pwdump, aiming to add features such as the ability to automatically stop the Symantec Anti-Virus service, collect the hashes and then restart AV. This is required as Symantec has (quite accurately) categorized pwdump and fgdump as “hacker tools.” In practice however, this feature rarely works and even if it does successfully stop the AV service, I have seen instances where it was unable to re-enable the service, potentially leaving your server without it. For this reason, I highly recommend telling fgdump to not disable AV (by using the –n switch) and instead disable and re-enable AV yourself. Packers are another solution but I won’t discuss them here.

Patch ‘er up!
John does not support the hashing algorithm (NTLM or md4) used by Windows XP and up out of the box. We will need to apply a source code patch to add this functionality. Don’t worry if you don’t know what I’m talking about yet, as the command (from the john-1.7.3.4 directory) to accomplish it is simply:

gzip -dc ../john-1.7.3.4-jumbo-3.diff.gz | patch -p1


If you are on Ubuntu, the patch command and the required openssl library are likely not installed. You can get them by running:

sudo apt-get install patch libssl-dev


Finally, we just need to compile john from source to get our binary.

cd src
make linux-x86-sse2


If you are on a 64bit linux computer, use this make option:

make linux-x86-64

If you are on a modern OS X 10.5+ computer, use this make option instead:

make macosx-x86-64

Done? Test john out with:

cd ../run
./john -test

If you are targeting a Windows system, make sure you see a line with output similar to the following:

Benchmarking: NT MD4 [128/128 X2 SSE2-16]... DONE
Raw:    25628K c/s real, 25628K c/s virtual

Great, now it’s time to punish your box! I mean, start up john. Don't forget to change the last argument to your hash file.

...
./john --format=NT ~/my-windows-hashes-file.pw
Loaded 10 password hashes with no different salts (NT MD4 [128/128 X2 SSE2-16])
...

Run john, Run! See passwords fly! We can now walk away and come back in a few days to check progress! Or you could just go grab a beer... Or a dozen, as you'll be waiting for quite some time (possibly infinitely.)

Hope that helped. Please feel free to email me with any questions.

Quick note on sorting john.pot files

I run john the ripper on multiple boxes at once by splitting up the input password hash files by type (NT, LanMan, DES, md5, etc.)

Once I've let the individual systems run for a while, I run reports using  john - -show. This means I need to combine the john.pot files from each system into a unique file on the system I'm running the reports on.

So, I ran the (fairly standard) commands:

$ cat john1.pot john2.pot john3.pot > combined.pot


$ cat combined.pot | sort | uniq > john.pot
sort: string comparison failed: Illegal byte sequence
sort: Set LC_ALL='C' to work around the problem.
sort: The strings compared were `1234567892031276d66b123456789:user' and `abcdefghijklmnop4ab38:l\4327367qrstuvwxyz'.

Oops! I wasn't expecting that command to bomb out like that. To fix the sort, do exactly as sort suggests. Type:

$ export LC_ALL='C'




What's going on here?

I found this reference webpage on Google. Here's the relevant bits:

LC_ALL

This variable determines the values for all locale categories. The value of the LC_ALL environment variable has precedence over any of the other environment variables starting with LC_ (LC_COLLATE, LC_CTYPE, LC_MESSAGES, LC_MONETARY, LC_NUMERIC, LC_TIME) and the LANG environment variable.

If the LANG environment variable is not set or is set to the empty string, the implementation-dependent default locale is used.

If the locale value is "C" or "POSIX", the POSIX locale is used and the standard utilities behave in accordance with the rules in POSIX Locale , for the associated category.

Updated Google Docs Drawing

The new Google Docs Drawing type is pretty sweet! Looks like I won't be needing to haul around Portable Dia on my flash drive anymore...



It's also got real-time collaboration! Check out the official introduction video on youtube:

Network Scanning with nmap

Shopping List
Before we start, make sure you have the following ingredients ready to go:

• Fydor’s nmap – The true king of network scanners


• OpenVAS – The glamor queen of vulnerability assessors

Start Here
The first thing you need to determine is if you will be performing a black box test or a glass box (often referred to as crystal box) test. I will only quickly go over the differences here.

The Fairest Box of All
Black box testing is the "simplest" as you are only provided the IP Addresses in scope of the assessment. That’s it. You are not told how many Windows systems to expect, how many firewalls may be in your way or even if an IPS may shut down your scan if you are too aggressive with your timing options (we’ll touch on that later in this article.) It is difficult to accurately predict how long this may take as the composition of every network is different. An extreme example is if you are scanning through ocean floor fiber it could take quite some time when each packet suffers 250ms+ latency.

Glass box testing is typically much quicker to perform as your client will be providing architectural details of how the network is bolted together. It’s best to ask for network diagrams plus a real live human being to help you decipher them. (I mean seriously, how many UML network diagrams have you seen?) This is important not only because they may use abbreviations unknown to you or how closely the diagram aligns with reality or even all the labels they scribbled on by hand are subject to the lost art that is penmanship. Err, graphmanship.

Scope Up!
Once the transparency of your target environment is agreed upon, you will need to define the scope of which portions of the target organization’s network is within scope for the assessment.

Ask for hosts which must NOT be scanned as they will fall over seizing due to poor IP/TCP stack implementations or other programming bugs. Place into “exclude.list”

Place the authorized IP address ranges and domains you were provided into “inscope.list”

Just a Minor Technicality
Ok, we are now ready to dominate the world with our awesomeness! Or at the very least impress your client with a thorough report...

If you are scanning a class B network (10.10.X.X) or smaller, you will likely want to choose -T4 for your speed setting. If you are attempting to map the Internet, stop now. Fydor has beaten you to it :-)

The Need For Speed
The –n switch will disable DNS lookups, reducing overall start to finish time as you no longer need to wait for all those UDP DNS request / responses to file in.

Setting how fast nmap chucks packets onto the network is controlled with:
-T, --scan-delay and —min-hostgroup
-T5 is balls to the wall and –T0 means you never want it to finish...

A safe choice is –T3 (the default) but again you will likely want to use -T4 to save time.

Top Heavy Servers
Scanning for every in-scope IP Address plus 65,535 port combinations to gather an inventory of live hosts could last longer than your contract engagement. You may need to pick 20 to 30 ports. In a perfect scenario, at least one of these ports will be open on every server that you need to find. Use -p to specify destination ports; a good starting list are the following UDP (U:) and TCP (T:) ports:
-p U:53,111,137,T:21-25,80,135,139,443,445,3389,8080

If your client has no idea what services they expect to find on their network, you now have permission to cry like a baby because they will have problems with your report regardless of what you find. T-T
All seriousness aside, thanks to Fydor’s Internet mapping project I mentioned earlier, you can just substitute the defined list with Fydor’s results by adding:
--top-ports 30 (where 30 is the number of ports you want to scan)

I Know That Look...
nmap is more than just a simple inventory scanner. It can also guess what the target operating system and it's network accessible service names and versions are. It does this by looking at distinguishing features of a packet or connection (such as what the starting TTL value is or how it responds to an unsolicited RST packet sent to a closed port.) This technique is referred to as fingerprinting. If nmap is unsure about a target host, it will guess (and tell you as much) and if it is completely baffled it will politely ask you to submit a fingerprint to add to the nmap database along with what you determine the target to be (by other means.)

-sV will attempt to match the name and version of each open network service that is discovered
-O will attempt to match the operating system of target boxen

New Implants
nmap has received some really neat upgrades since version 4.something such as the ability to run lua scripts during the scan process. An example:
--script smb-os-discovery

Make sure you are using the latest scripts by running:
nmap —script-updatedb

Putting it all Together


Thus far, we have compiled the following nmap command:

nmap –iL inscope.list —excludefile exclude.list –v –sS –n —reason –T4 —script smb-os-discovery –sV –O —top-ports 30

Sweep 1: I highly recommend swapping out –sS with –sL before you run your final scan. Why? A list of hosts to be scanned is a great piece of information to include in your report to PROVE that you only touched systems that you were expressly authorized to touch. Yes, this is just feeding back information they originally provided you... Hey, this is why consultants get paid the big bucks right?

Sweep 2: swap out –sS with –sP to icmp scan for which hosts you are able to receive echo replies from. Consider sending this list to your client immediately to verify that your are in the ballpark prior to the "real" scan. Obviously, this can be skipped if a networking device (such as a router or firewall) is dropping all your icmp echo request probes.

Sweep 3: -sS for the win! This is your traditional half open SYN scan. Nmap will send out a TCP SYN packet to each IP address and port combination as you specified to determine if it returns a SYN+ACK packet, indicating an open port. If it doesn't hear back from the target, it's assumed to be closed. (Yes, this is where the increased timing option pays off.)

Wrapping Up

If you are running up against an IPS blocking your aggressive scans by sending you RSTs or dropping your packets silently, try setting –f to fragment (split up) your packets to a smaller portion of their maximum transmission unit (MTU) or size. This may just be enough to fool (or crash) that pesky IPS into letting them all pass on their merry way for total target box domination. Make sure you have permission to try this from the client first...

This was intended to be a pick up and go article and not dig too deep into how network scanning works nor the intricacies of nmap. If you want WAY more info than I've provided here, I highly recommend buying Fydor's nmap book. If you can't spare a few bucks, it's also available online for free.

Stay tuned for Part 2, which covers OpenVAS.