Minidump Crash Analysis with PowerShell

Our commercial application mails home a minidump report when it crashes, that is if the customers presses the "Send" button on the error dialog.  Analyzing these has helped us fix defects that had existed years earlier but were apparently hard to reproduce.  Unfortunately the analysis and debugging of these can be tedious and is exacerbated by receiving many duplicates.  A few weeks ago we brainstormed some ways to quickly identify duplicates.  The key to this is gaining quick access to the stack trace.   What we needed was something like WinDbg but would output to the console.  Well, say hello to my little debugger friend – CDB.EXE.  CDB is freely available via the wonderful Debugging Tools for Windows package from Microsoft.  This little console debugger will do just what we want.  And you can script it via the command line even better.  So here’s a taste of where I’m heading with this:

function ExtractStackTrace([string[]]$output) {
    $startCapture
    $xml = "<StackTrace>"
    $reEip = "(?<eip>[a-fA-F0-9]{8})"
    $reRet = "(?<ret>[a-fA-F0-9]{8})"
    $reArg1 = "(?<arg1>[a-fA-F0-9]{8})"
    $reArg2 = "(?<arg2>[a-fA-F0-9]{8})"
    $reArg3 = "(?<arg3>[a-fA-F0-9]{8})"
    $reMod = "(?<mod>\w+)"
    $reFun = "(?<fun>\w+)"
    $reOff = "0x(?<off>[a-fA-F0-9]+)"
    for ($i = 0; $i -lt $cdbOutput.Length; $i++) {
        if ($startCapture) {
            $cdbOutput[$i] -match "^$reEip\s+$reRet\s+$reArg1\s+$reArg2\s+$reArg3\s+$reMod\!$reFun\+$reOff" > $null
            if (!$?) { break }
            $xml += "<Frame eip=`"0x" + $matches.eip + "`""
            $xml += " module=`"" + $matches.mod + "`""
            $xml += " function=`"" + $matches.fun + "`""
            $xml += " offset=`"0x" + $matches.off + "`""
            $xml += ">"
            $xml += "<ReturnAddress>0x" +$matches.ret + "</ReturnAddress>"
            $xml += "<Arg1>0x" + $matches.arg1 + "</Arg1>"
            $xml += "<Arg2>0x" + $matches.arg2 + "</Arg2>"
            $xml += "<Arg3>0x" + $matches.arg3 + "</Arg3>"
            $xml += "</Frame>"
        }
        elseif ($cdbOutput[$i] -match "ChildEBP RetAddr") {
            $startCapture = $true
        }
    }
    $xml += "</StackTrace>"
    $xml
}
 
function GetNativeStackTrace([string]$minidumpPath) {
    $cdbOutput = cdb.exe -z $minidumpPath -c ".ecxr ; KB ; q"
 
    $stackTraceXml = "<Minidump>"
    $stackTraceXml += ExtractExceptionCode $cdbOutput
    $stackTraceXml += ExtractStackTrace $cdbOutput
    $stackTraceXml += "</Minidump>"
    $stackTraceXml
}

Note that the hard work of decoding the minidump is done by this CDB command:

cdb.exe -z $minidumpPath -c ".ecxr ; KB ; q"

The rest of the PowerShell script is used to encode the returned text information into a XML document. Hopefully I will get some time in the next month or so to work on a signature matching facility to help us quickly identify duplicate minidumps. 

Oh yeah, here is what the output looks like:

<?xml version="1.0" encoding="utf-16"?>
<Minidump>
  <ExceptionCode>e0434f4d</ExceptionCode>
  <StackTrace>
    <Frame eip="0x0012de98" module="kernel32" function="RaiseException" offset="0x53">
      <ReturnAddress>0x79f97065</ReturnAddress>
      <Arg1>0xe0434f4d</Arg1>
      <Arg2>0xe0434f4d</Arg2>
      <Arg3>0xe0434f4d</Arg3>
    </Frame>
    <Frame eip="0x0012def8" module="mscorwks" function="RaiseTheExceptionInternalOnly" offset="0x226">
      <ReturnAddress>0x7a0945a4</ReturnAddress>
      <Arg1>0x013a3c08</Arg1>
      <Arg2>0x013a3c08</Arg2>
      <Arg3>0x013a3c08</Arg3>
    </Frame>
    <Frame eip="0x0012dfbc" module="mscorwks" function="JIT_Throw" offset="0xd0">
      <ReturnAddress>0x06f79674</ReturnAddress>
      <Arg1>0x013a3bec</Arg1>
      <Arg2>0x013a3bec</Arg2>
      <Arg3>0x013a3bec</Arg3>
    </Frame>
    <Frame> … </Frame>
  </StackTrace>
</Minidump>