Effective PowerShell Item 7: Understanding “Output”

In shells that you may have used in the past, everything that appears on the stdout and stderr streams is considered "the output".  In these other shells you can typically redirect stdout to a file using the redirect operator ‘>’.  And in some shells like Korn shell, you can capture stdout output to a variable like so:

DIRS=$(find . | sed.exe -e ‘s/\//\\/g’)

If you wanted to capture stderr in addition to stdout then you can use the stream redirect operator like so:

DIRS=$(find . | sed.exe -e ‘s/\//\\/g’ 2>&1)

You can do the same in PowerShell:

$Dirs = Get-ChildItem -recurse
$Dirs = Get-ChildItem -recurse 2>&1

Looks about the same in PowerShell so what’s the big deal?  Well there are a number of differences and subtleties in PowerShell that you need to be aware of. 

Output is Always a .NET Object

First, remember that PowerShell output is always a .NET object.  That output could be a System.IO.FileInfo object or a System.Diagnostics.Process object or a System.String object.  Basically it could be any .NET object whose assembly is loaded into PowerShell even your own .NET objects.  Be sure not to confuse PowerShell output with the text you see rendered to the screen when you don’t capture output.  In Effective PowerShell Item 3: Know Your Output Formatters we covered this notion that when a .NET object is about to "hit" the host (console) PowerShell uses some fancy formatting technology to try to determine the best "textual" representation for the object.  When you capture output to a variable, you are *not* capturing the text that was rendered to the host.  You are catching the .NET object.  Let’s look at an example:

PS C:\> Get-Process PowerShell

Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName
——-  ——    —–      —– —–   ——     — ———–
    425       9    32660      16052   181    31.63   5128 powershell

Now let’s capture that output and examine its type:

PS C:\> $Proc = Get-Process PowerShell
PS C:\> $Proc.GetType().Fullname
System.Diagnostics.Process

As you can see, a System.Diagnostics.Process object has been stored in $Proc and not the text that was rendered to the screen.  But what if we really wanted to capture the rendered text?  In this case, we could use the Out-String cmdlet to render the output as a string which we could then capture in a variable e.g.:

PS C:\> $Proc = Get-Process PowerShell | Out-String
PS C:\> $Proc.GetType().Fullname
System.String
PS C:\> $Proc

Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName
——-  ——    —–      —– —–   ——     — ———–
    479       9    32660      16052   181    31.72   5128 powershell

Another nice feature of Out-String is that it has a Width parameter that allows you to specify the maximum width of the text that is rendered.  This is handy when there is wide output that you don’t want wrapped or truncated to the width of your host.

Function Output Consists of Everything That Isn’t Captured

I’ve seen this problem bite folks time and time again on the PowerShell newsgroup.  It usually happens to those of us with programming backgrounds who are familiar with C style functions.  What you need to be aware of is that in PowerShell, a function is quite a bit different.  While a function in PowerShell does provide a separate scope for variables and a convenient way to invoke the same functionality multiple times without breaking the DRY principle, the way it deals with output can be confusing at first.  Essentially a function handles output in the same way as any PowerShell script that isn’t in a function.  What the heck does that mean?  Let’s look at an example.  For instance a programmer might look at this function definition:

function foo { "hi"; read-host "press enter"; "there" }

And expect it to prompt with "press enter" and then display "hi" and "there" but you would be wrong:

PS C:\> foo
hi
press enter:

there

Even though you can think of "hi" and "string" as outputs of the function, those outputs are output "immediately".  They aren’t returned from the function and then output to the host or to a capturing variable.  This is probably not surprising to those familiar with other shells but if your background is in programming it goes against your preconceived notions of what a function is.  PowerShell also allows us to use a C style construct – the return statement – in a way that furthers this incorrect impression that PowerShell functions are like C functions e.g.:

PS C:\> function bar {
>>          $Proc = Get-Process svchost
>>          "Returning svchost process objects"
>>          return $Proc
>>      }
>>

That should return us an array of System.Diagnostic.Process objects, right?  We told PowerShell to "return $Proc".  Let’s check the output:

PS C:\> $result | foreach {$_.GetType().Fullname}
System.String
System.Diagnostics.Process
System.Diagnostics.Process
System.Diagnostics.Process
…

Whoa!  Why is the first object System.String?  Well a quick look at its value and you’ll see why:

PS C:\> $result[0]
Returning svchost process objects

Notice that the informational message we thought we were displaying to the host actually got returned as part of the output of the function.  There are a couple of subtleties to understand here.  First, the return keyword allows you to exit the function at any particular point.  You may also "optionally" specify an argument to the return statement that will cause the argument to be output just before returning.  "return $Proc" does *not* mean that the functions only output is the contents of the $Proc variable.  In fact this construct is semantically equivalent to "$Proc; return".

The second subtlety to understand is this:

The line:
"Returning svchost process objects"

is equivalent to this line:
Write-Output "Returning svchost process objects"

That makes it clear that the string is considered part of the "output".  Now what if we wanted to make that information available to the end user but not the script consuming the output of the function?  Then we could have used Write-Host like so:

PS C:\> function bar {
>>          $Proc = Get-Process svchost
>>          Write-Host "Returning svchost process objects"
>>          return $Proc
>>      }
>>

Write-Host does not contribute to the output of the function.  It writes directly to the host.  This might all seem obvious now but you have to be diligent when you write a PowerShell function to ensure you get only the output you want.  This usually means redirecting unwanted output to $null (or optionally type casting the expression with the unwanted output to [void]).  Here’s an example:

PS C:\> function LongNumericString {
>>          $strBld = new-object System.Text.StringBuilder
>>          for ($i=0; $i -lt 20; $i++) {
>>              $strBld.Append($i)
>>          }
>>          $strBld.ToString()
>>      }
>>

Note that we don’t *need* to use the return keyword like we do in C style function.  Whatever expressions and statements that have output will contribute to the output of our function.  In the function above, we obviously want the output of $strBld.ToString() to be the function’s output.  So what is the output?

PS C:\> LongNumericString

                  Capacity                MaxCapacity                    Length
                  ——–                ———–                    ——
                        16                 2147483647                         1
                        16                 2147483647                         2
                        16                 2147483647                         3
                        16                 2147483647                         4
                        16                 2147483647                         5
                        16                 2147483647                         6
                        16                 2147483647                         7
                        16                 2147483647                         8
                        16                 2147483647                         9
                        16                 2147483647                        10
                        16                 2147483647                        12
                        16                 2147483647                        14
                        16                 2147483647                        16
                        32                 2147483647                        18
                        32                 2147483647                        20
                        32                 2147483647                        22
                        32                 2147483647                        24
                        32                 2147483647                        26
                        32                 2147483647                        28
                        32                 2147483647                        30
012345678910111213141516171819

Yikes! That is probably more than what you were expecting.  The problem is that the StringBuilder.Append() method returns the StringBuilder object so that you can cascade appends.  Unfortunately now our function outputs 20 StringBuilder objects and one System.String object.  It is simple to fix though, just throw away the unwanted output like so:

PS C:\> function LongNumericString {
>>          $strBld = new-object System.Text.StringBuilder
>>          for ($i=0; $i -lt 20; $i++) {
>>              [void]$strBld.Append($i)
>>          }
>>          $strBld.ToString()
>>      }
>>
PS C:\> LongNumericString
012345678910111213141516171819

Other Types of Output That Can’t Be Captured

In the previous section we saw one instance of a particular output type – Write-Host – that doesn’t contribute to the stdout output stream.  In fact, this type of output can’t be captured.  The argument to Write-Host’s -object parameter is sent directly to the host console bypassing the "stdout" output stream.  So unlike stderr output that can be captured as shown below, Write-Host output doesn’t use streams and therefore can’t be redirected. 

PS C:\> $result = remove-item ThisFilenameDoesntExist 2>&1
PS C:\> $result | foreach {$_.GetType().Fullname}
System.Management.Automation.ErrorRecord

Write-Host output can only be captured using the big stick – the Start-Transcript cmdlet.  Start-Transcript logs everything that happens during a PowerShell session.  If you need to create a comprehensive log file that captures everything then Start-Transcript is the only game in town.  [Update: 05/25/2008 – Well except for one major hole.  Transcripts don’t capture any output from EXEs.  Thanks to Shay for pointing that out.]  Keep in mind that Start-Transcript is meant more for session logging than individual script logging.  For instance, if you normally invoke Start-Transcript in your profile to log your PowerShell session, a script that calls Start-Transcript will generate an error because you can’t start another transcript if one has already been started.  You have to stop the previous one first.  

Here is the run down on the forms of output that can’t be captured except via Start-Transcript:

1. Direct to Host output via Write-Host & Out-Host
2. Debug output via Write-Debug or -Debug on a cmdlet
3. Warning output via Write-Warning
4. Verbose output via many cmdlets that output extra information to the host when -Verbose is specified
5. EXE stdout or stderr output

If you happen to agree with me that this situation should be better in a future version of PowerShell, please feel free to vote on these two issues:

Capture Warning, Verbose, Debug and Host Output Via Alternate Streams
Script Logging Needs to Be Improved

That’s it.  Just remember to keep an eye on what statements and expressions are contributing to the output of your PowerShell functions.  Testing is always a good way to verify that you are getting the output you expect.