PowerShell V3 Beta–Better NTFS Alternate Data Stream Handling

One of the many new features in Windows PowerShell V3 is better support for alternate data streams (ADS) in NTFS files.  ADS allows an NTFS file to contain additional data that is not part of the “main” stream i.e. the file’s primary content.  Tools like Windows Explorer or even PowerShell’s Get-ChildItem cmdlet don’t show these extra data streams.  In fact the file size reported by both of these tools does not take into account the data stored in the alternate streams.  For more information on ADS check out the NTFS topic on Wikipedia.

A common use of ADS is to indicate that a file downloaded by Internet Explorer came from the Internet Zone.  Files coming from the internet could be potentially dangerous.  Various applications check for this stream and if it is present and contains information indicating the “Internet” zone, they might block access or in the case of PowerShell’s RemoteSigned execution policy, only execute the file if it is signed.

Previous to PowerShell V3, you could use the SysInternals streams.exe tool to list and remove alternate data streams.  A common application of this tool was to delete all streams in a file.  That was a rather crude but effective way to “unblock” a file downloaded from the internet.

This is also one area where CMD.EXE was one up on PowerShell.  From a CMD prompt, you can use “dir /r” to list files and their alternate data streams.  You can also create/overwrite streams with CMD.exe like so “echo.>test.exe:Zone.Identifier” which would “unblock” an internet zone file.  You can also unblock such files by selecting the file’s Properties in Windows Explorer and pressing the “Unblock” button at the bottom right of the general tab.  However this is not convenient if you need to do this to dozens or hundreds of files.  With the PowerShell Community Extensions 2.0, we introduced an Unblock-File cmdlet that would delete only the stream named Zone.Identifier.  That is the stream that Internet Explorer creates when you download a file.  Fortunately with PowerShell V3, we can obsolete that cmdlet because V3 offers several ways to manage alternate data streams.

First up is PowerShell’s own Unblock-File cmdlet which, like the PSCX equivalent, is quite easy to use:

C:\PS> Get-Command Unblock-File -All

Capability      Name   ModuleName
----------      ----                                     ----------
Cmdlet          Unblock-File                             Pscx
Cmdlet          Unblock-File                             Microsoft.PowerShell.Utility

C:\PS> Get-ChildItem *.ps1 | Microsoft.PowerShell.Utility\Unblock-File

Note that you wouldn’t normally need to prefix Unblock-File with Microsoft.PowerShell.Utility.  In this case, I wanted to make sure I was using the PowerShell Unblock-File and not the one from PSCX.

In addition to using the big gun of Unblock-File you can also manipulate streams with the following cmdlets:

C:\PS> Get-Command -ParameterName Stream | Where ModuleName -match 'Microsoft.*?Manag' 

Capability Name          ModuleName
---------- ----          ----------
Cmdlet     Add-Content   Microsoft.PowerShell.Management
Cmdlet     Clear-Content Microsoft.PowerShell.Management
Cmdlet     Get-Content   Microsoft.PowerShell.Management
Cmdlet     Get-Item      Microsoft.PowerShell.Management
Cmdlet     Remove-Item   Microsoft.PowerShell.Management
Cmdlet     Set-Content   Microsoft.PowerShell.Management

Here is how you can list all the alternate data streams in a file and the contents of any particular data stream:

C:\PS> Get-Item .\Pscx- -Stream *

   FileName: C:\Users\Keith\Downloads\Pscx-

Stream                   Length
------                   ------
:$DATA                  1799345
Zone.Identifier              26

C:\PS> Get-Content .\Pscx- -Stream Zone.Identifier

Note that :$DATA is the main stream i.e. the file’s primary contents.

If you need to clear the contents of a data stream without removing the stream completely, you can use Clear-Content’s –Stream parameter e.g.:

C:\PS> Clear-Content .\Pscx- -Stream Zone.Identifier
C:\PS> Get-Content .\Pscx- -Stream Zone.Identifier
C:\PS> Get-Item .\Pscx- -Stream *

   FileName: C:\Users\Keith\Downloads\Pscx-

Stream                   Length
------                   ------
:$DATA                  1799345
Zone.Identifier               0

To completely remove the stream, use Remove-Item’s –Stream parameter e.g.:

C:\PS> Remove-Item .\Pscx- -Stream Zone.Identifier
C:\PS> Get-Item .\Pscx- -Stream *

   FileName: C:\Users\Keith\Downloads\Pscx-

Stream                   Length
------                   ------
:$DATA                  1799345

And if you need to create an alternate stream, you can do so using Add-Content’s –Stream parameter e.g.:

C:\PS> Add-Content Pscx- -Str Zone.Identifier "[ZoneTransfer]`r`nZoneId=3"
C:\PS> Get-Item Pscx- -Stream *

   FileName: C:\Users\Keith\Downloads\Pscx-

Stream                   Length
------                   ------
:$DATA                  1799345
Zone.Identifier              26

C:\PS> Get-Content .\Pscx- -Stream Zone.Identifier

Finally, Set-Content –Stream can be used to modify the content of an existing stream.

The new  Unblock-File cmdlet as well as the upgrades to the *-Content and Get/Remove-Item  cmdlets are a very welcome enhancement to PowerShell’s file handling capabilities.

This entry was posted in PowerShell 3.0. Bookmark the permalink.

8 Responses to PowerShell V3 Beta–Better NTFS Alternate Data Stream Handling

  1. Hi Keith,

    I noticed these (in a reduced state) in the dev preview. I then read the Windows 8 blog on ReFS and that appears to deprecate alternate data streams (i.e. it seems that ReFS doesn’t support alternate streams).

    So worth bearing in mind if you’re planning to work with ReFS. There are other restrictions – e.g. DFS/R doesn’t replicate shares on ReFS volumes.


    • rkeithhill says:

      Good point for the server but since ReFS won’t be available on the client OS, at least not for Win8, we’ll still be dealing with alternate data streams for a while.

  2. Travis Jones says:

    Excellent overview of this new feature Keith.

  3. Pingback: PowerShell Get-Script -Name Peter Kriegel | Management - PowerShell Get-Script -Name Peter Kriegel | Management

  4. You mention that you could use streams.exe to remove alternate data streams, but in fact, with a bit of fantasy and creativity, you can actually do almost everything with ADS’s with just basic command prompt commands.

    I wrote a couple of blog posts on the subject. And about deleting a data stream, if you want to delete all, the easiest you can do is “type filename > newfilename”. Then simply rename newflinename back to the original filename and all the alternate datastreams are gone.

  5. Lionel says:

    Oh, that’s nice! Is there something for hardlinks and symlinks, too?

    Regarding ReFS, I’m sure some functionality is planned for a future version. Windows’ update system make heavy use of hardlinks (with winsxs), for example, which are not supported by the current ReFS. I’m sure that’s one of the main reasons why you can’t use ReFS for boot, and that we will see hardlinks in ReFS soon enough (since it will support booting from, eventually). For streams, I can’t see Microsoft doing away completely with zone identifiers in the near future. They could use another way to annotate files, but I think that adding support for streams to ReFS is more likely.

    (One filesystem feature that I think will not be added to ReFS is extended attributes).

  6. Pingback: Find all files with Alternate Data Streams | Obligatory Moniker

  7. Pingback: The ‘Date modified’ of my .eml Files got Changed When I First Select The File. Why? | Second Life of a Hungarian SharePoint Geek

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s