Tail-File Cmdlet Coming in PSCX 1.2

Occasionally I need to tail some very large log files over the network.  In this scenario, the standard PowerShell approach:

PS> Get-ChildItem *.log | %{ Get-Content $_ | Select –Last 1 }

just isn’t practical for performance reasons.  So the final drop of PSCX 1.2 will have a new cmdlet called Tail-File that is optimized for tailing the end of large (even huge) log files.  Here’s a performance comparison of the two approachs:

19# measure-command { tail-file \\Keith-PC\C\*.txt -count 1 }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 7
Ticks             : 70813
TotalDays         : 8.19594907407407E-08
TotalHours        : 1.96702777777778E-06
TotalMinutes      : 0.000118021666666667
TotalSeconds      : 0.0070813
TotalMilliseconds : 7.0813

20# measure-command {Get-ChildItem \\Keith-PC\C\*.txt | %{ Get-Content $_ | Select -Last 1 }}

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 17

Milliseconds      : 656
Ticks             : 176567027
TotalDays         : 0.000204359984953704
TotalHours        : 0.00490463963888889
TotalMinutes      : 0.294278378333333
TotalSeconds      : 17.6567027
TotalMilliseconds : 17656.7027

That is a speed-up of approximately 2500x.  That is pretty significant especially if you need to do this to a number of large log files.  For reference, the above test tailed two medium size log files (total size: 37.4 MB):

-a—     5/23/2009  6:33 AM      4173558 dirlist.txt
-a—     5/23/2009  6:33 AM     35053946 filelist.txt

Tail-File handles ASCII and Unicode encoding as well UTF8 as long as it contains only ASCII characters i.e. it will not choke on the UTF8 byte order mark.  Oh yeah, it also supports active tailing (tail –f) via the Wait parameter.  Use Ctrl+C to return control back to the console.

Advertisements
This entry was posted in PSCX. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s