MSBuild: Groking Item Lists, Flattening, Transforms and Batching

One of the more important aspects of MSBuild that you need to grok is how item lists can be manipulated.  An item list typically corresponds to all the files you add to a project in Visual Studio via "Add New Item…" or "Add Existing Item…".  Therefore an item list might look something like this:

<ItemGroup>
  <Compile Include="foo.cs;bar.cs;baz.cs"/>
  <Reference Include="System.Web.dll"/>
</ItemGroup>

Flattening:

An item list can be flattened into a single string.  Keep in mind that an item list is internally stored as an array (ITaskItem[]).  Flattening can happen implicitly by using an item list in a context that expects a string e.g. <Exec Command="csc.exe @(Compile) /r:@(Reference)"/>.  MSBuild flattens item lists into a string using a semi-colon as the default separator.  For instance the task <Message Text="@(Compile)"/> results in this output:

foo.cs;bar.cs;baz.cs

Note: that this essentially converts from type ITaskItem[] to a string.  Of course the default separator can be changed using (@Compile, ‘ ‘).  In this case, the separator has been changed to a single space.  You can specify any string as the separator.  The act of changing the default separator causes an "explicit" flatten to happen.  That is, if you were to use this form where ITaskItem[] is expected e.g. <Csc Sources="@(Files, ‘ ‘)"/> then the Sources parameter (type ITaskItem[]) will get an item list that is flattened to a single string whose items are space separated.  Typically you do *not* want to provide a flattened item list where ITaskItem[] is expected i.e. almost any task parameter that expects an item list.

Transforms:

You can create a new (unnamed) item list by transforming the contents of an existing item list.  This eliminates the need for redundant item lists that vary only by certain attributes like file extension. The syntax for a transform is @(Compile->’%(filename).exe’) which results in the following operation:

unnamedItemList[0] = foo.exe
unnamedItemList[1] = bar.exe
unnamedItemList[2] = baz.exe

You can also change the separator at the same time that you perform a transform e.g. @(Compile->’%(filename).exe’, ‘ ‘) but this also does an explicit flatten of the item list into a single string.  Remember: changing the default separator always flattens the item list to a string.

Batching:

Finally you can cause MSBuild to operate on item lists in batches (or buckets) by specifying a direct metadata reference in a Target or Task attribute.  This will cause MSBuild to separate all the items in the specified item list into buckets corresponding to each unique combination of the specified metadata.  MSBuild will then loop over the Target or Task once for each bucket passing in the subset of items in the item list that correspond to that bucket i.e. those that match the specified metadata.  This feature is commonly used to "loop" tasks i.e. get MSBuild to perform the task once for each item in the item list e.g. by referencing the identity metadata.  It can also be used for filtering tasks by using metadata in a comparison inside a Task’s Condition attribute.  As an example, consider the following item list:

<ItemGroup>
  <Resource Include="foo.resx;bar.resx;baz.resx">
    <Culture>en-US</Culture>
  </Resource>
  <Resource Include="foo.de.resx;bar.de.resx;baz.de.resx">
    <Culture>de</Culture>
  </Resource>
  <Resource Include="foo.fr.resx;bar.fr.resx;baz.fr.resx">
    <Culture>fr</Culture>
  </Resource>
</ItemGroup>

Consider the buckets that get created by the following task:

<Copy SourceFiles="@(Resource)" DestinationFolder="%(Resource.Culture)" />

MSBuild will analyze the inputs to this task and note that DestinationFolder directly references metadata, in this case the Culture metadata.  This will cause MSBuild to bucketize the Resource item list into three buckets:

Metadata    Bucket contents
[en]        foo.resx    bar.resx    baz.resx
[de]        foo.de.resx bar.de.resx baz.de.resx
[fr]        foo.fr.resx bar.fr.resx baz.fr.resx

MSBuild will then execute the <Copy …/> task once for each bucket passing in the items in that specific bucket.  In this scenario the copy task would effectively execute like so:

copy from: foo.resx    bar.resx    baz.resx     to: en
copy from: foo.de.resx bar.de.resx baz.de.resx  to: de
copy from: foo.fr.resx bar.fr.resx baz.fr.resx  to: fr

Advertisements
This entry was posted in MSBuild. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s