Mono.Cecil - Optional Parameters

Earlier this week I posted a link to the Mono.Cecil 0.4.1 release but didn't include the sample source code I intially intended to. Below is the original post with sample source code included. It's an example of rather limited use for day to day development work but is in part based on a requirement we had to manipulate an assembly written in a language that don't support optional parameters into one that exposes optional parameters where required. Hopefully the example shows how Cecil can reduce a rather complex task into something very manageable and without the developer having to understand or learn the intricacies of modifying the IL directly. It's heavily based on Mono's innovative implementation of the Microsoft.VisualBasic assembly :-)

---

I’ve been doing some work this week with Mono’s brilliant assembly manipulation library Mono.Cecil. From the Cecil description, “Cecil is a library under development to generate and inspect programs and libraries in the ECMA CIL format. In simple English, with Cecil, you can load existing managed assemblies, browse all the contained types, modify them on the fly and save back to the disk the modified assembly.” It’s basically reflection on several banned substances.

One of the differences that exist between a language like Visual Basic.NET and C# is the availability of optional parameters in VB.NET. Optional parameters and default parameter values are features that form part of the CLI specifications but are not supported in all .NET languages. Support for it is available in Visual Basic.NET most probably due to the fact that it’s a more appealing feature for developers coming from a Visual Basic background and not so appealing to C# developers who typically achieves the same functionality with method overloads or other similar constructs.

A good reason to avoid the use of optional parameters is the fact that the default value for the optional parameter is inserted into the call site so should your library’s default values change both the client and library would need to be recompiled.

The lack of optional parameters becomes an issue when you have to author a library in C# to be consumed by a Visual Basic.NET client and part of the API specification states that certain parameters should be optional. There is no easy way to do this in C# so you either have to author the library in VB.NET or jump through some other hoops to get the optional parameter attributes set. I suspect Microsoft followed the former route in their implementation of the Microsoft.VisualBasic assembly and namespace and Mono the latter.

Looking at the Mono approach: If you take a look at the Mono make file for the Microsoft.VisualBasic assembly you’ll see that the assembly is compiled as usual but where optional parameters is required a special internal marker attribute Microsoft.VisualBasic.CompilerServices.__DefaultArgumentValueAttribute is applied to the target parameter. Below is the signature of the attribute:

[AttributeUsage(AttributeTargets.Parameter)]

internal sealed class __DefaultArgumentValueAttribute : Attribute

An example of its usage can be found on one of the Add methods on the Microsoft.VisualBasic.Collection class. In this case the Key, Before and After parameters are all optional with a default value of null.

public void Add (System.Object Item,
                  [__DefaultArgumentValue(null)] String Key,
                  [__DefaultArgumentValue(null)] System.Object Before,
                  [__DefaultArgumentValue(null)] System.Object After)

After compilation the resulting assembly is disassembled to IL, and then fixed up by a Perl script and the IL reassembled back into Microsoft.VisualBasic.dll.

Using Mono.Cecil the same effect can be achieved by directly manipulating the CIL and making the required modifications without resorting to disassembling and reassembling the assembly.

There are two steps involved in this process:

Modify the Param metadata table and set the optional flag (opt in ILASM) on the relevant parameters.
Set a special flag for which there is no ILASM equivalent to indicate the presence of an associated Constant record in the constants table and add the actual associated entry.

This is where Cecil really shines; previously there haven't been an easy way to manipulate an assembly in this way directly from managed code. I’ll explain the process below and show the changes in IL we want to make and then the couple of lines of Cecil code to make it happen. Magic.

As a starting point an example of the above approach using Mono’s special marker attribute in C# would look something like the following:

public void Bar(string name,[__DefaultArgumentValue("doe")] string surname, [__DefaultArgumentValue(1)]int categoryId){}

We’ve got a method Bar which takes three parameters and we’ve marked the last two parameters for post-processing using the marker attributes. The relevant resulting IL with our custom attribute present in the metadata for this method would be the following:

.method public hidebysig instance void  Bar(string name,
                                            string surname,
                                            int32 categoryId) cil managed
{
  .param [2]
  .custom instance void Microsoft.VisualBasic.CompilerServices.__DefaultArgumentValueAttribute::.ctor(string)
= ( 01 00 03 64 6F 65 00 00 )                         // ...doe..
  .param [3]
  .custom instance void Microsoft.VisualBasic.CompilerServices.__DefaultArgumentValueAttribute::.ctor(int32)
= ( 01 00 01 00 00 00 00 0

The same method written in Visual Basic.NET supporting optional parameters is shown below. This should give an indication of what we need to modify to make our code appear the same as code developed in a language supporting optional parameters. Below is the Visual Basic method definition and the resulting IL code.

Public Sub Bar(ByVal name As String, Optional ByVal surname As String = "doe", Optional ByVal categoryId As Int32 = 1)

End Sub

.method public instance void  Bar(string name,
                                  [opt] string surname,
                                  [opt] int32 categoryId) cil managed
{
  .param [2] = "doe"
  .param [3] = int32(0x00000001)

Here we can see that the optional and default value flags are set on parameters 2 and 3. The [opt] keyword indicates that the Optional ParamsAttribute flag was set and the link between the .param Idea [I] and the const value indicates that the parameter has an associated constant entry which is shown inline in the output above.

The first time I wrote this code everything appeared to work but the parameters never showed up as optional when consumed by a client that supports optional parameters. After a little digging it turned out that the initial CLI specifications had the optional flag value as 0x0004 and this value was also used by Mono.Cecil. Which if you compare it to the 0x0013 mask given for these flags by Serge Lidin in his Inside Microsoft.NET assembler don’t add up. In later versions of the CLI specifications this value has been updated to the correct value of 0x0010. Big thanks to Jb for updating this in svn so quickly!

On the code above, upon first inspection the parameter indices would appear a little unnatural as they seem to start at 1 but this is due to the fact that the 0 index is used to represent the return type of the method.

Now that the target result is known it’s simply a matter of loading the assembly with Cecil, making the required changes and saving the assembly back. All this can be accomplished with only a couple of lines of code in Cecil and its pretty self explanatory but comments have been added for good measure. To keep the sample simple some assumptions have been made with regards to the marker attribute name and type.

using System;
using System.Collections;
using System.IO;
using System.Text;
using Mono.Cecil;
using Mono.Cecil.Cil;

namespace Samples.Mono.Cecil
{
       public class Converter : BaseReflectionVisitor
       {
              public void Convert(string input, string output)
              {
                     // Load assembly for processing by Mono.Cecil
                     AssemblyDefinition assembly = AssemblyFactory.GetAssembly(input);

                     // Get all the types defined in the main module. Typically you won't need
                     // to worry that your assembly contains more than one module
                     foreach(TypeDefinition type in assembly.MainModule.Types){
                           // Iterate over the methods defined for the type and send
                           // this (BaseReflectionVisitor) as a visitor
                           foreach(MethodDefinition methodDef in type.Methods){
                                  methodDef.Accept(this);
                           }
                     }
                     // Save the modified assembly back
                     AssemblyFactory.SaveAssembly(assembly,output);
              }

              public override void VisitMethodDefinition(MethodDefinition methodDef)
              {
                     // Iterate over the parameters and search for our custom marker attribute
                     // When found mark the attribute as optional and as having a default value.
                     // Remove the marker attribute.
                     foreach (ParameterDefinition parmDef in methodDef.Parameters) {
                           CustomAttribute[] attribs = new CustomAttribute[parmDef.CustomAttributes.Count];
                           parmDef.CustomAttributes.CopyTo(attribs,0);

                           for(int i=0;i<attribs.Length;i++){

                                 if(attribs Idea [I]

.Constructor.DeclaringType.FullName !=
"Microsoft.VisualBasic.CompilerServices.__DefaultArgumentValueAttribute"){
                                         continue;
                                  }
                                  // Set opt parameter attribute and add constant entry
                                  parmDef.Attributes = parmDef.Attributes | ParamAttributes.Optional | ParamAttributes.HasDefault;
                                  parmDef.Constant = attribs Idea [I]

.ConstructorParameters[0];
parmDef.CustomAttributes.Remove(attribs Idea [I]

);
                           }
                     }
              }
}

This is all there is to it. This example code will take an assembly which has parameters marked with the __DefaultArgumentValueAttribute and make them optional. A rather complex task reduced to a couple of lines of C# code with Mono.Cecil. That is one of the strengths of Cecil, while this is an example of rather limited use the same holds true for more complex examples. Cecil, while deceptively simple to use allows you to do some very powerful manipulations of your assemblies or partial assemblies.

The sample source can be downloaded from here.

For more information see the following resources:

Jb Evain's blog.

The Cecil project page the Mono site.
CLI Specifications

[Update: Formatted the code sample a little better]