Friday, June 24, 2011

Macros in C#

[beginner level, originally posted on an internal company blog on Feb 1st]

Hi,


Today I want to talk about C# macros. Those are not 
macros in the
sense of C's #define statements but instead a syntactic sugar which
enables us to write simple, more readable and shorter code. My point
is that all those 
macros can be avoided by writing other code which
will result in the exact same IL. You might have not noticed these macros before or took them for C# keywords but in this article I will use Reflector (edit: Reflector was still free back then) to see what the compiler does in the case of four commonly used statements.

Extension Method

An extension method is a method defined on an existing class which can
be called using the dot operator. For example I can define the
following extension method on the string class:


public static class StringExtensions
{
  public static void MyStringExtensionMethod(this string input, int number)
  {
    Console.Out.WriteLine(String.Format(input, number));
  }
}


Then I call it in my code
string s = "My number is {0}";
s.MyStringExtensionMethod(10);

And the output is: My number is 10

Well, we don't expect the compiler to edit the existing string class
and add a method to it nor do we think a new class is created which
inherits the string class (it is sealed!).
So what did the compiler do? A hint to that is the definition of an
extension method. It is static in a static class. In fact the only
thing that makes this method an extension is the fact we added the
"this" keyword in the definition. The simple answer is that the
compiler did nothing. The only thing happened is that the "this"
keyword enabled the dot syntax but the call remained the same. In this
case the actual call to the method is
MyStringExtensionMethod(s,10);

Let's compare the following code with Reflector
static void Main(string[] args)
{
   string s = "My number is {0}";
   s.MyStringExtensionMethod(10);
   StringExtensions.MyStringExtensionMethod(s, 10);
}

The disassembly is somewhat strange:

private static void Main(string[] args)
{
   string s = "My number is {0}";
   s.MyStringExtensionMethod(10);
   s.MyStringExtensionMethod(10);
}

But we see that the IL code is exactly the same and that this is only
the interpretation of the tool

   L_0007: ldloc.0
   L_0008: ldc.i4.s 10
   L_000a: call void
Macros.StringExtensions::MyStringExtensionMethod(string, int32)
   L_000f: nop
   L_0010: ldloc.0
   L_0011: ldc.i4.s 10
   L_0013: call void
Macros.StringExtensions::MyStringExtensionMethod(string, int32)
   L_0018: nop
"using" statement 

The "using" statement is used to create, work with, and dispose an
IDisposable object. Probably the most common case to meet this
statement is when working with streams.
Lets look at a typical piece of code that uses the "using" statement.

using (StreamReader reader = new StreamReader("MyFile.txt"))
{
    Console.Out.WriteLine(reader.ReadToEnd());
}

In this code we print the content on a text file onto the console. We
don't want to lock the file after we finish reading so we free it.
This is exactly what the using statement does for us. We could simply
write the following code and not use the using statement:

    StreamReader myReader = new StreamReader("MyFile.txt");
    try
    {
      Console.Out.WriteLine(myReader.ReadToEnd());
    }
    finally
    {
      myReader.Dispose();
    }
Lets view the result in Reflector:

    using (StreamReader reader = new StreamReader("MyFile.txt"))
    {
      Console.Out.WriteLine(reader.ReadToEnd());
    }
    using (StreamReader myReader = new StreamReader("MyFile.txt"))
    {
      Console.Out.WriteLine(myReader.ReadToEnd());
    }


Again, the tool was smarter and formatted the code to the same statement.


"event" statement

Yes, you might not have noticed this but the "event" keyword is actually a
macro. I will not try to recreate the code because, as you will soon
see, it is quite complex but it is nice to have a look at it.

We will include this simple statement in our code and see the resulting IL code:

public event EventHandler<EventArgs> MyEvent;

This is a shorthand notation for an event with the signature void
Func(object sender, EventArgs args)
Now lets look at the resulting C# code:

  // Fields
   private EventHandler<EventArgs> MyEvent;
 
   // Events
   public event EventHandler<EventArgs> MyEvent;

First of all the compiler added an additional member of type
EventHandler<EventArgs> with the same name as the event (this is the
member you are actually using).
If we check further we can see that this is simply a MulticastDelegate
(the IL of its definition is):
.class public auto ansi serializable sealed
EventHandler<(System.EventArgs) TEventArgs> extends
System.MulticastDelegate


 So we are actually working with a MulticastDelegate but we also get
two custom method for adding and removing event handlers to it:
public void add_MyEvent(EventHandler<EventArgs> value)
{
   EventHandler<EventArgs> handler2;
   EventHandler<EventArgs> myEvent = this.MyEvent;
   do
   {
       handler2 = myEvent;
       EventHandler<EventArgs> handler3 = (EventHandler<EventArgs>)
Delegate.Combine(handler2, value);
       myEvent =
Interlocked.CompareExchange<EventHandler<EventArgs>>(ref this.MyEvent,
handler3, handler2);
   }
   while (myEvent != handler2);
}

and

public void remove_MyEvent(EventHandler<EventArgs> value)
{
   EventHandler<EventArgs> handler2;
   EventHandler<EventArgs> myEvent = this.MyEvent;
   do
   {
       handler2 = myEvent;
       EventHandler<EventArgs> handler3 = (EventHandler<EventArgs>)
Delegate.Remove(handler2, value);
       myEvent =
Interlocked.CompareExchange<EventHandler<EventArgs>>(ref this.MyEvent,
handler3, handler2);
   }
   while (myEvent != handler2);
}

And this is the code that get executed when you call "+= " or "-=" on
your event member.

[edit: The next section was not in the original post]
"lock" statement

Perhaps the most well known macro in C# is the lock statement. The lock statement is used to create a critical section in your code which relies on some object which is used as a lock on that section of code. The "lock" keyword is just a macro for a try-finally block which uses the Monitor class to create a critical section.

For example the following code:

      object myLock = new object();
      lock (myLock)
      {
        Console.WriteLine("Inside MyLock");
      }
 
      Monitor.Enter(myLock);
      try
      {
        Console.WriteLine("Inside MyLock");
      }
      finally
      {
        Monitor.Exit(myLock);
      }
 
When compiled and decompiled using Telerik justDecompile will result in the following code:

 object myLock = new object();
 Monitor.Enter(object obj = myLock);
 try
 {
  Console.WriteLine("Inside MyLock");
 }
 finally
 {
  Monitor.Exit(obj);
 }
 Monitor.Enter(myLock);
 try
 {
  Console.WriteLine("Inside MyLock");
 }
 finally
 {
  Monitor.Exit(myLock);
 }

We can clearly see that the two statements are identical (except this somewhat strange syntax generated by Mono.Cecil in the first Monitor.Enter call.


I hope you enjoyed this post.
Thank you for reading,
Boris

Friday, June 17, 2011

Helpful tools for the weary programmer Part 2

Hi All,


This is the second part of my review of the tools I use in my daily work. If you likes the first part you will surely like the second.


Code decompilation or disassembly
The third category of useful tools are disassembly tools. If I were to make this post couple of months ago it would probably contain only Red-Gate Reflector but recently Red-Gate decided to charge money for even the basic version of Reflector so I will survey two alternative tools which are currently in the beta stage. 

The first tool comes from the guys at #Develop and is called ILSpy. ILSpy can take a compiled .NET assembly and regenerate the original code (or at least something close to the original). In the current beta it can show you the code in both C# and IL and it is quite accurate in the generated code. You can save the generated code to a code file easily from the menu. ILSpy supports search and navigation directly from the code editor and browsing of dependent dlls. One of the big tests of a decompiler is the ability to decompile itself which ILSpy does perfectly.

Another contender to the crown comes from the guys at JetBrains (the ones who make ReSharper) in the form of dotPeek. The UI of dotPeek looks similar to ILSpy (and to Reflector) but offers richer menus. I found it somewhat difficult to navigate through the code editor and had to search through the context menu items. At times the navigation didn't work at all. The generated code was quite accurate and at some places clearer than the code generated by ILSpy. It had no difficulty in decompiling itself 


The third tool comes from Telerik and is called justDecompile. Currently it is in the Beta stage and free. The UI is again similar although somewhat slick. I didn't manage to get any useful data with this tool as it failed to decompile even the simplest of codes. It is clearly not ready for real user but definitively worth to keep in mind.


This list cannot be complete without ILDasm (the tool that comes with .NET, search ildasm in your Everything). I like this tool because you are able to view the manifest of a dll in a convenient way. You can also use it to check the dependencies of a specific dll but only for one level. The biggest disadvantages of this tool is that the code is decompiled into IL and there is no easy navigation between assemblies or even within the same assembly.


ILSpy decompilation of the Object class

dotPeek decompilation of the Object class


justDecompile decompilation of the Object class


Snooping/Spying a running application
You probably know the following scenario: you add some control to your application and see it well in the designer but when the application is run it is nowhere to be found,or a scrollbar appears out of nowhere and you have no idea which control shows it. It is simple to see the dynamic structure of your running application while it runs using one of the following tools:
Spy++ for Win32 based applications (anything which is GDI based and not drawn by DirectX). This tool is installed with Visual Studio and is easy to use. When you run it, it shows all the window handles of your top level windows in a tree and you can drill down to the smallest of windows (in GDI everything is a window). You can highlight a certain window by using its handle or find a certain window by pointing at it with the mouse (using the find target tool). When the window is found you can track all the messages sent to that window.


If you try to use Spy on a WPF application you will have a surprise on your hands. Any WPF Window contains only one GDI window because the rendering is done internally using DirectX. In order to inspect a WPF application I use a different tool called Snoop. This tool is essentially like Spy (but somewhat closer to Firebug). Pointing Snoop on a WPF application will give you the full Visual Tree of this application. For each item in this tree you will be able to see and sometimes edit its properties. Snoop will also tell you who set each property so you will finally be able to see which animation stuck the height of your button.


Snooping justDecompile which decompiles the Object class




These are the main tools I use in my daily work. I very much recommend that you try each of the mentioned tools at least once to see if it is useful for you.


Thank you for reading
Boris.

Friday, June 10, 2011

Helpful tools for the weary programmer Part 1

Hi All,

[beginner level]
Today I am going to talk about some tools which make my life as a programmer somewhat easier. I know there are tons of articles such as this but I want to give my personal view on the way I work. A nice lecture about useful tools was given in the 2010 TechEd in Eilat and can be found here. The tools I am going to talk about are either free or come with the installation of Visual Studio and they are ordered by their usefulness to me and grouped by functionality.

Locating your files
If you are still using windows search function then this tool is for you. The tool is called Everything. (This tool was shown to me by David Elentok couple years back, you can find his blog with very interesting posts on my favorite blogs section on the right side of the page.)
After installing, the tool will "index" your hard drive. I write "index" in double quotes because it doesn't really read  your entire hard drive like some other search tools, it simply reads the NTFS table and maps it to an internal database (which is about 4Mb in size, depends on the size and content of your hard drive). This process runs the first time you start Everything and takes around 1-2 minutes. Every time you start Everything again it will update the local database within several seconds (the default behaviour is to let it start with windows and let it run in the background, then you don't even have to wait for it to start every time)  After the "indexing" is done each search takes... well... it searches as fast as you can type. The search results are updated as soon as you make changes to the files on your hard drive which can be very useful. One problem I found with this tool is that it sometimes doesn't show correct file metadata, this is a minor issue which should be taken into account.
I think this tool is a must for anyone working on a computer.

Who store my referenced dll (or who accessed my file)?
How many times did you get this exception message:
Could not load file or assembly <SomeDllName> implementation version=1.0.0.0, culture=neutral, publickeystoken = 90ba9c70f846762e or one of its dependencies. the system cannot find the specified files.
What do you do then? You open Everything (see previous paragraph) type in <SomeDllName> and of course find this file in your bin directory. This leaves you with one of its dependencies... doh!

As an example I created a console application called LoadExample which is unable to load a dll called MissingDll (because I deleted it... :))

The easiest thing to do is to run "Fusion". Note that it needs to be run as administrator and you need permissions to edit the registry! Just run the tool (find it using Everything on your drive) as administrator (right-click on the executable file and select Run As Administrator) then in the settings set it to log failures and select some (existing) custom path to be able to delete the cache. Run the problematic application and you will see the missing dll.

This is what you see when you run LoadExample with Fusion

There is a missing file but it is not an assembly? You can use the Sysinternals Process Monitor which is downloadable as part of the Sysinternals Suite. Start procmon.exe and set the filter path to either the missing file name or if you don't know the missing file name then set the process name to your application process name (use the "contains" condition so that you don't have to write exact names and paths). Now run the application and check the Results column. You are looking for anything that might be a hint for a problem and usually contains the words "NOT FOUND". Note that the Process Monitors monitors five types of operations as denoted by small icons on the right side of the main toolbar. If you are looking for a missing file you might want to leave only the file system activity turned on.

I found the missing dll

Missing dll in native code (or CLI)? There is a simple solution in the form of Dependency Walker. Once you load a dll in Dependency Walker, it will show all the dlls it depends on in a hierarchical tree. If there are missing dlls they will he highlighted using a special icon. This is especially useful when you fail to register a dll using regsvr32 because it will show all the dlls which are needed (and missing) for the registration.

You can get more (or less) information on this topic by reading this blog post.

Next week I will publish the second part of this post with some more useful tools.
Thanks for reading,
Boris

Friday, June 3, 2011

Displaying Buffer Data in a .NET Application

Hi Everyone,


Recently I bought a new computer. It isn't high end but enough for the things I regularly do.
I got I5 760 2.8Ghz CPU with 4Gb of memory (1333Mhz) and ~2Tb of hard disk space.


Why am I telling you this? Because today I am going to tackle a simple problem: Loading a buffer into a text editing element. 
The setup is as follows: I have created a 8Mb file which contains the letter "A" 2^23 times (basically it is a huge line of As :)) and created some sample programs which contain only a text editing control and a button which loads the file into the control. All I do is click the button.


Legend:
Time - The time it took between pressing the button and the time the control is finished loading (if the control supports virtualization I scrolled all the way down).
Time to show data - The time it took beteen pressing the button and the time something was visible for user interaction (this is also the time until the UI became responsive).
Memory consumption - The amount of private bytes used by the application after the control is fully loaded.
Is it possible to work - Was it possible to interact with the loaded data in an acceptable manner.



  1. WPF TextBox (No text wrapping) - 
    1. Time - 11min 50sec
    2. Time to show data - less than 1 second
    3. Memory consumption - 930Mb
    4. Is it possible to work - Not really, I can interact with the TextBox but it is hard to do something.
    5. Although I set the TextBox to no wrapping the text was wrapped.
  2. WPF TextBox (with text wrapping) -
    1. Time - 15 sec
    2. Time to show data - less than 1 second
    3. Memory consumption - 330Mb
    4. Is it possible to work - Yes, the UI was quite responsive
  3. WPF RichTextBox  
    1. Time - 17 sec
    2. Time to show data - 17 sec
    3. Memory consumption - 440Mb
    4. Is it possible to work - Not really, everything I did made the UI not respond for a while
  4. Winforms TextBox  
    1. Time - I stopped waiting after 25 minutes
Now onto more advanced editors
  1. Avalon Edit May 2011 (no text wrapping)
    1. Time - 3 min 5 sec
    2. Time to show data - 3 min 5 sec
    3. Memory consumption - 650Mb
    4. Is it possible to work - Not really, everything I did made the UI not respond for a while
  2. Avalon Edit May 2011 (with text wrapping)
    1. Time - 3 min 5 sec
    2. Time to show data - 3 min 5 sec
    3. Memory consumption - 650Mb
    4. Is it possible to work - Not really, everything I did made the UI not respond for a while
  3. ICSharpCode.TextEditor
    1. Time - 4 sec
    2. Time to show data - 4 sec
    3. Memory consumption - 60Mb
    4. Is it possible to work - Not really. It was very slow although somewhat responsive
I would like to note that the above editors are very advanced editors which were not designed for the purpose of loading buffers. These editors excel in editing and displaying code.

Some well known editors
  1. Notepad (Windows 7)
    1. Time - 1 sec
    2. Time to show data - 1 sec
    3. Memory consumption - 20Mb
    4. Is it possible to work - Yes, the UI was completly responsive as if you are editing a 10 byte file
  2. Notepad++ 5.8.7
    1. Time - 1 sec
    2. Time to show data - 1 sec
    3. Memory consumption - 100Mb
    4. Is it possible to work - No, it hung after I scrolled around
  3. Visual Studio 2010 Ultimate SP1
    1. Time - 12 sec
    2. Time to show data - 12 sec
    3. Memory consumption - 220Mb 
    4. Is it possible to work - Yes, the UI was slow but still possible to work with
Conclusion
It seems that the default editors of .NET have problems with handling this case. Even when they are successful in loading the file, the memory consumption is quite large and the UI is not responsive. It is also obvious that Microsoft has solved this problem internally because their editors handle this case without a problem.

As usual please feel free to comment (especially if you know of a good editing control which can handle this case).

Thanks for reading,
Boris.