February 2007 - Posts - Coding Sanity

February 2007 - Posts

A must-read article on why programmers can’t program from Coding Horror. The scariest factoid from the post:  “199 out of 200 applicants for every programming job can’t write code at all. I repeat: they can't write any code whatsoever.”

Terrifying.

There's a story at Slashdot about a teenaged hacker named Brad Willman who broke into 3,000 computers using a trojan he wrote. He created some infected pictures, and then distrbuted them on child porn groups, using them to gain access to the downloaders compluters. Anyway, he managed to track down a judge who was contemplating abusing a boy, and had him arrested. The Computer Crime Research Center has an article about him. Anyway, it looks like the judge could have got off scott free due to this "hackers" actions, Willman said in response:

“In one way, he’s won because he won’t have to go to prison. But in another way, other people know about him now, and they can watch out for him. He’s not as free as he used to be before all of this happened.”

So, he's effectively destroyed any chance of convicting a pedophile, and this is his pathetic response? Since Willman had complete control over the judge's computer, of course the judge couldn't be convicted because of evidence on said computer, For all we know Willman planted it there. It gets even more worrying when you hear more about Willman. He is a serious loner, spending almost all his free time monitoring the computers he has hacked, reading emils, monitoring downloads. He calls himself Omni-Potent, sme illustrative quotes:

  • "I judged these people by reading their incoming and outgoing e-mails"
  • "Sure, a violation of privacy you must cry, but if you have nothing hurting kids, the future of the world, then there's no reason to worry as that is all that Omni-Potent protects"
  • "Omni-Potent's service thus far has been provided without cost to the public. Not one dime has been provided to Omni-Potent and yet there has been tremendous success in providing accurate information."
  • "It is after all, technology which helped to find me and mess up important investigations by attempting to lift my veil"

There's no doubt that this young man is seriously disturbed. Considering that he would have had to view the child porn he downloaded in order to "judge" it, one wonders about his motives.

Mor importantly he has committed a massive hacking attack, and should be in jal. However, it appears that if your hacking is targeted against people who society disapproves of (admittedly quite rightly so in this case), then you can hack people at your leisure. The reality is that none of us would welcome such intrusive surveillance, no matter what the aim, especially from a self-appointed teenaged "judge"

Happily the judge was sentenced to jail, but easily might not have been.

Introduction

In my blog post about marshalling calls, I discussed the basic pattern I use to create an asynchronous component. Well, in this article I’ll discuss a specific example. I was inspired by a post to a forum where someone was asking about the simplest of all possible things: searching the file system. He mentioned that he’d be searching the C:\ drive and all subdirectories, and I pointed out that such a search could take a while. I suggested that in such a situation he run the search in a background thread.

 

Well, anyway I decided to go and write a component that would do such a background file search for you, and handle all the marshalling to the UI. So, I created the FileSystemSearcher component. I wanted it to be a little more hard-core than the standard .NET file system searching, so this component allows for multiple starting search locations, and multiple search patterns.

 

Creating the Interface

The first step was pretty easy, I decided on the interface to the control. There are three main properties:

  • Locations – This contains the locations from which the search will progress. If the user decides on a recursive search, each one of these locations will be a root directory.
  • Search Patterns – This contains a set of search patterns, and file system objects which match any of the patterns will be returned.
  • Options – This property contains an enumeration which controls the search options. Basically it boils down to whether to look for files and/or directories, and whether to search recursively in child directories.

 

Next, I had to consider how the actual search would be performed. I decided that there would be a public method (BeginSearch) which would check that a search could be started, do any preparation work, and then invoke a background thread to do the actual work. The background thread would report on its progress by events marshaled to the context of the thread that called BeginSearch.

 

First off BeginSearch needed to ensure that there wasn’t already a search in progress. To support this I created a boolean variable which would hold the running state, and also created a lock variable to control access to it. If it wasn’t running, the next thing the BeginSearch needed to do was to capture the thread context. All the events would be marshaled to this context. I accomplished this the same way BackgroundWorker does, by getting an AsyncOperation object from AsyncOperationManager.CreateOperation. Finally, I used a delegate’s BeginInvoke to asynchronously kick off the method that did the actual work: PerformSearch.

 

Reporting Results

I won’t go into the details of the actual searching, suffice it to say that I handle the directory recursion myself and use Directory.GetFiles and Directory.GetDirectories on each directory. This allowed me to report detailed progress. I created the following events for reporting progress:

  • SearchCompleted – Fired when the search has completed.
  • SearchError – Fired whenever an error was encountered during the search, but the search continues nevertheless, skipping that directory.
  • BeginSearchDirectory – Fired when the search system is about to search a given directory.
  • EndSearchDirectory – Fired when the search has finished searching a directory.
  • ItemFound – Fired when the search has found a matching item.

 

Firing each event is a pretty involved process because of the thread marshalling that must occur. I’ll use the ItemFound as an example. The code that started the event firing looks similar to this:

 

_asyncOperation.Post(new SendOrPostCallback(AsyncItemFound), new DirectoryInfo(directory));

 

So, it’s using the AsyncOperation that was created in BeginSearch to post the directory information to the AsyncItemFound method. In other words, AsyncItemFound will be called on the UI thread. So let’s have a look at that:

 

private void AsyncItemFound(object state)

{

    OnItemFound((FileSystemInfo)state);

}

 

Well, that’s easy enough. It’s going to call OnItemFound, which will raise the event. I could have raised the event right here, but then people inheriting from my control would not have an OnItemFound to override.

 

And OnItemFound is also nice and simple:

 

protected virtual void OnItemFound(FileSystemInfo item)

{

    if (ItemFound != null)

    {

        ItemFound(this, new FileSystemInfoEventArgs(item));

    }

}

 

I’ve made it virtual so child classes can override it, and I simply fire the ItemFound event.

 

Stopping the Search

After completing the component and testing it for a bit, I decided to add the ability to stop the search. So I created a method called StopSearch, which like BeginSearch tested the running flag in a thread-safe manner, and if the search was still running, set a stop flag. Then, I just needed to add checks in the search code to test the stop flag and exit if it had been set to true.

 

Since my search algorithm is fairly fine-grained (one directory at a time), I decided not to bother with putting locking around the stop flag when I tested it during the search. Quite simply, at worst the search might search a couple of extra directories due to a race condition. Such a minor issue didn’t warrant adding the locking code.

 

Now there was only one snag on the horizon, I wanted to ensure that the StopSearch method would block until such time as the search had actually stopped. This is where ManualResetEvent comes into its own. I went along and created an event that would signal that the search was complete.

 

Inside the BeginSearch I set the event to non-signalled, so that threads would block if they waited on it. I then set it to signaled in AsyncSearchComplete, allowing blocked threads to continue. Finally, in StopSearch, I put a wait on the event.

 

Conclusion

So there you go; a brief description of what I did, and hopefully a bit of insight into why I did it. Plus, as an added bonus, you get an asynchronous search component. I haven't tested it too throuroughly I'm sorry to say, so please let me know if you find any bugs.

 

You can download the source code here.

Lutz Roeder has just released the latest version of Reflector, it's got a whole bunch of cool changes, go check it out. In addition he's set up a single site where you can get access to all the add-ins here.

You'll probably have to get new versions of your add-ins, since there were breaking changes. You can find the update to Reflector.Diff at the usual place or at Lutz's add-in page. Oh yeah, a big thank you to Richard Childress for tracking down an array-handling bug for me..

In the MSDN Forums not too long ago I had an argument with someone who felt that it was silly not to allow static methods on an interface. So, you could create the following interface:

 

interface IFoo

{

    static void DoThis();

}

 

Any class implementing IFoo would then have to implement the DoThis static function:

 

class MyClass1 : IFoo

{

    static void DoThis()

    {

        Console.WriteLine("MyClass1");

    }

}

 

Quite a neat idea, just taking the concept of interfaces and applying them to static members, right?

 

Not so fast. A very important point of static members is that they are defined on the type, not the instance, so the question arises where these static members should be implemented. I’ve placed an implementation on MyClass1, but what if I didn’t have MyClass1? This code should still be legal in that circumstance:

 

IFoo.DoThis();

 

So, if there is no MyClass1, what code does that call? Even worse, what if we go and define a MyClass2?

 

class MyClass2 : IFoo

{

    static void DoThis()

    {

        Console.WriteLine("MyClass2");

    }

}

 

Now, if we have both MyClass1 and MyClass2 defined, which one should IFoo.DoThis() call?

 

Okay, what if we create a new kind of modifier, not static, but muststatic, and this modifier, applicable only to interfaces indicates that the implemented member must be a static, but our muststatic does not imply the ability to call the member from the interface.

 

Well, yes, that would be possible, but the question is why would we need it? It would provide no polymorphism, since the types could not be used interchangeably, you cannot pass an instance of IFoo, if there’s nothing to pass, and with statics, there really is nothing to pass.

 

It would certainly provide no advantages I could see, but maybe I’m a bit dim, as was implied in the forum argument. Can any one of you see any advantage to this idea?

A work colleague was wondering as to the contents of my technical bookshelf, well as an inveterate reader, it's a wee bit stuffed:

Advanced .NET Programming

Advanced .NET Remoting Second Edition

A First Look at SQL Server 2005 for Developers

Algorithmics: The Spirit of Computing

An Introduction to Object-Oriented Programming with Visual Basic .NET

Application Architecture for .NET: Designing Applications and Services

Applied .NET Framework Programming

ASP.NET v. 2.0 – The Beta Version

Assembly Language for Intel-based Computers Fourth Edition

ATL COM Programming

Best Kept Secrets in .NET

Build your own .NET Language and Compiler

Compiling for the .NET Common Language Runtime

Compiling with C# and Java

Compressed Image File Formats

Compression Algorithms for Real Programmers

Computer Organization and Architecture: Designing for Performance Fourth Edition

CLR via C#

C# Network Programming

COM and .NET Interoperability

Data Compression the Complete Reference 3rd Edition

Deploying .NET Applications

Design Patterns: Elements of Reusable Object-Oriented Software

Developing Application Frameworks in .NET

Digital Image Compression Algorithms and Standards

Domain-Driven Design: Tackling Complexity in the Heart of Software

Encyclopedia of Graphics File Formats Second Edition

Enterprise Solution Patterns Using Microsoft .NET

Essential ASP.NET with Examples in C#

Expert C# Business Objects

Expert .NET Delivery Using NAnt and CruiseControl.NET

Expert Service-Oriented Architecture in C#: Using the Web Services Enhancements 2.0

eXtreme .NET: Introducing eXtreme Programming Techniques to .NET Developers

Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries

Graphics Programming Black Book Special Edition

Hacking Exposed Fourth Edition

Hacking: the Art of Exploitation

Imperfect XML

Improving .NET Application Performance and Scalability

Improving Web Application Security: Threats and Countermeasures

Inside Microsoft IL Assembler

Inside Microsoft SQL Server 2005: T-SQL Querying

Integration Patterns

Joel on Software

Linkers & Loaders

Managing Gigabytes: Compressing and Indexing Documents and Images

Maximizing .NET Performance

Microsoft Windows Internals Fourth Edition

MMURTL v1.0

Modern Operating Systems Second Edition

.NET and COM The Complete Interoperability Guide

.NET and XML

.NET Framework Solutions: In Search of the Lost Win32 API

.NET Framework Standard Library Annotated Reference, Volume 1: Base Class Library and Extended Numerics Library, 1/e

.NET Framework Standard Library Annotated Reference Volume 2: Networking Library, Reflection Library and XML Library

Operating System Concepts Fifth Edition

Operating Systems Design and Implementation Second Edition

Operating Systems: Internals and Design Principles Third Edition

PDF Reference 1.6 Fifth Edition

Peer-to-Peer with VB.NET

Pragmatic Unit Testing In C# with NUnit

Presenting Windows Workflow Foundation Beta Edition

Production Workflow: Concepts and Technologies

Professional WinFX Beta

Programming .NET Components

Programming with Managed Extensions for Microsoft Visual C++ .NET

Pro .NET 1.1 Network Programming Second Edition

Pro Service-Oriented Smart Clients with .NET 2.0

Protected Mode Software Architecture

Rapid Development

Shared Source CLI Essentials

Smart Client Architecture and Design Guide

TCP/IP Sockets in C#: Practical Guide for Programmers

Test-Driven Development in Microsoft .NET

The Art of Assembly Language

The C# Programming Language

The Common Language Infrastructure Annotated Standard

The Definitive Guide to Windows Installer

The Guru's Guide to SQL Server Architecture and Internals

The Guru's Guide to SQL Server Stored Procedures, XML, and HTML

The Guru's Guide to Transact-SQL

The .NET Developer's Guide to Windows Security

The Pragmatic Programmer: From Journeyman to Master

Windows Forms Programming in C#

Workflow Management: Models Methods, and Systems

Working Effectively with Legacy Code

Working with Microsoft Visual Studio 2005 Team System

Writing Secure Code Second Edition

Whew! Quite a list. Some, I must admit I have yet to read, my "inbox" is sitting at about 7 books right now. Some, I have used largely as reference books, and not read through so much as skimmed to get an overview, but the vast majority I've read. Now if you were to test me on their contents, I'd probably fail badly though…

This one is a common request on the forums, and it has a simple, if somewhat inelegant response:

try

{

    using (FileStream stream = File.OpenRead(path))

    {

      // Do stuff

    }

}

catch (IOException exception)

{

    // Handle locked file

}

There is no File.IsLocked (string) method, which would appear to make our code much more elegant:

if (!File.IsLocked(path))

{

    using (FileStream stream = File.OpenRead(path))

    {

        // Do stuff

    }

}

However, there is a bug in this idea. What if another process opened the file exclusively right after we called IsLocked, and before we called OpenRead? Well, in that circumstance, we would fail at opening the file when we expected to be able to open it. In fact, I challenge you to find any sequence involving IsLocked that would not open you up to a race condition. The only way you could be sure that you could open the file successfully was if no other process was executing at the same time. Well, there's only one thing that can guarantee that in you operating system, and that's the kernel. So, what Windows does, is allow you to make the attempt to open the file, and that attempt either succeeds or fails.

When writing a program it's often all too easy to slip into the mindset that nothing else is going on whilst your code executes. We are so used to the idea that we control the sequence of events that we all too rarely consider what's going on beyond our control, or how drastically conditions can change between two lines of code.

PS. I am trying out Word 2007 for this post, so let's hope the formatting doesn't get screwed up.