Work around limitations of VS 2005 Deployment Projects with Windows Services

I needed to deploy a Windows Service using an MSI. The project was built in VS 2005 using .Net 2.0; so far, so good.

The problem arose when I wanted to have the Service write to its own Event Log, rather than the default Application log. The standard Installer class from System.Configuration.Install, which otherwise does a fine job of installing the Windows Service, proved less than ideal because it automatically creates an EventLogSource with the same name as the Service executable, and sets its event log to be Application.

The solution lies in the way in which the Installer class contains a series of other installers in the Installer.Installers collection, which in turn contain other installer objects.

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Configuration.Install;
using System.Diagnostics;

namespace MyService
{
  [RunInstaller(true)]
  public partial class ProjectInstaller : Installer
  {
    public ProjectInstaller()
    {
      InitializeComponent();

      //
      // ProjectInstaller
      //
      this.Installers.AddRange(new System.Configuration.Install.Installer[] {
            this.serviceProcessInstaller1,
            this.serviceInstaller1});
    }

    private void serviceInstaller1_BeforeInstall(object sender, InstallEventArgs e)
    {
      RemoveEventLogInstallers(this, 0);
    }

    ///
    /// Recursive method to remove EventLogInstaller Installers from the Installers collection.
    /// This is needed to avoid automatic setting up of the event logs
    private static void RemoveEventLogInstallers(Installer installer, int depth)
    {
      Console.Out.WriteLine(String.Format("Number of installers at depth {0}: {1}", installer.Installers.Count, depth));
      for (int i = installer.Installers.Count - 1; i >= 0; --i)
      {
        Installer childInstaller = installer.Installers[i];
        Console.Out.WriteLine(String.Format("Installer {0} at depth {1} is of type: {2}", i, depth, installer.GetType().Name));

        RemoveEventLogInstallers(childInstaller, depth + 1); //recurse

        if (childInstaller is EventLogInstaller)
        {
          Console.Out.WriteLine(String.Format("Removing Installer {0} at depth {1} because it is of type: {2}", i, depth, childInstaller.GetType().Name));
          installer.Installers.Remove(childInstaller);
        }

      }
    }

    private void serviceInstaller1_BeforeUninstall(object sender, InstallEventArgs e)
    {
      RemoveEventLogInstallers(this, 0);
    }
  }
}

Search recursively through the entire tree of Installer objects and remove any installer whose type is EventLogInstaller, and the problem is solved! Make sure to do this before Install and before Uninstall, otherwise the (un)installation does not complete successfully.

Microsoft Application Blocks

Microsoft Application Blocks form a collection of ready-built ‘clumps’ of code which solve common problems such as security management, data access, logging, etc. They are the tangible result of the Microsoft Patterns and Practices advice: all of it sound and solid.

John Jakovich, one of the 4 Guys From Rolla, gives a useful An Introduction and Overview of the Microsoft Application Blocks. He summarises the utility of the Application Blocks thus:

…you don’t have to worry about the tedious details required to return a DataReader or write an exception to the System Log. This allows us to concentrate on the business logic in our applications.

I have written too many logging frameworks in the past: it’s boring above all else. I just want to log exceptions in a thread-safe manner, with a unique ID.which I can display to the user if necessary. If someone (i.e MS) has already written code to do (most of) this, then fine – I’ll use it.

The Security block is particularly useful for ASP.NET 1.1, where security and profile management is not as simple as in version 2.0. All that boring stuff about storing Role information in cookies? Solved! Better still, any security holes will be fixed by MS. Again, more time to concentraste on business logic.

Design Guidelines for Class Library Developers

The Application Blocks tie in nicely with a set of guidelines from MS on class library development. They include advice on:

  • array usage
  • exposure to COM
  • casting
  • threading

and several other subjects. This is basically just a gloop of Common Sense, but well worth a read.

MBR BootFX Application Framework

An alternative to the MS Application Blocks comes from Matt Baxter-Reynolds (he of DotNet247) in the form of BootFX:

The MBR BootFX Application Framework is a best-of-breed application
framework that we offer to all our clients who engage us to develop
software applications or software components for them. It’s designed
to give us a “leg up” on new projects by providing a tried and tested
code base for common software development activities.

There are lots of goodies there, including Object Relational Mapping (ORM), and support for SQL Server, Oracle, and MySQL databases. To top it all, it’s open source, via the Mozilla Public Licence 1.1. I met Matt 18 months or so back at a seminar run by Iridium; very personable guy.

Mail: Enabled

At this point, I am inclined to decide that SMTP stands for “Stoopid Mother Truckin’ Program”, after some frankly bizarre shenanigens with SMTP on one particluar server. Thankfully, MailEnable came to the rescue. This is a well-engineered bit of software, and best of all, it’s free, even for commercial use (Enterprise users pay for high-end features).

The difficulties arose with a DotNetNuke installation, incombination with some stubbornly unconfigurable SMTP services. FOr example, the main SMTP server on the machine could not be configured to talk to web applications, and a second SMTP server program was available to do this, on another port (i.e. not port 25).

HOWEVER, it turns out that DNN does not allow you to specify an SMTP server on a port except port 25. In other words, a setting like this will not have the desired effect:

localhost:587

The port is ignored by the System.Web.Mail.SmtpMail class as used by DNN. I checked the source for DNN 3.2.2, and the port number is NOT parsed out from the servername string. This seems to me to be a serious limitation. A line or two of extra code would enable mail sending on alternative ports: parse out the port number, and if its there, set the port explicitly for the mail message before sending:

' line 152 or Mail.vb:

' external SMTP server
If SMTPServer  "" Then

' TODO: parse out port number here

    Web.Mail.SmtpMail.SmtpServer = SMTPServer
        Select Case SMTPAuthentication
            Case "", "0" ' anonymous
            Case "1" ' basic
                If SMTPUsername  "" And SMTPPassword  "" Then
                    objMail.Fields("http://schemas.microsoft.com/ cdo/configuration/smtpauthenticate") = 1
                    objMail.Fields("http://schemas.microsoft.com/ cdo/configuration/sendusername") = SMTPUsername
                    objMail.Fields("http://schemas.microsoft.com/ cdo/configuration/sendpassword") = SMTPPassword

' TODO: Insert port here...

                 End If
            Case "2" ' NTLM
                    objMail.Fields("http://schemas.microsoft.com/ cdo/configuration/smtpauthenticate") = 2
        End Select
End If

So I was left having to use a server on the default port 25, but with only a single machine it’s tricky to get SmartHost relaying working properly (in fact, it might be impossible – I haven’t checked the RFCs).

In the end, I disabled the two existing SMTP servers, and installed MailEnable from http://www.mailenable.com/standard_edition.asp – it is very easy to use and configure. Crucially, it can listen on two different ports, and had no limitation with respect to web applications.

I highly recommend it if you are having similar problems.

Issue Tracking Software

A crucial component of a successful software project is feature tracking. Bugs, issues, defects, requirements, tasks: whatever they’re called, they need to be recorded, tracked, and dealt with.

At DevDoctor.com we started off with the free IssueTracker Starter Kit from the folks at Microsoft. This is a very simple, bare-bones application which gets the job done, but has some really useful features missing, at least in the version out-of-the-box. For example, you cannot view all tasks across all projects, only grouped by project. The Search feature is basic too, requiring SQL-like syntax (e.g. LIKE %crash%) to find matches.

Eventum is a well-featured system, running best on LAMP, but working on Win32 also. Particularly nice are the built-in time-tracking features, which allow a project manager to see at a glance where time is being spent on the project. It has email integration to allow issues to be created automatically, and for users to receive notification when they are assigned an issue. It also features integration with CVS, though Subversion support would bring it more up-to-date.

There are obviously free offerings like Bugzilla, but the interface sucks, and it’s very unfriendly. A well-featured system I have used at a previous company is FogBugz (from Fog Creek software, of Joel On Software). It’s therefore very usable: to bring up the page for Issue #51, for example, you simply type ’51’ in the search box, and hit Enter. It sports RSS feeds of issues, and has email, CVS, and Subversion (and even VSS) integration, allowing you to match up bugs fixed in code with the entries in the issue tracker. Sadly, it’s not free.

Another system which looks nice is OnTime from AxoSoft. The single-user version is free, though additional users are rather pricey. It has a WinForms client in addition to the Web interface, so provides for a much richer interaction. It also has Visual Studio integration, so bugs can be managed from the location where they will be fixed: inside the IDE. Its feature list is pretty impressive; clearly, the focus is on .Net development teams.

All in all, what I want from issue tracking depends on the context. The ASP.NET IssueTracker works well for my own needs for personal projects. When projects become more comples, and there is a need for time tracking too, then Eventum or OnTime look good options. For ease-of-use, FogBugz probably comes out top.

ACCU Conference 2005

Introduction

I attended six presentations at the ACCU Conference this year:

  1. IronPython – Python for .NET (Microsoft)
  2. Python at Google (Greg Stein)
  3. C++ and the CLR (Microsoft)
  4. Unit Testing XSLT (Jez Higgins)
  5. Modern 3-Tier Architectures (T-Mobile)
  6. .TEST Testing Framework (Parasoft)

I also made some random observations.

IronPython – Python for .NET

The experience of seeing Python being implemented on the CLR (by Jim Hugunin) has influence Microsoft’s design of .NET 2.0 – better support for dynamic languages, generative programming, dynamic classes, and so on, are all coming to .NET 2.0. In fact, so much so, that Microsoft head-hunted Hugunin last year.

IronPython targets the CLI, the open-standard implementation of the CLR, so it runs on Mono and other, non-Microsoft platforms. We saw a live demmonstration of IronPython importing classes written in C# from an Assembly, and calling methods and events on those classes. Even the IronPython source code itself if a Visual Studio project!

Python at Google

Greg Stein has worked for several major organisations, including Microsoft, Apache Foundation and now Google. He revealed that Python is fundamental to Google, and is used not only in back-end management, but as the main language for both Google Groups (http://groups.google.com/) and Google Code (http://code.google.com/). Some of the other areas for which Google uses Python include:

  • Wrappers for Version Control systems – to enforce check-in rules etc.
  • Build system – entirely in Python
  • Packaging of applications prior to deployment
  • Deployment to web-farms and clusters
  • Monitoring of servers (CPU Load, fan speed, NIC utilization, etc.)
  • Auto-restarting of applications
  • Log reporting and analysis

He referred to Python a the “special sauce” which makes a company more ‘tasty’ or appealing than another. Because Python lends itself to readability and maintainability, it also allows rapid changes to code, following shifting requirements. C and C++ tend to be used (in Google) only for specialist, speed-critical tasks. Otherwise, Python (or sometimes Java) is used – it’s simpler and quicker to write, and easier to modify.

Software Development

Greg made some interesting comments about software development. At Google, the typical project is “3 people, 3 months”, forcing many small applications to co-operate, rather than having just a single monolithic blob, aiding scalablility. Every Google developer has 20% of their time to “play” with their own ideas. Google Maps, and the Google Toolbar both started off with employees tinkering with code in the 20% “spare” time.

Employees are also encouraged to submit patches to any part of the Code Base, not just that for their project. THis more open approach contrasts with the style at Microsoft (apparently), where teams have access only to the source for their project and no other. Greg argues that this openness explains why Google innovates so well.

As an aside, he said that he has a machine running Unit Tests continuously on the latest code, so that as soon as code breaks, they are notified ;o)

C++ and the CLR

The bod from Redmond pitched C++ as the Systems programming language for the foreseeable future, underpinning the .NET Framework, and allowing developers to achieve applications of the highest performance.

The C++ compiler can compile manages code directly to a native platform executable, and allows free mixing of Managed and Unmanaged code throughout an application. In this way, the .NET Framework is treated as just another class library, which can be loaded on demand.

Performance can thus be highly tuned. We were shown a demo of Doom 2 running on the CLR, and running faster than the native code, due to the highly optimized code produced by the C++ compiler. For example, in Managed C++, when a variable goes out of scope, it is automatically set to null by the compiler, making the job of the Garbage Collector much easier. In C#, by contrast, this does not happen automatically, so that objects marked for GC may sit in the GC queue much longer, using up resources.

The Sampling Profiler was recommended as a good tool to use when beginning investigations into performance problems. No code changes are required, and it can be used on live systems with practially no detrimental effects to application speed.

Unit Testing XSLT

XSLT is a fully-specified programming language; it is Turing Complete. However, unlike most programming languages, it has very few testing and debugging tools. Often, in a mixed system of Java/C# + XML + HTML, the XSLT is something of an unknown: output must be inspected visually, not programatically, to attempt to verify correctness.

Jez Higgins demonstrated some simple code (written in XSLT) allow Unit-Testing of XSLT.

Basically, like any Unit Testing, you define expected output data for given input data, and test that the software (here, the XSLT transform) produces the correct output. There is an Open Source project called XSLTUnit (http://xsltunit.org/), which, although really proof-of-concept, is stable and useful. Starting with this framework, Jez added extra functionality so that Unit Testing of XSLT files can be included as part of an JUnit test script [the same could be done in other languages, e.g. C# and NUnit].

The XML looked intially complicated, but the scheme is actually remarkable simple:

    1. Define input data for a given XSLT file (in XML).
    2. Define expected outputs for the transform (in XML).*
    3. Write a special XSLT file which, upon transform, will produce Java code for each test case. The Java will inspect the XML output file.
    4. Write a small amount of Java to load in the XSLT and XML files, and perform the transformations.
    5. Using Ant and JUnit, compile and run the Java code generated by the XSLT to test the output data.

*Note: output from transform must be in XML

The beauty of this approach is that existing standard Unit Test tools (JUnit/NUnit/etc.) can be used to test XSLT! XSLT is used to generate code required to test XSLT itself; Java (or C#, etc.) is used dynamically as “scaffolding” to make use of standard testing frameworks.

Example:

import junit.framework.*; public class extends TestCase { public (String name) { super(name); } public static Test suite() { return new TestSuite(.class); } } // class public void () { Helper.runTest( “”, “”, “”); }

Once the initial test definitions are written, no more work is needed – no Java, no editing XML configuration files. Crucially, the fact that test definaitions are defined in XSLT allows non-programmers to write the Test definitions, in addition to the XSLT itself; this follows the standard pattern of Unit Testing, where the implementer also writes the Unit Tests.

In terms of the complications introduced by complex, once-unverifiable XSLT, this scheme could bring massive improvements.

Modern 3-Tier Architectures

The most salient point made by Nico Josuttis was with regard to Project Management: that in a large n-tier system, the architecture will change over its lifetime, and trying to lock down the technical design is recipe for disaster. Clients like to think that the architecture remains the same, but – even if that is what they are led to believe – developers must be willing to refactor or completely redesign.

The speaker argued that the back-end (database layer) should maintain data consistency itself, and therefore perform checks for:

  • Authentification (who am I)
  • Authorization (what I am allowed to do)
  • Validation (does the data make sense)

There was lots of discussion of the granularity of security and validation, and where exactly this should take place. Most people agreed that having two separate levels of validation is essential. This granularity issue was exemplified by comments from some people who work for large telecoms comapnies, whose databases contain thousands and (soon) millions of logins – one for each customer – not just a few logins used by the application for all connections, irrespective of the customer. This clearly has performance issues.

Project Management

Nico then spoke about development procedures within a large company, T-Mobile, where (typically) software is released every 3 months; they have 6 weeks development time, followed by 6 weeks of regression testing – a 50-50 split between coding and testing!

As for releasing software, he reported that in his experience, nightly builds are essential (echoing Greg Stein’s comments about the continuously running build machine). He suggested to plan for ten per cent of development effort to go into releasing software (build, deploy, integration, etc.).

He was also scathing about UML, MDA, and other “fad” techniques, at least as far as whole projects go. For small tasks they are okay, but a project driven from these modelling approaches is doomed, according to Josuttis.

Parasoft .TEST Testing Framework

Parasoft .TEST is a testing tool for .NET code. It has Visual Studio integration, but can also be run as a stand-alone app (for the QA people) and from the command-line for build process integration.

It addresses three main areas: Coding Standards, QA testing, and Unit Testing. For the latter, it integrates with NUnit, allowing existing Unit tests to be run as part of the testing cycle.

It works by inspecting the ILASM in existing Assemblies, so can work with Assemblies created with any .NET language. It can automatically generate NUnit test harness projects for the assembly and (with the VS-Plugin version) automatically add these to the Solution. This is probably the biggest bonus – all the tedious Unit Test scaffolding is done for you – all the developer needs to do is to fill in the implementation of each generated test. This then addresses one of the chief complaints of people using Unit Testing; namely, that it takes a long time to code up all the Test classes.

In addition, the suite generates internal Unit Tests for common boundary conditions (null reference, index out of bounds, etc., etc.) across all or part of the Assembly/Assemblies under test.

.TEST ships with about 250 pre-defined Coding Standards, but others can be defined by non-programmers, so that (for example) Security and Performance QA issues can be addressed at the code level, and as part of the testing cycle.

It was pretty impressive; seemingly all the gain and none of the pain of Unit Testing.

Other random observations

Python

There are Python Meetups in London (Putney); the UK Python mailing list is uk-python@python.org.

Version Control

Greg Stein said he wants to convert Google from Perforce to Subversion¬†for Source Code Control. Now, he might be a bit biased (he helped develop SVN), but if SVN is ready for Google, it’s ready for anyone, I reckon!