CLR-COM Interop

[This is a very old article I wrote back in 2002 when I worked for a company which built MRI scanners and was subsequently bought by Oxford Instruments. With COM being once again relevant with the introduction of WinRT, I thought it might be useful to revisit some core COM and .NET concepts.]

Details

CLR-to-Native Win32

The CLR subsystem responsible for managing access to the native platform is known as P/Invoke. These services are included in an application by use of the namespace System.Runtime.InteropServices, and the [DllImport(“<DLL_NAME>”)] attribute prepended to the function prototype.

When a call goes out to a piece of Unmanaged code from Managed code (CLR), a flag is set on the application’s CLR (pseudo-) stack. This causes the GC not to collect during the duration of the Unmanaged call. The Security Subsystem responds to the flag by searching up the stack for permissions to enter Unmanaged code. The security check can be suppressed by a call to System.Security.SuppressUnmanagedCodeSecurity, which prevents stack crawling by the Security Subsystem beyond the point at which the call was made.

Continue reading CLR-COM Interop

Networking at Iridium

I attended a seminar on .Net hosted by Iridium today. Met some interesting people, including a guy from Philips Belgium, a Micro$oftie developer, and another .Net code junkie (some useful links there).

The seminar covered Classic ASP-ASP.NET Interop; a bit on COM Interop; Windows DNA plus .Net for 3-tier development; Windows XP SP2; and VS.Net 2005/.Net 2.0.

Classic ASP-ASP.NET Interop

Early-adopters tend to stumble upon problems with new technology (sounds familiar!?!). Co-existence between Classic ASP and ASP.NET is possible, but a bit of a hack. In particular, the sharing of Session information is difficult; easier if data types in the Session are simple, and can be stored in a SQL DB, but otherwise tricky (storing big objects in the Session is bad practice anyhow, but that’s another story). Direct interaction between Classic ASP and ASP.NET is possible by using COM objects – either wrapping a .Net class as a COM object, or by using a RCW on a VB-style COM object. Btw, keeping the 3-tier model alive in ASP.NET perhaps needs some discipline, with “visual” components such as DataGrid so easily accessible from the Presentation layer.

COM Interop

.Net can expose functionaloty as COM objects with ease. To wrap a C# class as a COM object, give it Guid and InterfaceType attributes:

using System.Runtime.InteropServices;
...
[Guid("F09684D5-D3EF-4838-8BBC-726EF122566F),
InterfaceType(ComInterfaceType.InterfaceIsIDispatch)]
public interface MyComExposedNetInterface
{
...
}

Then declare a class which will act as the CoClass:

[Guid("0D53A3E8-E51A-49C7-944E-E72A2064F938"),
ClassInterface(ClassInterfaceType.None)]
public class MyNetComClass : MyComExposedNetInterface
{
...
}

The ClassInterfaceType.None directive restricts the methods exposed via COM to those defined in that particular .Net class (rather than those in base classes etc.). Compile and go! .Net registers the COM object internally, so that clients callinng into the COM object actually call into the .Net runtime, which manages the location and versioning of the component – nice.

Windows DNA plus .Net for 3-tier development

In a Three-Tier model (Data, Business, Presentation), the Business layer should be stateless. This means that the Presentation layer should pass in all the parameters it needs in order to retrieve the information it requires. The Data layer in .Net is simplified compared to DNA because ADO.NET handles much of the Database connectivity issues for the programmer.

Windows XP SP2

Windows Firewall, NetSh.exe,
blah, blah, Popup-blocking in IE, blah. More interesting was news that the entire of Windows XP SP2 (i.e. a good chunk of the Windows code base) has been recompiled with the /gs switch.
Some cunningness with “magic” numbers placed on the stack between function calls means that buffer overflows can be trapped more reliably. SP2 may break apps due to DLL versioning, but this can be fixed by using Side-by-Side deployment and application .manifest files to direct the app (well, the application loader) to the correct DLL version. Managing the XP Firewall could be a pain for software developers. I asked about MSXML versioning – it’s a real pain to be stuck with an ancient parser.

VS.Net 2005/.Net 2.0

Lots of nice goodies in VS.Net 2005 [side-note: some of this testing functionality already exists in Application Center Test], though only if you pay Big Bucks. The Team System will have integration IDE, Bug/Issue Tracking, Bug Reporting, Testing, Deployment, etc. Bugs
will be automatically added to a database, complete with line-number and filename where the error occurred. Generics will enable generation of specific classes on-the-fly, rather that at compile-time. ASP.NET 2.0 will have built-in Controls for Authentication, Login, User Profile management, to name a few, along with ability to add new .aspx pages after deployment.

Understanding Delphi’s perception of COM

Delphi is great for COM if you want to do something simple; as soon as someting a little more complex is needed, COM in Delphi can be a real headache. Here is a braindump from a recent investigation:

The LocalServer class registration function CoRegisterClassObject is called for each Class Factory registered with the Singleton ComServer object in TComServer.Initialize. Class Factories are registered in the initialization section of units in which they are defined, and TComServer.Initialize does not get called until after Application.Initialize has been called (using some InitProc hackery in first ComObj and then ComServ). Crucially, ComServer is a helper object to guarantee registration of all the Class Factories in the application (or DLL), so ComServer.ObjectCount refers to the number of objects it has helped to create (via Class Factories) not the number of references held (say, externally) to any particular COM object.

A post from b.p.d.activex.controls.writing by one Patrice Corteel explains further:

Now let’s go a step further in COM threading model discussion and look at local and remote servers versus in-proc servers.
A Delphi local or remote exe server has a main thread which calls CoInitialize which makes it an apartment. And as a default behaviour, every COM component based on the delphi COM framework is created in this thread and marked as apartment threaded. If you explicitly spawn new threads you can initialize them as you want but you must get a marshalled interface to call components created by your main thread.
If you want to create free threaded components it’s far more difficult since you can’t rely directly on Delphi implementation and you have to rewrite large parts of ComServ and subclass many of ComObj classes. I’m currently working on this topic and I’ll post more on this when I am over with tests.

Note that if lengthy initialisation is needed, set ComServer.StartSuspended to True before Application.Initialize; this causes the class factories to be registered with the REGCLS_SUSPENDED flag, and ComServ.InitComServer makes the required call to CoResumeClassObjects to activate them again.

References

 

ACCU Conference 2004

 

Just back from the ACCU Conference. Good sessions on OOP in Python 2.2+ [text], Game development, C++ Templates, PsyCo [screenshot], and even publishing.

Interesting job openings with CMed (Python on Linux for clinical systems). C++ Threading slides available at www.curbralan.com, and Organic Programming at (d’oh) www.organicprogramming.com. The OOP in Python talk looked at Descriptors, and how they are used (since 2.3) in everyday normal method calls: they sit between the caller and the returned method object, a bit like Delegates/Remoting in .NET. Or in fact, standard properties in C# and Delphi, though a little more complex. Essentially, Descriptors allow for ‘behind-the-scenes’ side effects. It is possible to ‘decorate’ classes so that for certain sorts of attributes, special actions will be performed. Read-only attributes can be implemented by having __set__ raise an exception. There was also some stuff about Method Resolution Order, given that Python supports multiple inheritance.

Incidentally, PyGame was described by several people os one of the best games dev packages around.

COM, Delphi and Python

I decided to write a command-line COM library registration tool, choosing C++ to flex some language muscles. This tool is needed to expand my Python+Excel -based automated calibration framework.

The first hurdle was char * style versus wchar_t * style character strings. What a nightmare! There are plenty of string types in C++, and plenty seem to work well, but ONLY on the condition that you stick with that one type and don’t mix. The standard output stream cout doesn’t support wchar_t, for example.

The thing that had me really stumped for a while was that the functions LoadTypeLib() and RegisterTypeLib() were returning S_OK but nothing was being changed in the Registry. Searching in some despair in the \Delphi5\Demos directory, I came across a project called TRegSvr, which did pretty much what I wanted. That worked first time – both reg and unreg – so what was different? Automatic calls to AddRef() and Release() in the Delphi code, of course. Adding in a call to pTypeLib->Release(); fixed the problem, although it is slightly baffling why NOT releasing the interface would cause the type lib NOT to be registered. Hmm.

Then I delved into Python’s COM support again, missing (until rather late into the process) this crucial information from Mark Hammond’s Python Programming on Win32:

…Python can’t support arbitrary COM interfaces; the pythoncom module must have built-in support for the interface…

Urg. In essence, even with the COM extensions, Python can only bind to IDispatch-supporting COM objects. Why the makepy utility wasn’t extended to all interfaces (given a suitable type library) I dunno. This, compounded by Delphi’s obfuscation of its COM support (e.g. an IDispatch-based object is actually an ‘Automation Object’ not just ‘COM Object’) made things a little slooow, but hey. Oh, and that’s forgeting that, in my installation, the \gen_py folder under win32com was missing, so that the generated Python file was dumped into %TEMP%.

Here is the COM reg/unreg program as it currently stands in all its ugliness:

// RegCOMSrv.cpp : Defines the entry point for the console application.
 //
 // An application to register COM Servers and their type libraries
 //
 // At the moment, only works with EXEs, not DLLs, but see:
 // ms-help://MS.VSCC/MS.MSDNVS/com/register.htm
 //
#include "stdafx.h"
 #include
 #include
 #include
 #include
using namespace std;
/*
 CComHelper
Used to automate and guarantee the COM initialisation gubbins.
 */
 class CComHelper {
 public:
 CComHelper() {
 HRESULT hr = CoInitialize(NULL); // for some reason, CoInitializeEx() fails
 if (!SUCCEEDED(hr)) {
 throw "CoInitialize failed!";
 }
 cout << "Initialised COM" << endl;
 }
~CComHelper() {
 CoUninitialize();
 cout << "Uninitialised COM" << endl;
 }
 };
static bool REGISTER = true;
static int INDEX_SWITCH = 1;
 static int INDEX_APP = 2;
 static int INDEX_TYPELIB = 3;
int _tmain(int argc, _TCHAR* argv[])
 {
 cout << "RegCOMSvr.exe - Copyright 2004 (c) Matthew P Skelton"
 << endl << endl;
if (argc < 4) {
 cout << "Usage: RegCOMSrv " << endl << "e.g. 'RegComSrv /r MyApp.exe MyApp.tlb'" << endl << " 'RegComSrv /u MyApp.exe MyApp.tlb'" << endl << "where /r Registers and /u Unregisters both COM server and TypeLib" << endl;
 return 1;
 }
CComHelper _com_helper; // destroyed at end of main()
 bool mode;
 ITypeLib** ppTypeLib = NULL;
 ITypeLib* pTypeLib = NULL;
 TLIBATTR* pTLibAttr = NULL;
wchar_t theApp[255];
 wchar_t theTypeLib[255];
wchar_t* w_theApp = theApp;
 wchar_t* w_theTypeLib = theTypeLib;
//OLECHAR o_theTypeLib[255];
 //LPCOLESTR po_theTypeLib = o_theTypeLib;
//MultiByteToWideChar(CP_ACP, 0, argv[INDEX_APP], strlen(argv[INDEX_APP]), theApp, 255);
 mbstowcs(theApp, argv[INDEX_APP], 255);
 //theApp = argv[INDEX_APP];
 //wstring s_theApp(theApp, wcslen(theApp));
 string s_theApp = argv[INDEX_APP];
 cout << "App: " << s_theApp.c_str() << endl;
//MultiByteToWideChar(CP_ACP, 0, argv[INDEX_TYPELIB], strlen(argv[INDEX_TYPELIB]), theTypeLib, 255);
 mbstowcs(theTypeLib, argv[INDEX_TYPELIB], 255);
 //theTypeLib = argv[INDEX_TYPELIB];
 //wstring s_theTypeLib(theTypeLib, wcslen(theTypeLib));
 string s_theTypeLib = argv[INDEX_TYPELIB];
 cout << "TypeLib: " << s_theTypeLib.c_str() << endl;
//wcscpy(o_theTypeLib, theTypeLib);
if (0 == strcmp(argv[INDEX_SWITCH], "/r")) {
 mode = REGISTER;
 cout << "Registering..." << endl;
 }
 else {
 mode = !REGISTER;
 cout << "Unregistering..." <Release() before app exits !!!
 if (SUCCEEDED(hr)){
 cout << "LoadTypeLib() succeeded" << endl;
 //cout << **ppTypeLib
 }
 else {
 cout << "LoadTypeLib() failed with result: " << hex << hr <Release();
 return 2;
 }
if (mode) {
 //HRESULT hr = LoadTypeLibEx(o_theTypeLib, REGKIND_REGISTER, ppTypeLib);
cout << "Contents of 'o_theTypeLib' as passed to LoadTypeLib" << endl;
 int max_chars = wcslen(theTypeLib);
 for (int i=0;i<max_chars;i++) {
 cout << (char)(theTypeLib[i]);
 }
 cout << endl << endl;
hr = RegisterTypeLib(pTypeLib, theTypeLib, NULL);
 if (SUCCEEDED(hr)){
 cout << "RegisterTypeLib() succeeded" << endl;
 //cout << **ppTypeLib
 }
 else {
 cout << "RegisterTypeLib() failed with result: " << hex << hr <Release();
 return 4;
 }
/*
 hr = LoadTypeLibEx(theTypeLib, REGKIND_REGISTER, ppTypeLib);
 if (SUCCEEDED(hr)){
 cout << "LoadTypeLibEx() succeeded" << endl;
 //cout << **ppTypeLib
 }
 else {
 cout << "LoadTypeLibEx() failed with result: " << hex << hr << endl;
 return 2;
 }
 */
 }
 else {
 //cout << "UnRegisterTypeLib() rather tricky. Not unregistering at this time™." <GetLibAttr(&pTLibAttr);
 // __try {
 if (SUCCEEDED(hr)) {
 hr = UnRegisterTypeLib(pTLibAttr->guid, pTLibAttr->wMajorVerNum, pTLibAttr->wMinorVerNum, pTLibAttr->lcid, pTLibAttr->syskind);
 if (SUCCEEDED(hr)) {
 cout << "UnRegisterTypeLib() succeeded" << endl;
 }
 else {
 cout << "UnRegisterTypeLib() failed with code" << hex << hr <Release();
 return 7;
 }
 }
 else {
 cout <GetLibAttr() failed" <Release();
 return 6;
 }
 // }
 // __finally {
 pTypeLib->ReleaseTLibAttr(pTLibAttr);
 // }
 }
 /*
 NOTE: These calls to pTypeLib->Release are ESSENTIAL for this app to work properly
It seems that if we don't explicitly call it, COM does not update the registry after the
 program exits.
 */
 pTypeLib->Release();
int error;
 char szEXE[MAX_PATH];
wsprintf(
 szEXE,
 "%s %s",
 s_theApp.c_str(),
 mode ? "/REGSERVER" : "/UNREGSERVER");
cout << "Command line is: " << szEXE << endl;
error = WinExec(szEXE, SW_HIDE);
 if (error < 32) {
 cout << "(Un)RegServer failed." << endl;
 return 3;
 }
 else {
 cout << "(Un)RegServer succeeded." <

Another thing which was confusing earlier on was that VBA can create IUnknown-based objects, so long as the Reference (to the type library) is added first. It resolves the ProgID to the actual interface definition. Non-IDispatch objects must be typed correctly, thus:

Option Explicit
 ' seems like we have to declare this explicitly and include
 ' a Reference to it - this is because it is not IDispatch-based
 Dim obj As TestPythonCOM.TestPython ' <<

IDispatch-based objects can be declared with just Dim obj As Object. Anyhow, the solution is to create an Automation-style (i.e. IDispatch) enabled COM object, which will be controllable by both Excel and Python. COM Events can be hooked by Python (using win32com.client.DispatchWithEvents()), so the operator can hit a button in Excel, firing an event in the application/COM Server, which triggers code in Python, which then updates Excel! Eventually, the manual interventaion will be unneccesary, and the sched event scheduler class can be used instead, allowing Python to snag the data from the app when it sees fit.

For the VBA side of things, as related to the Delphi server, there is an excellent site at http://www.gekko-software.nl/Delphi/, particularly the article entitled Firing events. It shows how to declare a WithEvents class instance which will be used to sink events from the server.