Achievements

StaffBank

"Well done all involved, especially Usha!"

Arwin - Senior Consultant - Baum Hart & Partners, after the Newham StaffBank upgrade.

"All It looks like we have now got Scot Nursing up and running for Leicester branch and the associated interfaces, its been a long old road this one and still a little way to go but thank you all for your efforts."

Chris Lee - Partner - Baum Hart & Partners

Technical Column                                                                                               (top) 
COM - The Basics

Introduction

Hi and welcome! In this article, we will look at a few basics of COM, the 'What's and 'how's of it. For most developers COM remains a mystery unless they are either curious enough to take a look at the internals or are unfortunate enough to be forced to do so. Given all that, I personally don't think everyone needs to know everything that happens under the COM bonnet. Learning COM is quite a tedious job as every COM book or document worth its salt is full of techno-jargon, which explains memory management and virtual table pointers. The aim of this article is to separate the wheat from the chaff and present only the stuff which will help us to understand how to work with components and not how to make components work.

Since COM is so robust, I don't think I will be doing it enough justice by wrapping it all up in a 500(0)-word article. However, since attention spans are not as good as they used to be I will split up the topic in small episodes. So (hopefully) in the next newsletter, this article will be followed up with another article with some advanced features of COM.

In this installment, we will try to identify the various players in the COM game. I have deliberately left out the confusing C++ and IDL code but have the article with a few code samples in VB for illustrative purposes.

So much for the introduction, let's start...

What is COM?

COM, of course, stands for Component Object Model. But, as with all technical terms, the acronym hardly explains what kind of a beast COM is. COM can have many definitions depending on what your context is. COM is essentially a specification (and an open one at that) which lays down to rules for components to interact with their clients using interfaces. When you have rolled up your sleeves to write code, COM is a layer of API. When you are creating an install disk for deployment, COM becomes a runtime requirement present in DLLs and other dependencies. In any case, COM is NOT a programming language.

Why COM?

To understand why a framework like COM came in being, we will have to look at the problem to which COM provided a solution. We all know how programming paradigms evolved over time. From monolithic COBOL applications on Mainframes, the programming community slowly migrated towards modular applications, which involved structural programming languages like C and Pascal. Next came the concept of object-oriented systems, which created a rage. C++ and Java are still hot. (For the rest of the article, C++ will be a reference base for object oriented languages. There's no code involved - so don't go away yet!).

Object-oriented programming languages offered features like encapsulation, polymorphism and inheritance, which the structural programming languages lacked. It was easier to write, modify and maintain code.

A Class became a very useful way for abstracting an element in the real-world problem domain. But as more and more players came in picture, there was a flood of C++ compilers. The code written by one compiler was often not compatible with another. At the same time, other programming languages like Visual Basic and Delphi hit the market and gained immense popularity. As a result, programmers had to rewrite their classes in various languages. The limitation of Object Oriented programming was clear - it was difficult to divorce the implementation from the respective programming languages. The 'Class' lost its charm. That's when Component-based programming came in the picture. Objects were out - components were in.

Some Problems Adressed by Components

The problem with Classes (or rather, objects) is that they are still an integral part of a whole program. A class is an excellent way to simulate a real life element, but every time you make a change in a class, you will need to recompile the code and create a new deployable. For example, say you have written a Payroll class. The class has all the code for calculating the payroll for a given month for a given employee. You have to provide the work hours the employee has put in, the compensation rate, etc. and the class will do the rest for you. So far, so good. Now consider, you have already deployed the application, which has the payroll class, at 10 different sites. One stormy night, you get a call from a client who wants a change in the payroll calculation. What would you do? Make the change and send a new version to that particular client? This means that you will be forced to maintain a new version just for this client. Which further means that any general change you make to the application will have to be replicated in the new version. This, in spite of configuration management tools and versioning schemes, is a pain at best.

COM was designed keeping in mind all these things and for introducing a new paradigm for component based programming.

Run-Time Linking

Use of libraries is a good alternative - but these too require a recompile. The best solution for this problem is to keep business logic in an intermediate layer so that you can make changes to the intermediate layer without changing you application (the ideology behind multi-tiered architecture). We need some equivalent of libraries, which can be updated without recompiling the application (using run time linking). COM provides runtime linking via DLLs. Components are stored in a DLL from which an application can invoke them at runtime.

Encapsulation and Self-sufficiency

The implementation of a component is totally separated from the client application. A component is essentially a black box to which the client application sends some input and receives some output. The client application need not know which language the component was written in, or on what OS or hardware was it written. If a COM component is made available to a client application, then COM will ensure that it will run.

Binary Dependency and sharing

This is a corollary to the last point. Since the components have to hide the implementation details (which are the source language, etc.), they should be best distributed as pre-compiled binary files. Also since the component is a stand-alone binary, more than one client application can share it. Thus, the need to have a separate file for each application using a component is eliminated.

Lifetime Management

Since the components can be shared by more than one application, the component will be loaded in memory. It is necessary to have a mechanism, which will unload the component once it is clear that all applications have finished interacting with the component. COM provides this mechanism through Reference Counting (will be covered later in this article).

Location Transparency

As an added feature, COM even hides the details about where the component is stored. The component can be on the same machine as the client, or can be on another machine on the LAN - to the client application it makes no difference. The client need not know where the component is. The client application asks the COM layer which component it wants and COM will fetch it!

Upgrading via Versioning

COM is very strict when it comes to making changes to the component. COM stipulates that upgrading a component should not break existing client applications. To handle this, COM provides a standard way of upgrading by providing different versions to the component.

An illustration

The following diagrams illustrate how modules and the application interact.

A Monolithic application where the various modules merge into the application and their boundaries are not visible

An object oriented application wherein the modules are encapsulated within Classes but the objects corresponding to the modules are still inseparable from the application

Component based design of the application wherein the modules are now external to the application and reside in components which can be easily updated without changing the application itself. As you can see, COM forms a layer between the client and the component.

Components are similar to Integrated Circuit Chips (ICs). You must have seen electronic devices, which have dozens of ICs, laid out on a PCB. Each IC is a component - a replaceable part of the whole, which has a pre-defined purpose. So if in case an IC which say, controls the LCD display of your calculator, breaks down - you don't have to replace the calculator, you can just replace that IC and get back to your calculations. COM is a set of rules which standardizes the interface between the component and the client application. Taking the analogy a bit further, it is the specification, which dictates how many PINs (or legs) the IC should have, what should be the distance between 2 PINs,etc. If an IC doesn't follow these standards, then obviously it cannot fit in a regular IC slot.

The (Almighty) Interface   

Earlier in this article, I mentioned that COM is a specification. This specification tells how the component should be written, how should it expose its functionality and how should clients use the functionality exposed by the component. Between a client application and a component there is a contract. This contract promises that as long as the component is available, it will support the given set of functionalities. The functionalities are exposed using what is called an Interface. An interface is an abstraction of the component. Technically speaking, an interface is basically a set of function declarations with a list of parameters in a particular order. COM stipulates that once an interface is designed and released to clients, its interface should not change. If a component changes its interface in any way, then the contract is broken and the client application will break.

Let's take ADODB components as an example. ADODB is a collection of components - Connection, Recordset, Record and so on. Each of these components has an interface (i.e., promises something). A Recordset interface says - "(One of the things I can do is...) I can open a recordset for you (using the 'Open' function) if you can provide me the SQL Query, the connection to use, the Cursor type, the Lock type and other Options; in the order given.". Now we know how can we make the Recordset object open a recordset for us, thanks to the interface.

So we write code assuming that the Recordset component will keep up this promise. Imagine what could happen if someone at Microsoft accidentally changes the order of the parameters! Or worse, removes the 'Open' function altogether! Our application will crash, hard.

Sometimes, components are aggregations of other components. For example, say you want to design a component for a Car. Now a car is a land vehicle, which has 4 wheels and an engine. So you go ahead and design a component for a vehicle - which might look like this: -

Simple Vehicle Component with an interface

This diagram shows that the Vehicle component has 2 functions (in Italics) - Start and Stop. The component exposes an interface (shown with a 'lollypop') called IVehicle (by convention, prefix 'I' will be used for Interfaces). These diagrams are referred to as 'Lollypop diagrams'. (A disclaimer - the diagrams here are slight variants of the conventional Lollypop diagrams - just to make things look simpler)

The Car component can utilize the Vehicle component as shown: -

Derived Car

Since a Car is a vehicle, we have derived the Car component from the Vehicle component. In addition the car component can have additional properties and methods which are not present in the bare bones Vehicle base class. In COM lingo, we say that the Car component supports both the IVehicle Interface and the ICar interface. This means that you get the functionality of a Vehicle component as well as the functionality of a Car component - depending on which one you want to use.

Similarly, you can design an Airplane component, which inherits from the Vehicle component.

Derived Airplane

So the Aiplane component now supports IAirplane and IVehicle interfaces.

What do the 'Lollypops' signify?

The box is your component, the internals of which are totally shielded from the clients. However, the component uses the interfaces (lollypops) to tell the client community what are they capable of. Clients can only see the interface (hence the interface sticks out of the black box of the component). Looking at the interface, the client application determines what can the component do. The interface doesn't tell the client how the component does it. The client will always interact with the component via the interface. The interface will route all the client's requests to the actual code implemented in the component.

COM Entities and Identifiers

To communicate with an interface, you should know what is it called. Unfortunately, with just 26 alphabets, it is difficult to guarantee that you will come up with a name, which has not been used by anyone to name his or her component. So there is this remote possibility that your component and its namesake will be installed on the same machine - which means that the behaviour of the client application using either component will be indeterminate. To handle this situation COM uses GUIDs to name components. GUID (pronounced as "goo-id") stands for Globally Unique Identifier. It is a 128-bit no. which is formed using information like the hardware address and the configuration details of the computer and the date time stamp. Though it will be technically inaccurate to say that a GUID is truly unique, the chances of a clash are one in 2128!

The code which actually implements functionality promised by an Interface is wrapped in a CoClass (for Component Class). The COM objects are instances of these CoClasses. While Interfaces specify what the component can do, the corresponding CoClasses specify how the component can do it. Logically related CoClasses are often grouped together to form a Library - ADODB is such a library which has CoClasses like Recordset, Connection, etc. For client application developers, the library serves as a namespace for the component group. Each of the above mentioned entities are identified by a GUID.

However, since the 128-bit GUID is not very friendly looking, many platforms do not implement a GUID as a datatype. To circumvent this limitation, we assign a readable name to our CoClass; this is generally called as a Program ID or ProgID. When you use the CreateObject function to create components, you provide this ProgID (we will see an example in the next section). The COM runtime will resolve the ProgID to the corresponding CoClass ID (called CLSID) and invoke the CoClass.

Binding and Automation

Let's take at a look at the concept of Binding and Automation. When we use resources, which are external to our program, we need to 'bind' the external resource. The concept of using resources belonging to another application is called as Automation.

Our VB application is called the Automation Controller or Automation Controller and the external resource (which will typically be in an OCX file or a DLL) is called an Automation Server. The Controller application requests the server for an object, which does a particular task. The server responds by sending it an Interface Pointer. The Interface Pointer acts as a placeholder for the COM component. When a request for an Interface Pointer is made, the Controller application's runtime needs to verify whether the DLL or OCX supports this interface or not. This verification process is called Binding.

Binding can be of two types: Late Binding and Early Binding. In case of Late binding, the compiler doesn't have enough information about the type of the object, which is being declared, and/or used, the verification is deferred till the point of execution. For example, check out this code in VB, which will create an ADO Recordset object.

But the application crashes during execution when the runtime realizes that the Recordset interface doesn't have a 'MakeCoffee' function! You will get to see this unsightly error message box.

Interface not Supported

Some development environments like ASP, which are 'type-less' (have only generic data types), support only Late Binding and this error is quite common especially while working with 3rd party components.

In case of Early Binding, the application will know the capabilities of the components (and the interfaces therefore) at compile time. That is, the component vendors provide a look-up for the interfaces and functions supported by the component. To achieve this, COM components generate Type Libraries. A type library lists out all the interfaces, Libraries and CoClasses present in a given component file. In VB, the type libraries are added as Project References. For example, the type library for the ADODB resides in the msado15.dll, which we add in our project references.

Type Libraries as References

Adding this 'adds' the interface wrappers to the project. Now that we have the datatypes of the ADODB objects, we can declare our variables to be of a specific type instead of declaring it as a generic Object type.

We can see that now VB development environment recognizes the Recordset interface and the Intellisense kicks in whenever you try to access the member functions and variables of the Recordset. So if you try to call the hypothetical MakeCoffee function from a Recordset object - the compiler will not allow you.

IUnknown - the basic interface

To describe how a client 'talks' to a component, we will have to take a peek behind the scenes. COM specifies that all components should derive from a standard interface called IUnknown. IUnknown is simple interface, which has just 3 functions :- QueryInterface, AddRef and Release. Since ALL components support this interface, these 3 functions will be available for ALL components. So the vehicle component, which we designed, can be depicted like this :-

A COM Vehicle

The other lollypop diagrams will also change accordingly to support the IUnknown interface.

QueryInterface (QI)

QI is a mechanism through which the Client asks the component whether it (the component) supports a particular interface. Every COM component should be able to tell its clients, which are the interfaces, it supports. If the object is present, then QI returns an interface to access the interface object. This way, the client doesn't have to know about all the interfaces exposed by a component. So a client has information about the component on a need to know basis.

When you declare a variable of an interface wrapper data type in VB, VB will call QI behind the scenes and will fetch an object of the interface for you. Similarly, CreateObject() function will also make a call to QI and get you the interface object.

So QI is basically client asking the component - "Do you support this interface?"; to which the component can say - "Yes! I do - here's an Interface Pointer, which is wrapped around the interface. Use it and destroy it after use." or "Sorry! I don't. I will throw an exception now!"

Lifetime Management through Reference Counting - AddRef() and Release()

One of the problems, which COM addresses, is Lifetime Management of the component. When a client invokes a component (via an Interface Pointer), the component is loaded into the memory. Since COM components can be shared by various clients at the same time, we need to make sure that none of the clients accidentally unloads the component from memory - causing the other clients to crash. To avoid this problem, COM takes the responsibility of loading and unloading a component in memory. How does COM know when to load and unload the component? It is done using a mechanism called Reference Counting. Though there are a lot of other things, which are involved, Reference Counting is fairly simple to understand.

Each component maintains a reference count. When the first client requests for a Interface Pointer of a component, COM will load the component in memory and will set this reference count to one. As more requests for interfaces are made and more Interface Pointers rolled out, reference count is incremented (using the AddRef() function). In Raw C++, the client should include the code for calling AddRef() explicitly. However, VB handles most of the AddRef() grunt work implicitly. FYI, a call to QueryInterface will automatically make a call to AddRef() if the interface asked for is supported.

Similarly, as soon as a client is done with an Interface Pointer, the reference count will be decremented by one (using the Release() function). As with AddRef(), the calls to Release() are handled implicitly by VB. Release() is called whenever an interface component goes out of scope or is explicitly set to Nothing.

The COM rule for life time management of a component is simple - if the reference count of the component falls to zero - unload the component from memory, otherwise don't. Hence no client is allowed to load or unload a component in memory, a client can merely instruct COM when it has started or finished using an Interface Pointer.

COM Client/Server Model

Now that we have seen what roles do the various COM elements play, we will look at how all of them interact with each other. Let us do a brief recap...

An Interface is an abstraction of a COM object - it doesn't contain the implementation. The implementation of the object is present in CoClasses, which are grouped together in a Component File(a COM DLL or OCX). The clients talk to the objects using their Interface Pointers - these pointers do the actual mapping (via COM) to the CoClass, which contains the object implementation. Type libraries help the client application developers by exposing the interfaces at compilation time. All interfaces are derived from a basic interface call IUnknown, which has a set of features, expected of every COM interface.

So now let's see how it all works. A client just knows about the interface, through which it invokes the component. The client application first establishes connection with the COM runtime, which is responsible for invoking the component (luckily, most programming tools like VB do this automatically). COM makes sure that it maps the calls made by the client (via the Interface Pointer) to the corresponding CoClass, which resides in a DLL (or OCX).

Simplified COM Client/Server Model

The above diagram is a crude but simple representation of how the mapping works. The component file on the extreme right is the actual file (DLL or OCX) which houses the binary component(s). You can also see that the applications only need know about the interfaces of the CoClasses they want - they don't have to know about all the CoClasses (and interfaces thereof) present in the component file. Hence Client App 2 has no Interface Pointers for CoClasses 3 & 4.

Some keywords we missed out

It is easy to get confused with all the acronyms and fancy brand names that surround the COM world. Let's take a look at some of the commonly heard terms.

OLE - Object Linking and Embedding. This is often confused with COM. OLE is an application of COM, is built over COM and is not COM itself. Years ago, when Microsoft Office were competing with other established word processing and spread sheet products, someone decided that it would be really nice if you could open a spread sheet from within a word processor and even a picture editor, or a bar chart. This was the beginning of OLE. OLE 1.0 was an (arguably) ad hoc effort to share information between 2 applications. The underlying technology, which enabled applications to share information, was called DDE (Dynamic Data Exchange).

Soon some smart guys at Microsoft realized that this was an opportunity to implement a robust framework, which has applications far beyond OLE. This gave rise to COM. OLE 2.0 was implemented with COM as the backbone. Soon came VBX controls (see ActiveX below) and the scope for COM and component based architecture looked bright. Competition came in form of OMG's CORBA (Object Management Group's Common Object Request Broker Architecture) and IBM's SOM (System Object Model), but thanks to the large base of Windows users and the powerful support from VB and VC++, COM emerged as a leader.

ActiveX - When VB was still in the nascent stages (this was in age of 16-bit Windows), a new concept of extending applications by using extension controls called VBXs (Visual Basic extension) emerged and became an instant hit. C++ was suddenly sidelined when this new method of quick and simple programming bared its fangs. The market for VBXs was so good that the Microsoft guys decided to redesign the whole thing and add 32-bit support. To console the whining C++ community, C++ support was included too. The result was ActiveX Controls. The term 'ActiveX' was kind of catchy so Microsoft decided to add it to every blessed product, which rolled out of Redmond. So we had ActiveX Data Objects, ActiveX Scripts, ActiveX Documents, etc.. Just as windows developers were chanting the ActiveX mantra, Microsoft decided to confuse the hell out of them by releasing OLE DB (which should have been called ActiveX DB to keep up with the trend). Anyway, ActiveX and OLE are both blanket names for COM based technology. COM is the backbone. COM rules!

DLLs - Dynamically Linked Libraries have been around for quite some time. Years ago, C came up with the concept of using Libraries, which were compiled binaries of programs containing functions which a C program can use. The main advantage was that these libraries could be reused. The main disadvantage was that if a C program used a library then the library had to be 'statically linked', which resulted in bloated Executables. The early DLLs were the same as the C libraries but allowed the linking part to be deferred till the point of execution. To the developers this meant that they could keep their EXE footprint small and supply the needed libraries as DLLs which can be distributed independently. With time, features like sharing and thread-safety were thrown in and all that ended in the DLLs we have now.

COM used the run-time linking feature offered by the DLLs and the result was Registrable COM DLLs. Logically, COM DLLs are designed to export objects, in contrast to normal DLLs which export just functions. Also COM DLLs should be 'registered'. Because of the registration process, COM DLLs can be placed on any drive, any folder; however, the normal DLLs must be stored either in a specified directory which the client application will probe.

OCXs - OLE Control eXtensions are special types of COM DLLs which house ActiveX Controls. ActiveX Controls are nothing but COM components nicely wrapped in an easy to use package. Drag and drop, add 2 lines of code and presto! - you are done! By convention, OCXs contain components which extend the GUI - but that's not a rule (remember the invisible Crystal Report Control?).

ATL - Active Template Library (Microsoft dropped the 'X' just for fun) is a robust (and equally complex) framework in C++ which provides powerful support for COM development. It is a common choice for developing COM based services and lightweight controls.

DCOM - Distributed COM was devised as an extension to COM to support component architecture over a LAN. It was Microsoft's reaction after it realized that it has almost missed the boat on distributed component architecture and CORBA was becoming the obvious choice of developers. We will cover this in a subsequent article in this long(?) series.

Stay tuned for....

After these basics, it's time we look at that the advanced COM stuff. In the next installment we will cover the following :-

The Registry the Service Control Manager (SCM) and their roles

Location Transparency - how does COM locate the Component file

COM versioning support (and necessity)

Marshalling and Unmarshalling techniques - how COM transfers data between the clients and the components.

And (hopefully) more...

Feedback

There are things which I couldn't explain in detail and few more things which I deliberately left out (not to mention the mistakes I might have made). Please direct your queries, suggestions, corrections and brickbats to me.

References

Inside COM by Dale Rogerson (Microsoft Press)

Essential COM by Don Box (Addison-Wesley)

The Component Object Model Specification, Draft Version 0.9 (Microsoft Corporation)

Design Patterns by Gamma, Helm, et al (Addison Wesley)

Developer's Workshop to COM and ATL 3.0 by Andrew Troelsen (WordWare Publishing/Bpb Publications)

MSDN (Technical Articles -> Component Object Model)

by K Guruprasad

Holidays for the Month                                                                                          (top)

26th November 2003 Wednesday Ramzan

Please note that, in order to compensate for the loss of working days when we go on a vacation (31st December

2003 to 3rd January 2004), we will be working on the following holidays:

26th November 2003 (the day of Ramzan) and 25th December 2003 (Christmas day).