There are a number of tools I use to create, manipulate and burn CD / DVD images on Windows, and a lot of people I talk to aren’t aware of at least some of these utilities. Nero and other commercial packages are nice, but it’s amazing how much functionality is available for free.

To mount CD or DVD images and make them appear as virtual CD drives, there’s nothing I like better than Microsoft’s virtual CD control panel. There’s lots of software out there to do this (daemon tools is also good), but the Microsoft tool is simple, small, and works great.

To read and extract CD or DVD images, the tool I like best is called cdmage. Unfortunately the development on this tools seems to have stopped, but it’s still unbeatable in the freeware category. It also has support for lots of different types of image formats. IsoBuster is another good tool for image manipulation - it has a freeware version that includes all the basics and a payware version with advanced features.

For creating CD or DVD images from files on a filesystem, I use mkisofs. It’s easy to use mkisofs to create a hybrid CD or DVD that contains ISO9660, Joliet, and RockRidge metadata. Lots of help and tips are available for it on the Internet. There have been lots of compiles of this tool for Windows - it shouldn’t be too hard to find a copy.

On occasion, I’ve used a Microsoft premastering tool called cdimage.exe. However, I’m not sure if Microsoft condones use of this tool - you can find it all over the Internet, but it’s not officially available from Microsoft anywhere I’ve seen. It basically does everything that mkisofs does.

To capture bootsectors from CDs, I use a tool called BBIE, or Bart’s Boot Image Extractor. I don’t know of any other tool that is command-line based and can extract images from both CDs in the drive and ISO images on the filesystem. When I’m making a custom Windows installation CD, I’ll use BBIE to capture the bootsector from a standard Windows installation CD.

CD and DVD media can differ greatly in quality, and looking at the brand the media is sold as doesn’t tell you anything. The only reliable way to tell where media came from is to look at the media ID that’s put onto the discs when they’re made at the factory. There are lots of programs that can read the media ID - I like DVD Identifier. I also use CDRID, which you’ll have to search for to find a copy.

For burning CD or DVD images, I use ImgBurn almost exclusively. ImgBurn works great for burning CDs and DVDs, and I haven’t run into any problems with it yet. It’s nice and small, fast, and has a good interface. ImgBurn only burns ISO images, and it can’t create the images from files on the filesystem, so you’ll have to use another tool to do that. I sometimes use burnatonce to do image burning, as it’s more full-featured than ImgBurn. However, burnatonce doesn’t support DVD burning (out of the box) and it seems like development has stopped on it.

Once the image is burned to media, it’s a good idea to verify that the burn was solid and the media is OK. To do this I use CDCheck, which is free for personal use. CDCheck scans the burned media to make sure that all the data can be read off of it. I’ve found that especially with DVDs this is important to do since even high quality DVD media usually contains a few lemons in a package.

The nice thing is that most of the tools I mentioned can be run from a single executable, or at least don’t require installation. They’re easy to copy around and don’t clutter up a system. I don’ t know of any CD or DVD image manipulation tasks that you can’t do with free software on Windows.

The Principle of Audience states that an API must be designed with its audience in mind. The audience of an API is the people that will be writing code against it.

All too commonly, APIs are designed primarily as a way to expose some underlying functionality at some level of abstraction. This style of API design results in APIs that are either too general or too large for easy use - they’re functionality-oriented instead of audience-oriented.

Let’s say that you’re writing an API around a large, complex piece of software. You could ask one of two questions in order to drive the API design:
1) What functional areas of this software do I want to expose?
2) What kinds of users wish to interact with this software, and what are they trying to get done with it?

Driving an API with the first question can result in an API that is complete but not easy to use and learn. Driving with the second question results in an API that is scenario-focused and was written with purpose and intention. It all boils down to one fact: people are writing to an API because they’re trying to get some work done and then move on.

How Audience Drives API Design

Once you’ve identified your audience for your API, you can start to make audience-focused decisions to drive the design of the API.

One issue to consider is whether the API should be high-level or low-level. A low-level API may do little more than wrap existing code and provide a consistent interface to it. This low-level API usually has an interface style that’s identical to the code being wrapped (for example, procedural code wrapped by a procedural API), and it often maps one-to-one with concepts in the underlying code. The main purpose of such a low-level API is to provide a level of indirection so that future changes can be made to the underlying code without breaking existing clients of the API.

A higher-level API may provide a totally different way of working with the existing functionality. A high-level API may have a completely different interface style than the wrapped code (like an object-oriented interface to a procedural library). It may not map one-to-one with concepts in the wrapped code: the API may invent concepts of its own and expose them to clients, and it may hide or suppress concepts in the underlying code that clients of the API shouldn’t be concerned with.

The high-level or low-level decision comes down to audience. If your audience is already familiar with the software the API is being written for, a low-level API probably makes sense. If the audience is unfamiliar with the existing software, a high-level API provides a needed conceptual level of indirection.

Another audience issue to consider is whether an API is going to be internal (to an organization) or external. An external API probably needs to be versioned, and definitely needs different documentation than an internal API. People using an internal API probably have access to the authors of the API, and often can make changes to the API to fix bugs or add additional functionality.

Audience Considerations

When you write an audience-oriented API, there are some considerations to keep in mind. Be sure to provide starting points. Think about the API from the perspective of a user who is coming to it for the first time. Where should they start? There should be jumping-off points both in terms of the documentation and the API itself. The documentation should at least have an index, a 5-minute tutorial section, and a FAQ. The FAQ should consist of real questions that are frequently asked - it’s going to be hard to come up with this list until the API has been used by people other than its authors.

As an author of an API, you will find it difficult to predict where these jumping-off points should be, or even if there are enough of them. The best way is to team up with another developer you know who is completely new to the API, and have them attempt to write a program against it. Observe where they start, and at what points they struggle with the API. Getting another person to sit down with your API and critique it is going to be much more valuable than trying to predict where the pain points are by guessing.

A great way to provide jumping-off points is a suite of tests that ship with your API and exercise all of its functionality. These tests could be very simple (they probably aren’t unit tests), just calling the API and verifying the result, perhaps mocking up the back-end to some degree. Developers using your API can consult the tests to see how various things are done.

Ensure that your API ships with as many development aids as possible. Understand what IDEs and environments your audience will be using to write against your API and deploy code in. Make sure that your API ships with whatever help and documentation is appropriate for these environments. For example, if your audience is going to be using an IDE that offers code completion or contextual code hints, make sure that your API ships with the necessary hooks to integrate into this IDE and provide that kind of support. The whole idea is to make it as easy as possible for your audience to code against your API.

Multiple APIs

So what happens when you’ve thought about your audience, and you realize that there are multiple audiences each with different needs? Instead of trying to build a single, one-size-fits-all API that covers all the needs, consider writing multiple, distinct APIs for your product.

Let’s say that you’re writing an API for a some software that manages a large amount of data and performs interesting computations on it. One audience of the API might be people who want to write functionally equivalent client applications (perhaps on unsupported platforms), so you would want to write an API that exposes most of your core functionality and concepts. However, another audience may simply want to get data in or out of your data store. For this audience, an import / export oriented API would make a lot more sense.

A lot of business integration is about just moving data out of one system and into another. Users of an API for this reason don’t care about all the features and concepts of your application - they simply want a clean API to either get their data out of your system, or put their data into your system. Given that business data far outlasts the applications that process it, every application that manages customer data should have an API for importing and exporting that data in as neutral a form as possible.

Web Services

The Principle of Audience is one reason I believe a currently popular approach to web services won’t work. I call this concept the “build it and they will come” approach to web services. The idea being that a company will invest lots of time and money into writing web services that expose their data or services. This is done without a clear customer (audience) in mind, with the hope of it being useful for future business integration. I’d be willing to bet that most of the time these web services are only used internally in a company (probably with both endpoints on the same platform), and are either scrapped or completely revised if integration with external entities is ever needed.

The problem with “build it and they will come” web services is that no one knows exactly who “they” are. These web service projects are often written either because web services are popular right now, or because having web service support becomes a marketing bullet point. Without a clear audience in mind, these web services become at best toys to try out new technology, and at worst a maintenance time sink with little business value.

A final note - one counter-point to an audience-oriented API might be phrased: Isn’t the whole point of an API that we can write a library and then people will use it in all sorts of interesting ways we’ve never thought of? My response is that an API will only be useful for an unforeseen purpose by coincidence. You can certainly take steps to make the API flexible and extendable, but in the end the best APIs are those that are designed ahead of time for their audience.

The Common Language Runtime (CLR - virtual machine for .NET) and the Java Virtual Machine (JVM) share similar architectures, but there are lots of differences, especially when it comes to memory management and allocation. Both virtual machines have automatic memory management in the form of garbage collection - the programmer is not normally responsible for allocating and releasing memory. Both virtual machines implement multiple generations of garbage collection for performance reasons, and both manage a “heap” of memory internally for application code use. However, there are a few subtle differences that don’t seem to be discussed much so I’m going to highlight them here. A caveat: what I’m going to discuss would probably be considered implementation details of Microsoft’s CLR and Sun’s JVM on the Windows platform. The behavior may be different in future releases, on other platforms, or on VM implementations from other vendors.

Difference #1: Fixed upper limit on the heap size

Both the CLR and the JVM manage an internal heap of memory that is used for allocations. Both VMs will grow the heap by allocating more memory from the operating system when needed. However, the JVM places a fixed upper limit on the heap size. This limit is specified by using the -Xmx switch when starting the runtime. If the JVM tries to satisfy an allocation that would result in the heap growing beyond that limit, and no garbage can be collected, then an OutOfMemoryError is thrown and the allocation fails. As far as I can tell, the CLR has no such artificial upper limit on the heap size. The CLR heap maximum size will be dependent on how much memory can be allocated from the operating system.

Difference #2: Releasing allocated memory

Something that surprises a lot of programmers about the JVM is that it does not release allocated memory back to the operating system, even if it could. For a hypothetical example, imagine that a JVM process starts and allocates 25 MB of memory from the operating system initially. Application code then attempts allocations that require an additional 50MB of heap. The JVM will allocate an additional 50MB from the operating system. Once the application code has stopped using that memory, it is garbage collected, and the JVM heap size will decrease. However, the JVM will not free the allocated operating system memory. For the rest of the lifetime of the process, that memory will remain allocated, even if the heap is never grown again.

In practice, this doesn’t tend to matter too much. Normally this “unused” memory will be paged out by the operating system so it doesn’t tend to impact other processes.

The CLR, on the other hand, will release allocated memory back to the operating system if it is no longer needed. In the example above, the CLR would have released the memory once the heap had decreased.

Sometimes graphs can illustrate concepts much better, so to illustrate this point further I made some graphs. I wrote roughly equivalent C# and Java programs. The program attempts a series of object allocations, each one larger than the last. After each allocation in the series, the program drops all references to the allocated objects and forces a GC to happen. The graphs show the private bytes and working set (process memory usage from the OS point of view) and heap size (heap memory usage from the JVM or CLR point of view).

The Java program:

And the C# program:

Remember that these graphs are showing equivalent programs. In each graph you can read “private bytes” as the amount of memory the process has allocated from the operating system. The “working set” shows the portion of the memory pages in physical RAM. Note that the JVM will never release memory back to the OS even if its internal heap decreases. For the CLR real memory usage follows the heap size. These graphs also show an example of the fixed heap size difference: in the JVM graph, the program encounters an OutOfMemoryError at the end of the graph (this is where the heap size levels off).

For each graph, I used performance counters to capture the private bytes and working set data. For the JVM graph, I used JSE 5 JMX memory monitoring (MemoryMXBean) to get the heap size. For the CLR graph, I tried using the CLR performance counters to capture the heap size, but it didn’t work for me. I ended up using GC.GetTotalMemory() which worked fine.

I’m going to start a series of entries about API design. Being a consumer of many APIs and producer of a few, I’ve accumulated a number of opinions about how to design APIs. But before I dig into the nuts and bolts of this topic, a little background is in order.

What is an API?

API is an acronym that stands for Application Programming Interface. You can drop the application and programming part and think of an API as an interface. An API is a public interface to a system (where “system” could be at many different levels of granularity). Often the term API is used to describe the interface of a third party library or a programming environment, but the most common APIs you will encounter are the APIs you and your coworkers write on a daily basis. Even if you’re writing code that’s only going to be used inside your own codebase or organization, you’re still writing an API for yourself and your colleagues.

Types of APIs

If an API cannot change in sync with its consumers, then it’s a fundamentally different kind of API than one that can. In other words, if you control both the API code and the client code, you have a different set of problems than if you only control the API. In both situations, the principles of API design that I’ll discuss apply, but in the situation where you don’t control the clients, there are many additional challenges (like versioning).

If you are publishing an API which will be used by clients you don’t control, then the costs of bad API design are much greater. Once you’ve published an API, you must choose between backwards compatibility and breaking clients, which isn’t an easy choice. But even when you’re writing both the API and the clients, it is still usually beneficial to put some thought into the APIs you’re writing. An API designed with intention is better than an implicit or ad-hoc API.

However, proper API design is hard and costly. It’s not always the right thing to do: you wouldn’t normally design an API for prototype code. It’s often better to extract APIs from v1 code rather than to try to begin by designing an API.