Dear Spring community,

In the past months I've been working with various clients on projects using Google Web Toolkit [GWT]. I like GWT primarily because of the Java to javascript compiler. This is the key to the door letting mere mortal Java developers create RIA's without having to learn a new language.

I've allways been a fan of test driven development, and to my disappointment at first sight it looked like TDD and GWT were not going to play together.

Testing GWT code is a bit problematic. The core of the problem is that GWT code is compiled to javascript before it is run. In many cases a GWT.create() statement…

When approaching OSGi, one of the first concepts that have to be learned is the notion of a bundle. In this entry, I'd like to take a closer look of what a bundle actually is and how a vanilla jar can be transformed into an OSGi bundle.  So, without further ado,

What is a bundle?

The OSGi spec describes the bundle as a "unit of modularization" that "is comprised of Java classes and other resources which together can provide functions to end users.". So far so good, but what exactly is a bundle? Quoting the spec again:

a bundle is a JAR file that:

• Contains [...] resources
• Contains a manifest file describing the contents of the JAR file and providing information about the bundle
• Can contain optional documentation in the OSGI-OPT directory of the JAR file or one of its sub-directories

In short, a bundle = jar + OSGI information (specified in the JAR manifest file - META-INF/MANIFEST.MF), no extra files or predefined folder layout are required. This means that all it takes to create a bundle from a jar, is to add some entries to the JAR manifest.

OSGi metadata

The OSGi metadata is represented by manifest entries that dictate to the OSGi framework what the bundle provides or/and requires. The specification indicates around 20 manifest headers but we will just take a look at the ones you are most likely to use.

Export-Package

As the name implies, this header indicates what packages (available in the bundle) are exported so they can be imported by other bundles. Only the packages specified by the header will be exported, the rest will be private and will not be seen outside the containing bundle.

Import-Package

Similar to Export-Package, this header indicates the packages that are imported by a bundle. As well, only the packages specified by this header will be imported. By default, imported packages are mandatory - the importing bundle will fail to start, if the imported package is not available.

Bundle-SymbolicName

Bundle-Name

Bundle-Activator

Bundle-Classpath

Bundle-ManifestVersion

Below is an example, taken from Spring 2.5.x core bundle manifest that uses some of the headers mentioned above:

 
Bundle-Name: spring-core 
Bundle-SymbolicName: org.springframework.bundle.spring.core 
Bundle-ManifestVersion: 2 
Export-Package:org.springframework.core.task;uses:="org.springframework.core,org.springframework.util";version=2.5.1 org.springframework.core.type;uses:=org.springframework.core.annotation;version=2.5.1[...] 
Import-Package:org.apache.commons.logging,edu.emory.mathcs.backport.java.util.concurrent;resolution:=optional[...] 

Most of the time used on OSGi metadata is likely to be spent on Export/Import package entries as they describe the relationship between bundles (that is, between your modules). When it comes to packages, nothing is implicit - only packages that are mentioned are imported/exported, the rest aren't. This applies also to sub-packages: exporting org.mypackage will export just this package and nothing else (like org.mypackage.util). Same goes for importing - even if a package is available inside the OSGi space, it will not be seen by a certain bundle unless it is explicitly imported by it.

To summarize, if a bundle A exports package org.mypackage and bundle B wants to consume it, then the META-INF/MANIFEST.MF for bundle A should specify the package inside its Export-Package header, while bundle B should include it in its Import-Package entry.

Package consideration

While exporting is fairly straight forward, importing is slightly more complex. It is common for applications to degrade nicely by searching the environment for certain libraries and using only what is available, or for libraries to include code that is not used by the user. Such examples include logging (using JDK 1.4 or Log4j), regular expressions (Jakarta ORO or JDK 1.4+) or concurrent utilities (java.util in JDK 5 or backport-util-concurrent library for JDK 1.4).

In OSGi terms, relying on a package based on its availability translates to an optional Package-Import. You have already seen such a package in the previous example:

Since in OSGi, multiple versions of the same class can exist, it is best practice to specify the version of the class package both when exporting and importing a package. This is done through the version attribute which is added after each package declaration. The version format supported by OSGi is <major>.<minor>.<micro>.<qualifier> where major, minor and micro are numbers and qualifier is alphanumeric.

The meaning of the version is completely up to the bundle provider however, it is recommend to use a popular numbering scheme such as the one from Apache APR project where:

• <major> - indicates a significant update which doesn't guarantee any compatibility
• <minor> - indicates an update which preserves compatibility with older minor versions
• <micro> - represents an insignificant update from the user point of view which is perfectly compatible both forwards and backwards
• <qualifier> - is a user defined string - it is not widely used and can provide an additional label to the version number, such as the build number or target platform without a standardized meaning

The default version (if the attribute is missing) is "0.0.0".

While the exported packages have to indicate a specific version, the importers can indicate a range using the mathematical interval notation - for example

[1.0.4, 2.0) will match version 1.0.42 and upwards up to 2.0 (excluding). Note that an specifying only a version instead of an interval will match all packages that are at greater or equal then the specified version, that is :

Import-Package: com.mypackage;version="1.2.3"

is equivalent to

Import-Package: com.mypackage;version="[1.2.3, ∞)"

As a last tip, make sure to always use quotes when specifying an version, whether it is a range or not.

Working with OSGi metadata

Now that we have some information on what bundles are, let's see what tools we can use for osgi-fying an existing jar:

by hand

This do-it-yourself approach is discouraged as typos and extra spaces can easily sneak in and render the manifest useless. Even when working with a smart editor, the manifest format itself can cause some problems since it has a limit of 72 spaces per line which, if broken, can cause some cryptic problems. Manually creating or updating the jar is not a good idea since the jar format requires the META-INF/MANIFEST.MF entry to be the first one in the archive - if it's not, even though it's present in the jar, the manifest file will not be read. The manual approach is really recommended for cases where there are no other alternatives.

However, if one really wants/needs to work directly with the manifest, then a editor that can deal with UNIX/DOS spaces should be used along with a proper jar creation utility (such as the jar tool that comes with the JDK) to cope with all the MANIFEST requirements.

Bnd

Bnd stands for BuNDle tool and is a nice utility created by Peter Kriens (OSGi Technical Officer) that "helps [...] create and diagnose OSGi R4 bundles". Bnd parses the java classes to understand the available and imported packages so it can create the equivalent OSGi entries. Bnd offers a series of directives and options which can customize the resulting artifact. The nice thing about bnd.jar itself is that is can be run from command line, by Ant through dedicated tasks or integrated into Eclipse as a plug-in.

Bnd can create jars from the classes available on the classpath or inside Eclipse projects or can osgi-fy existing jars by adding the needed OSGi artifacts. Additionally, it can print and verify the OSGi information about the given jar making it quite a powerful, yet simple to use tool.

First time users, can use Bnd to see what OSGi manifest will be added to a vanilla jar. Let's pick a vanilla jar such as c3p0 (which is an excellent connection pool library) and issue a print command:

```code java -jar bnd.jar print c3p0-0.9.1.2.jar ```

The output is fairly big and contains of several sections:

1. Generic manifest information:

   [MANIFEST c3p0-0.9.1.2.jar]
   Ant…

Spring Batch 1.0.0.m4 is available today via the s3 Milestone repository (browse at http://s3browse.com/explore/maven.springframework.org/milestone/org/springframework/batch).  See the Spring Batch downloads page for more information (http://static.springframework.org/spring-batch).

We have re-jigged the release schedule to squeeze in an additional milestone, to which Ben and Lucas are dedicated, so 1.0.0.m5 will be out in the next 10-14 days. Then we just have time for an rc1, with some contingency for rc2 if necessary, before the expected March 20 final release. 

We've been working really hard on Spring Batch getting ready for the Spring Portfolio 2.5 release train, and I thought it would be a good time to update everyone on what is happening. In this article I'm going to expand a bit on the domain modelling, and our decision to raise the profile of some of the core domain objects, and increase their responsibilities. I will also give a few tastes of what is coming in the next couple of releases leading up to 1.0, so people have a chance to comment if they want to.

By way of an apology: there have been some quite significant changes in the internals…

My last blog showed how Spring is soaring past EJB. Research by BZ Media and others shows that Apache Tomcat is the leading open source application server with a 64% market penetration. The dominance of Spring and Tomcat is well-known. What people may not know as well is that thousands of organizations are running Spring on Tomcat as their middleware infrastructure. Those organizations want one company to turn to for the products and services they need to be successful.

Today we announced our acquisition of Covalent Technologies. Not only does Covalent bring Apache leadership, but our combined company now has significant leadership on Apache Tomcat and HTTP. Two weeks ago, Sun paid $1bn for the "M" in LAMP. Now that Covalent's outstanding Apache expertise and services are part of SpringSource, we are strong leaders in the “A”. We have always been about technology leadership, so we're very excited about what we can do together with Covalent. Over the last few years, Covalent has earned a great reputation in the marketplace with its support for Apache projects, including Tomcat and Apache HTTP. Its hundreds of support customers include more than half of the Fortune 500, and household names like Pfizer, Johnson & Johnson, British Telecom (BT), NASA, Intel, Royal Bank of Scotland and Bear Stearns. Our announcement…

One of the central themes behind Spring 2.5 is comprehensive annotation-based configuration. We've been talking and blogging a lot about @Autowired, about Spring MVC's @RequestMapping and also about the new support for annotated tests written with JUnit4 or TestNG. @Autowired is certainly the central one of Spring 2.5's annotations, being available for use in service components, web components, unit tests - even domain objects when using Spring's @Configurable with AspectJ weaving. Spring MVC's @RequestMapping is equally flexible, supporting many variants of handler method signatures.

Today I…

Well, it took a lot longer than we initially anticipated, but I'm really pleased to say that the Spring Dynamic Modules project reached its 1.0 milestone today. When I first posted on this topic back in September of 2006 ("Spring OSGi support gaining momentum") the initial specification was just an attachment to an issue against the Spring Framework, and connections to the wider OSGi community were only just beginning to be formed.

Fast forward eighteen months, and Spring Dynamic Modules has become a full-fledged project in the Spring portfolio with committers from SpringSource, BEA, and Oracle. Both BEA and Oracle are using Spring Dynamic Modules to build their own OSGi-based products (see for example "WebLogic Event Server - why we used Spring"), and the Spring Dynamic Modules discussion group has almost 1000 members. The OSGi Alliance itself has formed an Enterprise Expert Group…

I am glad to report (along side Adrian) that after 3 milestones and 2 release candidates, Spring Dynamic Modules (formerly known as Spring OSGi) 1.0 has been released.

A lot of features have been improved or added since my previous post (about 1.0 M1); I'll talk more about them in future entries (there is also the reference documentation that explains the library at length) so I'll just name a few:

- consistency

We want to provide a powerful, simple and consistent programming model. That's why Spring Dynamic Modules builds on top of Spring and uses its proven concepts, reliability and ubiquity.

- highly non-intrusive nature

The recommended way to use Spring DM is to not use its classes inside your code or have any imports for them inside your bundle manifests. If you are not using Spring inside your code and only for your application configuration, the same rule applies. Spring DM creates the application context for you, so there is no need for you to depend on Spring or Spring DM. And don't worry about things such as custom namespace or XML schemas - we've already covered them.

- OSGi service dynamics life cycle management

This is one of the most important Spring DM features - the ability to interact with OSGi services just as you would with normal beans. You can publish and consume OSGi services without writing any code; we'll deal with the dynamics for you - and you have full control (more about this in the future).

- smarter integration testing framework

Since we used Spring-DM integration testing extensively internally, we improved the defaults, the maven integration and made the automatic manifest generation faster and smarter then before. For example, the framework automatically determines the classes available in the test bundle and will not generated imports for it.

- simple bundle interaction

Andy Piper (blog) added a simple, declaratively way to install/start/stop/update bundles based on the module life cycle and Spring beans dependency.

- managed startup/shutdown context creation

In OSGi applications are broken into various modules (also known as bundles) which rely on each other services. This creates a dependency tree between the module which becomes important during startup and shutdown. Traditionally this can be addressed by installing and starting the bundles based on the dependency order however, this doesn't solve entirely solve the problem. As the OSGi specification recommends, OSGi services that take a long time to initialize (such as connection pools) should rely on a different thread then the one used for starting and stopping the bundle. This means that if a bundle is started, it does not mean its services are. And not every application is ready to wait for its required service during start up - in fact, few do. This means that a bundle will fail since it relied on a service published several milliseconds later (OSGi is, by default, an in-VM platform where things happen really fast).

This behavior is not rare - in fact, it's quite common at startup on multi-core platforms with multiple bundles. Spring DM addresses this problem by determining the dependencies (from the Spring configuration) and waiting for them to become available before creating the application context. A similar process will be used at shutdown, when Spring DM will stop the contexts based on their dependency order so you don't have to worry about starting or stopping your bundles.

- thread-less dependency waiting

I cannot discuss the dependency mechanism without mentioning the 'thread-less' approach used for dependency waiting (sounds a bit like an oxymoron, I know - we're working on a fancy title for it) implemented by Hal Hildebrand (see his blog). Since various services need to be available for a module to properly start some sort of waiting/monitoring is required which traditionally implies using a thread.

However, on an OSGi platform can be (and there will be) multiple modules (several dozens easily) - using a waiting thread per module simply does not scale. One thing which we worked hard on was improve this model and I believe we provided a very nice solution - using no thread at all for the waiting process. This means that no matter if 3 bundles are deployed or 300, no CPU time will be spent unless your modules actually start.

Spring Dynamic Modules is not just about 'spring'-ifying an API but rather dealing with a different runtime environment.

With regard to tooling, Spring IDE supports Spring DM namespaces and (thanks to Christian) also provides Spring-DM specific targets for Eclipse PDE, a features available in Spring IDE nightly builds (more info on installing and using the plug-in is available in the reference documentation).

Future directions

So now that 1.0 has been released, what's next? Plenty of areas to cover:

Web Support

OSGi platforms provides a dedicated Http Service but using it requires coding. Things such as resource loading, JSP generation and deployment can be significantly simplified. This is the main are of focus of the 1.1 release.

Persistence

Modern persistent tools provide advanced features such as lazy-loading which bend the modularity borders enforced by the OSGi environment as they rely on class generation and proxying. We want to address this problem and, just like with web support, provide a smooth experience whether plain JDBC or/and ORM tools are being used.

AOP

Following the persistence problem, we are seeking solutions for doing generic AOP inside OSGi. It's a hard nut to crack and to do it properly, internal OSGi platform support is required. The good news is that projects like Equinox Aspects have already led the way and OSGi Enterprise Expert Group (EEG) have the problem on their radar.

Enough talking

If you want to know more about Spring Dynamic Modules, see the project page and the reference documentation and do use our mailing list (the forum will appear shortly). Moreover, lately we worked on some OSGi/Spring DM screencasts which are available on the Spring DM home page. The first one (composed by two parts), made by yours truly, shows how to quickly create a project to do integration testing with Spring DM.
Why integration testing? Since with Spring DM it's a very simple and fast process and a very effective way to learn about OSGi (especially with regard to modularity).

There will be more screencasts in the future - just let us know what you'd like to see and based on the number of requests, we'll queue them accordingly.

Without further ado, "Using Spring DM for OSGi integration testing":