Engineering | Glyn Normington | May 08, 2008 | ...

The SpringSource Application Platform is constructed from OSGi bundles and supports applications which are also constructed from OSGi bundles. The Platform supports the standard features of OSGi, but it also supports some additional manifest headers. Several people have asked Why did SpringSource add proprietary headers? and What are the semantics of the new headers?, so this post explains the background motivation and the semantics of Import-Library and Import-Bundle.

Standard OSGi Bundle Support

The Platform is built on the OSGi R4.1 standard, or JSR 291 if you prefer, and uses Equinox as its OSGi implementation. The result is that you can develop standard OSGi bundles using the Platform's tooling and deploy those bundles on the Platform, as a number of users have been doing since the Platform's launch.

So OSGi savvy developers can use the Platform as a standard OSGi container and benefit from Platform features such as:

the ability to deploy bundles using the Admin Console or by dropping bundles in the Platform's pickup directory,
diagnostics such as resolution failure diagnosis, application specific trace, and automatic deadlock detection,
strong integration with Spring and Spring Dynamic Modules, for developers who want to use these frameworks, and
automatic provisioning of dependencies from a repository.

However, the Platform also aims to make it easy for enterprise application developers with little or no prior exposure to OSGi to benefit from OSGi, which places some extra requirements on the Platform.

Additional Requirements of Enterprise Applications

As Sam's recent blog on the Platform's deployment options explains, you can deploy existing monolithic WAR files on the Platform with no need to understand OSGi - the Platform takes care of everything for you. But to benefit from shared libraries, shared services and, ultimately, PAR file scoping, it is necessary to break monolithic WAR files into OSGi bundles. How hard can that be?

Well, some steps in the process are relatively easy, especially if good software engineering practices have been followed and the code has been organised into service, domain, and infrastructure components. These components can be converted into bundles and the dependencies between them expressed using standard OSGi Import-Package and Export-Package headers in META-INF/MANIFEST.MF.

A more difficult step is expressing dependencies on enterprise frameworks such as Spring and Hibernate. It is entirely possible to express these dependencies using standard OSGi Import-Package and Require-Bundle headers, and this is exactly what you should do if your aim is to create OSGi bundles which will run in other OSGi containers, but this approach has some hidden costs.

Firstly, the developer has to decide precisely which packages comprise a given framework. It isn't sufficient merely to import the packages the application code uses, as several enterprise frameworks weave further dependencies into the bytecode of the application when the application is loaded. The developer has to discover, probably by trial and error, which additional implementation packages to import to ensure correct behaviour of the woven application.

Then there is the chore of migrating from one version of a framework to the next where the precise set of packages comprising the framework has changed. The additional packages required for weaving are typically not defined by a public contract and so are subject to change.

Additionally, the resultant package imports don't properly capture the design intent, which makes maintaining or extending the application more difficult in the future.

We really don't want to impose these burdens on our users, so we created some additional SpringSource Application Platform specific manifest headers, Import-Library and Import-Bundle, as convenient ways of expressing dependencies on enterprise frameworks. As you'll see below, these headers are really just syntactic sugar which are expressed in terms of standard OSGi package imports.

Import-Library

The basic syntax is similar to that of other manifest headers:

    Import-Library: <librarySymbolicName>;version=<versionRange>

where <librarySymbolicName> is the symbolic name of the library and <versionRange> is a range of acceptable versions of the library using OSGi version range notation. A library definition specifies the library's symbolic name and version and these together uniquely identify the library to the Platform.

If you are unfamiliar with OSGi version range notation, by far the most commonly used forms are minimum version ranges such as 2, meaning version 2 or later, and half-open ranges such as [2.2.1,2.2.2), meaning any version between 2.2.1 inclusive and 2.2.2 exclusive. If version=<versionRange> is omitted, together with the semicolon delimiter of course, then the default range includes all versions.

For each library import, the Platform selects the library with the given symbolic name and the highest version in the given version range available in the Platform's repository. The Platform then replaces the library import with a set of package imports which match all the packages exported by the bundles of the library. The Platform detects the situation where a bundle imports two or more libraries which export a common package, issues an appropriate log message, and fails to install the importing bundle.

So, for example, the following header imports some version of the Spring Framework library between 2.5.4 inclusive and 2.5.5 exclusive:

    Import-Library: org.springframework.spring;version="[2.5.4,2.5.5)"

Optional Library Import

You can indicate that a library import is optional using the following syntax. Note the special separator := which indicates a directive that modifies the semantics of the manifest header, as opposed to the separator = which indicates a matching attribute, like version.

    Import-Library: <librarySymbolicName>;version=<versionRange>;resolution:=optional

If resolution is not specified, or is specified as mandatory, the bundle containing the import library header will fail to install if there is no library with the given symbolic name and a version in the given range. But if resolution:=optional is specified, the library import will be ignored if no suitable library is available.

So, for example, the following header imports some version of the Spring Framework library from 2.5 onwards, but is ignored if no suitable library is available:

    Import-Library: org.springframework.spring;version="2.5";resolution:=optional

Importing More than One Library

If you need to import more than one library, then specify a comma-separated list of library imports in a single Import-Library manifest header as in the following example:

    Import-Library: org.foo.p;version="[1,2)",org.bar.q;version="[2,3)"

Import-Bundle

Import-Bundle is a further convenience for cases where a library would consist of only a single bundle and a library definition is inconvenient to create. The syntax is very similar to that of Import-Library except that it refers to a bundle's symbolic name and version instead of those of a library.

As you would expect, for each imported bundle, the Platform selects the bundle with the given symbolic name and the highest version in the given version range available in the Platform's repository. The Platform then replaces the bundle import with a set of package imports matching the packages exported by the bundle.

So, for example, the following header imports the Hibernate Object-Relational Mapper bundle:

    Import-Bundle: com.springsource.org.hibernate;version="[3.2.6,3.2.7)"

Why not Overload Require-Bundle?

If you are familiar with OSGi, you may be asking yourself why we didn't overload Require-Bundle instead of introducing Import-Bundle.

Well, we wanted Require-Bundle to retain its standard semantics, including the ability to marry together pieces of a split package. But we wanted Import-Library and Import-Bundle to have the same underlying semantics as Import-Package which avoid the complexities of split packages.

We also anticipate that, as the Platform evolves over time, we'll need to add further directives to Import-Library and Import-Bundle which would not be appropriate to add to Require-Bundle.

What Next?

The Platform beta program is in progress and we'll be listening to all feedback on Platform features, including the new manifest headers.

For users who want to take advantage of the Platform's headers, but who need to produce bundles which will run on other OSGi containers, we plan to produce a tool which will replace the Import-Library and Import-Bundle…

SpringSource Application Platform Deployment Options

Engineering | Sam Brannen | May 06, 2008 | ...

Since we released the SpringSource Application Platform last Wednesday, numerous developers have downloaded the 1.0.0 beta and started taking the Platform for a test drive. As a result, people have begun asking, "How can I deploy my apps on the Platform, and what kind of deployment and packaging options do I have?" Moreover, developers are eagerly requesting to see working samples. In response, the S2AP team will be releasing several sample applications over the coming weeks demonstrating these features and more, but before you get your hands on these samples, I'd like to give you a high-level…

Running Spring Applications on OSGi with the SpringSource Application Platform

Engineering | Rob Harrop | May 02, 2008 | ...

A lot of people have been asking what exactly the SpringSource Application Platform does for Spring applications to make them run well under OSGi, over and above what you can get out of the box with OSGi and Spring Dynamic Modules. Adrian’s post yesterday highlighted some of the general issues, now lets look at a few of the details.

The three most challenging aspects of running Spring applications on OSGi are:

Load-time weaving
Classpath scanning
Thread context classloader management

The remaining, but less interesting, issues include: JSP support, TLD scanning, annotation matching and resource lookups. Overall, there was a decent-sized set of issues that needed to be solved to make applications deploy smoothly.

Load-time weaving

Load-time weaving was one of the most problematic features to support in a robust manner. At the basic level, it requires hooking into the Equinox ClassLoader so that standard ClassFileTransformers can be attached and used during the defineClass calls. On top of this, many uses of LTW require access to a throwaway ClassLoader that can be used to inspect types to decide what needs to happen during the weave, without affecting the real ClassLoader.

This base level of support was actually reasonably simple to achieve. The difficulty comes in when weaving is driven by classes in one bundle, but classes in another bundle need to be woven. This is pretty common in enterprise applications where one bundle contains domain entities and another contains types that use a JPA EntityManager. The Platform takes care of this complexity by ensuring that all bundles in an application can be woven with the appropriate ClassFileTransformers.

When you start propagating weaving across to other bundles, you really need to know when to stop. If you simply apply weaving to all bundles, then applications will interfere with each other. The Platform prevents this from happening by explicitly scoping weaving so that it applies only to modules in the application.

Another issue with LTW is that it complicates refresh. When a bundle is refreshed, OSGi will refresh all the bundles that depend on it. This means that, in the example I gave above, refreshing the domain bundle will cause the EntityManager bundle to be refreshed. However, refreshing the EntityManager does not refresh the domain bundle, meaning that weaving is possibly out of sync. The Platform handles this by propagating refresh to other bundles that are affected by weaving.

Classpath scanning

With classpath scanning, the main issue is that Equinox doesn’t expose standard jar: and file: resources. The Platform puts an adapter in the middle so that libraries see the resource protocols that they expect. This has a nice side-effect of making a lot of third-party libraries work - it’s not just a fix for classpath scanning.

Thread context classloader management

Many third-party libraries use the thread context ClassLoader to access application types and resources. Each bundle in OSGi has it’s own ClassLoader, so therefore, only one bundle can be exposed as the thread context ClassLoader at any time. This means that if a third-party library needs to see types that are distributed across multiple bundles, it isn’t going to work as expected.

The Platform fixes this by creating a ClassLoader that imports all the exported packages of every module in your application. This ClassLoader is then exposed as the thread context ClassLoader, enabling third-party libraries to see all the exported types in your application.

This is just a small cross-section of the issues that are addressed by the Platform but hopefully it gives you an idea of what the Platform means for Spring Framework users.

Completing the picture: Spring, OSGi, and the SpringSource Application Platform

Engineering | Adrian Colyer | May 01, 2008 | ...

** Updated May 2nd with case study :- see the bottom of this post for details ** I'm sure most of you reading this blog will have seen the announcement of the SpringSource Application Platform yesterday. If not, be sure to check out Rob's blog post which describes some of the motivation, programming model, and roadmap.

A couple of common questions are being asked that I'd like to address straight away in this post. After that I'll describe two other exciting announcements that complement the SpringSource Application Platform itself but that didn't grab the headlines yesterday: the…

Introducing the SpringSource Application Platform

Engineering | Rob Harrop | April 30, 2008 | ...

After many months of feverish coding, I am pleased to announce the beta release of the SpringSource Application Platform 1.0.

At the beginning of 2007 we began discussing possible alternatives to the monolithic and heavyweight application servers with which Enterprise Java has become synonymous. Customers were looking for a platform that was lightweight, modular and flexible enough to meet their development and deployment needs.

The Spring and Tomcat pairing demonstrates that developers and operators can successfully use a lightweight platform in production. Despite the success of this combination, the lack of modularity and explicit support for non-web applications limits its applicability and flexibility.

We set about building the SpringSource Application Platform to address these requirements and remove these limitations.

At the heart of the Platform is the Dynamic Module Kernel (DMK). The DMK is an OSGi-based kernel that takes full advantage of the modularity and versioning of the OSGi platform. The DMK builds on Equinox and extends its capabilities for provisioning and library management, as well as providing core function for the Platform.

SpringSource Application Platform Architecture

To maintain a minimal runtime footprint, OSGi bundles are installed on demand by the DMK provisioning subsystem. This allows for an application to be installed into a running Platform and for its dependencies to be satisfied from an external repository. Not only does this remove the need to manually install all your application dependencies, which would be tedious, but it keeps memory usage to a minimum.

The DMK itself requires a minimal set of bundles to run, and is configured with a profile to control exactly the set of additional modules that are loaded. For example, the DMK does not require Tomcat to be present, but the default Platform profile includes Tomcat to allow web applications to be deployed. If you want to run a Platform without Tomcat, you can simply edit the profile and it won’t be installed. (If you try this - remember that removing web support means that web modules will no longer deploy, so delete the contents of the pickup directory so the platform won’t attempt to install the Admin and splash screen applications when it starts.) The default Platform configuration with the admin console installed takes only 15MB of memory.

One frustration I have always had with Enterprise Java is that applications are often shoe-horned into contrived silos, and that explicit support for different application types is lacking. Consider an application for an online store. This application has a web front-end, a message-driven order processing module, a batch-driven stock reordering module and a B2B web service module. Today, many applications like this would be packaged as a WAR or an EAR and the modules would look very similar, with little support for the differences in the module types. Interestingly, many people would refer to this as a web application, rather than an application with a web module.

In the SpringSource Application Platform, applications are modular and each module has a personality that describes what kind of module it is: web, batch, web service, etc. The Platform deploys modules of each personality in a personality-specific manner. For example, web modules are configured in Tomcat with web context. Each module in the application can be updated independently of the other modules whilst retaining the identity of being part of the larger application. Whatever kind of application you are building, the programming model remains standard Spring and Spring DM.

In the 1.0 Platform release we support the web and bundle personalities, which enable you to build sophisticated web applications. Future releases will include support for more personalities as detailed later.

Building Applications

The Platform supports applications packaged in three forms:

Java EE WAR
Raw OSGi bundles
Platform Archive (PAR)

Standard WAR files are supported directly in the Platform. At deployment time, the WAR file is transformed into an OSGi bundle and installed into Tomcat. All the standard WAR contracts are honoured, and your existing WAR files should just drop in and deploy without change.

Any OSGi-compliant bundle can be deployed directly into the Platform, and can take full advantage of on-the-fly provisioning for any dependencies referred to by Import-Package and Require-Bundle.

The PAR format is the recommended approach for packaging and deploying applications for the Platform. A PAR is simply a collection of OSGi bundles (modules) grouped together in a standard JAR file, along with a name and a version that uniquely identify the application. The PAR file is deployed as a single unit into the Platform. The Platform will extract all the modules from the PAR and install them. Third-party dependencies will be installed on-the-fly as needed.

The PAR format has three main benefits over deploying the bundles directly into the Platform. Firstly, it’s just easier. An average-sized enterprise application might contain 12+ bundles - deploying these by hand will be far too cumbersome. Secondly, the PAR file forms an explicit scope around all the bundles in the application which prevents applications that use overlapping types or services from clashing with each other. This scope is also used by some advanced features such as load-time weaving to ensure that weaving of one application doesn’t interfere with weaving of another. Lastly, the PAR forms a logical grouping to define what modules are part of an application and what third-party dependencies the application has. This grouping is used by the management tools to provide a detailed view of the application. A typical PAR application looks like this:

PAR File Structure

Dependencies between modules in an application are typically expressed using Import-Package and Export-Package. Dependencies on third-party libraries can be expressed in the same way, but for many libraries this can be error-prone and time-consuming. When using a library such as Hibernate, you will typically have to import more packages than you initially expect. To get around this you could use Require-Bundle, but this has some semantic rough edges such as split packages where a logical package is split across two or more class loaders causing problems at runtime. The Platform introduces two new mechanisms for referring to third-party dependencies: Import-Bundle and Import-Libary. Import-Bundle is analogous to Require-Bundle except it prevents split packages and the other problems with Require-Bundle. Import-Library provides a mechanism to refer to all the packages exported by a group of bundles, for instance all bundles in the Spring Framework, in a single declaration:

Bundle-SymbolicName: com.myapp.dao.jdbc
Bundle-Version: 1.0.0
Import-Bundle: org.apache.commons.dbcp;version="1.2.2.osgi"
Import-Library: org.springframework.spring;version="2.5.4.A"

Here I have a module bundle that depends on the Commons DBCP bundle and the Spring Framework library. The Spring Framework library contains all the bundles needed to use Spring in your application.

Import-Library and Import-Bundle expand under the covers into Import-Package and are therefore consistent with standard OSGi semantics.

The Platform understands the personality of a module, and from this can infer how to configure the module’s execution environment. When deploying a web module, all the servlet infrastructure needed for a typical Spring MVC application is created automatically, removing the need to recreate this boilerplate code across applications. In the 1.0 final release, more smarts will be added to the web module personality to support additional technologies such as Spring Web Flow.

Whatever packaging format you choose, the programming model is simply Spring Framework and Spring Dynamic Modules, with the other Spring Portfolio products running on top.

Serviceability

Serviceability was a critical consideration for the whole engineering team. We have spent an inordinate amount of time worrying about the format of log messages and the size of stack traces to make sure that diagnosing problems with your applications is as easy as possible. Any time we find a task that is repetitive and time-consuming we look for a way to automate it or remove it altogether.

To aid with problem diagnosis the Platform has a strong split between log and trace messages. Log messages are intended for end-user consumption and allow you to get access to the most important failure information without having to trawl through gigabytes of trace content. All application failures are displayed and coded in the log output - the codes serving as a convenient way to access knowledge base or support content. To understand why this is so useful, consider the following output from Platform startup:

[2008-04-29 12:12:01.124] main                     <SPKB0001I> Platform starting.
[2008-04-29 12:12:04.037] main                     <SPKE0000I> Boot subsystems installed.
[2008-04-29 12:12:06.013] main                     <SPKE0001I> Base subsystems installed.
[2008-04-29 12:12:07.396] platform-dm-1            <SPPM0000I> Installing profile 'web'.
[2008-04-29 12:12:07.674] platform-dm…

Web Applications and OSGi

Engineering | Costin Leau | April 29, 2008 | ...

Since the first milestones of Spring Dynamic Modules, requests for running web applications in OSGi started to come in. It has been probably one of the most requested features and no wonder, once 1.0 final was released, web support has been the main focus of the 1.1 branch. I am pleased to report that, with the just released M2, as already hinted by Juergen, Spring-DM supports not just vanilla wars (available since 1.1.0 M1) but also Spring-MVC applications running inside OSGi. In this entry, I would like to briefly discuss the typical OSGi web scenario and Spring-DM's approach. But first,

Why deploy WARs in OSGi?

Easy question: OSGi natively provides versioning, package wiring and hot reloading. Imagine taking advantage of these features within your applications: you can stop embedding libraries under WEB-INF/lib and start sharing them between your web apps, avoid taglibs duplications (while keeping multiple versions running) and update, at runtime, only certain parts of your application. This is especially useful as web applications tend to be tiered and thus subject to a significant number of changes during their life cycle.

Why are web applications in OSGi problematic?

The Servlet specification revolves around the idea of a web container: a runtime environment for web components that provides standard services such as life cycle management (object creation and disposal, thread allocation), concurrency, HTTP request handling and so on. The OSGi platform on the other hand, acts also as a managed environment with its Service Registry, package wiring and versioning (to name just a few). To deal with this problem, the OSGi committee designed, part of the compendium specification, the Http Service.

…

Today, Portability Matters More Than Ever

Engineering | Juergen Hoeller | April 29, 2008 | ...

Yesterday, I blogged about how Spring helps maximize application portability. Even if the portability problem has been an ongoing topic in enterprise Java land for many years, that blog was timely. Today, Oracle announced that its $6.7 billion acquisition of BEA Systems has closed. There is substantial overlap between the product sets of the two companies, so this is bound to bring uncertainty to the WebLogic and OC4J customer bases. WebLogic and OC4J may both fall into the "J2EE server" category but they are very different products with very different characteristics.

Since many enterprise…

Portability at the Framework Level

Engineering | Juergen Hoeller | April 28, 2008 | ...

Portability is a key factor in the Spring universe. We believe in portability at the framework level: Application components are written against a specific framework (or framework generation), such as Spring 2.5; the framework is then in turn responsible for adapting onto any underlying hosting environment. However, the specific application framework is above and distinct from the hosting environment. A brand-new framework version may be deployed onto an established generation of the hosting platform, as long as the fundamental capabilities of the environment are sufficient. This approach…

The Conference Season Rolls On

Engineering | Rod Johnson | April 24, 2008 | ...

Yesterday I gave the opening keynote at the JAX conference in Wiesbaden, Germany. JAX is one of Europe’s largest Java conferences, with over 2,000 attendees. The topic was The Future of Enterprise Java, and I expanded on the themes of my recent blog of predictions, going into more detail about the implications of Java EE 6 and the future of the application server.
I’ve uploaded the slides, which include 8 predictions for an interesting period in the evolution of enterprise Java. This is the first time I've referred to Joseph Stalin, Monica Lewinsky and Monty Python in the same presentation.
…

Spring Security 2.0 Final Release: No More Dead Fairies

Engineering | Rod Johnson | April 17, 2008 | ...

Spring Security 2.0 has been released. This is a major step forward for the Spring Portfolio. Spring (Acegi) Security is already the Java platform's most widely used enterprise security framework, with over 250,000 downloads on SourceForge and over 20,000 downloads per release. Through making it so much simpler to use, this release will undoubtedly take adoption to a new level.

I'm particularly pleased about this release for a number of reasons:

It's a great thing for the Spring community. It's (a lot) simpler to use, as well as more powerful. It puts the most powerful enterprise Java security solution within the reach of many more users, pretty much eliminating the hurdles to adoption. See this tutorial for an example of just how much easier it makes it to secure a typical web application. The proliferation of XML bean definitions is a thing of the past.
It's a continuation of the work of Spring 2.x, through applying the power of a custom XML namespace to enable aggressive defaulting, while still allowing for customization.
Like Spring 2.5, it exhibits the current Spring Portfolio trend toward radical reduction in the need for XML.
It's a proof of the value of the SpringSource business model. Our revenue model enables us to invest more than ever in creating open source software. Without being able to hire both Acegi/Spring Security creator Ben Alex and the other major committer, Luke Taylor, this release either wouldn't have occurred or would have been much less extensive.
It's good for the fairy kingdom.

Acegi/Spring Security creator Ben Alex and Luke Taylor have done a great job. Ben will be talking about Spring Security at Java One next month. If…

Spring blog

SpringSource Application Platform Manifest Headers