Spring AI 1.0.0 M4 Release

Releases | Mark Pollack | November 20, 2024 | ...

We are happy to announce the 1.0.0 Milestone 4 release of Spring AI.

This release has squashed most of the reported bugs and brings significant enhancements across various areas.

Amazon Bedrock Converse

Spring AI now supports the Amazon Bedrock Converse API, which provides a unified interface for the AI chat models that Amazon offers. Unlike the older Bedrock Invoke API, Converse introduces exciting new features such as Tool Calling and Multimodality/Visual capabilities (for models that support these features). This makes it a more powerful and versatile option for working with Amazon’s chat models.

In many ways, the Converse API serves a role similar to Spring AI itself, offering portability across model APIs but limited to Amazon's chat models. We recommend transitioning to the Converse module for chat models. Note that embedding models remain unaffected.

For details on this new capability, refer to the Spring AI Bedrock Converse documentation. You can also follow upcoming improvements for future enhancements. Many thanks to Max Jiang for starting this large undertaking.

Function Calling improvements

Function Types and Methods

Recent improvements expand support for various Function types and methods through the FunctionCallback Builder class. This allows invocation of java.util.Function, Supplier, and Consumer interfaces.

Basic Function Callback

FunctionCallback callback = FunctionCallback.builder()
    .description("Process a new order")
    .function("processOrder", (Order order) -> processOrderLogic(order))
    .inputType(Order.class)
    .build();

Using ToolContext {#using-toolcontext}

Access additional state or context using ToolContext with BiFunction<I, ToolContext, O>

@Bean
@Description("Get the weather in location")
public BiFunction<WeatherService.Request, ToolContext, WeatherService.Response> weatherFunctionWithContext() {
    return (request, context) -> new MockWeatherService().apply(request);
}

Method Invocation

Method invocation support provides the foundation for the upcoming @ToolMapping annotation in M5:

public static class LightControl {
    private static final Logger logger = LoggerFactory.getLogger(LightControl.class);
    private final Map<String, Object> arguments = new HashMap<>();

    public void controlLight(String roomName, boolean on) {
        arguments.put("roomName", roomName);
        arguments.put("on", on);
        logger.info("Setting light in room '{}' to: {}", roomName, on ? "ON" : "OFF");
    }
}

Usage:

LightControl lightControl = new LightControl();

String response = ChatClient.create(this.chatModel)
    .prompt("Turn light on in the living room.")
    .functions(
        FunctionCallback.builder()
            .description("Controls lights by room name, allowing them to be turned on or off.")
            .method("controlLight", String.class, boolean.class)
            .targetObject(lightControl)
            .build()
    )
    .call()
    .content();

For additional features, consult the FunctionCallback documentation which covers:

Schema type configuration (JSON Schema and OpenAPI Schema support)
Custom response handling and object mapping
Support for generic input types using ParameterizedTypeReference
Automatic schema generation from Java types, including Jackson annotation support
Guidelines for writing effective function descriptions and error handling

Kotlin Support

Spring AI introduces support for Kotlin, making it even easier for Kotlin developers to integrate AI capabilities into their applications. This release introduces idiomatic Kotlin extensions and type-safe APIs. Many thanks to Sebastien Deleuze for taking on this work.

Type-Safe Response Handling

The new Kotlin extensions enable more concise and type-safe ways to handle AI responses. Instead of using Java-style type declarations, you can now use Kotlin's reified generics:

import org.springframework.ai.chat.client.entity

data class Joke(val setup: String, val punchline: String)

@SpringBootApplication
class KotlinHelloWorldApplication {

   @Bean
   fun jokeRunner(chatModel: ChatModel) = CommandLineRunner {
      val response = ChatClient.create(chatModel).prompt().user("Tell me a joke").call().entity<Joke>()

      println("\nJoke:")
      println("Setup: ${response.setup}")
      println("Punchline: ${response.punchline}")
   }
}

fun main(args: Array<String>) {
   runApplication<KotlinHelloWorldApplication>(*args)
}

Function Registration

Kotlin functions can now be directly registered as AI tools. The syntax is straightforward:

@Configuration
class Config {

   @Bean
   fun weatherFunctionInfo(currentWeather: (WeatherRequest) -> WeatherResponse): FunctionCallback {
      return FunctionCallback.builder()
         .description(
            "Find the weather conditions, forecasts, and temperatures for a location, like a city or state."
         )
         .function("WeatherInfo", currentWeather)
         .inputType(WeatherRequest::class.java)
         .build()
   }

   @Bean
   @Description("Get current weather")
   fun currentWeather(): (WeatherRequest) -> WeatherResponse = { request ->
      MockKotlinWeatherService().invoke(request)
   }
}

Then, it can be used as below.

@Bean
open fun init(chatModel: ChatModel) = CommandLineRunner {

   try {
      val userMessage = UserMessage(
         "What are the weather conditions in San Francisco, Tokyo, and Paris? Find the temperature in Celsius for each of the three locations."
      )

      val response = chatModel.call(
         Prompt(
            listOf(userMessage),
            OpenAiChatOptions.builder().withFunction("WeatherInfo").build()
         )
      )

      println("Response: $response")
   } 
   catch (e: Exception) {
      println("Error during weather check: ${e.message}")
   }
}

The reference documentation has been updated to include Kotlin examples, and you can find other examples in the spring-ai-examples repository.

Additional Project Resources

The repository awesome-spring-ai keeps the theme of other “awseome” repositories and collects community related resources for Spring AI such as book, presentation, example code and so on. Please make PRs to the repository for any useful material you a have found.

The repository spring-ai-integration-tests is now available and is making progress to run all the integration tests for the entire project twice daily.

The repository spring-ai-examples is now available for hosting “official” examples. Check out the reflection agent example!

Also, as an introduction to the project, check out the blog post Why Spring AI

Advanced and Modular RAG

Spring AI introduces experimental support for advanced Retrieval Augmented Generation (RAG) based on recent research in modular RAG system design, in particular the papers Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks and Retrieval-Augmented Generation for Large Language Models: A Survey. A big thanks to Thomas Vittale for leading this effort.

Here are the key building blocks:

Pre-retrieval Components

QueryTransformer: Transforms queries to improve retrieval effectiveness (e.g., translation, rewrites)
QueryExpander: Expands single queries into multiple variations for broader context capture

Orchestration Components

QueryRouter: Routes queries to appropriate retrievers based on metadata or semantic analysis

Retrieval Components

DocumentRetriever: Core interface for retrieving relevant documents
DocumentJoiner: Combines results from multiple retrievers/queries

Post-retrieval Components

DocumentCompressor: Reduces document content while preserving key information
DocumentRanker: Reorders documents by relevance
DocumentSelector: Filters retrieved documents based on criteria

Augmentation Components

QueryAugmenter: Enhances queries with retrieved context
ContextualQueryAugmenter: Default implementation focused on document context integration

Building on our modular RAG components, the RetrievalAugmentationAdvisor provides an implementation to get you started and provides a linear RAG flow:

Query Creation: Creates Query from user text using PromptTemplate
Query Transformation: Applies QueryTransformers chain
Query Expansion: Creates multiple queries via QueryExpander
Document Retrieval: Routes and executes parallel retrievals
Document Integration: Joins results from all sources
Context Augmentation: Enhances query with retrieved context

You stitch together these collaborating building block components in a bean definition such as

@Bean
public RetrievalAugmentationAdvisor customRagAdvisor(VectorStore vectorStore) {
    return RetrievalAugmentationAdvisor.builder()
        .queryTransformers(List.of(new TranslationQueryTransformer(...)))
        .queryExpander(new MultiQueryExpander(...))
        .queryRouter(
            AllRetrieversQueryRouter.builder()
                .documentRetrievers(new VectorStoreDocumentRetriever(...))
                .build()
        )
        .documentJoiner(new ConcatenationDocumentJoiner())
        .queryAugmenter(new ContextualQueryAugmenter(...))
        .build();
}

While RetrievalAugmentationAdvisor implements a linear flow, users can implement custom Advisors for advanced patterns like conditional, branching, recursive, and adaptive flows. More documentation is to come and we encourage you to send feedback on our github issue tracker.

Other improvements

There were a few locations where Spring Boot being used in the model modules. This is no longer the case. The only dependencies on Spring Boot are not isolated to the autoconfigure module.

Azure OpenAI SDK upgraded to version 12 beta.

Documentation

Added a comprehensive Chat Model comparison page
Restructured documentation navigation for better accessibility
Updated diagrams for chat model architecture
Added guide for programmatic configuration of Vertex embeddings with service accounts
Enhanced integration guides:
Ollama updates:
- New auto-pull functionality
- Llama32-Vision support
Spring AI features:
- Out-of-the-box support for Llama32-Vision
- Portable multimodality API
Vector database configuration:
- Milvus
- Chroma
- OpenSearch

Expanded configuration and setup documentation, focusing on:

Property overrides
Code examples
Integration patterns across different model types

Vector Store & Embedding Improvements

Added new vector store implementations for Oracle Coherence and Azure Cosmos
Enhanced existing vector stores:
- Azure: Provided ability to match pre-existing field names and use of keyless authentication
- Milvus: Added support for non-default databases
- Chroma: Introduced builder pattern for better initialization
- OpenSearch: Improved configuration structure
Introduced builder pattern for TokenTextSplitter

Contributors

There were other refactoring, bug fixing, documentation enhancements across the board by a wide range of contributors. If we haven’t gotten to your PR yet, we will, please be patient. Thanks to

Roadmap

We are planning to have one more milestone release, 1.0.0 M5 targeted for late December that focuses on design issues, for example returning more information in the ChatResponse, overhaul the VectorStore API to add/delete and support different query types, unify the API styles of various builders, and more. Then in January we will make a 1.0.0 RC1 release followed quickly by a 1.0.0 GA release.

Spring blog