Spring AI now supports NVIDIA's Large Language Model API, offering integration with a wide range of models. By leveraging NVIDIA's OpenAI-compatible API, Spring AI allows developers to use NVIDIA's LLMs through the familiar Spring AI API.

We'll explore how to configure and use the Spring AI OpenAI chat client to connect with NVIDIA LLM API.

• The demo application code is available in the nvidia-llm GitHub repository.
• The SpringAI / NVIDIA integration documentation.

Prerequisite

• Create NVIDIA account with sufficient credits.
• Select your preferred LLM model from NVIDIA's offerings. Like the meta/llama-3.1-70b-instruct in the screenshot below.
• From the model's page, obtain the API key for your chosen model.
…

OpenAI recently introduced a powerful feature called Structured Outputs, which ensures that AI-generated responses adhere strictly to a predefined JSON schema. This feature significantly improves the reliability and usability of AI-generated content in real-world applications. Today, we're excited to announce that Spring AI (1.0.0-SNAPSHOT) has fully integrated support for OpenAI's Structured Outputs, bringing this capability to Java developers in a seamless, Spring-native way.

Following diagram shows how the new Structured Outputs feature extends the OpenAI Chat API:

Note: Spring AI already provides a powerful, Model-agnostic Structured Output utilities that can be use with various AI models including the OpenAI. The OpenAI Structured Outputs feature offers an additional, consistent, but a model specific solution, currently available only to the gpt-4o, gpt-4o-mini and later models.

…

Faster information processing not only informs - it transforms how we perceive and innovate.

Spring AI, a powerful framework for integrating AI capabilities into Spring applications, now offers support for Groq - a blazingly fast AI inference engine with support for Tool/Function calling.

Leveraging Groq's OpenAI-compatible API, Spring AI seamlessly integrates by adapting its existing OpenAI Chat client. This approach enables developers to harness Groq's high-performance models through the familiar Spring AI API.

We'll explore how to configure and use the Spring AI OpenAI chat client to connect with Groq. For detailed information, consult the Spring AI Groq documentation and related tests…

Earlier this week, Ollama introduced an exciting new feature: tool support for Large Language Models (LLMs).

Today, we're thrilled to announce that Spring AI (1.0.0-SNAPSHOT) has fully embraced this powerful feature, bringing Ollama's function calling capabilities to the Spring ecosystem.

Ollama's tool support allows models to make decisions about when to call external functions and how to use the returned data. This opens up a world of possibilities, from accessing real-time information to performing complex calculations. Spring AI takes this concept and integrates it seamlessly with the…

UPDATE: (04.06.2024) Adde snippets for using structured output with the new, fluent ChatClient API .

UPDATE: (17.05.2024) Generic Types support for BeanOutputConverter added.

Science works with chunks and bits and pieces of things with the continuity presumed, and Art works only with the continuities of things with the chunks and bits and pieces presumed. - Robert M. Pirsig

The ability of LLMs to produce structured outputs is important for downstream applications that rely on reliably parsing output values. Developers want to quickly turn results from an AI model into data types, such as…

UPDATE 20.07.2024 : Update Message API hierarchy diagram and update the model names supporting multimodality

UPDATE 02.06.2024 : Add an additional code snippet showing how to use the new ChatClient API.

Humans process knowledge, simultaneously across multiple modes of data inputs. The way we learn, our experiences are all multimodal. We don't have just vision, just audio and just text.

These foundational principles of learning were articulated by the father of modern education John Amos Comenius, in his work, "Orbis Sensualium Pictus", dating back to 1658.

"All things that are naturally connected ought to be taught in combination"

…

UPDATE: As of March 13, 2024, Mistral AI has integrated support for parallel function calling into their large model, a feature that was absent at the time of this blog's initial publication.

Mistral AI, a leading developer of open-source large language models, unveiled the addition of Function Calling support to its cutting-edge models.

Function Calling is a feature that facilitates the integration of LLM with external tools and APIs. It enables the language model to request the execution of client-side functions, allowing it to access necessary run-time information or perform tasks…

What is the Spring Cloud Function?

Spring Cloud Function is a SpringBoot-based framework allowing users to concentrate on their business logic by implementing them as Java Functions (i.e., Supplier, Function, Consumer). In turn the framework provides necessary abstraction to enable execution of these functions in various environments (e.g., REST, Streaming) as well as serverless environments such as AWS Lambda or Azure Functions, without having to worry about the underlying platform-specific details. This allows developers to focus on writing their business logic and let the framework handle…

This article is part of a blog series that explores the newly redesigned Spring Cloud Stream applications based on Java Functions.

Here are all the previous parts of this blog series.

• Introducing Function Based Streaming Applications

• Function Composition with Streaming Applications

• How to Build a Supplier and Source Application

• How to Build a Consumer and Sink Application

• Build and Run a Simple Stream Application

• Case Study: HTTP Request Function and Processor

• Case Study: File Source and MongoDB Sink

• Case Study: Relational Database Source and File Sink

• Case Study: Remote File Ingest with Spring Cloud Data Flow

• Case Study: Aggregator Function and Processor

• Case Study: Elasticsearch sink

…