Spring AI with Docker Model Runner
This blog post is authored by Eddú Meléndez.
Docker recently released a Model Runner in Docker Desktop for Mac 4.40.0 on Apple silicon. The Docker Model Runner provides a local Inference API designed to be compatible with the OpenAI API, enabling easy integration with Spring AI as part of the Spring AI 1.0.0-M7 release. Models are distributed as standard OCI artifacts on Docker Hub under the ai namespace.
Prerequisites
-
Download Docker Desktop for Mac 4.40.0.
-
Choose one of the following options to enable the Model Runner:
Option 1:
-
Enable Model Runner `docker desktop enable model-runner --tcp 12434`.
-
Set the base-url to `http://localhost:12434/engines\`
Option 2:
-
Enable Model Runner `docker desktop enable model-runner`.
-
Use Testcontainers and set the base-url as follows:
…