Total Pageviews

Monday 29 April 2024

Spring-AI

 An Application Framework for AI Engineering.

https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/index.html

 build status

Welcome to the Spring AI project!

The Spring AI project provides a Spring-friendly API and abstractions for developing AI applications.

Let's make your @Beans intelligent!

For further information go to our Spring AI reference documentation.

Project Links

Educational Resources

Some selected videos. Search YouTube! for more.

  • Spring Tips: Spring AI
    Watch Spring Tips video
  • Overview of Spring AI @ Devoxx 2023
    Watch the Devoxx 2023 video
  • Introducing Spring AI - Add Generative AI to your Spring Applications
    Watch the video

Getting Started

Please refer to the Getting Started Guide for instruction on adding your dependencies.

Note, the new Spring CLI project lets you get up and running in two simple steps, described in detail here.

  1. Install Spring CLI
  2. Type spring boot new --from ai --name myai in your terminal

Adding Dependencies manually

Note that are two main steps.

  1. Add the Spring Milestone and Snapshot repositories to your build system.
  2. Add the Spring AI BOM
  3. Add dependencies for the specific AI model, Vector Database or other component dependencies you require.

Overview

Despite the extensive history of AI, Java's role in this domain has been relatively minor. This is mainly due to the historical reliance on efficient algorithms developed in languages such as C/C++, with Python serving as a bridge to access these libraries. The majority of ML/AI tools were built around the Python ecosystem. However, recent progress in Generative AI, spurred by innovations like OpenAI's ChatGPT, has popularized the interaction with pre-trained models via HTTP. This eliminates much of the dependency on C/C++/Python libraries and opens the door to the use of programming languages such as Java.

The Python libraries LangChain and LlamaIndex have become popular to implement Generative AI solutions and can be implemented in other programming languages. These Python libraries share foundational themes with Spring projects, such as:

  • Portable Service Abstractions
  • Modularity
  • Extensibility
  • Reduction of boilerplate code
  • Integration with diverse data sources
  • Prebuilt solutions for common use cases

Taking inspiration from these libraries, the Spring AI project aims to provide a similar experience for Spring developers in the AI domain.

Note, that the Spring AI API is not a direct port of either LangChain or LlamaIndex. You will see significant differences in the API if you are familiar with those two projects, though concepts and ideas are fairly portable.

Feature Overview

This is a high level feature overview. The features that are implemented lay the foundation, with subsequent more complex features building upon them.

You can find more details in the Reference Documentation

Interacting with AI Models

ChatClient: A foundational feature of Spring AI is a portable client API for interacting with generative AI models. With this portable API, you can initially target one AI chat model, for example OpenAI and then easily swap out the implementation to another AI chat model, for example Amazon Bedrock's Anthropic Model. When necessary, you can also drop down to use non-portable model options.

Spring AI supports many AI models. For an overview see here. Specific models currently supported are

  • OpenAI
  • Azure OpenAI
  • Amazon Bedrock (Anthropic, Llama, Cohere, Titan, Jurassic2)
  • HuggingFace
  • Google VertexAI (PaLM2, Gemini)
  • Mistral AI
  • Stability AI
  • Ollama
  • PostgresML
  • Transformers (ONNX)
  • Anthropic Claude3

Prompts: Central to AI model interaction is the Prompt, which provides specific instructions for the AI to act upon. Crafting an effective Prompt is both an art and science, giving rise to the discipline of "Prompt Engineering". These prompts often leverage a templating engine for easy data substitution within predefined text using placeholders.

Explore more on Prompts in our concept guide. To learn about the Prompt class, refer to the Prompt API guide.

Prompt Templates: Prompt Templates support the creation of prompts, particularly when a Template Engine is employed.

Delve into PromptTemplates in our concept guide. For a hands-on guide to PromptTemplate, see the PromptTemplate API guide.

Output Parsers: AI model outputs often come as raw java.lang.String values. Output Parsers restructure these raw strings into more programmer-friendly formats, such as CSV or JSON.

Get insights on Output Parsers in our concept guide.. For implementation details, visit the OutputParser API guide.

Incorporating your data

Incorporating proprietary data into Generative AI without retraining the model has been a breakthrough. Retraining models, especially those with billions of parameters, is challenging due to the specialized hardware required. The 'In-context' learning technique provides a simpler method to infuse your pre-trained model with data, whether from text files, HTML, or database results. The right techniques are critical for developing successful solutions.

Retrieval Augmented Generation

Retrieval Augmented Generation, or RAG for short, is a pattern that enables you to bring your data to pre-trained models. RAG excels in the 'query over your docs' use-case.

Learn more about Retrieval Augmented Generation.

Bringing your data to the model follows an Extract, Transform, and Load (ETL) pattern. The subsequent classes and interfaces support RAG's data preparation.

Documents:

The Document class encapsulates your data, including text and metadata, for the AI model. While a Document can represent extensive content, such as an entire file, the RAG approach segments content into smaller pieces for inclusion in the prompt. The ETL process uses the interfaces DocumentReader, DocumentTransformer, and DocumentWriter, ending with data storage in a Vector Database. This database later discerns the pieces of data that are pertinent to a user's query.

Document Readers:

Document Readers produce a List<Document> from diverse sources like PDFs, Markdown files, and Word documents. Given that many sources are unstructured, Document Readers often segment based on content semantics, avoiding splits within tables or code sections. After the initial creation of the List<Document>, the data flows through transformers for further refinement.

Document Transformers:

Transformers further modify the List<Document> by eliminating superfluous data, like PDF margins, or appending metadata (e.g., primary keywords or summaries). Another critical transformation is subdividing documents to fit within the AI model's token constraints. Each model has a context-window indicating its input and output data limits. Typically, one token equates to about 0.75 words. For instance, in model names like gpt-4-32k, "32K" signifies the token count.

Document Writers:

The final ETL step within RAG involves committing the data segments to a Vector Database. Though the DocumentWriter interface isn't exclusively for Vector Database writing, it the main type of implementation.

Vector Stores: Vector Databases are instrumental in incorporating your data with AI models. They ascertain which document sections the AI should use for generating responses. Examples of Vector Databases include Chroma, Postgres, Pinecone, Qdrant, Weaviate, Mongo Atlas, and Redis. Spring AI's VectorStore abstraction permits effortless transitions between database implementations.

Cloning the repo

This repository contains large model files. To clone it you have to either:

  • Ignore the large files (won't affect the spring-ai behaviour) : GIT_LFS_SKIP_SMUDGE=1 git clone git@github.com:spring-projects/spring-ai.git.
  • Or install the Git Large File Storage before cloning the repo.

Building

To build with running unit tests

./mvnw clean package

To build including integration tests. Set API key environment variables for OpenAI and Azure OpenAI before running.

./mvnw clean verify -Pintegration-tests

To run a specific integration test allowing for up to two attempts to succeed. This is useful when a hosted service is not reliable or times out.

./mvnw -pl vector-stores/spring-ai-pgvector-store -Pintegration-tests -Dfailsafe.rerunFailingTestsCount=2 -Dit.test=PgVectorStoreIT verify

To build the docs

./mvnw -pl spring-ai-docs antora

The docs are then in the directory spring-ai-docs/target/antora/site/index.html

To reformat using the java-format plugin

./mvnw spring-javaformat:apply

To update the year on license headers using the license-maven-plugin

./mvnw license:update-file-header -Plicense

To check javadocs using the javadoc:javadoc

./mvnw javadoc:javadoc -Pjavadoc
from https://github.com/spring-projects/spring-ai 
-------------------------------------------------------

Getting Started

This section offers jumping off points for how to get started using Spring AI.

You should follow the steps in each of the following section according to your needs.

Spring CLI

The Spring CLI, simplifies creating new applications directly from your terminal. Like the 'create-react-app' command for those familiar with the JavaScript ecosystem, Spring CLI provides a spring boot new command to create Spring-based projects. Spring CLI also offers features to integrate external code bases into your current project, and many other productivity features.


 It is important to understand that the "Spring CLI" is a distinct project from the "Spring Boot CLI", each with its own set of functionalities.

To begin creating a Spring AI application, follow these steps:

  1. Download the latest Spring CLI Release and follow the installation instructions.

  2. To create a simple OpenAI-based application, use the command:

    spring boot new --from ai --name myai

  1. Consult the generated README.md file for guidance on obtaining an OpenAI API Key and running your first AI application.

To add the same simple AI application to an existing project, execute:

spring boot add ai

 Spring CLI allows users to define their own project catalogs that define which projects you can create or add to your existing code base.

Spring Initializr

Head on over to start.spring.io and select the AI Models and Vector Stores that you want to use in your new applications.

Add Milestone and Snapshot Repositories

If you prefer to add the dependency snippets by hand, follow the directions in the following sections.

To use the Milestone and Snapshot version, you need to add references to the Spring Milestone and/or Snapshot repositories in your build file.

For Maven, add the following repository definitions as needed:

  <repositories>
    <repository>
      <id>spring-milestones</id>
      <name>Spring Milestones</name>
      <url>https://repo.spring.io/milestone</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
    </repository>
    <repository>
      <id>spring-snapshots</id>
      <name>Spring Snapshots</name>
      <url>https://repo.spring.io/snapshot</url>
      <releases>
        <enabled>false</enabled>
      </releases>
    </repository>
  </repositories>

For Gradle, add the following repository definitions as needed:

repositories {
  mavenCentral()
  maven { url 'https://repo.spring.io/milestone' }
  maven { url 'https://repo.spring.io/snapshot' }
}

Dependency Management

The Spring AI Bill of Materials (BOM) declares the recommended versions of all the dependencies used by a given release of Spring AI. Using the BOM from your application’s build script avoids the need for you to specify and maintain the dependency versions yourself. Instead, the version of the BOM you’re using determines the utilized dependency versions. It also ensures that you’re using supported and tested versions of the dependencies by default, unless you choose to override them.

If you’re a Maven user, you can use the BOM by adding the following to your pom.xml file -

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>0.8.1-SNAPSHOT</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Gradle users can also use the Spring AI BOM by leveraging Gradle (5.0+) native support for declaring dependency constraints using a Maven BOM. This is implemented by adding a 'platform' dependency handler method to the dependencies section of your Gradle build script. As shown in the snippet below this can then be followed by version-less declarations of the Starter Dependencies for the one or more spring-ai modules you wish to use, e.g. spring-ai-openai.

dependencies {
  implementation platform("org.springframework.ai:spring-ai-bom:0.8.1-SNAPSHOT")
  // Replace the following with the starter dependencies of specific modules you wish to use
  implementation 'org.springframework.ai:spring-ai-openai'
}

Add dependencies for specific components

Each of the following sections in the documentation shows which dependencies you need to add to your project build system.

Embeddings Models

Chat Models

 

 

Posted by at
Labels: , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)