Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Release 0.1.6 #45

Merged
merged 1 commit into from
Aug 13, 2024
Merged

feat: Release 0.1.6 #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
242 changes: 160 additions & 82 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,11 @@ This client works with Chroma Versions `0.4.3+`

### Embeddings Support

- [x] OpenAI API
- [x] Cohere API (including Multi-language support)
- ✅ Default Embedding Function (all-mini-lm model)
- ✅ OpenAI Embedding Function
- ✅ Cohere Embedding Function
- ✅ HuggingFace Embedding Function
- ✅ Ollama Embedding Function
- [ ] Sentence Transformers
- [ ] PaLM API
- [ ] Custom Embedding Function
Expand All @@ -36,7 +39,7 @@ This client works with Chroma Versions `0.4.3+`

- [x] Push the package to Maven
Central - https://docs.github.com/en/actions/publishing-packages/publishing-java-packages-with-maven
- ⚒️ Fluent API - make it easier for users to make use of the library
- ⚒️ Fluent API - make it easier for users to make use of the library
- [ ] Support for PaLM API
- [x] Support for Sentence Transformers with Hugging Face API
- ⚒️ Authentication ⚒️
Expand All @@ -46,10 +49,11 @@ This client works with Chroma Versions `0.4.3+`
Add Maven dependency:

```xml

<dependency>
<groupId>io.github.amikos-tech</groupId>
<artifactId>chromadb-java-client</artifactId>
<version>0.1.5</version>
<version>0.1.6</version>
</dependency>
```

Expand All @@ -59,6 +63,43 @@ Ensure you have a running instance of Chroma running. We recommend one of the tw
- If you are a fan of Kubernetes, you can use the Helm chart - https://github.com/amikos-tech/chromadb-chart (Note: You
will need `Docker`, `minikube` and `kubectl` installed)

### Default Embedding Function

Since version `0.1.6` the library also offers a built-in default embedding function which does not rely on any external
API to generate embeddings and works in the same way it works in core Chroma Python package.

```java
package tech.amikos;

import tech.amikos.chromadb.*;
import tech.amikos.chromadb.Collection;

import java.util.*;

public class Main {
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
client.reset();
EmbeddingFunction ef = new DefaultEmbeddingFunction();
Collection collection = client.createCollection("test-collection", null, true, ef);
List<Map<String, String>> metadata = new ArrayList<>();
metadata.add(new HashMap<String, String>() {{
put("type", "scientist");
}});
metadata.add(new HashMap<String, String>() {{
put("type", "spy");
}});
collection.add(null, metadata, Arrays.asList("Hello, my name is John. I am a Data Scientist.", "Hello, my name is Bond. I am a Spy."), Arrays.asList("1", "2"));
Collection.QueryResponse qr = collection.query(Arrays.asList("Who is the spy"), 10, null, null, null);
System.out.println(qr);
} catch (Exception e) {
System.out.println(e);
}
}
}
```

### Example OpenAI Embedding Function

In this example we rely on `tech.amikos.chromadb.OpenAIEmbeddingFunction` to generate embeddings for our documents.
Expand All @@ -81,7 +122,7 @@ public class Main {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
String apiKey = System.getenv("OPENAI_API_KEY");
EmbeddingFunction ef = new OpenAIEmbeddingFunction(apiKey,"text-embedding-3-small");
EmbeddingFunction ef = new OpenAIEmbeddingFunction(apiKey, "text-embedding-3-small");
Collection collection = client.createCollection("test-collection", null, true, ef);
List<Map<String, String>> metadata = new ArrayList<>();
metadata.add(new HashMap<String, String>() {{
Expand Down Expand Up @@ -109,7 +150,8 @@ The above should output:

#### Custom OpenAI Endpoint

For endpoints compatible with OpenAI Embeddings API (e.g. [ollama](https://github.com/ollama/ollama)), you can use the following:
For endpoints compatible with OpenAI Embeddings API (e.g. [ollama](https://github.com/ollama/ollama)), you can use the
following:

> Note: We have added a builder to help with the configuration of the OpenAIEmbeddingFunction

Expand Down Expand Up @@ -145,28 +187,28 @@ import tech.amikos.chromadb.Collection;
import java.util.*;

public class Main {
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
client.reset();
String apiKey = System.getenv("COHERE_API_KEY");
EmbeddingFunction ef = new CohereEmbeddingFunction(apiKey);
Collection collection = client.createCollection("test-collection", null, true, ef);
List<Map<String, String>> metadata = new ArrayList<>();
metadata.add(new HashMap<String, String>() {{
put("type", "scientist");
}});
metadata.add(new HashMap<String, String>() {{
put("type", "spy");
}});
collection.add(null, metadata, Arrays.asList("Hello, my name is John. I am a Data Scientist.", "Hello, my name is Bond. I am a Spy."), Arrays.asList("1", "2"));
Collection.QueryResponse qr = collection.query(Arrays.asList("Who is the spy"), 10, null, null, null);
System.out.println(qr);
} catch (Exception e) {
e.printStackTrace();
System.out.println(e);
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
client.reset();
String apiKey = System.getenv("COHERE_API_KEY");
EmbeddingFunction ef = new CohereEmbeddingFunction(apiKey);
Collection collection = client.createCollection("test-collection", null, true, ef);
List<Map<String, String>> metadata = new ArrayList<>();
metadata.add(new HashMap<String, String>() {{
put("type", "scientist");
}});
metadata.add(new HashMap<String, String>() {{
put("type", "spy");
}});
collection.add(null, metadata, Arrays.asList("Hello, my name is John. I am a Data Scientist.", "Hello, my name is Bond. I am a Spy."), Arrays.asList("1", "2"));
Collection.QueryResponse qr = collection.query(Arrays.asList("Who is the spy"), 10, null, null, null);
System.out.println(qr);
} catch (Exception e) {
e.printStackTrace();
System.out.println(e);
}
}
}
}
```

Expand All @@ -176,8 +218,6 @@ The above should output:
{"documents":[["Hello, my name is Bond. I am a Spy.","Hello, my name is John. I am a Data Scientist."]],"ids":[["2","1"]],"metadatas":[[{"type":"spy"},{"type":"scientist"}]],"distances":[[5112.614,10974.804]]}
```



### Example Hugging Face Sentence Transformers Embedding Function

In this example we rely on `tech.amikos.chromadb.HuggingFaceEmbeddingFunction` to generate embeddings for our documents.
Expand All @@ -193,27 +233,27 @@ import tech.amikos.chromadb.Collection;
import java.util.*;

public class Main {
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
client.reset();
String apiKey = System.getenv("HF_API_KEY");
EmbeddingFunction ef = new HuggingFaceEmbeddingFunction(apiKey);
Collection collection = client.createCollection("test-collection", null, true, ef);
List<Map<String, String>> metadata = new ArrayList<>();
metadata.add(new HashMap<String, String>() {{
put("type", "scientist");
}});
metadata.add(new HashMap<String, String>() {{
put("type", "spy");
}});
collection.add(null, metadata, Arrays.asList("Hello, my name is John. I am a Data Scientist.", "Hello, my name is Bond. I am a Spy."), Arrays.asList("1", "2"));
Collection.QueryResponse qr = collection.query(Arrays.asList("Who is the spy"), 10, null, null, null);
System.out.println(qr);
} catch (Exception e) {
System.out.println(e);
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
client.reset();
String apiKey = System.getenv("HF_API_KEY");
EmbeddingFunction ef = new HuggingFaceEmbeddingFunction(apiKey);
Collection collection = client.createCollection("test-collection", null, true, ef);
List<Map<String, String>> metadata = new ArrayList<>();
metadata.add(new HashMap<String, String>() {{
put("type", "scientist");
}});
metadata.add(new HashMap<String, String>() {{
put("type", "spy");
}});
collection.add(null, metadata, Arrays.asList("Hello, my name is John. I am a Data Scientist.", "Hello, my name is Bond. I am a Spy."), Arrays.asList("1", "2"));
Collection.QueryResponse qr = collection.query(Arrays.asList("Who is the spy"), 10, null, null, null);
System.out.println(qr);
} catch (Exception e) {
System.out.println(e);
}
}
}
}
```

Expand All @@ -223,6 +263,44 @@ The above should output:
{"documents":[["Hello, my name is Bond. I am a Spy.","Hello, my name is John. I am a Data Scientist."]],"ids":[["2","1"]],"metadatas":[[{"type":"spy"},{"type":"scientist"}]],"distances":[[0.9073759,1.6440368]]}
```

### Ollama Embedding Function

In this example we rely on `tech.amikos.chromadb.embeddings.ollama.OllamaEmbeddingFunction` to generate embeddings for
our documents.

```java
package tech.amikos;

import tech.amikos.chromadb.*;
import tech.amikos.chromadb.embeddings.ollama.OllamaEmbeddingFunction;
import tech.amikos.chromadb.Collection;

import java.util.*;

public class Main {
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
client.reset();
EmbeddingFunction ef = new OllamaEmbeddingFunction();
Collection collection = client.createCollection("test-collection", null, true, ef);
List<Map<String, String>> metadata = new ArrayList<>();
metadata.add(new HashMap<String, String>() {{
put("type", "scientist");
}});
metadata.add(new HashMap<String, String>() {{
put("type", "spy");
}});
collection.add(null, metadata, Arrays.asList("Hello, my name is John. I am a Data Scientist.", "Hello, my name is Bond. I am a Spy."), Arrays.asList("1", "2"));
Collection.QueryResponse qr = collection.query(Arrays.asList("Who is the spy"), 10, null, null, null);
System.out.println(qr);
} catch (Exception e) {
System.out.println(e);
}
}
}
```

### Example Auth

> Note: This is a workaround until the client overhaul is completed
Expand All @@ -238,18 +316,18 @@ import tech.amikos.chromadb.Collection;
import java.util.*;

public class Main {
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
String encodedString = Base64.getEncoder().encodeToString("admin:admin".getBytes());
client.setDefaultHeaders(new HashMap<>() {{
put("Authorization", "Basic " + encodedString);
}});
// your code here
} catch (Exception e) {
System.out.println(e);
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
String encodedString = Base64.getEncoder().encodeToString("admin:admin".getBytes());
client.setDefaultHeaders(new HashMap<>() {{
put("Authorization", "Basic " + encodedString);
}});
// your code here
} catch (Exception e) {
System.out.println(e);
}
}
}
}
```

Expand All @@ -264,18 +342,18 @@ import tech.amikos.chromadb.Collection;
import java.util.*;

public class Main {
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
String encodedString = Base64.getEncoder().encodeToString("admin:admin".getBytes());
client.setDefaultHeaders(new HashMap<>() {{
put("Authorization", "Bearer test-token");
}});
// your code here
} catch (Exception e) {
System.out.println(e);
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
String encodedString = Base64.getEncoder().encodeToString("admin:admin".getBytes());
client.setDefaultHeaders(new HashMap<>() {{
put("Authorization", "Bearer test-token");
}});
// your code here
} catch (Exception e) {
System.out.println(e);
}
}
}
}
```

Expand All @@ -290,18 +368,18 @@ import tech.amikos.chromadb.Collection;
import java.util.*;

public class Main {
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
String encodedString = Base64.getEncoder().encodeToString("admin:admin".getBytes());
client.setDefaultHeaders(new HashMap<>() {{
put("X-Chroma-Token", "test-token");
}});
// your code here
} catch (Exception e) {
System.out.println(e);
public static void main(String[] args) {
try {
Client client = new Client(System.getenv("CHROMA_URL"));
String encodedString = Base64.getEncoder().encodeToString("admin:admin".getBytes());
client.setDefaultHeaders(new HashMap<>() {{
put("X-Chroma-Token", "test-token");
}});
// your code here
} catch (Exception e) {
System.out.println(e);
}
}
}
}
```

Expand Down
3 changes: 1 addition & 2 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<name>Chroma Vector DB Java Client Library</name>
<groupId>io.github.amikos-tech</groupId>
<artifactId>chromadb-java-client</artifactId>
<version>0.1.5</version>
<version>0.1.6</version>
<description>
Chroma Vector DB Java Client
</description>
Expand All @@ -27,7 +27,6 @@
<name>Amikos Tech OOD</name>
<url>https://amikos.tech</url>
</organization>

<scm>
<connection>scm:git:git://github.com/amikos-tech/chromadb-java-client.git</connection>
<developerConnection>scm:git:ssh://[email protected]:amikos-tech/chromadb-java-client.git</developerConnection>
Expand Down
Loading