Skip to content

Commit 5c695de

Browse files
whiteghostDevfelix
whiteghostDev
and
felix
committed
Initial 1.0.0 Java-Bindings PR/release (#805)
* Initial 1.0.0 Java-Bindings PR/release * Initial 1.1.0 Java-Bindings PR/release * Add debug ability * 1.1.2 release --------- Co-authored-by: felix <[email protected]>
1 parent 165020e commit 5c695de

File tree

12 files changed

+1090
-0
lines changed

12 files changed

+1090
-0
lines changed

‎README.md

+1
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ Find the most up-to-date information on the [GPT4All Website](https://gpt4all.io
6161
* <a href="https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-bindings/typescript">:computer: Official Typescript Bindings</a>
6262
* <a href="https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-bindings/golang">:computer: Official GoLang Bindings</a>
6363
* <a href="https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-bindings/csharp">:computer: Official C# Bindings</a>
64+
* <a href="https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-bindings/java">:computer: Official Java Bindings</a>
6465

6566

6667
## Contributing

‎gpt4all-bindings/java/.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Make sure native directory never gets commited to git for the project.
2+
/src/main/resources/native

‎gpt4all-bindings/java/README.md

+105
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# Java bindings
2+
3+
Java bindings let you load a gpt4all library into your Java application and execute text
4+
generation using an intuitive and easy to use API. No GPU is required because gpt4all system executes on the cpu.
5+
The gpt4all models are quantized to easily fit into system RAM and use about 4 to 7GB of system RAM.
6+
7+
## Getting Started
8+
You can add Java bindings into your Java project by adding dependency to your project:
9+
10+
**Maven**
11+
```
12+
<dependency>
13+
<groupId>com.hexadevlabs</groupId>
14+
<artifactId>gpt4all-java-binding</artifactId>
15+
<version>1.1.2</version>
16+
</dependency>
17+
```
18+
**Gradle**
19+
```
20+
implementation 'com.hexadevlabs:gpt4all-java-binding:1.1.2'
21+
```
22+
23+
To add the library dependency for another build system see [Maven Central Java bindings](https://central.sonatype.com/artifact/com.hexadevlabs/gpt4all-java-binding/).
24+
25+
To download a model binary weights file use an url such as https://gpt4all.io/models/ggml-gpt4all-j-v1.3-groovy.bin.
26+
27+
For information about other models available see [Model file list](https://github.com/nomic-ai/gpt4all/tree/main/gpt4all-chat#manual-download-of-models).
28+
29+
### Sample code
30+
```java
31+
public class Example {
32+
public static void main(String[] args) {
33+
34+
String prompt = "### Human:\nWhat is the meaning of life\n### Assistant:";
35+
36+
// Replace the hardcoded path with the actual path where your model file resides
37+
String modelFilePath = "C:\\Users\\felix\\AppData\\Local\\nomic.ai\\GPT4All\\ggml-gpt4all-j-v1.3-groovy.bin";
38+
39+
try (LLModel model = new LLModel(Path.of(modelFilePath))) {
40+
41+
// May generate up to 4096 tokens but generally stops early
42+
LLModel.GenerationConfig config = LLModel.config()
43+
.withNPredict(4096).build();
44+
45+
// Will also stream to Standard out
46+
String fullGeneration = model.generate(prompt, config, true);
47+
48+
} catch (Exception e) {
49+
// Exception generally may happen if model file fails to load
50+
// for a number of reasons such as file not found.
51+
// It is possible that Java may not be able to dynamically load the native shared library or
52+
// the llmodel shared library may not be able to dynamically load the backend
53+
// implementation for the model file you provided.
54+
//
55+
// Once the LLModel class is successfully loaded into memory the text generation calls
56+
// generally should not throw exceptions.
57+
e.printStackTrace(); // Printing here but in production system you may want to take some action.
58+
}
59+
}
60+
61+
}
62+
```
63+
64+
For a maven based sample project that uses this library see [Sample project](https://github.com/felix-zaslavskiy/gpt4all-java-bindings-sample)
65+
66+
### Additional considerations
67+
#### Logger warnings
68+
The Java bindings library may produce a warning:
69+
```
70+
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
71+
SLF4J: Defaulting to no-operation (NOP) logger implementation
72+
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
73+
```
74+
If you don't have a SLF4J binding included in your project. Java bindings only use logging for informational
75+
purposes, so logger is not essential to correctly use the library. You can ignore this warning if you don't have SLF4J bindings
76+
in your project.
77+
78+
To add a simple logger using maven dependency you may use:
79+
```
80+
<dependency>
81+
<groupId>org.slf4j</groupId>
82+
<artifactId>slf4j-simple</artifactId>
83+
<version>1.7.36</version>
84+
</dependency>
85+
```
86+
87+
#### Loading your native libraries
88+
1. Java bindings package jar comes bundled with native library files for Windows, macOS and Linux. These library files are
89+
copied to a temporary directory and loaded at runtime. For advanced users who may want to package shared libraries into Docker containers
90+
or want to use a custom build of the shared libraries and ignore the once bundled with the java package they have option
91+
to load libraries from your local directory by setting a static property to the location of library files.
92+
There are no guarantees of compatibility if used in such a way so be careful if you really want to do it.
93+
94+
For example:
95+
```java
96+
class Example {
97+
public static void main(String[] args) {
98+
// gpt4all native shared libraries location
99+
LLModel.LIBRARY_SEARCH_PATH = "C:\\Users\\felix\\gpt4all\\lib\\";
100+
// ... use the library normally
101+
}
102+
}
103+
```
104+
2. Not every avx only shared library is bundled with the jar right now to reduce size. Only the libgptj-avx is included.
105+
If you are running into issues please let us know using the gpt4all project issue tracker https://github.com/nomic-ai/gpt4all/issues.

‎gpt4all-bindings/java/pom.xml

+202
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,202 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
5+
<modelVersion>4.0.0</modelVersion>
6+
7+
<groupId>com.hexadevlabs</groupId>
8+
<artifactId>gpt4all-java-binding</artifactId>
9+
<version>1.1.2</version>
10+
<packaging>jar</packaging>
11+
12+
<properties>
13+
<maven.compiler.source>11</maven.compiler.source>
14+
<maven.compiler.target>11</maven.compiler.target>
15+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
16+
</properties>
17+
18+
<name>${project.groupId}:${project.artifactId}</name>
19+
<description>Java bindings for GPT4ALL LLM</description>
20+
<url>https://github.com/nomic-ai/gpt4all</url>
21+
<licenses>
22+
<license>
23+
<name>The Apache License, Version 2.0</name>
24+
<url>https://github.com/nomic-ai/gpt4all/blob/main/LICENSE.txt</url>
25+
</license>
26+
</licenses>
27+
<developers>
28+
<developer>
29+
<name>Felix Zaslavskiy</name>
30+
<email>[email protected]</email>
31+
<organizationUrl>https://github.com/felix-zaslavskiy/</organizationUrl>
32+
</developer>
33+
</developers>
34+
<scm>
35+
<connection>scm:git:git://github.com/nomic-ai/gpt4all.git</connection>
36+
<developerConnection>scm:git:ssh://github.com/nomic-ai/gpt4all.git</developerConnection>
37+
<url>https://github.com/nomic-ai/gpt4all/tree/main</url>
38+
</scm>
39+
40+
<dependencies>
41+
<dependency>
42+
<groupId>com.github.jnr</groupId>
43+
<artifactId>jnr-ffi</artifactId>
44+
<version>2.2.13</version>
45+
</dependency>
46+
47+
<dependency>
48+
<groupId>org.slf4j</groupId>
49+
<artifactId>slf4j-api</artifactId>
50+
<version>1.7.36</version>
51+
</dependency>
52+
53+
<dependency>
54+
<groupId>org.junit.jupiter</groupId>
55+
<artifactId>junit-jupiter-api</artifactId>
56+
<version>5.9.2</version>
57+
<scope>test</scope>
58+
</dependency>
59+
</dependencies>
60+
61+
<distributionManagement>
62+
<snapshotRepository>
63+
<id>ossrh</id>
64+
<url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
65+
</snapshotRepository>
66+
<repository>
67+
<id>ossrh</id>
68+
<url>https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/</url>
69+
</repository>
70+
</distributionManagement>
71+
72+
<build>
73+
<resources>
74+
<resource>
75+
<directory>src/main/resources</directory>
76+
</resource>
77+
<resource>
78+
<directory>${project.build.directory}/generated-resources</directory>
79+
</resource>
80+
</resources>
81+
<plugins>
82+
<plugin>
83+
<groupId>org.apache.maven.plugins</groupId>
84+
<artifactId>maven-surefire-plugin</artifactId>
85+
<version>3.0.0</version>
86+
<configuration>
87+
<forkCount>0</forkCount>
88+
</configuration>
89+
</plugin>
90+
<plugin>
91+
<groupId>org.apache.maven.plugins</groupId>
92+
<artifactId>maven-resources-plugin</artifactId>
93+
<version>3.3.1</version>
94+
<executions>
95+
<execution>
96+
<id>copy-resources</id>
97+
<!-- Here the phase you need -->
98+
<phase>validate</phase>
99+
<goals>
100+
<goal>copy-resources</goal>
101+
</goals>
102+
<configuration>
103+
<outputDirectory>${project.build.directory}/generated-resources</outputDirectory>
104+
<resources>
105+
<resource>
106+
<directory>C:\Users\felix\dev\gpt4all_java_bins\release_1_1_1_Jun8_2023</directory>
107+
</resource>
108+
</resources>
109+
</configuration>
110+
</execution>
111+
</executions>
112+
</plugin>
113+
114+
115+
<plugin>
116+
<groupId>org.sonatype.plugins</groupId>
117+
<artifactId>nexus-staging-maven-plugin</artifactId>
118+
<version>1.6.13</version>
119+
<extensions>true</extensions>
120+
<configuration>
121+
<serverId>ossrh</serverId>
122+
<nexusUrl>https://s01.oss.sonatype.org/</nexusUrl>
123+
<autoReleaseAfterClose>true</autoReleaseAfterClose>
124+
</configuration>
125+
</plugin>
126+
<plugin>
127+
<groupId>org.apache.maven.plugins</groupId>
128+
<artifactId>maven-source-plugin</artifactId>
129+
<version>2.2.1</version>
130+
<executions>
131+
<execution>
132+
<id>attach-sources</id>
133+
<goals>
134+
<goal>jar-no-fork</goal>
135+
</goals>
136+
</execution>
137+
</executions>
138+
</plugin>
139+
<plugin>
140+
<groupId>org.apache.maven.plugins</groupId>
141+
<artifactId>maven-javadoc-plugin</artifactId>
142+
<version>3.5.0</version>
143+
<executions>
144+
<execution>
145+
<id>attach-javadocs</id>
146+
<goals>
147+
<goal>jar</goal>
148+
</goals>
149+
</execution>
150+
</executions>
151+
</plugin>
152+
<plugin>
153+
<groupId>org.apache.maven.plugins</groupId>
154+
<artifactId>maven-gpg-plugin</artifactId>
155+
<version>1.5</version>
156+
<executions>
157+
<execution>
158+
<id>sign-artifacts</id>
159+
<phase>verify</phase>
160+
<goals>
161+
<goal>sign</goal>
162+
</goals>
163+
<!--
164+
<configuration>
165+
<keyname>${gpg.keyname}</keyname>
166+
<passphraseServerId>${gpg.keyname}</passphraseServerId>
167+
</configuration>
168+
-->
169+
</execution>
170+
</executions>
171+
</plugin>
172+
<plugin>
173+
<groupId>org.apache.maven.plugins</groupId>
174+
<artifactId>maven-assembly-plugin</artifactId>
175+
<version>3.6.0</version>
176+
<configuration>
177+
<descriptorRefs>
178+
<descriptorRef>jar-with-dependencies</descriptorRef>
179+
</descriptorRefs>
180+
<archive>
181+
<manifest>
182+
<mainClass>com.hexadevlabs.gpt4allsample.Example4</mainClass>
183+
</manifest>
184+
</archive>
185+
</configuration>
186+
<executions>
187+
<execution>
188+
<id>make-assembly</id>
189+
<phase>package</phase>
190+
<goals>
191+
<goal>single</goal>
192+
</goals>
193+
</execution>
194+
</executions>
195+
</plugin>
196+
</plugins>
197+
198+
199+
</build>
200+
201+
202+
</project>

0 commit comments

Comments
 (0)