AIDevMonster
diff --git a/‎.codespellrc
+1-1 b/‎.codespellrc
+1-1
diff --git a/‎gpt4all-bindings/typescript/.gitignore
+1 b/‎gpt4all-bindings/typescript/.gitignore
+1
diff --git a/‎gpt4all-bindings/typescript/.npmignore
+2-1 b/‎gpt4all-bindings/typescript/.npmignore
+2-1
diff --git a/‎gpt4all-bindings/typescript/README.md
+45-3 b/‎gpt4all-bindings/typescript/README.md
+45-3
diff --git a/‎gpt4all-bindings/typescript/binding.gyp
+29-19 b/‎gpt4all-bindings/typescript/binding.gyp
+29-19
@@ -1,3 +1,3 @@
 [codespell]
 ignore-words-list = blong, belong
-skip = .git,*.pdf,*.svg
+skip = .git,*.pdf,*.svg,*.lock
@@ -1,2 +1,3 @@
 node_modules/
 build/
+prebuilds/
@@ -1,3 +1,4 @@
 test/
 spec/
-
+scripts/
+build
@@ -2,12 +2,32 @@
 The original [GPT4All typescript bindings](https://github.com/nomic-ai/gpt4all-ts) are now out of date.
 
 - created by [jacoobes](https://github.com/jacoobes) and [nomic ai](https://home.nomic.ai) :D, for all to use.
-- will maintain this repository when possible, new feature requests will be handled through nomic
 
+### Code (alpha)
+```js
+import { LLModel, createCompletion, DEFAULT_DIRECTORY, DEFAULT_LIBRARIES_DIRECTORY } from '../src/gpt4all.js'
+
+const ll = new LLModel({
+    model_name: 'ggml-vicuna-7b-1.1-q4_2.bin',
+    model_path: './', 
+    library_path: DEFAULT_LIBRARIES_DIRECTORY
+});
+
+const response = await createCompletion(ll, [
+    { role : 'system', content: 'You are meant to be annoying and unhelpful.'  },
+    { role : 'user', content: 'What is 1 + 1?'  } 
+]);
+
+```
+### API 
+- The nodejs api has made strides to mirror the python api. It is not 100% mirrored, but many pieces of the api resemble its python counterpart.
+- [docs](./docs/api.md)
 ### Build Instructions
 
-- As of 05/21/2023, Tested on windows (MSVC) only. (somehow got it to work on MSVC 🤯)
+- As of 05/21/2023, Tested on windows (MSVC). (somehow got it to work on MSVC 🤯)
     - binding.gyp is compile config
+- Tested on Ubuntu. Everything seems to work fine
+- MingW works as well to build the gpt4all-backend. HOWEVER, this package works only with MSVC built dlls.
 
 ### Requirements
 - git
@@ -31,6 +51,15 @@ cd gpt4all-bindings/typescript
  ```sh
 git submodule update --init --depth 1 --recursive
  ```
+**AS OF NEW BACKEND** to build the backend,
+```sh
+yarn build:backend
+```
+This will build platform-dependent dynamic libraries, and will be located in runtimes/(platform)/native The only current way to use them is to put them in the current working directory of your application. That is, **WHEREVER YOU RUN YOUR NODE APPLICATION**
+- llama-xxxx.dll is required.
+- According to whatever model you are using, you'll need to select the proper model loader.
+    - For example, if you running an Mosaic MPT model, you will need to select the mpt-(buildvariant).(dynamiclibrary)
+
 ### Test
 ```sh
 yarn test
@@ -48,9 +77,22 @@ yarn test
 
 #### spec/
 - Average look and feel of the api
-- Should work assuming a model is installed locally in working directory
+- Should work assuming a model and libraries are installed locally in working directory
 
 #### index.cc
 - The bridge between nodejs and c. Where the bindings are.
+#### prompt.cc 
+- Handling prompting and inference of models in a threadsafe, asynchronous way.
+#### docs/
+- Autogenerated documentation using the script `yarn docs:build`
+
+### Roadmap
+This package is in active development, and breaking changes may happen until the api stabilizes. Here's what's the todo list:
 
+- [x] prompt models via a threadsafe function in order to have proper non blocking behavior in nodejs
+- [ ] createTokenStream, an async iterator that streams each token emitted from the model. Planning on following this [example](https://github.com/nodejs/node-addon-examples/tree/main/threadsafe-async-iterator)
+- [ ] proper unit testing (integrate with circle ci)
+- [ ] publish to npm under alpha tag `gpt4all@alpha`
+- [ ] have more people test on other platforms (mac tester needed)
+- [x] switch to new pluggable backend
 
@@ -1,45 +1,55 @@
 {
   "targets": [
     {
-      "target_name": "gpt4allts", # gpt4all-ts will cause compile error
-      "cflags!": [ "-fno-exceptions" ],
-      "cflags_cc!": [ "-fno-exceptions" ],
+      "target_name": "gpt4all", # gpt4all-ts will cause compile error
+      "cflags_cc!": [ "-fno-exceptions"],
       "include_dirs": [
         "<!@(node -p \"require('node-addon-api').include\")",
-        "../../gpt4all-backend/llama.cpp/", # need to include llama.cpp because the include paths for examples/common.h include llama.h relatively
         "../../gpt4all-backend",
       ],
-      "sources": [ # is there a better way to do this 
-        "../../gpt4all-backend/llama.cpp/examples/common.cpp",
-        "../../gpt4all-backend/llama.cpp/ggml.c",
-        "../../gpt4all-backend/llama.cpp/llama.cpp",
-        "../../gpt4all-backend/utils.cpp", 
+      "sources": [ 
+        # PREVIOUS VERSION: had to required the sources, but with newest changes do not need to
+        #"../../gpt4all-backend/llama.cpp/examples/common.cpp",
+        #"../../gpt4all-backend/llama.cpp/ggml.c",
+        #"../../gpt4all-backend/llama.cpp/llama.cpp",
+        # "../../gpt4all-backend/utils.cpp", 
         "../../gpt4all-backend/llmodel_c.cpp",
-        "../../gpt4all-backend/gptj.cpp",
-        "../../gpt4all-backend/llamamodel.cpp",
-        "../../gpt4all-backend/mpt.cpp",
-        "stdcapture.cc",
+        "../../gpt4all-backend/llmodel.cpp",
+        "prompt.cc",
         "index.cc",
        ],
       "conditions": [
         ['OS=="mac"', {
             'defines': [
-                'NAPI_CPP_EXCEPTIONS'
-            ],
+                'LIB_FILE_EXT=".dylib"',
+                'NAPI_CPP_EXCEPTIONS',
+            ]
         }],
         ['OS=="win"', {
             'defines': [
+                'LIB_FILE_EXT=".dll"',
                 'NAPI_CPP_EXCEPTIONS',
-                "__AVX2__" # allows SIMD: https://discord.com/channels/1076964370942267462/1092290790388150272/1107564673957630023
             ],
             "msvs_settings": {
                 "VCCLCompilerTool": {
                     "AdditionalOptions": [
                         "/std:c++20",
-                        "/EHsc"
-                    ], 
-                },  
+                        "/EHsc",
+                  ], 
+                },
             },
+        }],
+        ['OS=="linux"', {
+            'defines': [
+                'LIB_FILE_EXT=".so"',
+                'NAPI_CPP_EXCEPTIONS',
+            ],
+            'cflags_cc!': [
+                '-fno-rtti',
+            ],
+            'cflags_cc': [
+                '-std=c++20'
+            ]
         }]
       ]
     }]
Original file line number	Diff line number	Diff line change
`@@ -1,2 +1,3 @@`
`1`	`1`	`node_modules/`
`2`	`2`	`build/`
	`3`	`+prebuilds/`
-Original file line number
+Diff line change
 test/
 spec/
+-
 +scripts/
 +build