How do i use llama.cpp as an API?

[Shreyas-ITB]

Hello! Im making a cool project that involves llama.cpp, but i need it in the form of an API to make the communication easier rather than catching the output of llama.cpp in the console. Is there any way to use this as an API?

[TusharAgey]

#1443 - Use this.

[Green-Sky]

llama.cpp provides a C API.
Not sure what your mean.

[mirek190]

what do you mean c api?

[Green-Sky]

what do you mean c api?

https://github.com/ggerganov/llama.cpp/blob/master/llama.h

[mirek190]

you mean server?

[Shreyas-ITB]

@TusharAgey thankyou very much, i have been looking for it for ages! :)