Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic LLM Chain node calling out a model is extremly slow #13863

Open
gustaff-weldon opened this issue Mar 12, 2025 · 1 comment
Open

Basic LLM Chain node calling out a model is extremly slow #13863

gustaff-weldon opened this issue Mar 12, 2025 · 1 comment
Labels
in linear Issue or PR has been created in Linear for internal review

Comments

@gustaff-weldon
Copy link

gustaff-weldon commented Mar 12, 2025

Bug Description

We have noticed that using Basic LLM Chain node with a model (in this case Gemini 2.0) passing down binary image for vision analysis is extremely slow. Ie. during execution a lot time is spent in llm chain node.
For comparison we have done a rest call to Gemini REST api, with the very same set of images and the results are very unfavourable for llm chain + model nodes combo.

Image

Time to process 2 images for llm chain + gemini node:

Basic LLM Chain node: 35181ms
Model node: 4790ms

Image Image

Time spent on HTTP request node uploading the very same 2 images:

HTTP Request node: 5062ms

Image

Something is really fishy here, both use the same system prompt asking LLM to extract markdown from a given image of document.

To Reproduce

Use included workflow to process couple of invoice images and attached test invoices.

Obviously, I cannot share the documents we are processing, but sample images show 3-7x more times spent in llm chain + model nodes combo vs http request for the same images.

Extract_markdown_from_an_image.json

Image
Image
Image
Image

Expected behavior

We expected llm chain node to be equally performant to the direct REST api call to LLM API. In our tests, it is multiple times slower.

Operating System

ubuntu

n8n Version

1.80.4

Node.js Version

whatever n8n docker image comes with

Database

PostgreSQL

Execution mode

main (default)

@Joffcom
Copy link
Member

Joffcom commented Mar 12, 2025

Hey @gustaff-weldon,

We have created an internal ticket to look into this which we will be tracking as "GHC-1162"

@Joffcom Joffcom added the in linear Issue or PR has been created in Linear for internal review label Mar 12, 2025
@gustaff-weldon gustaff-weldon changed the title Lang chain node calling out a model is extremly slow Basic LLM Chain node calling out a model is extremly slow Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in linear Issue or PR has been created in Linear for internal review
Projects
None yet
Development

No branches or pull requests

2 participants