Skip to content

Commit 9d053d6

Browse files
authored
Templates sometimes have BOS in them, remove duplicate
1 parent 5ab40e6 commit 9d053d6

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

llama_cpp/llama.py

+4
Original file line numberDiff line numberDiff line change
@@ -1016,6 +1016,10 @@ def _create_completion(
10161016
)
10171017
model_name: str = model if model is not None else self.model_path
10181018

1019+
# User or template may have added an unwanted extra BOS
1020+
if prompt_tokens[:2] == [self.token_bos()] * 2:
1021+
del prompt_tokens[0]
1022+
10191023
# NOTE: This likely doesn't work correctly for the first token in the prompt
10201024
# because of the extra space added to the start of the prompt_tokens
10211025
if logit_bias is not None:

0 commit comments

Comments
 (0)