Code

Update to latest#1

Code

Merged

olexiyb merged 123 commits intosanctumSanctum-AI/llama.cpp:sanctumfrom

update-to-latestSanctum-AI/llama.cpp:update-to-latestCopy head branch name to clipboard

Nov 23, 2023

Commits

Commits on Oct 26, 2023

• server : do not release slot on image input (#3798)

  ggerganov

  committed
  34b2a5e

  Copy full SHA for 34b2a5e

Commits on Oct 27, 2023

• cuda : improve text-generation and batched decoding performance (#3776)

  Show description for 2f9ec7e
  ggerganov

  authored
  2f9ec7e

  Copy full SHA for 2f9ec7e

• simple : fix batch handling (#3803)

  tterrasson

  authored
  c8d6a1f

  Copy full SHA for c8d6a1f

• llama : correctly report GGUFv3 format (#3818)

  cebtenzzre

  authored
  6d459cb

  Copy full SHA for 6d459cb

• speculative : ensure draft and target model vocab matches (#3812)

  Show description for 41aee4d
  KerfuffleV2

  authored
  41aee4d

  Copy full SHA for 41aee4d

Commits on Oct 28, 2023

• starcoder : add GPU offloading (#3827)

  Show description for fdee152
  ggerganov

  authored
  fdee152

  Copy full SHA for fdee152

• common : print that one line of the syntax help *also* to standard output (#3823)

  HenkPoley

  authored
  1774611

  Copy full SHA for 1774611

• llama : add option for greedy sampling with probs (#3813)

  Show description for ee1a0ec
  ggerganov

  authored
  ee1a0ec

  Copy full SHA for ee1a0ec

• llama : allow quantizing k-quants to fall back when tensor size incompatible (#3747)

  Show description for bd6d9e2
  KerfuffleV2

  authored
  bd6d9e2

  Copy full SHA for bd6d9e2

• convert : ignore tokens if their IDs are within [0, vocab_size) (#3831)

  ggerganov

  authored
  8a2f2fe

  Copy full SHA for 8a2f2fe

• issues : change label from bug to bug-unconfirmed (#3748)

  ggerganov

  committed
  ba231e8

  Copy full SHA for ba231e8

• metal : try cwd for ggml-metal.metal if bundle lookup fails (#3793)

  Show description for 82a6646
  

  akx

  and

  ggerganov

  authored
  82a6646

  Copy full SHA for 82a6646

• flake : update flake.lock for newer transformers version + provide extra dev shell (#3797)

  Show description for ff3bad8
  Green-Sky

  authored
  ff3bad8

  Copy full SHA for ff3bad8

Commits on Oct 29, 2023

• ggml : quantization refactoring (#3833)

  Show description for d69d777
  

  ggerganov

  and

  cebtenzzre

  authored
  d69d777

  Copy full SHA for d69d777

• llama : fix kv shift bug (#3835)

  Show description for 71a09da
  ggerganov

  authored
  71a09da

  Copy full SHA for 71a09da

• make : remove unnecessary dependency on build-info.h (#3842)

  cebtenzzre

  authored
  2046eb4

  Copy full SHA for 2046eb4

• Extend llama_kv_cache_seq_rm to allow matching any sequence (#3843)

  Show description for 6e08281
  KerfuffleV2

  authored
  6e08281

  Copy full SHA for 6e08281

Commits on Oct 30, 2023

• ggml : move FP16 <-> FP32 code to ggml-impl.h (#3861)

  Show description for 207b519
  ggerganov

  authored
  207b519

  Copy full SHA for 207b519

Commits on Oct 31, 2023

• flake.nix: fix for rocm 5.7 (#3853)

  Tungsten842

  authored
  07178c9

  Copy full SHA for 07178c9

• samplers : Min-P sampler implementation [alternative to Top P/Top K] (#3841)

  Show description for 238657d
  

  3 peopleauthored
  238657d

  Copy full SHA for 238657d

Commits on Nov 1, 2023

• llama : refactor graph build code (#3837)

  Show description for 71e3718
  ggerganov

  authored
  71e3718

  Copy full SHA for 71e3718

• server : re-enable completion and embedded at the same time (#3876)

  a-h

  authored
  ca190bc

  Copy full SHA for ca190bc

• scripts : add server-llm.sh (#3868)

  Show description for f0e2093
  ggerganov

  authored
  f0e2093

  Copy full SHA for f0e2093

• finetune : add -ngl parameter (#3762)

  Show description for 73bdcb3
  

  3 peopleauthored
  73bdcb3

  Copy full SHA for 73bdcb3

• ggml : fix UNUSED macro (#3762)

  ggerganov

  committed
  9a3b4f6

  Copy full SHA for 9a3b4f6

• sampling : null grammar field after reset (#3885)

  l3utterfly

  authored
  e75dfdd

  Copy full SHA for e75dfdd

• log : make generating separate log files optional (#3787)

  Show description for a2758d0
  

  staviq

  and

  cebtenzzre

  authored
  a2758d0

  Copy full SHA for a2758d0

• common : allow caller to handle help/argument exceptions (#3715)

  Show description for 0e40806
  

  bandoti

  and

  ggerganov

  authored
  0e40806

  Copy full SHA for 0e40806

• llm : add llm_build_context (#3881)

  Show description for 5033796
  ggerganov

  authored
  5033796

  Copy full SHA for 5033796

• common : minor (#3715)

  ggerganov

  committed
  ff8f9a8

  Copy full SHA for ff8f9a8

• metal : multi-simd softmax (#3710)

  Show description for e16b9fa
  ggerganov

  authored
  e16b9fa

  Copy full SHA for e16b9fa

• llm : fix falcon norm after refactoring (#3837)

  ggerganov

  committed
  523e49b

  Copy full SHA for 523e49b

• llm : fix llm_build_kqv taking unused tensor (benign, #3837)

  ggerganov

  committed
  c43c2da

  Copy full SHA for c43c2da

• llama : implement YaRN RoPE scaling (#2268)

  Show description for 898aeca
  

  cebtenzzre

  and

  jquesnelle

  authored
  898aeca

  Copy full SHA for 898aeca

• ggml-cuda : compute ptrs for cublasGemmBatchedEx in a kernel (#3891)

  Show description for d02e98c
  slaren

  authored
  d02e98c

  Copy full SHA for d02e98c

• llama : fix llama_context_default_params after #2268 (#3893)

  cebtenzzre

  authored
  0eb332a

  Copy full SHA for 0eb332a

Commits on Nov 2, 2023

• cuda : fix RoPE after #2268 (#3897)

  cebtenzzre

  authored
  2fffa0d

  Copy full SHA for 2fffa0d

• metal : fix build errors and kernel sig after #2268 (#3898)

  ggerganov

  authored
  183b3fa

  Copy full SHA for 183b3fa

• cuda : check if this fixes Pascal card regression (#3882)

  ggerganov

  authored
  4d719a6

  Copy full SHA for 4d719a6

• build : link against build info instead of compiling against it (#3879)

  Show description for b12fa0d
  

  cebtenzzre

  and

  kassane

  authored
  b12fa0d

  Copy full SHA for b12fa0d

• llm : prevent from 1-D tensors being GPU split (#3697)

  ggerganov

  committed
  1efae9b

  Copy full SHA for 1efae9b

• gguf : remove special-case code for GGUFv1 (#3901)

  Show description for 2756c4f
  ggerganov

  authored
  2756c4f

  Copy full SHA for 2756c4f

• cmake : disable LLAMA_NATIVE by default (#3906)

  slaren

  authored
  21958bb

  Copy full SHA for 21958bb

• gguf : print error for GGUFv1 files (#3908)

  ggerganov

  authored
  4ff1046

  Copy full SHA for 4ff1046

• cuda : use CUDA memory pool with async memory allocation/deallocation when available (#3903)

  Show description for d606905
  

  young-developer

  and

  Oleksii Maryshchenko

  authored
  d606905

  Copy full SHA for d606905

• cuda : fix const ptrs warning causing ROCm build issues (#3913)

  ggerganov

  authored
  c7743fe

  Copy full SHA for c7743fe

• readme : add notice about #3912

  ggerganov

  authored
  224e7d5

  Copy full SHA for 224e7d5

• cmake : fix relative path to git submodule index (#3915)

  abetlen

  authored
  51b2fc1

  Copy full SHA for 51b2fc1

• cuda : add ROCM aliases for CUDA pool stuff (#3918)

  KerfuffleV2

  authored
  629f917

  Copy full SHA for 629f917

Commits on Nov 3, 2023

• llama : change yarn_ext_factor placeholder to -1 (#3922)

  cebtenzzre

  authored
  3fdbe6b

  Copy full SHA for 3fdbe6b

• common : YAYF (yet another YARN fix) (#3925)

  Show description for 0581602
  ggerganov

  authored
  0581602

  Copy full SHA for 0581602

• speculative : change default p_accept to 0.5 + CLI args (#3919)

  Show description for 8f961ab
  ggerganov

  committed
  8f961ab

  Copy full SHA for 8f961ab

• ggml-cuda : move row numbers to x grid dim in mmv kernels (#3921)

  slaren

  authored
  abb77e7

  Copy full SHA for abb77e7

• ggml-metal: fix yarn rope (#3937)

  jxy

  authored
  5ba3746

  Copy full SHA for 5ba3746

• metal : round up to 16 to fix MTLDebugComputeCommandEncoder assertion (#3938)

  psugihara

  authored
  d9b33fe

  Copy full SHA for d9b33fe

Commits on Nov 4, 2023

• gguf-py: Support 01.AI Yi models (#3943)

  KerfuffleV2

  authored
  f28af0d

  Copy full SHA for f28af0d

Commits on Nov 5, 2023

• cuda : revert CUDA pool stuff (#3944)

  Show description for 48ade94
  slaren

  authored
  48ade94

  Copy full SHA for 48ade94

• ci : use intel sde when ci cpu doesn't support avx512 (#3949)

  netrunnereve

  authored
  a7fac01

  Copy full SHA for a7fac01

• cmake : MSVC instruction detection (fixed up #809) (#3923)

  Show description for c41ea36
  

  3 peopleauthored
  c41ea36

  Copy full SHA for c41ea36

• llama : mark LLM_ARCH_STARCODER as full offload supported (#3945)

  Show description for 3d48f42
  wsxiaoys

  authored
  3d48f42

  Copy full SHA for 3d48f42

• cuda : fix disabling device with --tensor-split 1,0 (#3951)

  Show description for 132d25b
  

  cebtenzzre

  and

  slaren

  authored
  132d25b

  Copy full SHA for 132d25b

• server : fix typo for --alias shortcut from -m to -a (#3958)

  RoyalHeart

  authored
  bb60fd0

  Copy full SHA for bb60fd0

• Allow common process_escapes to handle \x sequences (#3928)

  Show description for d9ccce2
  KerfuffleV2

  authored
  d9ccce2

  Copy full SHA for d9ccce2

• ggml-cuda : fix f16 mul mat (#3961)

  Show description for 2833a6f
  slaren

  authored
  2833a6f

  Copy full SHA for 2833a6f

Commits on Nov 6, 2023

• llava : expose as a shared library for downstream projects (#3613)

  Show description for 381efbf
  

  3 peopleauthored
  381efbf

  Copy full SHA for 381efbf

Commits on Nov 7, 2023

• cuda : supports running on CPU for GGML_USE_CUBLAS=ON build (#3946)

  Show description for 46876d2
  wsxiaoys

  authored
  46876d2

  Copy full SHA for 46876d2

• Use params when loading models in llava-cli (#3976)

  Show description for 54b4df8
  tejom

  authored
  54b4df8

  Copy full SHA for 54b4df8

• ggml : fix backward rope after YaRN (#3974)

  Show description for e9c1cec
  xaedes

  authored
  e9c1cec

  Copy full SHA for e9c1cec

• make : do not add linker flags when compiling static llava lib (#3977)

  ggerganov

  authored
  413503d

  Copy full SHA for 413503d

• gguf : track writer state, free unneeded tensors, cleanup (#3871)

  cebtenzzre

  authored
  0a7c980

  Copy full SHA for 0a7c980

Commits on Nov 8, 2023

• ggml-alloc : fix backend assignments of views (#3982)

  slaren

  authored
  875fb42

  Copy full SHA for 875fb42

Commits on Nov 9, 2023

• server : add min_p param (#3877)

  Show description for 57ad015
  Mihaiii

  authored
  57ad015

  Copy full SHA for 57ad015

• scripts: Generalize convert scripts (#3838)

  Show description for a75fa57
  Galunid

  authored
  a75fa57

  Copy full SHA for a75fa57

Commits on Nov 10, 2023

• Unbreak persimmon after #3837 (#4010)

  Galunid

  authored
  df9d129

  Copy full SHA for df9d129

• server : allow continue edit on completion mode (#3950)

  Show description for 4a4fd3e
  jhen0409

  authored
  4a4fd3e

  Copy full SHA for 4a4fd3e

Commits on Nov 11, 2023

• gguf-py: Refactor and allow reading/modifying existing GGUF files (#3981)

  Show description for 34b0a08
  

  KerfuffleV2

  and

  cebtenzzre

  authored
  34b0a08

  Copy full SHA for 34b0a08

• server : fix crash when prompt exceeds context size (#3996)

  z80maniac

  authored
  d96ca7d

  Copy full SHA for d96ca7d

• Fix gguf-convert-endian script (#4037)

  Show description for e86fc56
  monatis

  authored
  e86fc56

  Copy full SHA for e86fc56

Commits on Nov 12, 2023

• Fix some documentation typos/grammar mistakes (#4032)

  Show description for 532dd74
  

  richardkiss

  and

  KerfuffleV2

  authored
  532dd74

  Copy full SHA for 532dd74

• gguf-py: gguf_writer: Use bytearray to build metadata (#4051)

  Show description for 21fd874
  KerfuffleV2

  authored
  21fd874

  Copy full SHA for 21fd874

Commits on Nov 13, 2023

• Add ReLU and SQR CUDA ops to (partially) fix Persimmon offloading (#4041)

  Show description for bb50a79
  KerfuffleV2

  authored
  bb50a79

  Copy full SHA for bb50a79

• sync : ggml (backend v2) (#3912)

  Show description for 4760e7c
  ggerganov

  authored
  4760e7c

  Copy full SHA for 4760e7c

• readme : update hot topics

  ggerganov

  authored
  c049b37

  Copy full SHA for c049b37

• ggml : sync (im2col, GPU conv, 32-bit arm compat) (#4060)

  Show description for 3d68f36
  ggerganov

  authored
  3d68f36

  Copy full SHA for 3d68f36

• llava : fix regression for square images in #3613 (#4056)

  monatis

  authored
  bd90eca

  Copy full SHA for bd90eca

Commits on Nov 14, 2023

• convert.py: also look for plain model.safetensors (#4043)

  Show description for b46d12f
  afrideva

  authored
  b46d12f

  Copy full SHA for b46d12f

• stablelm : StableLM support (#3586)

  Show description for 36eed0c
  Galunid

  authored
  36eed0c

  Copy full SHA for 36eed0c

• Fix MacOS Sonoma model quantization (#4052)

  Show description for 6bb4908
  

  3 peopleauthored
  6bb4908

  Copy full SHA for 6bb4908

Commits on Nov 15, 2023

• ggml-cuda : increase max graph size (#4084)

  slaren

  authored
  1cf2850

  Copy full SHA for 1cf2850

• llama : restore prefix space in llama tokenizer (#4081)

  cebtenzzre

  authored
  a6fc554

  Copy full SHA for a6fc554

Commits on Nov 16, 2023

• gguf : fix potential infinite loops while parsing (#4100)

  Show description for 8da4627
  

  texmex76

  and

  Bernhard Gstrein

  authored
  8da4627

  Copy full SHA for 8da4627

Commits on Nov 17, 2023

• Respect tokenizer.ggml.add_bos_token value when tokenizing (#4040)

  Show description for 91f6499
  KerfuffleV2

  authored
  91f6499

  Copy full SHA for 91f6499

• llama : fix data units (#4101)

  Show description for 4f447a4
  ggerganov

  authored
  4f447a4

  Copy full SHA for 4f447a4

• cuda : get_row_rounding F32 (#4095)

  Show description for b83e149
  

  AndrewGodfrey

  and

  cebtenzzre

  authored
  b83e149

  Copy full SHA for b83e149

• finetune : zero the loraB initial vectors (#4082)

  Show description for 947f64f
  AndrewGodfrey

  authored
  947f64f

  Copy full SHA for 947f64f

• finetune : speed-up ggml_compute_forward_out_prod_f32 via BLAS (#4079)

  Show description for 3e916a0
  gwjr

  authored
  3e916a0

  Copy full SHA for 3e916a0

• llama : add functions to get the model's metadata (#4013)

  Show description for e85bb1a
  slaren

  authored
  e85bb1a

  Copy full SHA for e85bb1a

• train : move number of gpu layers argument parsing to common/train.cpp (#4074)

  Show description for ba4cf5c
  jpodivin

  authored
  ba4cf5c

  Copy full SHA for ba4cf5c

• py : remove superfluous import statements (#4076)

  Show description for f7d5e97
  jpodivin

  authored
  f7d5e97

  Copy full SHA for f7d5e97

• llava : fix compilation warning that fread return value is not used (#4069)

  huawei-lin

  authored
  c7cce12

  Copy full SHA for c7cce12

• common : improve yaml log escaping (#4080)

  Show description for 9e87ef6
  joennlae

  authored
  9e87ef6

  Copy full SHA for 9e87ef6

• py : Falcon HF compatibility (#4104)

  Show description for 11173c9
  cmp-nct

  authored
  11173c9

  Copy full SHA for 11173c9

• convert : use 'model' value if it exists. This allows karpathy/tinyllamas to load (#4089)

  Show description for 2ab0707
  dmahurin

  authored
  2ab0707

  Copy full SHA for 2ab0707

• examples : add tokenize (#4039)

  zakkor

  authored
  2fa02b4

  Copy full SHA for 2fa02b4

• tokenize : fix trailing whitespace

  ggerganov

  committed
  5ad387e

  Copy full SHA for 5ad387e

• build : support ppc64le build for make and CMake (#3963)

  Show description for 8e93610
  

  bufferoverflow

  and

  ggerganov

  authored
  8e93610

  Copy full SHA for 8e93610

• llama : increase max nodes (#4115)

  slaren

  authored
  bbecf3f

  Copy full SHA for bbecf3f

Commits on Nov 18, 2023

• Clean up ggml-cuda.cu warnings when compiling with clang (for ROCM) (#4124)

  Show description for 2923f17
  KerfuffleV2

  authored
  2923f17

  Copy full SHA for 2923f17

• scripts : Remove missed baichuan convert script (#4127)

  Galunid

  authored
  0b5c3b0

  Copy full SHA for 0b5c3b0

• tokenize example: Respect normal add BOS token behavior (#4126)

  Show description for 28a2e6e
  KerfuffleV2

  authored
  28a2e6e

  Copy full SHA for 28a2e6e

Commits on Nov 19, 2023

• gguf-py : export chat templates (#4125)

  Show description for e937066
  slaren

  authored
  e937066

  Copy full SHA for e937066

• gitignore : tokenize

  ggerganov

  committed
  35985ac

  Copy full SHA for 35985ac

• common : comma should be semicolon (#4137)

  kchro3

  authored
  262005a

  Copy full SHA for 262005a

• server : relay error messages (#4131)

  SoftwareRenderer

  authored
  936c79b

  Copy full SHA for 936c79b

• finetune : add --n-gpu-layers flag info to --help (#4128)

  csaben

  authored
  05e8301

  Copy full SHA for 05e8301

• Revert "finetune : add --n-gpu-layers flag info to --help (#4128)"

  Show description for dae06c0
  ggerganov

  committed
  dae06c0

  Copy full SHA for dae06c0

Commits on Nov 20, 2023

• speculative : fix prompt tokenization in speculative example (#4025)

  Show description for 40a34fe
  AutonomicPerfectionist

  authored
  40a34fe

  Copy full SHA for 40a34fe

• ci : add flake8 to github actions (python linting) (#4129)

  Show description for f23c035
  Galunid

  authored
  f23c035

  Copy full SHA for f23c035

• main : Add ChatML functionality to main example (#4046)

  Show description for 881800d
  Sebby37

  authored
  881800d

  Copy full SHA for 881800d

• readme : update ROCm Windows instructions (#4122)

  Show description for dfc7cd4
  

  jammm

  and

  cebtenzzre

  authored
  dfc7cd4

  Copy full SHA for dfc7cd4

• finetune - update readme to mention llama support only (#4148)

  Galunid

  authored
  0b871f1

  Copy full SHA for 0b871f1

Commits on Nov 21, 2023

• stablelm : simplify + speedup generation (#4153)

  Galunid

  authored
  8e672ef

  Copy full SHA for 8e672ef

Commits on Nov 22, 2023

• Merge remote-tracking branch 'ggreganov/master' into update-to-latest

  Andreybest

  committed
  d1252f8

  Copy full SHA for d1252f8