llama.cpp/index missing and needed when using llama.cpp as a submodule

[danemadsen]

Using llama.cpp as a submodule after b12fa0d1c13596869c512f49a526b979c94787cc seems to cause builds to fail because ninja throws an error ninja: error: '/home/dane_madsen/Maid/lib/.git/modules/lib/butler/llama.cpp/index', needed by '/home/dane_madsen/Maid/lib/butler/llama.cpp/common/build-info.cpp', missing and no known rule to make it

If you have flutter installed you can clone this: https://github.com/Mobile-Artificial-Intelligence/maid.git and attempt to switch to the latest llama.cpp to build

Expected Behavior

Compile without issue as llama.cpp has prior to b12fa0d1c13596869c512f49a526b979c94787cc

Current Behavior

Ninja throws an error ninja: error: '/home/dane_madsen/Maid/lib/.git/modules/lib/butler/llama.cpp/index', needed by '/home/dane_madsen/Maid/lib/butler/llama.cpp/common/build-info.cpp', missing and no known rule to make it

Environment and Context

$neofetch
       _,met$$$$$gg.
    ,g$$$$$$$$$$$$$$$P.
  ,g$$P"     """Y$$.".
 ,$$P'              `$$$.     dane_madsen@workstation 
',$$P       ,ggs.     `$$b:   ----------------------- 
`d$$'     ,$P"'   .    $$$    OS: Debian GNU/Linux trixie/sid x86_64 
 $$P      d$'     ,    $$P    Kernel: 6.5.0-3-amd64 
 $$:      $$.   -    ,d$$'    Uptime: 9 hours, 3 mins 
 $$;      Y$b._   _,d$P'      Packages: 2859 (dpkg) 
 Y$$.    `.`"Y$$$$P"'         Shell: bash 5.2.15 
 `$$b      "-.__              Resolution: 1920x1080, 3440x1440 
  `Y$$                        DE: GNOME 44.5 
   `Y$$.                      WM: Mutter 
     `$$b.                    WM Theme: Adwaita 
       `Y$$b.                 Theme: Adwaita [GTK2/3] 
          `"Y$b._             Icons: Adwaita [GTK2/3] 
              `"""            Terminal: vscode 
                              CPU: AMD Ryzen 9 5900X (24) @ 4.200GHz 
                              GPU: NVIDIA GeForce RTX 4070 Ti 
                              Memory: 8724MiB / 64173MiB 

                                                      
                                                      
$ python3 --version~
unknown option --version~
usage: python3 [option] ... [-c cmd | -m mod | file | -] [arg] ...
Try `python -h' for more information.
$ python3 --version
Python 3.11.6
$ make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
dane_madsen@workstation:~/Maid$ g++ --version
g++ (Debian 13.2.0-5) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Failure Logs

ninja: error: '/home/dane_madsen/Maid/lib/.git/modules/lib/butler/llama.cpp/index', needed by '/home/dane_madsen/Maid/lib/butler/llama.cpp/common/build-info.cpp', missing and no known rule to make it

[Ph0rk0z]

I commented out the build info stuff under common/makefile. Wanted to see what happened with the pascal commit. It compiled fine but now gives OOM errors from malloc in llama.cpp:642 even though the memory on the card is far below 24g. Not sure if this is from some other commit or just from omitting build-info so harshly.

Definitely broken for stuff like llama-cpp-python and your code.

[danemadsen]

Yeah honestly I'm good with it for now because I'm looking to keep GGUFv1 support for abit longer as most of the 3B and 4B GGUF models (which I see as the models people will most likely want to try on mobile) are still v1 and havnt been updated.

Glad its not just something wrong on my end though.

[Ph0rk0z]

You can convert those easily and just copy the data without re-quantizing.