Skip to content

Latest commit

 

History

History
32 lines (22 loc) · 1.04 KB

FAQs.md

File metadata and controls

32 lines (22 loc) · 1.04 KB

Frequently Asked Questions (FAQs)

ImportError: cannot import name 'LLaMATokenizer' from 'transformers'

https://stackoverflow.com/questions/75907910/importerror-cannot-import-name-llamatokenizer-from-transformers


Pytorch matmul - RuntimeError: "addmm_impl_cpu_" not implemented for 'Half'

  • map==gpu, torch_dtype=torch.float16(default)
  • map==cpu, torch_dtype=torch.float32.

https://stackoverflow.com/questions/73530569/pytorch-matmul-runtimeerror-addmm-impl-cpu-not-implemented-for-half


GPU inference with offloaded folder

open_llama_kor = LLMtranslator(model_path, target_lang='ko', torch_dtype=torch.float16, translator='google', offload_folder="offload")

CPU inference with offloaded folder

open_llama_kor = LLMtranslator(model_path, target_lang='ko', torch_dtype=torch.float32, translator='google', offload_folder="offload")

ValueError: The current device_map had weights offloaded to the disk. #239

Using offload_folder args. nomic-ai/gpt4all#239