Skip to content

Commit 1ddb52e

Browse files
authored
scripts : Use mmap in torch load (#4202)
* Use mmap in torch load, prefer .bin files when loading * Revert .bin > .safetensors preference
1 parent f837c3a commit 1ddb52e

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

convert-hf-to-gguf.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ def get_tensors(self) -> Iterator[tuple[str, Tensor]]:
5959
from safetensors import safe_open
6060
ctx = cast(ContextManager[Any], safe_open(self.dir_model / part_name, framework="pt", device="cpu"))
6161
else:
62-
ctx = contextlib.nullcontext(torch.load(self.dir_model / part_name, map_location="cpu"))
62+
ctx = contextlib.nullcontext(torch.load(str(self.dir_model / part_name), map_location="cpu", mmap=True, weights_only=True))
6363

6464
with ctx as model_part:
6565
for name in model_part.keys():

0 commit comments

Comments
 (0)