This change adds support for Intel Gaudi HPUs. #7275
Conversation
Several configuration files are provided in the examples directory for use with Gaudi. LLaMA-Factory features and optimizations including inferencing, training (sft, dpo, etc.), LoRA fine-tuning, distributed training with DeepSpeed and DDP are working. Please see README for details. Co-authored-by: Yaser Afshar [email protected] Co-authored-by: Edward Mascarenhas [email protected] Co-authored-by: Jianhong-Zhang [email protected] Co-authored-by: Wenbin Chen [email protected] Co-authored-by: Voas, Tanner [email protected]
There was a problem hiding this comment.
Could we avoid add too many examples? I think one for fine-tuning and another one for inference is sufficient
There was a problem hiding this comment.
Thanks for your comment. Yes we can. I will remove following files from examples.
deleted: examples/inference/llama3_gaudi.yaml
deleted: examples/train_full/llama3_full_sft_ds3_gaudi.yaml
deleted: examples/train_full/qwen2_full_sft_ds1_gaudi.yaml
deleted: examples/train_full/qwen2_full_sft_ds3_gaudi.yaml
deleted: examples/train_lora/llama3_lora_dpo_gaudi.yaml
deleted: examples/train_lora/llama3_lora_eval_gaudi.yaml
deleted: examples/train_lora/llama3_lora_ppo_gaudi.yaml
deleted: examples/train_lora/llama3_lora_pretrain_gaudi.yaml
deleted: examples/train_lora/llama3_lora_reward_gaudi.yaml
Only remaining will be 1 file in inference, and 3 in train_lora for llama3 with deepspeed and without deepspeed, and qwen2vl.
requirements.txt
Outdated
| @@ -1,7 +1,6 @@ | |||
| transformers>=4.41.2,<=4.49.0,!=4.46.*,!=4.47.*,!=4.48.*;python_version<'3.10' | |||
There was a problem hiding this comment.
Transformers 4.49 is required. We should not downgrade the version
LLaMA-Factory features and optimizations including inferencing, training (sft, dpo, etc.), LoRA fine-tuning, distributed training with DeepSpeed and DDP are working. Please see README for details. Co-authored-by: Yaser Afshar [email protected] Co-authored-by: Edward Mascarenhas [email protected] Co-authored-by: Jianhong-Zhang [email protected] Co-authored-by: Wenbin Chen [email protected]
Co-authored-by: Voas, Tanner [email protected]
Before submitting