llava_internlm2_chat_1_8b_qlora_clip_vit_large_p14_336_lora_e1_gpu8_finetune llava_internlm2_chat_20b_clip_vit_large_p14_336_e1_gpu8_finetune llava_internlm2_chat_20b_clip_vit_large_p14_336_e1_gpu8_pretrain llava_internlm2_chat_20b_qlora_clip_vit_large_p14_336_lora_e1_gpu...
QLoRA LoRA Full parameter fine-tune DPO ORPO Reward Model 🛠️ Quick Start Installation It is recommended to build a Python-3.10 virtual environment using conda conda create --name xtuner-env python=3.10 -y conda activate xtuner-env Install XTuner via pip pip install -U xtuner or with ...
这个https://github.com/lxe/simple-llm-finetuner资源有助于模型微调。 -提出了用于LLaMA模型实验的HuggingFace transformers库。 -可在HuggingFace上访问LLaMA的指令调优变体。 -“GPT4-x-vicuna”变体高效且兼容https://github.com/ggerganov/llama.cpp". -LLaMA模型已经通过GPT-4输出和数据集进行了微调。 -...
Resources: GitHub: xtuner HuggingFace LLaVA format model: xtuner/llava-llama-3-8b-v1_1-transformers Official LLaVA format model: xtuner/llava-llama-3-8b-v1_1-hf Details Model Visual Encoder Projector Resolution Pretraining Strategy Fine-tuning Strategy Pretrain Dataset Fine-tune Dataset LLaVA-...
internlm2_chat_7b_dpo_qlora_varlenattn internlm2_chat_7b_full_finetune_custom_dataset_e1 internlm2_chat_7b_orpo_qlora_varlenattn_ultrafeedback_e5 internlm2_chat_7b_qlora_alpaca_e3 internlm2_chat_7b_qlora_code_alpaca_e3 internlm2_chat_7b_qlora_custom_sft_e1 ...
For more examples, please see finetune.md. Step 2, convert the saved PTH model (if using DeepSpeed, it will be a directory) to Hugging Face model, by xtuner convert pth_to_hf ${CONFIG_NAME_OR_PATH} ${PTH} ${SAVE_PATH} Chat XTuner provides tools to chat with pretrained / fine-tune...
For more examples, please see finetune.md. Step 2, convert the saved PTH model (if using DeepSpeed, it will be a directory) to Hugging Face model, by xtuner convert pth_to_hf ${CONFIG_NAME_OR_PATH} ${PTH} ${SAVE_PATH} Chat XTuner provides tools to chat with pretrained / fine-tune...
Fine-tune XTuner supports the efficient fine-tune (e.g., QLoRA) for LLMs. Dataset prepare guides can be found ondataset_prepare.md. Step 0, prepare the config. XTuner provides many ready-to-use configs and we can view all configs by ...
For more examples, please seefinetune.md. Step 2, convert the saved PTH model (if using DeepSpeed, it will be a directory) to Hugging Face model, by xtuner convert pth_to_hf${CONFIG_NAME_OR_PATH}${PTH}${SAVE_PATH} Chat XTuner provides tools to chat with pretrained / fine-tuned LLM...