Skip to content
Snippets Groups Projects
Commit c1653711 authored by jbreton's avatar jbreton
Browse files

new hyper parameters and results

parent 435cf81a
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
modules/llm/Mixtral-8x7b/Mixtral-8x7b_fine_tune.py
+ 2
− 1
View file @ c1653711
...@@ -71,7 +71,8 @@ def fine_tuned(base_model, new_model): ...@@ -71,7 +71,8 @@ def fine_tuned(base_model, new_model):
train_dataset=train_data, train_dataset=train_data,
args=TrainingArguments( args=TrainingArguments(
per_device_train_batch_size=2, per_device_train_batch_size=2,
gradient_accumulation_steps=1, gradient_accumulation_steps=2,
num_train_epochs=6,
learning_rate=1e-4, learning_rate=1e-4,
logging_steps=2, logging_steps=2,
optim="adamw_torch", optim="adamw_torch",
......
results/LLM/Mixtral-8x7b/MIXTRAL_fine_tuned_raw_answers.json
+ 1
− 1
View file @ c1653711
Source diff could not be displayed: it is too large. Options to address this: view the blob.
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment