You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The scale numbers don't need to add up to one, and you can also use numbers creater than 1 to further increase the influence of an adapter. But making the values to big will sometimes result in worse output. Play around to find good values.
64
+
The scale numbers don't need to add up to one, and you can also use numbers greater than 1 to further increase the influence of an adapter. But making the values to big will sometimes result in worse output. Play around to find good values.
65
65
66
66
Gradient checkpointing reduces the memory requirements by ~50% but increases the runtime.
67
67
If you have enough RAM, you can make finetuning a bit faster by disabling checkpointing with `--no-checkpointing`.
0 commit comments