Log learning rate value at every epoch

[anais2390]

Hi everyone !

I would like to know if there is a way to access the value of the learning rate at every eopch, in order to log it into a csv file ?
I am using ReduceLROnPlateau and LrScheduleHandler:

# learning rate scheduler
lr_scheduler = ReduceLROnPlateau(opt, verbose=True, patience=patience, mode='min')
lr_handler = LrScheduleHandler(lr_scheduler, step_transform=lambda x: evaluator.state.metrics["loss"], print_lr=True)
lr_handler.attach(evaluator)

Thanks !

[Nic-Ma]

Hi @anais2390 ,

Thanks for your interest and feedback here.
Actually, we also have some similar requirements from the internal team for federated learning tasks.
Let me think about it and get back to you soon.

Thanks.

[anais2390]

Thanks @Nic-Ma,
We found this way to do it :

@trainer.on(Events.EPOCH_COMPLETED)
def print_lr_at_each_epoch(trainer):
lr = lr_scheduler.optimizer.param_groups[0]['lr']
print(f"INFO:Epoch[{epoch}]: current learning rate: {lr}")

Then we can manage to append the lr at each epoch in a list end write it into a csv file
Does this seems ok for you ?

[Nic-Ma]

Yes, it can do, I am enhancing the print_lr arg of LrScheduleHandler to allow flexible logging of LR values.

Thanks.

[Nic-Ma]

Submitted PR Project-MONAI/MONAI#3570 .

Thanks.