Cyclical Learning Rate (CLR)

From GM-RKB
Jump to navigation Jump to search

A Cyclical Learning Rate (CLR) is a learning rate that varies cyclically between a minimum and a maximum bound.



References

2018

(...)

schedule = CyclicalSchedule(CosineAnnealingSchedule, min_lr=0.01, max_lr=2, cycle_length=250, cycle_length_decay=2, cycle_magnitude_decay=0.5)

plot_schedule(schedule)

2017a

2017b