Created
October 25, 2015 16:31
-
-
Save taey16/fcc2120cc07605eb7029 to your computer and use it in GitHub Desktop.
Assign different learning rate for a layer
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-- suppose you have a model called model | |
lrs_model = model:clone() | |
lrs = lrs_model:getParameters() | |
lrs:fill(1) -- setting the base learning rate to 1 | |
-- now lets set the learning rate factor of the bias of module 5 to 2 | |
lrs_model:get(5).bias:fill(2) | |
-- same thing for the weights of module 2, let's set them to 3 | |
lrs_model:get(2).weight:fill(3) | |
-- now pass lrs_model to optimState, which was created previously | |
optimState.learningRates = lrs |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment