Guide to optimziers#
Custom optimziers#
Any custom optimizer class should inherit from torch.optim.Optimizer
(or its children) and implement its abstract methods.
Integration with fedsim-cli#
To automatically include your custom optimizer by the provided cli tool, you can define it in a python file and pass its path to --optimzier
or --local-optimzier
option (without .py) followed by column and name of the optimizer definition (class or method).
For example, if you have optimizer CustomOpt
stored in a foo/bar/my_custom_opt.py
, you can pass --optimizer foo/bar/my_custom_opt:CustomOpt
for setting global optimizer or --local-optimizer foo/bar/my_custom_opt:CustomOpt
for setting the local optimizer.
Note
Arguments of constructor of any optimzier could be given in arg:value
format following its name (or path if a local file is provided). Examples:
fedsim-cli fed-learn --optimzier SGD lr:0.1 weight_decay:0.001 ...
fedsim-cli fed-learn --local-optimizer foo/bar/my_custom_opt:CustomOpt lr:0.2 momentum:True ...