Guide to optimziers#
Any custom optimizer class should inherit from
torch.optim.Optimizer (or its children) and implement its abstract methods.
Integration with fedsim-cli#
To automatically include your custom optimizer by the provided cli tool, you can define it in a python file and pass its path to
--local-optimzier option (without .py) followed by column and name of the optimizer definition (class or method).
For example, if you have optimizer
CustomOpt stored in a
foo/bar/my_custom_opt.py, you can pass
--optimizer foo/bar/my_custom_opt:CustomOpt for setting global optimizer or
--local-optimizer foo/bar/my_custom_opt:CustomOpt for setting the local optimizer.
Arguments of constructor of any optimzier could be given in
arg:value format following its name (or path if a local file is provided). Examples:
fedsim-cli fed-learn --optimzier SGD lr:0.1 weight_decay:0.001 ...
fedsim-cli fed-learn --local-optimizer foo/bar/my_custom_opt:CustomOpt lr:0.2 momentum:True ...