Skip to content

Files

Latest commit

 

History

History
41 lines (36 loc) · 1.81 KB

File metadata and controls

41 lines (36 loc) · 1.81 KB

Federated-Learning-Knowledge-Distillation

repo in several methods FedAvg, FedMD, FedProto, FedProx, FedHKD.

Code structure

  • main.py //For training the model
  • models.py //Our VAEs model for FMNIST, CIFAR10/100
  • sampling.py // functions that generate non-iid datasets for federated learning
  • utils.py // define functions that compute accuracy, soft prediction and model averaging
  • mem_utils.py // Library for monitoring memory usage and training time
  • option.py // define hyper-parameters
  • Server/*.py // object definition for server in each method
  • Client/*.py // object definition for client in each method

Parameters

  • --dataset: 'CIFAR10', 'CIFAR100', ' SVHN'
  • --batch_size: 64 by defalut
  • --num_epochs: number of global rounds, 50 by defalut
  • --lr: learning rate, 0.001 by defalut
  • --lr_sh_rate: period of learning rate decay, 10 by defalut
  • --dropout_rate: drop out rate for each layer, 0.2 by defalut
  • --clip_grad: maximum norm for gradient
  • --num_users: number of clients, 10 by defalut
  • --sampling_rate: proportion of clients send updates per round, 1 by defalut
  • --local_ep: local epoch, 5 by defalut
  • --beta: concentration parameter for Dirichlet distribution: 0.5 by defalut
  • --seed: random seed(for better reproducting experiments): 0 by defalut
  • --std: standard deviation by Differential Noise, 2 by defalut
  • --code_len: length of latent vector, 32 by defalut
  • --alg: 'FedAvg, FedProx, Moon, FedMD, Fedproto, FedHKD'
  • --eval_only: only ouput the testing accuracy during training and the running time
  • --part: percentage of each local data
  • --temp: temperture for soft prediction
  • --lam: hyper-parameter for loss2
  • --gamma: hyper-parameter for loss3
  • --model: CNN resnet18 shufflenet
  • --save_model: save checkpoints of the model

Running the code for training and evaluation

python main.py