Skip to content

Model Search

The active examples below retain their ResNet paths. FedRLNAS, all pFedRLNAS search spaces, FedTP, the local AnyCostFL/FedRolex ViT variants, and HeteroFL MobileNetV3 are archived research examples. Their old commands require a separate historical checkout and environment.

Run each active example from its own directory using Python 3.13. The AnyCostFL, FedRolex, and HeteroFL workspace manifests provide their ptflops dependency.

HeteroFL

HeteroFL is an algorithm aimed at solving heterogeneous computing resources requirements on different federated learning clients. They use five different complexities to compress the channel width of the model. In the implementation, we need to modify the model to implement those five complexities and scale modules. The retained example uses the ResNet family. The custom MobileNetV3 branch is archived. The core operations of assigning different complexities to the clients and aggregate models of complexities are in function get_local_parameters and aggregation respectively, in heterofl_algorithm.py.

cd examples/model_search/heterofl
uv run python heterofl.py -c heterofl_resnet18_dynamic.toml

Reference: Diao et al., "HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients," in Proc. International Conference on Learning Representations (ICLR), 2021.

FedRolex

FedRolex argues that the statistical method of pruning channels in HeteroFL will cause unbalanced updates of the model parameters. In this algorithm, they introduce a rolling mechanism to evenly update the parameters of each channel in the system-heterogeneous federated learning. The retained implementation uses ResNet; its local ViT variant is archived.

cd examples/model_search/fedrolex
uv run python fedrolex.py -c example_ResNet.toml

Reference: Alam et al., "FedRolex: Model-Heterogeneous Federated Learning with Rolling Sub-Model Extraction," in Proc. NeurIPS, 2022.

AnyCostFL

AnyCostFL is an on-demand system-heterogeneous federated learning method to assign models of different architectures to meet the resource budgets of devices in federated learning. In this algorithm, it adopts the similar policy to assign models of different channel pruning rates as the HeteroFL. But they prune the channel on the basis of the magnitude of the l2l_2 norms of the channels. The retained implementation uses ResNet; its local ViT variant is archived.

cd examples/model_search/anycostfl
uv run python anycostfl.py -c example_ResNet.toml

Reference: Li et al., "AnycostFL: Efficient On-Demand Federated Learning over Heterogeneous Edge Device," in Proc. INFOCOM, 2022.

SysHeteroFL

SysHeteroFL is a system-heterogeneous federated learning algorithm that assigns models of different architectures to the clients to achieve better performance when there are resource budgets on the clients. In this implementation, subnets of ResNet model with different architectures are sampled.

SysHeteroFL remains an active, separate ResNet example. It has no local workspace manifest, so the command supplies its ptflops dependency explicitly:

cd examples/model_search/sysheterofl
uv run --with 'ptflops>=0.7.5' python sysheterofl.py -c config_ResNet152.toml

Reference: D. Yao, "Exploring System-Heterogeneous Federated Learning with Dynamic Model Selection," arXiv:2409.08858.

  • FedRLNAS searched a shared architecture using reinforcement learning and the DARTS search space. See its archive.
  • pFedRLNAS / PerFedRLNAS personalized architectures and weights across clients. Its NASViT, MobileNetV3, and DARTS modes are preserved together in the pFedRLNAS archive because their historical imports depend on the sibling layout.
  • FedTP personalized transformer attention through a server hypernetwork. Its archive preserves both original configs and links to the retired legacy ViT factory.

The archive guide also covers the local ViT and MobileNetV3 variants, provenance, and restoration boundaries. These retirements concern specific Plato implementations; they do not retire generic Hugging Face or Torchvision model families.