The official implementation of the ICML 2023 paper OFQ-ViT

From-scratch structured attention-head pruning — reproduces Michel et al. on GPT-2, shows where the method fails on a distilled model, and adds redundancy-scaling and layer-profile analyses.

Papers for deep neural network compression and acceleration