Low-Rank Decomposition Title & Authors Introduction Links LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation Yixiao Li, Yifan Yu, Qingru Zhang, Chen Liang, Pengcheng He, Weizhu Chen, Tuo Zhao Github Paper Matrix Compression via Randomized Low Rank and Low Precision Factorization Rajarshi Saha, Varun Srivastava, Mert Pilanci Github Paper TensorGPT: Efficient Compression of the Embedding Layer in LLMs based on the Tensor-Train Decomposition Mingxue Xu, Yao Lei Xu, Danilo P. Mandic Paper LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression Ayush Kaushal, Tejas Vaidhya, Irina Rish PaperProject Rethinking Compression: Reduced Order Modelling of Latent Features in Large Language Models Arnav Chavan, Nahush Lele, Deepak Gupta Github Paper Data-free Weight Compress and Denoise for Large Language Models Runyu Peng, Yunhua Zhou, Qipeng Guo, Yang Gao, Hang Yan, Xipeng Qiu, Dahua Lin Paper SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression Xin Wang, Yu Zheng, Zhongwei Wan, Mi Zhang Github Paper Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization Yixin Ji, Yang Xiang, Juntao Li, Wei Chen, Zhongyi Liu, Kehai Chen, Min Zhang Github Paper Surgical Feature-Space Decomposition of LLMs: Why, When and How? Arnav Chavan, Nahush Lele, Deepak Gupta Github Paper MCNC: Manifold Constrained Network Compression Chayne Thrash, Ali Abbasi, Parsa Nooralinejad, Soroush Abbasi Koohpayegani, Reed Andreas, Hamed Pirsiavash, Soheil Kolouri Paper MoDeGPT: Modular Decomposition for Large Language Model Compression Chi-Heng Lin, Shangqian Gao, James Seale Smith, Abhishek Patel, Shikhar Tuli, Yilin Shen, Hongxia Jin, Yen-Chang Hsu Paper ESPACE: Dimensionality Reduction of Activations for Model Compression Charbel Sakr, Brucek Khailany Paper CompAct: Compressed Activations for Memory-Efficient LLM Training Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki, Assaf Schuster Paper Natural GaLore: Accelerating GaLore for memory-efficient LLM Training and Fine-tuning Arijit Das Github Paper
