Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

A curated list for Efficient Large Language Models

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models

A research library for pytorch-based neural network pruning, compression, and more.

[AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models

Model optimizer used in Adlik.

Caffe/Neon prototxt training file for our Neurocomputing2017 work: Fuzzy Quantitative Deep Compression Network

This project provides tools to load and prune large language models using a structured pruning method.

KEN: Unleash the power of large language models with the easiest and universal non-parametric pruning algorithm

Official code for "EC-SNN: Splitting Deep Spiking Neural Networks on Edge Devices" (IJCAI2024)

Hierarchical Ensemble Pruning

[EMNLP 2024] Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning

This repository has the porpouse of give a solution to the travelling sales man problem

[JCST 2023] "Inductive Lottery Ticket Learning for Graph Neural Networks" by Yongduo Sui, Xiang Wang, Tianlong Chen, Meng Wang, Xiangnan He, Tat-Seng Chua.

This repository contains scripts to prune Wav2vec2 using a neuroevolution-based method. More details about this method can be found in the paper Compressing Wav2vec2 for Embedded Applications.

[PRL 2024] This is the code repo for our label-free pruning and retraining technique for autoregressive Text-VQA Transformers (TAP, TAP†).

Play the Othello board game against the algorithm and try to win!

This project implements Genetic Programming (GP) to evolve and optimize mathematical expressions that best fit given data. It leverages tree-based evolutionary algorithms to generate, evaluate, and refine expressions using selection, crossover, and mutation.

attention pruning