2016. Research on denoising sparse autoencoder. International Journal of Machine Learning and Cybernetics (2016), 1-11. DOI:h p://dx.doi.org/10.1007/s13042-016-0550-yL. Meng, S. Ding, and Y. Xue, "Research on d
load(path, map_location=sparse_autoencoder.cfg.device) Training a Sparse Autoencoder on a Language Model Sparse Autoencoders can be intimidating at first but it's fairly simple to train one once you know what each part of the config does. I've created a config class which you instantiate...
This library is used to train and evaluate Sparse Autoencoders (SAEs). It handles the following training types: e2e (end-to-end): Loss function includes sparsity and final model kl_divergence. e2e + downstream reconstruction: Loss function includes sparsity, final model kl_divergence, and MSE ...
DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation research areaComputer Vision|conferenceICLRPublished year2025 AuthorsJiatao Gu, Yuyang Wang, Yizhe Zhang, Qihang Zhang†‡, Dinghuai Zhang§, Navdeep Jaitly, Josh Susskind, Shuangfei Zhai ...
Sparse Finetuning for Inference Acceleration of Large Language Models We consider the problem of accurate sparse finetuning of large language models (LLMs), that is, finetuning pretrained LLMs on specialized tasks, while inducing sparsity in their weights. On the accuracy side, we observe that ...
CLIP-UP: A Simple and Efficient Mixture-of-Experts CLIP Training Recipe with Sparse Upcycling research area Computer Vision | Published year 2025 AuthorsXinze Wang, Chen Chen, Yinfei Yang, Hong-You Chen, Bowen Zhang, Aditya Pal, Xiangxin Zhu, Xianzhi Du CtrlSynth: Controllable Image-Text Synthes...
In 2015, Mousavi27presented the Stacked Denoising Autoencoder (SDA) network model, which for the first time combined deep learning with compressed sensing to achieve high-quality reconstruction. In 2017, Yao et al28. integrated residual learning with the ReconNet framework29to further refine the ...
European Conference on Computer Vision (ECCV) 2012 Thabo Beeler (Disney Research/ETH Joint PhD) Derek Bradley (Disney Research) Henning Zimmer (Disney Research) Markus Gross (Disney Research/ETH Zurich) Coupled 3D Reconstruction of Sparse Facial Hair and Skin August 5, 2012 ACM SIGGRAPH 2012 Thabo...
Train Sparse Autoencoders Efficiently by Utilizing Features Correlation (Read more on arXiv or HuggingFace) Nikita Balagansky, Daniil Gavrilov, Daniil Laptev, Yaroslav Aksenov, Vadim Kurochkin i) This paper introduces KronSAE, an efficient sparse autoencoder (SAE) architecture leveraging Kronecker fact...
We developed a framework based on Phase Coherence Graph Autoencoder (PCGAE) that employs graph autoencoders (GAE) for non-linear dimensionality reduction of phase coherence matrices. This approach clusters to identify more distinct metastable brain states and is applied to the analysis of resting-...