Sparse autoencoders This repository hosts: sparse autoencoders trained on the GPT2-small model's activations. a visualizer for the autoencoders' features Install pip install git+https://github.com/openai/sparse_autoencoder.git Code structure ...
Files main sae-viewer public src .gitignore README.md package-lock.json package.json tailwind.config.js tsconfig.json sparse_autoencoder .gitignore .pre-commit-config.yaml LICENSE README.md SECURITY.md pyproject.tomlBreadcrumbs sparse_autoencoder / sae-viewer/ Directory actions More options...
BreadcrumbsHistory for sparse_autoencoder sparse_autoencoder onmain User selector All users DatepickerAll time Commit History Commits on Jun 27, 2024 n2g impl WuTheFWasThatcommittedJun 28, 2024 0b47601 Commits on Jun 17, 2024 add training code leogao2committedJun 17, 2024 4a32f5e Commi...
Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Cancel Create saved search Sign in Sign up Reseting focus {{ message }} openai / sparse_autoencoder Public Notifications You must be signed in to change notification settings Fork 46 ...
Files main sae-viewer public src .gitignore README.md package-lock.json package.json tailwind.config.js tsconfig.json sparse_autoencoder .gitignore .pre-commit-config.yaml LICENSE README.md SECURITY.md pyproject.tomlBreadcrumbs sparse_autoencoder /sae-viewer / .gitignore ...
首先他们给了一个autoencoder (就是所谓的逐层SAE),Transcode(给定L预测L+1)和crosscoder(一次预测一群)的对比,类似于防止混淆:basic setup就不翻译了,基本和SAE的loss差不多:Multi-Layer SAE Residual Stream Analysis with Multi-Layer SAEs GitHub - tim-lawson/mlsae: Multi-Layer Sparse Autoencoders ...
lukesalamone.github.io/ 什么是稀疏自动编码器(Sparse Autoencoders)? TLDR:稀疏自动编码器(sparse autoencoder)只是一种常规的自动编码器(regular autoencoder),它使用 L1 惩罚( L1 penalty )或 KL 散度损失(KL divergence loss)来鼓励稀疏性,而不是使用低维瓶颈( low-dimensional bottleneck.)。 如果您理解了...
Training Sparse Autoencoders on Language Models. Contribute to AlignmentResearch/mats_sae_training development by creating an account on GitHub.
Security Insights Additional navigation options Commits Breadcrumbs History for sparse_autoencoder sparse_autoencoder model.py onmain User selector All users Datepicker All time Commit History Commits on Jun 6, 2024 updates (#3) WuTheFWasThat ...
@misc{bloom2024saetrainingcodebase, title = {SAELens}, author = {Bloom, Joseph and Tigges, Curt and Duong, Anthony and Chanin, David}, year = {2024}, howpublished = {\url{https://github.com/jbloomAus/SAELens}}, } About Training Sparse Autoencoders on Language Models ...