Docs

Setup, reproduction, architecture, and known limitations.

Setup

git clone https://github.com/ana-lan/activation-lens
cd activation-lens
conda create -n activation-lens python=3.11 -y
conda activate activation-lens
pip install -r requirements.txt

Build the SAE

python3 -m model.build_activation_cache
python3 -m model.train_sae

Dead-feature diagnosis: early runs showed 52–82% of features going permanently dead. Decoder weight normalization + periodic resampling reduced this to 0.77%.

Reproduce the experiments

# Phase 5 — quantization benchmark
python3 -m benchmarks.feature_survival

# Phase 6 — safety monitor
python3 -m safety.find_harm_feature
python3 -m safety.multi_feature_classifier
python3 -m safety.calibrate_live_multifeature
python3 -m safety.benchmark_overhead

# Phase 7 — steering
python3 -m steering.screen_candidates
python3 -m steering.stage2_test_candidates
python3 -m steering.stage2_persistent_test

Run the dashboard locally

uvicorn backend.api.main:app --reload

# in a second terminal:
cd frontend
npm install
npm run dev

The public demo runs in precomputed mode. Set NEXT_PUBLIC_DEMO_MODE=live locally for real inference on arbitrary prompts and features.

Environment & hardware

• Apple Silicon Mac; all final experiments run CPU-only
• TransformerLens warned the installed PyTorch/MPS combination could produce incorrect results
• Python 3.11, conda environment (see requirements.txt)
• Overhead benchmark: 4 PyTorch CPU threads, no KV-cache

Limitations

• Quantization is simulated via uniform min-max weight rounding, not a production backend (GPTQ/AWQ)
• Single model (GPT-2-small), single layer (6), single SAE training seed
• Steering tested at one layer only; negative result may not generalize
• Safety monitor has no baseline comparison against simpler methods