SAELens
CallumMcDougallGDM commited on
Commit
0d8f4f6
·
verified ·
1 Parent(s): e29753b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -10,7 +10,7 @@ Gemma Scope 2 is a comprehensive, open suite of sparse autoencoders and transcod
10
 
11
  Sparse Autoencoders are a "microscope" of sorts that can help us break down a model's internal activations into the underlying concepts, just as biologists use microscopes to study the individual cells of plants and animals.
12
 
13
- See our landing page for details on the whole suite.
14
 
15
  # 2. What Is In This Repo?
16
 
@@ -44,10 +44,10 @@ Unless you're doing full circuit-style analysis, we recommend using SAEs / trans
44
  - **Width**: our SAEs have widths 16k, 64k, 256k, 1m. You can visit Neuronpedia to get a qualitative sense of what kinds of features you can find at different widths, but we generally recommend using 64k or 256k.
45
  - **L0**: our SAEs have target L0 values "small" (10-20), "medium" (30-60) or "large" (60-150)". You can also look at the `config.json` file saved with every SAE's parameters to check exactly what the L0 is (or just visit the Neuronpedia page!). We generally recommend using "medium" which is useful for most tasks, although this might vary depending on your exact use case. Again you can visit Neuronpedia to get a sense of what kind of features each model type finds.
46
 
47
- # 4. Point of Contact
48
 
49
  Point of contact: Callum McDougall
50
  Contact by email: [email protected]
51
 
52
- # 5. Citation
53
- Paper: (link to go here)
 
10
 
11
  Sparse Autoencoders are a "microscope" of sorts that can help us break down a model's internal activations into the underlying concepts, just as biologists use microscopes to study the individual cells of plants and animals.
12
 
13
+ You can read more in our [blog post](https://deepmind.google/blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior), and also see our [landing page](https://huggingface.co/google/gemma-scope-2) for details on the whole suite.
14
 
15
  # 2. What Is In This Repo?
16
 
 
44
  - **Width**: our SAEs have widths 16k, 64k, 256k, 1m. You can visit Neuronpedia to get a qualitative sense of what kinds of features you can find at different widths, but we generally recommend using 64k or 256k.
45
  - **L0**: our SAEs have target L0 values "small" (10-20), "medium" (30-60) or "large" (60-150)". You can also look at the `config.json` file saved with every SAE's parameters to check exactly what the L0 is (or just visit the Neuronpedia page!). We generally recommend using "medium" which is useful for most tasks, although this might vary depending on your exact use case. Again you can visit Neuronpedia to get a sense of what kind of features each model type finds.
46
 
47
+ # 5. Point of Contact
48
 
49
  Point of contact: Callum McDougall
50
  Contact by email: [email protected]
51
 
52
+ # 6. Citation
53
+ Paper link [here](https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior/Gemma_Scope_2_Technical_Paper.pdf)