Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -10,7 +10,7 @@ Gemma Scope 2 is a comprehensive, open suite of sparse autoencoders and transcod
|
|
| 10 |
|
| 11 |
Sparse Autoencoders are a "microscope" of sorts that can help us break down a model's internal activations into the underlying concepts, just as biologists use microscopes to study the individual cells of plants and animals.
|
| 12 |
|
| 13 |
-
|
| 14 |
|
| 15 |
# 2. What Is In This Repo?
|
| 16 |
|
|
@@ -44,10 +44,10 @@ Unless you're doing full circuit-style analysis, we recommend using SAEs / trans
|
|
| 44 |
- **Width**: our SAEs have widths 16k, 64k, 256k, 1m. You can visit Neuronpedia to get a qualitative sense of what kinds of features you can find at different widths, but we generally recommend using 64k or 256k.
|
| 45 |
- **L0**: our SAEs have target L0 values "small" (10-20), "medium" (30-60) or "large" (60-150)". You can also look at the `config.json` file saved with every SAE's parameters to check exactly what the L0 is (or just visit the Neuronpedia page!). We generally recommend using "medium" which is useful for most tasks, although this might vary depending on your exact use case. Again you can visit Neuronpedia to get a sense of what kind of features each model type finds.
|
| 46 |
|
| 47 |
-
#
|
| 48 |
|
| 49 |
Point of contact: Callum McDougall
|
| 50 |
Contact by email: [email protected]
|
| 51 |
|
| 52 |
-
#
|
| 53 |
-
Paper
|
|
|
|
| 10 |
|
| 11 |
Sparse Autoencoders are a "microscope" of sorts that can help us break down a model's internal activations into the underlying concepts, just as biologists use microscopes to study the individual cells of plants and animals.
|
| 12 |
|
| 13 |
+
You can read more in our [blog post](https://deepmind.google/blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior), and also see our [landing page](https://huggingface.co/google/gemma-scope-2) for details on the whole suite.
|
| 14 |
|
| 15 |
# 2. What Is In This Repo?
|
| 16 |
|
|
|
|
| 44 |
- **Width**: our SAEs have widths 16k, 64k, 256k, 1m. You can visit Neuronpedia to get a qualitative sense of what kinds of features you can find at different widths, but we generally recommend using 64k or 256k.
|
| 45 |
- **L0**: our SAEs have target L0 values "small" (10-20), "medium" (30-60) or "large" (60-150)". You can also look at the `config.json` file saved with every SAE's parameters to check exactly what the L0 is (or just visit the Neuronpedia page!). We generally recommend using "medium" which is useful for most tasks, although this might vary depending on your exact use case. Again you can visit Neuronpedia to get a sense of what kind of features each model type finds.
|
| 46 |
|
| 47 |
+
# 5. Point of Contact
|
| 48 |
|
| 49 |
Point of contact: Callum McDougall
|
| 50 |
Contact by email: [email protected]
|
| 51 |
|
| 52 |
+
# 6. Citation
|
| 53 |
+
Paper link [here](https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/gemma-scope-2-helping-the-ai-safety-community-deepen-understanding-of-complex-language-model-behavior/Gemma_Scope_2_Technical_Paper.pdf)
|