Skip to content

Latest commit

 

History

History
971 lines (481 loc) · 55.3 KB

CHANGELOG.md

File metadata and controls

971 lines (481 loc) · 55.3 KB

CHANGELOG

v0.6.4 (2024-10-24)

Fix

  • fix: Merge pull request #33 from jbloomAus/fix/topk-selection-purview

Fix/topk selection purview (afccd5a)

Unknown

  • Update README.md (8235a9e)

  • updated formatting/typing (fb141ae)

  • Merge pull request #32 from jbloomAus/docs/readme-update

docs: updated readme (b5e5480)

  • TopK will now select from all latents regardless of feature batch size (c1f0e14)

v0.6.3 (2024-10-23)

Fix

  • fix: update cached_activations directory to include number of prompts (0308cb1)

v0.6.2 (2024-10-23)

Fix

Unknown

v0.6.1 (2024-10-22)

Fix

  • fix: update saelens to v4 (ef1a330)

Unknown

v0.6.0 (2024-10-21)

Feature

Unknown

  • Update README.md (a1546fd)

  • Removed outdated vis type (b0676af)

  • Update README.md (9b8446a)

  • Updated format (90e4a09)

  • Updated readme (f6819a6)

  • Merge pull request #31 from jbloomAus/fix/reduce-mem

fix: added mem cleanup (60bd716)

  • Fixed formatting (f1fab0c)

  • Added cleanup (305c46d)

  • Merge pull request #30 from jbloomAus/feat-mask-via-position

feat: prepending/appending tokens for prompt template + feat mask via Position (4c60e4c)

  • add a few tests (96247d5)

  • handle prefixes / suffixes and ignored positions (bff7fd9)

  • simplify masking (385b6e1)

  • add option for ignoring tokens at particular positions (ed3426d)

  • Merge pull request #29 from jbloomAus/refactor/optimize-dfa-speed

Sped up DFA calculation 60x (f992e3c)

  • Sped up DFA calculation (be11cd5)

  • Added test to check for decoder weight dist (head dist) (f147696)

  • Merge pull request #28 from jbloomAus/feature/np-topk-size-arg

Feature/np topk size arg (c5c1365)

  • Merge pull request #25 from jbloomAus/fix/dfa-for-gqa

Fix/dfa for gqa (85c345f)

  • Fixed formatting (48a67c7)

  • Removed redundant code from rebase (a71fb9d)

  • fixed rebase (57ee280)

  • Added tests for DFA for GQA (fcfac37)

  • Removed duplicate code (cc00944)

  • Fixed formatting (50b08b4)

  • Removed debugging statements (f7b949b)

  • more debug prints x3 (53536b0)

  • more debug prints x2 (6f2c504)

  • more debug prints (e1bef90)

  • temp print statements (fd75714)

  • updated ignore (c01062f)

  • Reduced memory load of GQA DFA (1ae40e9)

  • DFA will now work for models with grouped query attention (c66c90f)

  • Edited default chunk size (3c78bdc)

  • Fixed formatting (10a36e3)

  • Removed debugging statements and added device changes (0f51dd9)

  • more debug prints x3 (112ef42)

  • more debug prints x2 (ef154d6)

  • more debug prints (1b18d14)

  • temp print statements (2194d2c)

  • Lowered default threshold (a49d1e5)

  • updated ignore (2067655)

  • Reduced memory load of GQA DFA (8ec1956)

  • DFA will now work for models with grouped query attention (8f3cf55)

  • Added head attr weights functionality for when DFA is use (234ea32)

  • Added tests for DFA for GQA (3b99e36)

  • Simply updated default value for top K (5c855fe)

  • Testing variable topk sizes (79fe14b)

  • Merge pull request #27 from jbloomAus/fix/resolve-duplication

Removed sources of duplicate sequences (525bffe)

  • Updated location of wandb finish() (921da77)

  • Added two sets of tests for duplication checks (3e95ffd)

  • Restored original random indices function as it seemed ok (388719b)

  • Removed sources of duplicate sequences (853306c)

  • Removed duplicate code (7093773)

  • More rebasing stuff (59c6cd8)

  • Edited default chunk size (7d68f9e)

  • Fixed formatting (4d5f38b)

  • Removed debugging statements and added device changes (76e17c9)

  • more debug prints x3 (06535d3)

  • more debug prints x2 (26e8297)

  • more debug prints (9ded356)

  • temp print statements (024ad57)

  • Lowered default threshold (a3b5977)

  • updated ignore (d5d325a)

  • Reduced memory load of GQA DFA (93eb1a9)

  • DFA will now work for models with grouped query attention (6594155)

  • Added head attr weights functionality for when DFA is use (9312d90)

v0.5.1 (2024-08-27)

Fix

  • fix: multi-gpu-tlens

fix: handle multiple tlens devices (ed1e967)

  • fix: handle multiple tlens devices (ba5368f)

Unknown

  • Fixed formatting (ed7d3b1)

  • Removed debugging statements (6489d1c)

  • more debug prints x3 (5ba2b8a)

  • more debug prints x2 (e124ff9)

  • more debug prints (e2b0c35)

  • temp print statements (95df55b)

  • Lowered default threshold (dc1f31a)

  • updated ignore (eb0d56a)

  • Reduced memory load of GQA DFA (05867f1)

  • DFA will now work for models with grouped query attention (91a5dd1)

  • Added head attr weights functionality for when DFA is use (03a615f)

v0.5.0 (2024-08-25)

Feature

  • feat: accelerate caching. Torch load / save faster when files are small.

Refactor/accelerate caching (6027d0a)

Unknown

  • Updated formatting (c1ea288)

  • Sped up caching with native torch functions (230840a)

  • Increased cache loading speed (83fe5f4)

v0.4.0 (2024-08-22)

Feature

  • feat: Refactor json writer and trigger DFA release

JSON writer has been refactored for reusability and readability (664f487)

Unknown

  • Merge pull request #20 from jbloomAus/feature/dfa

SAEVisRunner DFA Implementation (926ea87)

  • Refactored JSON creation from the neuronpedia runner (d6bb24b)

  • Update ci.yaml (4b2807d)

  • Fixed formatting (a62cc8f)

  • Fixed target index (ca2668d)

  • Corrected DFA indexing (d5028ae)

  • Adding temporary testing notebook (98e4b2f)

  • Added DFA output to neuronpedia runner (68eeff3)

  • Fixed test typehints (d358e6f)

  • Fixed formatting (5cb19e2)

  • Corrected typehints (6173fbd)

  • Removed another unused import (8be1572)

  • Removed unused imports (9071210)

  • Added support for DFA calculations up to SAE Vis runner (4a08ffd)

  • Added activation collection flow for DFA (0ebb1f3)

  • Merge pull request #19 from jbloomAus/fix/remove_precision_reduction

Removed precision reduction option (a5f8df1)

  • Removed float16 option entirely from quantile calc (1b6a4a9)

  • Removed precision reduction option (cd03ffb)

v0.3.0 (2024-08-15)

Feature

  • feat: seperate files per dashboard html (cd8d050)

Unknown

  • Merge pull request #17 from jbloomAus/refactor/remove_enc_b

Removed all encoder B code (67c9c3f)

  • Merge pull request #18 from jbloomAus/feat-seperate-files-per-html-dashboard

feat: seperate files per dashboard html (8ff69ba)

  • Removed all encoder B code (5174e2e)

  • Merge pull request #16 from jbloomAus/performance_refactor

Create() will now reduce precision by default (fb07b90)

  • Removed line (d795490)

  • Removed unnecessary print (4544f86)

  • Precision will now be reduced by default for quantile calc (539d222)

  • Merge pull request #15 from jbloomAus/quantile_efficiency

Quantile OOM prevention (4a40c37)

  • Made quantile batch optional and removed sampling code (2df51d3)

  • Added device check for test (afbb960)

  • Added parameter for quantile calculation batching (49d0a7a)

  • Added type annotation (c71c4aa)

  • Removed unused imports (ec01bfe)

  • Added float16 version of quantile calculation (2f01eb8)

  • Merge pull request #13 from jbloomAus/hook_z_support

fix: restore hook_z support following regression. (ea87559)

  • format (21e3617)

  • make sure hook_z works (efaeec0)

  • Merge pull request #12 from jbloomAus/use_sae_lens_loading

Use sae lens loading (89bba3e)

v0.2.3 (2024-08-06)

Fix

  • fix: neuronpedia uses api_key for uploading features, and update sae_id->sae_set (0336a35)

Unknown

  • Merge pull request #11 from jbloomAus/ignore_bos_option

Ignore bos option (ae34b70)

  • change threshold (4a0be67)

  • type fix (525b6a1)

  • default ignore bos eos pad (d2396a7)

  • ignore bos tokens (96cf6e9)

  • jump relu support in feature masking context (a1ba87a)

  • depend on latest sae lens (4988207)

  • Merge pull request #10 from jbloomAus/auth_and_sae_set

fix: neuronpedia uses api_key for uploading features, and update sae_id -> sae_set (4684aca)

  • Combine upload-features and upload-dead-stubs (faac839)

  • Activation store device should be cuda when available (93050b1)

  • Activation store device should be cuda when available (4469066)

  • Better support for huggingface dataset path (3dc4b78)

  • Docker tweak (a1a70cb)

v0.2.2 (2024-07-12)

Fix

  • fix: don't sample too many tokens + other fixes

fix: don't sample too many tokens (b2554b0)

  • fix: don't sample too many tokens (0cbb2ed)

Unknown

    • Don't force manual overrides for dtype - default to SAE's dtype
  • Add n_prompts_in_forward_pass to neuronpedia.py
  • Add n_prompts_total, n_tokens_in_prompt, and dataset to neuronpedia artifact
  • Remove NPDashboardSettings for now (just save the NPRunnerConfig later)
  • Fix lint error
  • Consolidate minibatch_size_features/tokens to n_feats_at_a_time and n_prompts_in_fwd_pass
  • Update/Fix NP acceptance test (b6282c8)
  • Merge pull request #7 from jbloomAus/performance-improvement

feat: performance improvement (f98b3dc)

v0.2.1 (2024-07-08)

Fix

Unknown

  • Merge pull request #6 from jbloomAus/fix-bfloat16

fix bfloat 16 error (2f3c597)

  • fix bfloat 16 error (63c3c62)

  • Merge pull request #5 from jbloomAus/np-updates

Updates + fixes for Neuronpedia (9e6b5c4)

  • Fix SAELens compatibility (139e1a2)

  • Rename file (16709ad)

  • Fix type (6b20386)

  • Make Neuronpedia outputs an object, and add a real acceptance test (a5db256)

  • Np Runner: Multi-gpu defaults (07f7128)

  • Ensure minibatch is on correct device (e206546)

  • NP Runner: Automatically use multi-gpu, devices (bf280e6)

  • Allow dtype override (a40077d)

  • NP-Runner: Remove unnecessary layer of batching. (e2ac92b)

  • NP Runner: Allow skipping sparsity check (ef74d2a)

  • Merge pull request #2 from jbloomAus/multiple-devices

feat: Multiple devices (535e6c9)

v0.2.0 (2024-06-10)

Feature

  • feat: experimental release 2 (e264f97)

v0.1.0 (2024-06-10)

Feature

  • feat: experimental release (d79310a)

Unknown

  • fix pyproject.toml (a27c87d)

  • test deployment (288a2d9)

  • refactor np runner and add acceptance test (212593c)

  • Fix: Default context tokens length for neuronpedia runner (aefe95c)

  • Allow custom context tokens length for Neuronpedia runner (d204cc8)

  • Fix: Streaming default true (1b91dff)

  • Fix n_devices error for non-cuda (70b2dbd)

  • fix import path for ci (3bd4687)

  • make pyright happy, start config (b39ae85)

  • add black (236855b)

  • fix ci (12818d7)

  • add pytest cov (aae0571)

  • bring checks in line with sae_lens (7cd9679)

  • activation scaling factor (333d377)

  • Move Neuronpedia runner to SAEDashboard (4e691ea)

  • fold w_dec norm by default (b6c9bc7)

  • rename sae_vis to sae_dashboard (f0f5341)

  • rename feature data generator (e02ed0a)

  • update demo (8aa9e52)

  • add demo (dd3036f)

  • delete old demo files (3d86202)

  • remove unnecessary print statement (9d3d937)

  • set sae lens version (87a7691)

  • update older readme (c5c98e5)

  • test (923da42)

  • remove sae lens dep (2c26d5f)

  • Merge branch 'refactor_b' (3154d63)

  • pass linting (0c079a1)

  • format (6f37e2e)

  • run ci on all branches (faa0cc4)

  • don't use feature ablations (dc6e6dc)

  • mock information in sequences to make normal sequence generation pass (c87b82f)

  • Remove resid (ff83737)

  • adding a test for direct_effect_feature_ablation_experiment (a9f3d1b)

  • shortcut direct_effect_feature_ablation_experiment if everything is zero (2c68ff0)

  • fixing CI and replacing manual snapshots with syrupy snapshots (3b97640)

  • more refactor, WIP (81657c8)

  • continue refactor, make data generator (eb1ae0f)

  • add use of safetensors cache for repeated calculations (a241c32)

  • more refactor / benchmarking (d65ee87)

  • only run unit tests (5f11ddd)

  • fix lint issue (24daf17)

  • format (83e89ed)

  • organise tests, make sure only unit tests run on CI (21f5fb1)

  • see if we can do some caching (c1dca6f)

  • more refactoring (b3f0f41)

  • further refactor, possible significant speed up (ddd3496)

  • more refactor (a5f6deb)

  • refactoring in progress (d210b60)

  • use named arguments (4a81053)

  • remove create method (43b2018)

  • move chunk (0f26aa8)

  • use fixtures (7c11dd9)

  • refactor to create runner (9202c19)

  • format (abd8747)

  • target ci at this branch (ea3b2a3)

  • comment out release process for now (7084b5b)

  • test generated output (7b8b2ab)

  • commit current demo html (00a03a0)

v0.0.1 (2024-04-25)

Chore

  • chore: setting up pytest (2079d00)

  • chore: setting up semantic-release (09075af)

Fix

  • fix: remove circular dep with sae lens (1dd9f6c)

Unknown

  • Merge pull request #44 from chanind/pytest-setup

chore: setting up pytest (034eefa)

  • Merge pull request #43 from callummcdougall/move_saelens_dep

Remove dependency on saelens from pyproject, add to demo.ipynb (147d87e)

oops I forgot to switch back to main before pushing (1312cd0)

  • Merge branch 'main' into allow_disable_buffer (e7edf5a)

  • 16 (64e7018)

  • Merge pull request #40 from chanind/semantic-release-autodeploy

chore: setting up semantic-release for auto-deploy (a4d44d1)

  • version 0.2.16 (afca0be)

  • Merge pull request #38 from chanind/type-checking

Enabling type checking with Pyright (f1fd792)

  • Merge pull request #39 from callummcdougall/fix_loading_saelens_sae

FIX: SAELens new format has "scaling_factor" key, which causes assert to fail (983aee5)

  • Fix Formatting (13b8106)

  • Merge branch 'main' into fix_loading_saelens_sae (21b0086)

  • Allow SAELens autoencoder keys to be superset of required keys, instead of exact match (6852170)

  • enabling type checking with Pyright (05d14ea)

  • Fix version (5a43916)

  • format (8f1506b)

  • v0.2.17 (2bb14da)

  • Use main branch of SAELens (2b34505)

  • Update version 0.2.16 (bf90293)

  • Merge pull request #36 from callummcdougall/allow_disable_buffer

FEATURE: Allow setting buffer to None, which gives the whole activation sequence (f5f9594)

  • fix all indices view (5f87d52)

  • Merge pull request #35 from callummcdougall/fix_gpt2_demo

Fix usage of SAELens and demo notebook (88b5933)

  • Merge branch 'fix_gpt2_demo' into allow_disable_buffer (ea57bfc)

  • Import updated SAELens, use correct tokens, fix missing file cfg.json file error. (14ba9b0)

  • Merge pull request #34 from ArthurConmy/patch-1

Update README.md (3faac82)

  • Update README.md (416eca8)

  • Merge pull request #33 from chanind/setup-poetry-and-ruff

Setting up poetry / ruff / github actions (287f30f)

Update setup.py with eindex dependency (8d7ed12)

  • Merge pull request #32 from hijohnnylin/pin_older_sae_training

Demo notebook errors under "Multi-layer models" vis (9ac1dac)

  • Pin older commit of mats_sae_training (8ca7ac1)

  • two more deps (7f231a8)

  • Update setup.py with eindex

Without this, 'pip install sae-vis' will cause errors when e.g. you do 'from sae_vis.data_fetching_fns import get_feature_data' (a9d7de9)

  • update version number (72e584b)

  • add gifs to readme (1393740)

  • test gif (4fbafa6)

  • fix height issue (3f272f6)

  • fix pypi (7151164)

  • update setup (8c43478)

  • Merge pull request #26 from hijohnnylin/update_html_anomalies

Update and add some HTML_ANOMALIES (1874a47)

fixing repo URL in setup.py (14a0be5)

  • fixing repo URL in setup.py (4faeca5)

  • fix hook point bug (9b573b2)

  • Merge pull request #20 from chanind/fix-final-resid-layer

fixing bug if hook_point == hook_point_resid_final (d6882e3)

  • fixing bug using hook_point_resid_final (cfe9b30)

  • fix indexing speed (865ff64)

  • enable JSON saving (feea47a)

  • Merge pull request #19 from chanind/support-mlp-and-attn-out

supporting mlp and attn out hooks (1c5463b)

removing Python build artifacts and adding to .gitignore (b0e0594)

  • removing Python build artifacts and adding to .gitignore (b6486f5)

  • update readme (0ee3608)

  • update readme (f8351f8)

  • update version number (1e74408)

  • fix formatting and docstrings (b9fe2bb)

  • Merge pull request #17 from jordansauce/sae-agnostic-functions-new

Added SAE class agnostic functions (0039c6f)

  • add to pypi (02a5b9a)

  • Added sae class agnostic functions

Added parse_feature_data() and parse_prompt_data() (e2709d0)

Topk error handling for empty masks (2740c00)

  • Topk error handling for empty masks (1c2627e)

  • viz to vis (216d02b)

  • update readme links (f9b3f95)

  • update for TL (1941db1)

  • Merge pull request #5 from lucyfarnik/transformer-lens-models

Compatibility with TransformerLens models (8d59c6c)

  • Merge branch 'main' into transformer-lens-models (73057d7)

  • Merge pull request #4 from lucyfarnik/resid-saes-support

Added support for residual-adjacent SAEs (b02e98b)

  • Merge pull request #7 from lucyfarnik/fix-histogram-div-zero

Fixed division by zero in histogram calculation (3aee20e)

  • Merge pull request #6 from lucyfarnik/handling-dead-features

Edge case handling for dead features (9e43c30)

  • add features argument (f24ef7e)

  • fix image link (22c8734)

  • Merge pull request #1 from lucyfarnik/read-me-links-fix

Fixed readme links pointing to the old colab (86f8e20)

  • Fixed division by zero in histogram calculation (e986e90)

  • Added readme section about models (7523e7f)

  • Fixed readme links pointing to the old colab (28ef1cb)

  • Edge case handling for dead features (5197aee)

  • Compatibility with TransformerLens models (ba708e9)

  • Added support for MPS (196c0a2)

  • Added support for residual-adjacent SAEs (89aacf1)

  • black font (d81e74d)

  • fix html bug (265dedd)

  • add jax and dataset deps (f1caeaf)

  • remove TL dependency (155991f)

  • first commit (7782eb6)