v0.6.4

v0.6.4 (2024-10-24)

Fix

• fix: Merge pull request #33 from jbloomAus/fix/topk-selection-purview

Fix/topk selection purview (afccd5a)

Unknown

• Update README.md (8235a9e)

• updated formatting/typing (fb141ae)

• Merge pull request #32 from jbloomAus/docs/readme-update

docs: updated readme (b5e5480)

• TopK will now select from all latents regardless of feature batch size (c1f0e14)

v0.6.3

v0.6.3 (2024-10-23)

Fix

• fix: update cached_activations directory to include number of prompts (0308cb1)

v0.6.2

v0.6.2 (2024-10-23)

Fix

• fix: lint (3fc0e2c)

Unknown

• Merge branch 'main' of https://github.com/jbloomAus/SAEDashboard (8f74a96)

• Fix: divide by zero, cached_activations folder name (1792298)

v0.6.1

v0.6.1 (2024-10-22)

Fix

• fix: update saelens to v4 (ef1a330)

Unknown

• Merge branch 'main' of https://github.com/jbloomAus/SAEDashboard (508a74d)

v0.6.0

v0.6.0 (2024-10-21)

Feature

• feat: np sae id suffix (448b14e)

Unknown

• Merge pull request #31 from jbloomAus/fix/reduce-mem

fix: added mem cleanup (60bd716)

• Fixed formatting (f1fab0c)

• Added cleanup (305c46d)

• Merge pull request #30 from jbloomAus/feat-mask-via-position

feat: prepending/appending tokens for prompt template + feat mask via Position (4c60e4c)

• add a few tests (96247d5)

• handle prefixes / suffixes and ignored positions (bff7fd9)

• simplify masking (385b6e1)

• add option for ignoring tokens at particular positions (ed3426d)

• Merge pull request #29 from jbloomAus/refactor/optimize-dfa-speed

Sped up DFA calculation 60x (f992e3c)

• Sped up DFA calculation (be11cd5)

• Added test to check for decoder weight dist (head dist) (f147696)

• Merge pull request #28 from jbloomAus/feature/np-topk-size-arg

Feature/np topk size arg (c5c1365)

• Merge pull request #25 from jbloomAus/fix/dfa-for-gqa

Fix/dfa for gqa (85c345f)

• Fixed formatting (48a67c7)

• Removed redundant code from rebase (a71fb9d)

• fixed rebase (57ee280)

• Added tests for DFA for GQA (fcfac37)

• Removed duplicate code (cc00944)

• Fixed formatting (50b08b4)

• Removed debugging statements (f7b949b)

• more debug prints x3 (53536b0)

• more debug prints x2 (6f2c504)

• more debug prints (e1bef90)

• temp print statements (fd75714)

• updated ignore (c01062f)

• Reduced memory load of GQA DFA (1ae40e9)

• DFA will now work for models with grouped query attention (c66c90f)

• Edited default chunk size (3c78bdc)

• Fixed formatting (10a36e3)

• Removed debugging statements and added device changes (0f51dd9)

• more debug prints x3 (112ef42)

• more debug prints x2 (ef154d6)

• more debug prints (1b18d14)

• temp print statements (2194d2c)

• Lowered default threshold (a49d1e5)

• updated ignore (2067655)

• Reduced memory load of GQA DFA (8ec1956)

• DFA will now work for models with grouped query attention (8f3cf55)

• Added head attr weights functionality for when DFA is use (234ea32)

• Added tests for DFA for GQA (3b99e36)

• Simply updated default value for top K (5c855fe)

• Testing variable topk sizes (79fe14b)

• Merge pull request #27 from jbloomAus/fix/resolve-duplication

Removed sources of duplicate sequences (525bffe)

• Updated location of wandb finish() (921da77)

• Added two sets of tests for duplication checks (3e95ffd)

• Restored original random indices function as it seemed ok (388719b)

• Removed sources of duplicate sequences (853306c)

• Removed duplicate code (7093773)

• More rebasing stuff (59c6cd8)

• Edited default chunk size (7d68f9e)

• Fixed formatting (4d5f38b)

• Removed debugging statements and added device changes (76e17c9)

• more debug prints x3 (06535d3)

• more debug prints x2 (26e8297)

• more debug prints (9ded356)

• temp print statements (024ad57)

• Lowered default threshold (a3b5977)

• updated ignore (d5d325a)

• Reduced memory load of GQA DFA (93eb1a9)

• DFA will now work for models with grouped query attention (6594155)

• Added head attr weights functionality for when DFA is use (9312d90)

v0.5.1

v0.5.1 (2024-08-27)

Fix

• fix: multi-gpu-tlens

fix: handle multiple tlens devices (ed1e967)

• fix: handle multiple tlens devices (ba5368f)

v0.5.0

v0.5.0 (2024-08-25)

Feature

• feat: accelerate caching. Torch load / save faster when files are small.

Refactor/accelerate caching (6027d0a)

Unknown

• Updated formatting (c1ea288)

• Sped up caching with native torch functions (230840a)

• Increased cache loading speed (83fe5f4)

v0.4.0

v0.4.0 (2024-08-22)

Feature

• feat: Refactor json writer and trigger DFA release

JSON writer has been refactored for reusability and readability (664f487)

Unknown

• Merge pull request #20 from jbloomAus/feature/dfa

SAEVisRunner DFA Implementation (926ea87)

• Refactored JSON creation from the neuronpedia runner (d6bb24b)

• Update ci.yaml (4b2807d)

• Fixed formatting (a62cc8f)

• Fixed target index (ca2668d)

• Corrected DFA indexing (d5028ae)

• Adding temporary testing notebook (98e4b2f)

• Added DFA output to neuronpedia runner (68eeff3)

• Fixed test typehints (d358e6f)

• Fixed formatting (5cb19e2)

• Corrected typehints (6173fbd)

• Removed another unused import (8be1572)

• Removed unused imports (9071210)

• Added support for DFA calculations up to SAE Vis runner (4a08ffd)

• Added activation collection flow for DFA (0ebb1f3)

• Merge pull request #19 from jbloomAus/fix/remove_precision_reduction

Removed precision reduction option (a5f8df1)

• Removed float16 option entirely from quantile calc (1b6a4a9)

• Removed precision reduction option (cd03ffb)

v0.3.0

v0.3.0 (2024-08-15)

Feature

• feat: seperate files per dashboard html (cd8d050)

Unknown

• Merge pull request #17 from jbloomAus/refactor/remove_enc_b

Removed all encoder B code (67c9c3f)

• Merge pull request #18 from jbloomAus/feat-seperate-files-per-html-dashboard

feat: seperate files per dashboard html (8ff69ba)

• Removed all encoder B code (5174e2e)

• Merge pull request #16 from jbloomAus/performance_refactor

Create() will now reduce precision by default (fb07b90)

• Removed line (d795490)

• Removed unnecessary print (4544f86)

• Precision will now be reduced by default for quantile calc (539d222)

• Merge pull request #15 from jbloomAus/quantile_efficiency

Quantile OOM prevention (4a40c37)

• Made quantile batch optional and removed sampling code (2df51d3)

• Added device check for test (afbb960)

• Added parameter for quantile calculation batching (49d0a7a)

• Added type annotation (c71c4aa)

• Removed unused imports (ec01bfe)

• Added float16 version of quantile calculation (2f01eb8)

• Merge pull request #13 from jbloomAus/hook_z_support

fix: restore hook_z support following regression. (ea87559)

• format (21e3617)

• make sure hook_z works (efaeec0)

• Merge pull request #12 from jbloomAus/use_sae_lens_loading

Use sae lens loading (89bba3e)

• add settings.json (d8f3034)

• add dtype (0d8008a)

• cli util (9da440e)

• wandb logging improvement (a077369)

• add override for np set name (8906d10)

• auto add folder path to output dir (35e06ab)

• update tests (50163b0)

• first step towards sae_lens remote loading (415a2d1)

v0.2.3

v0.2.3 (2024-08-06)

Fix

• fix: neuronpedia uses api_key for uploading features, and update sae_id->sae_set (0336a35)

Unknown

• Merge pull request #11 from jbloomAus/ignore_bos_option

Ignore bos option (ae34b70)

• change threshold (4a0be67)

• type fix (525b6a1)

• default ignore bos eos pad (d2396a7)

• ignore bos tokens (96cf6e9)

• jump relu support in feature masking context (a1ba87a)

• depend on latest sae lens (4988207)

• Merge pull request #10 from jbloomAus/auth_and_sae_set

fix: neuronpedia uses api_key for uploading features, and update sae_id -> sae_set (4684aca)

• Combine upload-features and upload-dead-stubs (faac839)

• Activation store device should be cuda when available (93050b1)

• Activation store device should be cuda when available (4469066)

• Better support for huggingface dataset path (3dc4b78)

• Docker tweak (a1a70cb)