Code

Finetune section refactoring#199

Code

Merged

olegklimov merged 25 commits intov1.2.0smallcloudai/refact:v1.2.0from finetune_section_refactoringsmallcloudai/refact:finetune_section_refactoringCopy head branch name to clipboardNov 2, 2023

+2,668-393,308

Commits

Commits on Nov 1, 2023

• finetune filtering is almost working, no it is finetune's turn

  JegernOUTT

  committed
  3ab00a9

  Copy full SHA for 3ab00a9

• new datasource

  JegernOUTT

  committed
  c0d08bc

  Copy full SHA for c0d08bc

• rm finetune filter from data pipeline

  JegernOUTT

  committed
  cea773e

  Copy full SHA for cea773e

• massive datapipeline refactoring

  JegernOUTT

  committed
  498cc32

  Copy full SHA for 498cc32

• kind of working version

  JegernOUTT

  committed
  aacdd6b

  Copy full SHA for aacdd6b

• flash sa

  JegernOUTT

  committed
  2f9d68f

  Copy full SHA for 2f9d68f

• flash attention for starcoders

  JegernOUTT

  committed
  78ab3c5

  Copy full SHA for 78ab3c5

• relaxing requirements

  JegernOUTT

  committed
  0634253

  Copy full SHA for 0634253

• cherry-picking fixes

  JegernOUTT

  committed
  d937d7b

  Copy full SHA for d937d7b

• more logging

  JegernOUTT

  committed
  99030b4

  Copy full SHA for 99030b4

• trainable_embeddings

  JegernOUTT

  committed
  a9969ad

  Copy full SHA for a9969ad

• goodbye codify model

  Show description for 9851754
  JegernOUTT

  committed
  9851754

  Copy full SHA for 9851754

• being able to load safetensors

  JegernOUTT

  committed
  36088d2

  Copy full SHA for 36088d2

• fix flashattn dependency

  JegernOUTT

  committed
  12317b0

  Copy full SHA for 12317b0

• fix flashattn dependency

  JegernOUTT

  committed
  81f2bb7

  Copy full SHA for 81f2bb7

• fixing layer retrieval for some models (now we correctly find layers for some models to set a lora to them)

  JegernOUTT

  committed
  bfab54e

  Copy full SHA for bfab54e

• removed flash-att installation from docker file

  JegernOUTT

  committed
  86b4acd

  Copy full SHA for 86b4acd

• correct saving shared weights

  Show description for 98745c8
  JegernOUTT

  committed
  98745c8

  Copy full SHA for 98745c8

• dump status each iteration

  JegernOUTT

  committed
  c9cd5ed

  Copy full SHA for c9cd5ed

• log fix

  JegernOUTT

  committed
  9888d78

  Copy full SHA for 9888d78

• forward packages explicitly

  JegernOUTT

  committed
  e44e84d

  Copy full SHA for e44e84d

• rm encodings, legacy scratchpads

  JegernOUTT

  committed
  1ce6f88

  Copy full SHA for 1ce6f88

• rm legacy code contrast diff format

  JegernOUTT

  committed
  3428edf

  Copy full SHA for 3428edf

• codellama/7b fixes

  JegernOUTT

  committed
  d233683

  Copy full SHA for d233683

Commits on Nov 2, 2023

• add flash attention to codellama

  Show description for 674e944
  JegernOUTT

  committed
  674e944

  Copy full SHA for 674e944