Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[4] Clean train config by using hydra config overriding #6

Merged
merged 19 commits into from
Oct 14, 2024

Conversation

vinamarora8
Copy link
Member

@vinamarora8 vinamarora8 commented Oct 12, 2024

Hydra allows us to have a base config and override it with another config on top. We can use this for POYO where we have a mostly stable base config, and each different training target overrides/adds only some fields. Having a base config means we can add default config fields later without breaking any dataset-specific configs.

Also, use of proper hierarchies in base config cleans it up. Example:

optim:
  base_lr: 3.125e-5  # scaled linearly by batch size
  weight_decay: 1e-4
  lr_decay_start: 0.5  # fraction of epochs before starting LR decay

wandb:
  enable: true
  entity: null
  project: poyo
  run_name: null
  log_model: false

This PR adds base.yaml which is inherited by train_poyo_mp.yaml, train_allen_neuropixels.yaml, train_mc_maze_small.yaml. Does not update other configs as they don't work as it is with current code.

@vinamarora8 vinamarora8 changed the title Clean train config by using hydra config overriding [4] Clean train config by using hydra config overriding Oct 12, 2024
@vinamarora8 vinamarora8 requested a review from mazabou October 12, 2024 00:03
@mazabou mazabou changed the base branch from vinam/ckpt_dir to main October 14, 2024 16:57
@mazabou mazabou merged commit e5eee12 into main Oct 14, 2024
0 of 2 checks passed
@mazabou mazabou deleted the vinam/clean_cfg branch October 14, 2024 17:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants