Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release/2.4] Enable bf16 with fp32 weights for MIOpen batchnorm #1801

Merged
merged 1 commit into from
Jan 15, 2025
Merged

[release/2.4] Enable bf16 with fp32 weights for MIOpen batchnorm #1801

merged 1 commit into from
Jan 15, 2025

Conversation

jithunnair-amd
Copy link
Collaborator

This PR enables:

  • using MIOpen OCL_mix backend for bf16 batchnorm with fp32 weights (using torch autocast). This was required and tested for customer workload using NCHW (which is the only memory_layout enabled).
  • logging for MIOpen batchnorm using PYTORCH_MIOPEN_EXTRA_LOGGING env var.

TODO in separate PR: Need to implement PyTorch unit tests for this bf16/fp16 inputs + fp32 weights case.

(cherry picked from commit abbfe77)

@jithunnair-amd
[release/2.5] Enable bf16 with fp32 weights for MIOpen batchnorm (#1672)
d3a5429 
This PR enables:
* using MIOpen OCL_mix backend for bf16 batchnorm with fp32 weights
(using torch autocast). This was required and tested for customer
workload using NCHW (which is the only memory_layout enabled).
* logging for MIOpen batchnorm using `PYTORCH_MIOPEN_EXTRA_LOGGING` env
var.

TODO in separate PR: Need to implement PyTorch unit tests for this
bf16/fp16 inputs + fp32 weights case.

(cherry picked from commit abbfe77)
@jithunnair-amd jithunnair-amd requested review from dnikolaev-amd and removed request for jeffdaily December 18, 2024 16:23
@jithunnair-amd jithunnair-amd mentioned this pull request Dec 18, 2024
Closed
@rocm-repo-management-api
Copy link

rocm-repo-management-api bot commented Dec 18, 2024 •
edited
Loading

Jenkins build for d3a5429db61ff9e44ef7d2f2ef70a819297cbca1 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@rocm-repo-management-api
Copy link

rocm-repo-management-api bot commented Dec 19, 2024 •
edited
Loading

Jenkins build for d3a5429db61ff9e44ef7d2f2ef70a819297cbca1 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@rocm-repo-management-api
Copy link

rocm-repo-management-api bot commented Jan 6, 2025 •
edited
Loading

Jenkins build for d3a5429db61ff9e44ef7d2f2ef70a819297cbca1 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@rocm-repo-management-api
Copy link

rocm-repo-management-api bot commented Jan 7, 2025 •
edited
Loading

Jenkins build for d3a5429db61ff9e44ef7d2f2ef70a819297cbca1 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@rocm-repo-management-api
Copy link

rocm-repo-management-api bot commented Jan 13, 2025 •
edited
Loading

Jenkins build for d3a5429db61ff9e44ef7d2f2ef70a819297cbca1 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@rocm-repo-management-api
Copy link

rocm-repo-management-api bot commented Jan 13, 2025 •
edited
Loading

Jenkins build for d3a5429db61ff9e44ef7d2f2ef70a819297cbca1 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@rocm-repo-management-api
Copy link

rocm-repo-management-api bot commented Jan 14, 2025 •
edited
Loading

Jenkins build for d3a5429db61ff9e44ef7d2f2ef70a819297cbca1 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@pruthvistony pruthvistony merged commit f5fe136 into release/2.4 Jan 15, 2025
3 of 5 checks passed
@pruthvistony pruthvistony deleted the release/2.4_cherry-pick_pr-1672 branch January 15, 2025 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dnikolaev-amd dnikolaev-amd dnikolaev-amd approved these changes

@pruthvistony pruthvistony pruthvistony approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jithunnair-amd @pruthvistony @dnikolaev-amd