refactor: Refactor of L0_backend_python and the env subtest #7838
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This validates the change made to ../core wrt how model configuration mtime is handled.
* Run all cases wihh shm probe * Warmup test and then run multiple iterations * Log free shared memory on enter/exit of probe * Add shm probe to all tests * Add debug_str to shm_util * Refactor ensemble_io test, modify probe to check for growth rather than inequality * Improve stability of bls_tensor_lifecycle gpu memory tests * Add more visibility into failing model/case in python_unittest helper * [FIXME] Skip probe on certain subtests for now * [FIXME] Remove shm probe from test_restart on unhealthy stub * Start clean server run for each bls test case * Don't exit early on failure so logs can be properly collected * Restore bls test logic * Fix shm size compare * Print region name that leaked * Remove special handling on unittest * Remove debug str * Add enter and exit delay to shm leak probe --------- Co-authored-by: Ryan McCormick <[email protected]>
* Update trace_summery script * Remove GRPC_WAITREAD and Overhead
* Add gsutil cp retry helper function * Add max retry to GCS upload * Use simple sequential upload
* Handle empty output * Add test case for 0 dimension output * Fix up number of tests
* tensorrt-llm benchmarking test
…und (#6834) * Update miniconda version * Install pytest for different py version * Install pytest
* Add test for shutdown while loading * Fix intermittent failure on test_model_config_overwrite
Adding OpenTelemetry Batch Span Processor --------- Co-authored-by: Theo Clark <[email protected]> Co-authored-by: Ryan McCormick <[email protected]>
* Support Double-Type Infer/Response Parameters
* Base Python Backend Support for Windows
* Add unit test reports to L0_dlpack_multi_gpu * Add unit test reports to L0_warmup
* Add response statistics * Add L0_response_statistics * Enable http vs grpc statistics comparison * Add docs for response statistics protocol * Add more comments for response statistics test * Remove model name from config * Improve docs wordings * [Continue] Improve docs wordings * [Continue] Add more comments for response statistics test * [Continue 2] Improve docs wordings * Fix typo * Remove mentioning decoupled from docs * [Continue 3] Improve docs wordings * [Continue 4] Improve docs wordings Co-authored-by: Ryan McCormick <[email protected]> --------- Co-authored-by: Ryan McCormick <[email protected]>
* Switch to Python model for busyop test * Clean up * Address comment * Remove unused import
* Add cancellation into response statistics * Add test for response statistics cancel * Remove debugging print * Use is None comparison * Fix docs * Use default args None * Refactor RegisterModelStatistics()
…ublished containers (#7759)
…am metric buckets (#7752)
Co-authored-by: Kyle McGill <[email protected]>
…t version map to dictionary (#7500) Co-authored-by: Olga Andreeva <[email protected]> Co-authored-by: Kyle McGill <[email protected]>
Co-authored-by: GuanLuo <[email protected]>
Co-authored-by: Anant Sharma <[email protected]> Co-authored-by: Jacky <[email protected]> Co-authored-by: Kris Hung <[email protected]>
…map for the different subtests of the L0_backend_python test
nv-kmcgill53
added
PR: ci
18 tasks
pvijayakrish
force-pushed
the
kmcgill-refactor-L0_backend_python
branch
from
deb693b to
f6b3275
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does the PR do?
This PR refactors the L0_backend_python test to be able to incrementally run different parts of the test by using variables rather than having to comment out large sections of code.
Additionally this PR also refactors the
envsubtest to make use of configurable variables to run certain parts of the test on demand. This allows theL0_backend_pythontest to potentially be parallelized in the CI. This also turns silent failures into verbose failures, such as the Locale test in theenvsubtest currently failing even thought the CI says it's passing.Checklist
<commit_type>: <Title>Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
triton-inference-server/python_backend#260
triton-inference-server/python_backend#384
Where should the reviewer start?
Test plan:
Caveats:
Background
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)