refactor: Refactor of L0_backend_python and the env subtest#7838

Draft

nv-kmcgill53 wants to merge 3470 commits intomainfrom kmcgill-refactor-L0_backend_python

+215,514-31,796

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 17, 2024

Use current time when overwriting model configuration. (#6727 )
whoisj
authored

Commits on Jan 18, 2024

Commits on Jan 19, 2024

Commits on Jan 22, 2024

Remove boost::filesystem (#6810 )
rmccorm4
authored

Commits on Jan 23, 2024

Generate unittest xml reports from L0_python_api (#6822 )
rmccorm4
authored

Commits on Jan 27, 2024

Add test for shutdown while unloading in background (#6835 )
kthui
authored

Commits on Jan 29, 2024

Commits on Jan 30, 2024

Commits on Jan 31, 2024

Add test for shutdown while loading model (#6837 )
kthui
authored

Commits on Feb 1, 2024

Commits on Feb 7, 2024

Updating vllm version to 0.3.0 (#6858 )
oandreeva-nv
authored

Commits on Feb 8, 2024

Python Backend Windows Support (#6830 )
fpetrini15
authored

Commits on Feb 9, 2024

Add support for Oracle Cloud in deploy (#6850 )
bruno-garbaccio
authored

Commits on Feb 13, 2024

Add link to TRTLLM metrics docs (#6874 )
rmccorm4
authored

Commits on Feb 16, 2024

Update 'main' to track development of 2.44.0 / 24.03 (#6892 )
mc-nv
authored

Commits on Feb 17, 2024

Add response statistics (#6869 )

kthui
and
rmccorm4
authored

Commits on Feb 22, 2024

Fix busyop test for L0_memory_growth (#6900 )
krishung5
authored

Commits on Feb 23, 2024

Add cancellation into response statistics (#6904 )
kthui
authored

Commits on Feb 24, 2024

Install required pip pkgs (#6906 )
krishung5
authored

Commits on Feb 27, 2024

Match forward headers case insensitively. (#6889 )
yinggeh
authored

Commits on Mar 1, 2024

Add note on --cache-config spacing and fix typos (#6929 )
rmccorm4
authored

Commits on Mar 2, 2024

Commits on Mar 4, 2024

Expose tritonserver args in values.yaml (#5582 )
okyspace
authored

Commits on Mar 6, 2024

Parameterize git repository (#6934 )

nv-kmcgill53
and
kyle
authored

Commits on Mar 9, 2024

Test improved input validation errors (#6933 )
indrajit96
authored

Commits on Mar 13, 2024

Commits on Mar 15, 2024

Commits on Mar 19, 2024

Add documentation for mapping between Triton Errors and HTTP status codes (#6992 )
Tabrizian
authored

Commits on Mar 21, 2024

Commits on Mar 25, 2024

Commits on Mar 27, 2024

Add meetup invite banner (#7049 )
rmccorm4
authored

Commits on Apr 1, 2024

Update 'main' post-24.03 (#7051 )

authored

Commits on Apr 4, 2024

Commits on Apr 8, 2024

Fix html image rendering in sphinx documentation (#7084 )
tanmayv25
authored

Commits on Apr 10, 2024

Updated vllm version (#7095 )
oandreeva-nv
authored

Commits on Apr 12, 2024

Commits on Apr 13, 2024

Commits on Apr 23, 2024

Use a lower concurrency with more repetition for L0_memory_growth (#7127 )
krishung5
authored

Commits on Apr 24, 2024

Replace deprecated tritongrpcclient package (#7061 )
Tabrizian
authored

Commits on Apr 25, 2024

Commits on Apr 26, 2024

Commits on Apr 30, 2024

Update main post-24.04 (#7160 )

mc-nv
and
krishung5
authored

Commits on May 3, 2024

Remove meetup note now that the event has completed (#7179 )
Tabrizian
authored

Commits on May 8, 2024

Rename triton_tensorrtllm_worker -> trtllmExecutorWorker (#7194 )
krishung5
authored

Commits on May 9, 2024

Tests for Top Level Request Caching for Ensemble Models (#7074 )
lkomali
authored

Commits on May 10, 2024

Test cuda shared memory offset and byte size out of bounds(#7202 )
jbkyang-nvi
authored

Commits on May 13, 2024

Upgrade the golang version to 1.22.3 (#7208 )
tanmayv25
authored

Commits on May 14, 2024

Update 'Dockerfile' Python path to include DALI (#7216 )
mc-nv
authored

Commits on May 15, 2024

Remove the dependency on CUDA driver (#7224 )
krishung5
authored

Commits on May 16, 2024

Commits on May 17, 2024

Fix the L0_simple_go_client (#7239 )
tanmayv25
authored

Commits on May 18, 2024

Add section on ensemble model caching (#7234 )
rmccorm4
authored

Commits on May 20, 2024

Add testing for escaped log messages

nnshah1
and
rmccorm4
authored

Commits on May 21, 2024

Commits on May 22, 2024

Return an error if --load-model is specified without explicit model control mode (#7259 )
rmccorm4
authored

Commits on May 23, 2024

Exclude Jax example from Python 3.8 (#7260 )
krishung5
authored

Commits on May 24, 2024

Commits on May 28, 2024

Fix Python 3.11 env (#7274 )
krishung5
authored

Commits on May 29, 2024

Commits on Jun 3, 2024

Commits on Jun 4, 2024

fix: Fix L0_input_validation--base (#7304 )
yinggeh
authored

Commits on Jun 18, 2024

fix: Fix version for setuptools and grpcio-tools. Remove cudnn 8 installation (#7331 )
krishung5
authored

Commits on Jun 20, 2024

ci: Add INT64 Datatype Support for Shape Tensors in TensorRT Backend (#7329 )
pskiran1
authored

Commits on Jun 26, 2024

Update 15-container-copyright.txt (#7375 )
Tabrizian
authored

Commits on Jun 28, 2024

Update main post -24.06 (#7380 )
mc-nv
authored

Commits on Jul 3, 2024

test: Add input byte size tests using C APIs (#7372 )
yinggeh
authored

Commits on Jul 8, 2024

Commits on Jul 10, 2024

Commits on Jul 12, 2024

Commits on Jul 25, 2024

Commits on Jul 27, 2024

ci: Fix shape and reformat free tensor handling in the input byte size check (#7444 )
pskiran1
authored

Commits on Jul 29, 2024

Commits on Jul 30, 2024

test: Add BF16 test for python backend (#7483 )
rmccorm4
authored

Commits on Jul 31, 2024

Commits on Aug 2, 2024

Including 'tritonserver.lib' into final package (#7491 )
mc-nv
authored

Commits on Aug 5, 2024

build: Add default value for argument 'TRITON_REPO_ORGANIZATION' from sdk Dockerfile (#7437 )
zhanga5
authored

Commits on Aug 6, 2024

chore:Purge PA from Client Repo (#7488 )
fpetrini15
authored

Commits on Aug 7, 2024

Commits on Aug 8, 2024

Add vLLM x Triton user meetup announcement (#7509 )
harryskim
authored

Commits on Aug 10, 2024

Fix benchmarking tests (#7461 )
pskiran1
authored

Commits on Aug 17, 2024

test: Add python backend tests for the new histogram metric (#7540 )
yinggeh
authored

Commits on Aug 20, 2024

Commits on Aug 22, 2024

ci: Raise Documentation Generation Errors (#7559 )
fpetrini15
authored

Commits on Aug 23, 2024

docs: Add tensorrtllm_backend into doc generation (#7563 )
krishung5
authored

Commits on Aug 30, 2024

feat: Python Deployment of Triton Inference Server (#7501 )

authored

Commits on Sep 3, 2024

fix: Adding copyright info (#7591 )
KrishnanPrash
authored

Commits on Sep 4, 2024

test: Refactor core input size checks (#7592 )
yinggeh
authored

Commits on Sep 7, 2024

Don't Build tritonfrontend for Windows. (#7599 )
fpetrini15
authored

Commits on Sep 11, 2024

Commits on Sep 17, 2024

Fix: Add mutex lock for state completion check in gRPC streaming to prevent race condition (#7617 )
pskiran1
authored

Commits on Sep 19, 2024

Update fetch_models.sh (#7621 )
vd-nv
authored

Commits on Sep 20, 2024

ci: Set stability factor to a higher value (#7634 )
lkomali
authored

Commits on Oct 1, 2024

[docs] Removed vLLM meetup announcement (#7673 )
oandreeva-nv
authored

Commits on Oct 3, 2024

[doc]Adjusted formatting of the warning (#7675 )
oandreeva-nv
authored

Commits on Oct 8, 2024

Commits on Oct 9, 2024

build: Adding tritonfrontend to build.py (#7681 )

KrishnanPrash
and
rmccorm4
authored

Commits on Oct 11, 2024

feat: OpenAI Compatible Frontend (#7561 )
rmccorm4
authored

Commits on Oct 12, 2024

docs: Add beta note to OpenAI compatible API (#7695 )
rmccorm4
authored

Commits on Oct 17, 2024

docs: Clarify meanings of ensemble key and value (#7711 )
kthui
authored

Commits on Oct 21, 2024

Revert "Change compute capablity min value (#7708 )" (#7721 )
mc-nv
authored

Commits on Oct 24, 2024

test: Add small delay to L0_lifecycle test_load_new_model_version after each model reload (#7735 )
kthui
authored

Commits on Oct 29, 2024

Removing caching on windows. (#7717 )
mc-nv
authored

Commits on Nov 4, 2024

Commits on Nov 7, 2024

Commits on Nov 8, 2024

test: OpenAI frontend invalid chat tokenizer network issue WAR (#7779 )
kthui
authored

Commits on Nov 13, 2024

Update ONNX version for generated models (#7785 )
mc-nv
authored

Commits on Nov 14, 2024

test: RHEL Filesystem Tests (#7788 )
fpetrini15
authored

Commits on Nov 15, 2024

Update model generation scenario (#7793 ) (#7797 )
mc-nv
authored

Commits on Nov 19, 2024

fix: Fix L0_input_validation (#7800 )
pskiran1
authored

Commits on Nov 20, 2024

build: Support RHEL ORT TensorRT Execution Provider (#7812 )

fpetrini15
and
nv-kmcgill53
authored

Commits on Nov 21, 2024

Commits on Nov 22, 2024

fix: Resolve integer overflow in Load API file decoding (#7787 )

pskiran1
and
GuanLuo
authored

Commits on Nov 25, 2024

feat: Enable deferred unregistering of shared memory regions after inference (#7743 )
pskiran1
authored

Commits on Nov 26, 2024