refactor: Refactor of L0_backend_python and the env subtest

.clang-format

@@ -2,7 +2,8 @@
 BasedOnStyle: Google
 
 IndentWidth: 2
-ContinuationIndentWidth: 2
+ColumnLimit: 80
+ContinuationIndentWidth: 4
 UseTab: Never
 MaxEmptyLinesToKeep: 2
 
@@ -34,4 +35,5 @@ BinPackArguments: true
 BinPackParameters: true
 ConstructorInitializerAllOnOneLineOrOnePerLine: false
 
-IndentCaseLabels: true
+IndentCaseLabels: true
+

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,24 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Description**
+A clear and concise description of what the bug is.
+
+**Triton Information**
+What version of Triton are you using?
+
+Are you using the Triton container or did you build it yourself?
+
+**To Reproduce**
+Steps to reproduce the behavior.
+
+Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well).
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/PULL_REQUEST_TEMPLATE/pull_request_template_external_contrib.md

@@ -0,0 +1,50 @@
+#### What does the PR do?
+<!-- Describe your pull request here. Please read the text below the line, and make sure you follow the checklist.-->
+
+#### Checklist
+- [ ] I have read the [Contribution guidelines](#../../CONTRIBUTING.md) and signed the [Contributor License
+Agreement](https://github.com/NVIDIA/triton-inference-server/blob/master/Triton-CCLA-v1.pdf)
+- [ ] PR title reflects the change and is of format `<commit_type>: <Title>`
+- [ ] Changes are described in the pull request.
+- [ ] Related issues are referenced.
+- [ ] Populated [github labels](https://docs.github.com/en/issues/using-labels-and-milestones-to-track-work/managing-labels) field
+- [ ] Added [test plan](#test-plan) and verified test passes.
+- [ ] Verified that the PR passes existing CI.
+- [ ] I ran pre-commit locally (`pre-commit install, pre-commit run --all`)
+- [ ] Verified copyright is correct on all changed files.
+- [ ] Added _succinct_ git squash message before merging [ref](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
+- [ ] All template sections are filled out.
+- [ ] Optional: Additional screenshots for behavior/output changes with before/after.
+
+#### Commit Type:
+Check the [conventional commit type](https://github.com/angular/angular/blob/22b96b9/CONTRIBUTING.md#type)
+box here and add the label to the github PR.
+- [ ] build
+- [ ] ci
+- [ ] docs
+- [ ] feat
+- [ ] fix
+- [ ] perf
+- [ ] refactor
+- [ ] revert
+- [ ] style
+- [ ] test
+
+#### Related PRs:
+<!-- Related PRs from other Repositories -->
+
+#### Where should the reviewer start?
+<!-- call out specific files that should be looked at closely -->
+
+#### Test plan:
+<!-- list steps to verify feature works -->
+<!-- were e2e tests added?-->
+
+#### Caveats:
+<!-- any limitations or possible things missing from this PR -->
+
+#### Background
+<!-- e.g. what led to this change being made. this is optional extra information to help the reviewer -->
+
+#### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
+- closes GitHub issue: #xxx

.github/PULL_REQUEST_TEMPLATE/pull_request_template_internal_contrib.md

@@ -0,0 +1,50 @@
+#### What does the PR do?
+<!-- Describe your pull request here. Please read the text below the line, and make sure you follow the checklist.-->
+
+#### Checklist
+- [ ] PR title reflects the change and is of format `<commit_type>: <Title>`
+- [ ] Changes are described in the pull request.
+- [ ] Related issues are referenced.
+- [ ] Populated [github labels](https://docs.github.com/en/issues/using-labels-and-milestones-to-track-work/managing-labels) field
+- [ ] Added [test plan](#test-plan) and verified test passes.
+- [ ] Verified that the PR passes existing CI.
+- [ ] Verified copyright is correct on all changed files.
+- [ ] Added _succinct_ git squash message before merging [ref](https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
+- [ ] All template sections are filled out.
+- [ ] Optional: Additional screenshots for behavior/output changes with before/after.
+
+#### Commit Type:
+Check the [conventional commit type](https://github.com/angular/angular/blob/22b96b9/CONTRIBUTING.md#type)
+box here and add the label to the github PR.
+- [ ] build
+- [ ] ci
+- [ ] docs
+- [ ] feat
+- [ ] fix
+- [ ] perf
+- [ ] refactor
+- [ ] revert
+- [ ] style
+- [ ] test
+
+#### Related PRs:
+<!-- Related PRs from other Repositories -->
+
+#### Where should the reviewer start?
+<!-- call out specific files that should be looked at closely -->
+
+#### Test plan:
+<!-- list steps to verify -->
+<!-- were e2e tests added?-->
+
+- CI Pipeline ID:
+<!-- Only Pipeline ID and no direct link here -->
+
+#### Caveats:
+<!-- any limitations or possible things missing from this PR -->
+
+#### Background
+<!-- e.g. what led to this change being made. this is optional extra information to help the reviewer -->
+
+#### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
+- closes GitHub issue: #xxx

.github/pull_request_template.md

@@ -0,0 +1,13 @@
+Thanks for submitting a PR to Triton!
+Please go the the `Preview` tab above this description box and select the appropriate sub-template:
+
+* [PR description template for Triton Engineers](?expand=1&template=pull_request_template_internal_contrib.md)
+* [PR description template for External Contributors](?expand=1&template=pull_request_template_external_contrib.md)
+
+If you already created the PR, please replace this message with one of
+* [External contribution template](https://raw.githubusercontent.com/triton-inference-server/server/main/.github/PULL_REQUEST_TEMPLATE/pull_request_template_external_contrib.md)
+* [Internal contribution template](https://raw.githubusercontent.com/triton-inference-server/server/main/.github/PULL_REQUEST_TEMPLATE/pull_request_template_internal_contrib.md)
+
+and fill it out.
+
+

.github/workflows/codeql.yml

@@ -0,0 +1,84 @@
+# Copyright 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#  * Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#    contributors may be used to endorse or promote products derived
+#    from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+name: "CodeQL"
+
+on:
+  pull_request:
+
+jobs:
+  analyze:
+    name: Analyze
+    runs-on: ubuntu-latest
+    permissions:
+      actions: read
+      contents: read
+      security-events: write
+
+    strategy:
+      fail-fast: false
+      matrix:
+        language: [ 'python' ]
+        # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
+        # Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
+
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v3
+
+    # Initializes the CodeQL tools for scanning.
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v2
+      with:
+        languages: ${{ matrix.language }}
+        # If you wish to specify custom queries, you can do so here or in a config file.
+        # By default, queries listed here will override any specified in a config file.
+        # Prefix the list here with "+" to use these queries and those in the config file.
+
+        # Details on CodeQL's query packs refer to:
+        # https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
+        queries: +security-and-quality
+
+
+    # Autobuild attempts to build any compiled languages  (C/C++, C#, Go, or Java).
+    # If this step fails, then you should remove it and run the build manually (see below)
+    - name: Autobuild
+      uses: github/codeql-action/autobuild@v2
+
+    # Command-line programs to run using the OS shell.
+    # See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
+
+    #   If the Autobuild fails above, remove it and uncomment the following three lines.
+    #   modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
+
+    # - run: |
+    #   echo "Run, Build Application using script"
+    #   ./location_of_script_within_repo/buildscript.sh
+
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v2
+      with:
+        category: "/language:${{matrix.language}}"

.github/workflows/pre-commit.yaml

@@ -0,0 +1,45 @@
+# Copyright 2023-2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#  * Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+#  * Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#  * Neither the name of NVIDIA CORPORATION nor the names of its
+#    contributors may be used to endorse or promote products derived
+#    from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+name: pre-commit
+
+on:
+  pull_request:
+
+jobs:
+  pre-commit:
+    runs-on: ubuntu-22.04
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        fetch-depth: 2
+    - name: Get modified files
+      id: modified-files
+      run: echo "modified_files=$(git diff --name-only -r HEAD^1 HEAD | xargs)" >> $GITHUB_OUTPUT
+    - uses: actions/setup-python@v3
+    - uses: pre-commit/[email protected]
+      with:
+        extra_args: --files ${{ steps.modified-files.outputs.modified_files }}

.gitignore

@@ -1,11 +1,17 @@
-/bazel-bin
-/bazel-ci_build-cache
-/bazel-genfiles
-/bazel-trtserver
-/bazel-out
-/bazel-serving
-/bazel-tensorflow
-/bazel-tensorflow_serving
-/bazel-testlogs
-/bazel-tf
-/bazel-workspace
+/build
+/builddir
+/.vscode
+*.so
+__pycache__
+tmp
+*.log
+*.xml
+test_results.txt
+artifacts
+cprofile
+*.prof
+
+# Test exclusions
+qa/L0_openai/openai
+tensorrtllm_models
+custom_tokenizer