Merge branch 'main' into christophe-papazian/upgrade_to_python_3.13

.github/CODEOWNERS

@@ -5,7 +5,7 @@
 /utils/build/docker/golang*/ @DataDog/dd-trace-go-guild @DataDog/system-tests-core
 /utils/build/docker/java*/ @DataDog/apm-java @DataDog/asm-java @DataDog/system-tests-core
 /utils/build/docker/java_otel/ @DataDog/opentelemetry @DataDog/system-tests-core
-/utils/build/docker/nodejs*/ @DataDog/apm-js @DataDog/asm-js @DataDog/system-tests-core
+/utils/build/docker/nodejs*/ @DataDog/dd-trace-js @DataDog/system-tests-core
 /utils/build/docker/php*/ @DataDog/apm-php @DataDog/system-tests-core
 /utils/build/docker/python*/ @DataDog/apm-python @DataDog/asm-python @DataDog/system-tests-core
 /utils/build/docker/ruby*/ @DataDog/ruby-guild @DataDog/asm-ruby @DataDog/system-tests-core
@@ -23,7 +23,7 @@
 /manifests/dotnet.yml @DataDog/apm-dotnet @DataDog/asm-dotnet
 /manifests/golang.yml @DataDog/dd-trace-go-guild
 /manifests/java.yml @DataDog/asm-java @DataDog/apm-java
-/manifests/nodejs.yml @DataDog/apm-js @DataDog/asm-js
+/manifests/nodejs.yml @DataDog/dd-trace-js
 /manifests/php.yml @DataDog/apm-php @DataDog/asm-php
 /manifests/python.yml @DataDog/apm-python @DataDog/asm-python
 /manifests/ruby.yml @DataDog/ruby-guild @DataDog/asm-ruby

CHANGELOG.md

@@ -3,6 +3,13 @@
 All notable changes to this project will be documented in this file.
 
 
+### 2024-12 (138 PR merged)
+
+* 2024-12-31 [Allow a class to declare several scenarios](https://github.com/DataDog/system-tests/pull/3757) by @cbeauchesne
+* 2024-12-17 Optimization of java weblogs [1](https://github.com/DataDog/system-tests/pull/3701), [2](https://github.com/DataDog/system-tests/pull/3697), [3](https://github.com/DataDog/system-tests/pull/3694) and [4](https://github.com/DataDog/system-tests/pull/3693) by @smola
+* 2024-12-13 Use ruff instead of black as [formatter](https://github.com/DataDog/system-tests/pull/3656) and [linter](https://github.com/DataDog/system-tests/pull/3631) by @cbeauchesne
+* 2024-12-03 Lot ofrevamp of k8 tests by @robertomonteromiguel
+
 ### 2024-11 (207 PR merged)
 
 * 2024-11-22 [Docker SSI: report data to FPD](https://github.com/DataDog/system-tests/pull/3525) by @robertomonteromiguel

docs/scenarios/docker_ssi.md

@@ -14,25 +14,25 @@
 # Overall
 
 The Docker SSI tests are an easy and fast tests to check the SSI instrumentation.
-The Docker SSI tests don't pretend to reemplace the current AWS tests, there only try to complement them, providing a quick checks and verification for some features included in the SSI.
+The Docker SSI tests don't pretend to replace the current AWS tests, they only try to complement them, providing a quick checks and verification for some features included in the SSI.
 
 The main differences between the AWS/Onboarding tests and the Docker SSI tests are:
 
 * In Docker SSI, the SSI is installed inside a docker container, instead of using virtual machines.
-* The AWS/Onboarding tests represent better a real customer scenarios, but Docker SSI it doesn't.
-* Docker SSI relies on APM Test Agent, instead of rely on the backend.
+* The AWS/Onboarding tests better represent real customer scenarios, while Docker SSI does not.
+* Docker SSI relies on APM Test Agent, instead of relying on the Datadog's production backend.
 
 The main Docker SSI properties are:
 
 * Install the SSI host injection on the docker container that runs the weblog application.
-* Uses a APM Test Agent to make the assertions.
+* Uses APM Test Agent to make the assertions.
 * The weblog applications are containerized applications.
 * The weblog application container is built by the system-tests scenarios in separate steps.
-* The base image of the weblog application container should be parametrizable (it will detailed in the next sections)
+* The base image of the weblog application container should be parametrizable (detailed in the next sections)
 
 The Docker SSI are good for:
 
-* Test the instrumentation agains different runtime versions.
+* Test the instrumentation against different runtime versions.
 * Test the guardrail features (unsupported versions of the language).
 * Test the service naming features on SSI.
 * Test the crash tracking features.
@@ -42,18 +42,18 @@ The Docker SSI are good for:
 
 ## Prerequisites
 
-There are not special requirements to run the Docker SSI tests, you only need the docker engine and have the access to GHCR.
+There are no special requirements to run the Docker SSI tests, you only need the docker engine and have the access to GHCR.
 
 ### System-tests requirements
 
-All system-tests assertions and utilities are based on python and pytests. You need to prepare this environment before run the tests:
+All system-tests assertions and utilities are based on python and pytests. You need to prepare this environment before running the tests:
 
 - Python and pytests environment as described: [configure python and pytests for system-tests](../../README.md#requirements).
 - Ensure that requirements.txt is loaded (you can run "`./build.sh -i runner`")
 
 ## Run the scenario
 
-There is only one scenario declared: "`DOCKER_SSI`". But this unique scenario can run agains multiple variants of weblogs, conteinerized OS and architectures and therefore the matrix combinations might be very big.
+There is only one scenario declared: "`DOCKER_SSI`". But this unique scenario can run against multiple variants of weblogs, containerized OSes and architectures and therefore the number of matrix combinations might be very high.
 
 To help us to run a concrete case of the matrix variants, there are two scripts to be executed locally:
 
@@ -125,7 +125,7 @@ The following picture shows the main directories for the Docker SSI tests:
 
 ## Docker SSI definitions
 
-The key of the matrix of OSs (docker base image), architectures, language versions and weblogs resides on the file: `utils/docker_ssi/docker_ssi_definitions.py`. This file is the glue for all the components of this large matrix.
+The key of the matrix of OSes (docker base image), architectures, language versions and weblogs resides on the file: `utils/docker_ssi/docker_ssi_definitions.py`. This file is the glue for all the components of this large matrix.
 
 The first component is the Supported Images definition:
 
@@ -179,7 +179,7 @@ JS_APP = WeblogDescriptor(
 )
 ```
 
-Might be your weblog only supports one base image and it's required a runtime version to be installed, for example:
+Might be that your weblog only supports one base image and it requires a runtime version to be installed, for example:
 
 ```python
 TOMCAT_APP = WeblogDescriptor("tomcat-app", "java", [SupportedImages().TOMCAT_9_ARM64])
@@ -195,11 +195,11 @@ We can differentiate between two types of applications:
 - Applications that come already prepared/wrapped with a base image. For example, tomcat or jboss.
 - "Normal” applications that we encapsulate in a docker container and that in theory could be executed in more than one base image. For example, my HelloWorld application could run on Ubuntu22, Debian 12, RedHat.... images.
 
-The two previous points, will be better understood by following examplesThe previous.
+The two previous points will be better understood by following examplesThe previous.
 
 ### Create a new weblog using a prebuilt docker image
 
-For example, the Tomcat Java Web Server is distributed by a prebuilt docker images. If we want to test the tomcat 9, but we don't need to test Tomcat 9 installed on different Operating Systems, we could use the tomcat prebuilt images.
+For example, the Tomcat Java Web Server is distributed by prebuilt docker images. If we want to test the tomcat 9, but we don't need to test Tomcat 9 installed on different Operating Systems, we could use the tomcat prebuilt images.
 
 If we want to test the tomcat 9 image deploying a web application we'd do this:
 
@@ -215,7 +215,7 @@ FROM tomcat:9
 COPY --from=build app/payment-service/target/payment-service*.war /usr/local/tomcat/webapps/
 ```
 
-We can convert this standard image into "Docker SSI" image. We only need to parametrize the "From" clausule, and store the dockerfile as `utils/build/ssi/java/tomcat-app.Dockerfile`:
+We can convert this standard image into "Docker SSI" image. We only need to parametrize the "From" clause, and store the dockerfile as `utils/build/ssi/java/tomcat-app.Dockerfile`:
 
 ```yaml
 ARG BASE_IMAGE
@@ -257,7 +257,7 @@ ALL_WEBLOGS = [
 
 **NOTE:**
 
-When system-tests builds the docker images is going to install the SSI software inside of the container.
+When system-tests builds the docker images it is going to install the SSI software inside of the container.
 
 ---
 
@@ -269,9 +269,9 @@ If we want to run the "DOCKER_SSI" test cases for this new weblog:
 
 ### Create a new weblog deployable on different containers or operating systems
 
-We will use these types of weblogs to run an application agains different operating systems and distincs language runtime versions.
+We will use these types of weblogs to run an application against different operating systems and distinct language runtime versions.
 
-For example, we can define the PHP application that run the app into different OS and PHP runtime versions (`utils/build/ssi/php/php-app.Dockerfile`):
+For example, we can define a PHP application that runs on different OS and PHP runtime versions (`utils/build/ssi/php/php-app.Dockerfile`):
 
 ```yaml
 ARG BASE_IMAGE
@@ -338,7 +338,7 @@ ALL_WEBLOGS = [
 
 **NOTE:**
 
-When system-tests builds the docker images is going to install the SSI software inside of the container and the required runtime version.
+When system-tests builds the docker images it is going to install the SSI software inside of the container and the required runtime version.
 
 ---
 
@@ -351,7 +351,7 @@ If we want to run a "DOCKER_SSI" test case for this new weblog:
 ## Create a new test case
 
 The creation of test methods for the “Docker SSI” scenarios does not require anything special, we will do it as it is done for the rest of the system-tests scenarios.
-We can make request to the deployed weblog and make assertiong throught the "test agent" interface:
+We can make a request to the deployed weblog and make assertions through the "test agent" interface:
 
 ```python
 @scenarios.docker_ssi

docs/weblog/README.md

@@ -328,6 +328,50 @@ A POST request which will receive the following JSON body:
 
 An empty GET request that will execute two database queries, one to get a username and another to do a vulnerable SELECT using the obtained username.
 
+### POST /iast/sc/*
+
+These group of endpoints should trigger vulnerabilities detected by IAST with untrusted data coming from certain sources although the data is validated or sanitized by a configured security control
+
+#### POST /iast/sc/s/configured
+
+A post request using a parameter with a value that triggers a vulnerability. The value should be sanitized by a sanitizer security control configured for this vulnerability.
+
+#### POST /sc/s/not-configured
+
+A post request using a parameter  with a value that triggers a vulnerability. The value should be sanitized by a sanitizer security control that is not configured for this vulnerability.
+
+#### POST /sc/s/all
+
+A post request using a parameter with a value that triggers a vulnerability. The value should be sanitized by a sanitizer security control configured for all vulnerabilities.
+
+#### POST /sc/iv/configured
+
+A post request using a parameter with a value that triggers a vulnerability. The value should be validated by an input validator security control configured for this vulnerability.
+
+#### POST /sc/iv/not-configured
+
+A post request using a parameter with a value that triggers a vulnerability. The value should be validated by an input validator security control that is not configured for this vulnerability.
+
+#### POST /sc/iv/all
+
+A post request using a parameter  with a value that triggers a vulnerability. The value should be validated by an input validator security control configured for all vulnerabilities.
+
+#### POST /sc/iv/overloaded/secure
+
+A post request using two parameters that triggers a vulnerability. The values should be validated by an input validator security control with an overloaded method configured for all vulnerabilities.
+
+#### POST /sc/iv/overloaded/insecure
+
+A post request using two parameters that triggers a vulnerability. The values should be validated by an input validator security control with an overloaded method configured for other method signature.
+
+#### POST /sc/s/overloaded/secure
+
+A post request using a parameter with a value that triggers a vulnerability. The value should be sanitized by a sanitizer security control with an overloaded method configured for all vulnerabilities.
+
+#### POST /sc/s/overloaded/insecure
+
+A post request using a parameter with a value that triggers a vulnerability. The value should be sanitized by a sanitizer security control with an overloaded method configured for other method signature.
+
 ### GET /make_distant_call
 
 This endpoint accept a mandatory parameter `url`. It'll make a call to these url, and should returns a JSON response :

manifests/cpp.yml

@@ -54,6 +54,7 @@ tests/:
         test_path_parameter.py: irrelevant (ASM is not implemented in C++)
         test_sql_row.py: irrelevant (ASM is not implemented in C++)
         test_uri.py: irrelevant (ASM is not implemented in C++)
+      test_security_controls.py: irrelevant (ASM is not implemented in C++)
     rasp/:
       test_cmdi.py: irrelevant (ASM is not implemented in C++)
       test_lfi.py: irrelevant (ASM is not implemented in C++)

manifests/dotnet.yml

@@ -153,6 +153,8 @@ tests/:
           TestSqlRow: missing_feature
         test_uri.py:
           TestURI: irrelevant
+      test_security_controls.py:
+        TestSecurityControls: missing_feature
     rasp/:
       test_cmdi.py:
         Test_Cmdi_BodyJson: v3.7.0

manifests/golang.yml

@@ -161,6 +161,8 @@ tests/:
           TestSqlRow: missing_feature
         test_uri.py:
           TestURI: missing_feature
+      test_security_controls.py:
+        TestSecurityControls: missing_feature
     rasp/:
       test_cmdi.py: missing_feature
       test_lfi.py: missing_feature

manifests/java.yml

@@ -25,8 +25,12 @@ tests/:
   appsec/:
     api_security/:
       test_api_security_rc.py:
-        Test_API_Security_RC_ASM_DD_processors: missing_feature
-        Test_API_Security_RC_ASM_DD_scanners: missing_feature
+        Test_API_Security_RC_ASM_DD_processors:
+          '*': v1.38.0
+          spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
+        Test_API_Security_RC_ASM_DD_scanners:
+          '*': v1.38.0
+          spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
         Test_API_Security_RC_ASM_processor_overrides_and_custom_scanner: irrelevant (waf does not support it yet)
       test_apisec_sampling.py: missing_feature
       test_schemas.py:
@@ -42,7 +46,10 @@ tests/:
         Test_Schema_Request_Headers:
           '*': v1.31.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-        Test_Schema_Request_Json_Body: missing_feature
+        Test_Schema_Request_Json_Body:
+          '*': missing_feature
+          ratpack: v1.36.0
+          vertx3: v1.36.0
         Test_Schema_Request_Path_Parameters: missing_feature
         Test_Schema_Request_Query_Parameters:
           '*': v1.31.0
@@ -563,6 +570,12 @@ tests/:
             spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
             vertx3: missing_feature
             vertx4: missing_feature
+      test_security_controls.py:
+        TestSecurityControls:
+          '*': v1.45.0
+          play: missing_feature
+          ratpack: missing_feature
+          spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
     rasp/:
       test_cmdi.py:
         Test_Cmdi_BodyJson:
@@ -650,11 +663,9 @@ tests/:
         Test_Lfi_StackTrace:
           '*': v1.40.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
         Test_Lfi_Telemetry:
           '*': v1.40.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
         Test_Lfi_UrlQuery:
           '*': v1.40.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
@@ -694,15 +705,12 @@ tests/:
         Test_Shi_StackTrace:
           '*': v1.45.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
         Test_Shi_Telemetry:
           '*': v1.45.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
         Test_Shi_Telemetry_Variant_Tag:
           '*': v1.45.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
         Test_Shi_UrlQuery:
           '*': v1.45.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
@@ -749,13 +757,11 @@ tests/:
         Test_Sqli_StackTrace:
           '*': v1.39.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
           vertx3: v1.40.0 # issue in context propagation in 1.39.0
           vertx4: v1.40.0 # issue in context propagation in 1.39.0
         Test_Sqli_Telemetry:
           '*': v1.39.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
           vertx3: v1.40.0 # issue in context propagation in 1.39.0
           vertx4: v1.40.0 # issue in context propagation in 1.39.0
         Test_Sqli_UrlQuery:
@@ -804,13 +810,11 @@ tests/:
         Test_Ssrf_StackTrace:
           '*': v1.39.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
           vertx3: missing_feature (APPSEC-55781)
           vertx4: missing_feature (APPSEC-55781)
         Test_Ssrf_Telemetry:
           '*': v1.39.0
           spring-boot-3-native: missing_feature (GraalVM. Tracing support only)
-          spring-boot-payara: missing_feature (APPSEC-54966)
           vertx3: missing_feature (APPSEC-55781)
           vertx4: missing_feature (APPSEC-55781)
         Test_Ssrf_UrlQuery:

manifests/nodejs.yml

@@ -297,6 +297,8 @@ tests/:
             nextjs: missing_feature
         test_uri.py:
           TestURI: missing_feature
+      test_security_controls.py:
+        TestSecurityControls: missing_feature
     rasp/:
       test_cmdi.py:
         Test_Cmdi_BodyJson:

manifests/php.yml

@@ -147,6 +147,8 @@ tests/:
           TestSqlRow: missing_feature
         test_uri.py:
           TestURI: missing_feature
+      test_security_controls.py:
+        TestSecurityControls: missing_feature
     rasp/:
       test_cmdi.py: missing_feature
       test_lfi.py: missing_feature

manifests/python.yml

@@ -243,6 +243,8 @@ tests/:
           TestSqlRow: missing_feature
         test_uri.py:
           TestURI: missing_feature
+      test_security_controls.py:
+        TestSecurityControls: missing_feature
     rasp/:
       test_cmdi.py:
         Test_Cmdi_BodyJson: v2.20.0.dev

manifests/ruby.yml

@@ -148,6 +148,8 @@ tests/:
           TestSqlRow: missing_feature
         test_uri.py:
           TestURI: missing_feature
+      test_security_controls.py:
+        TestSecurityControls: missing_feature
     rasp/:
       test_cmdi.py: missing_feature
       test_lfi.py: missing_feature

tests/appsec/iast/sink/test_sql_injection.py

@@ -51,6 +51,5 @@ class TestSqlInjection_StackTrace:
     def setup_stack_trace(self):
         self.r = weblog.post("/iast/sqli/test_insecure", data={"username": "shaquille_oatmeal", "password": "123456"})
 
-    @missing_feature(context.weblog_variant == "jersey-grizzly2", reason="Endpoint responds 500")
     def test_stack_trace(self):
         validate_stack_traces(self.r)