[Feature Enhancement] Support Partial Plugin-Level Transformations in DynamicConfigTransformer

[srikanthjg]

Description

Currently, when a rule in our RuleEvaluator matches, the entire pipeline configuration is replaced using the template. This means that all processors before and after the matched plugin get lost, and we only end up with the new pipeline from the template. We want to support partial transformations in which only the matched plugin is replaced (or expanded) while retaining all other parts of the pipeline configuration.

UseCase

OSCF processor currently uses transformation to address different types of logs; but it comes with a limitation that no other processors can be used along with this processor.

Requirement

Based on rule and template yaml, Transform from:

  processors:
        processorA:
        processorB:
        processorC:

to

  processors:
        processorA:
        processorB1:
        processorB2:
        processorB3:
        processorC:

Proposed Solution

1. Extend RuleEvaluator, add partial transformation:

• Add fields in the rule YAML to indicate partial transformations (e.g., partialTransformation: true, transformOperation: "replaceProcessorWithList").

rule snippet:

pluginName: "ocsf"
applyWhen:
  - "$.pipelines.*.processors[?(@.ocsf)].ocsf"
partialTransformation: true
transformOperation: "replaceProcessorWithList"
templateFile: "new_processors_template.yaml"

template snippet:

processors:
   processorA1:
   processorA2:
   processorA3:

2. Store Plugin-Level Template Snippet

• Instead of holding a full pipeline in pipelineTemplateModel, load or parse a snippet containing only the replacement/expanded processors for the matched plugin.
• Keep placeholders or special logic in that snippet if needed.

3. Update DynamicConfigTransformer
   Locate the exact matched processor in the pipeline (by plugin name or JSON Path).
   Remove the old processor.
   Insert the new processor(s) from the partial template snippet at the same position.
   Keep all other processors (and sources/sinks) as-is.

Backward Compatibility: If partialTransformation is not specified (or is false), continue with the existing “full replacement” logic.
Existing users who rely on full pipeline replacement remain unaffected.

Acceptance Criteria

• Partial transformation replaces exactly one matched plugin with new processors from the template while preserving all other processors.
• If multiple plugins match, transform them all
• Full transformation logic remains intact for rules that do not specify partialTransformation: true.