feat: Add support for jinja based template rendering of the dataset #438
+138
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of the change
Added a handler
apply_custom_data_formatting_jinja_template
which does jinja based template rendering of the dataset.Handling of edge case:
Example template:"### Input: {{Tweet text}} \n\n ### Response: {{text_label}}"
Jinja2 by default, does not support placeholders variable names with spaces (e.g., {{Tweet text}}), which will raise an error.
Hence additional preprocessing check (function:
transform_placeholders
) has been done. This checks if there is space between the placeholder variable and then process it accordingly (by modifying variable by{{element["Tweet text"]}}
.Related issue number
Issue: https://github.ibm.com/ai-foundation/watson-fm-stack-tracker/issues/1470
How to verify the PR
Verify added test cases.
Was the PR tested