chore: OpenAIPrompt bug fixes #2334
Conversation
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2334 +/- ##
==========================================
- Coverage 84.52% 84.49% -0.04%
==========================================
Files 328 328
Lines 16844 16850 +6
Branches 1512 1516 +4
==========================================
- Hits 14238 14237 -1
- Misses 2606 2613 +7 ☔ View full report in Codecov by Sentry. |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
cognitive/src/main/scala/com/microsoft/azure/synapse/ml/services/openai/OpenAIPrompt.scala
Outdated
Show resolved
Hide resolved
| val results = completionNamed | ||
| .transform(dfTemplated) | ||
| .withColumn(getOutputCol, | ||
| getParser.parse(F.element_at(F.col(completionNamed.getOutputCol).getField("choices"), 1) | ||
| .getField("text"))) | ||
| .drop(completionNamed.getOutputCol) | ||
|
|
||
| val resultsFinal = results.select(results.columns.filter(_ != getErrorCol).map(col) :+ col(getErrorCol): _*) |
There was a problem hiding this comment.
is this necessary, do error colum,ns show up before other columns?
There was a problem hiding this comment.
Right now the Error column appears before the output column - we wanted it at the end so I added this
There was a problem hiding this comment.
is this change reflected in transform schema?
There was a problem hiding this comment.
After trying modifying SimpleHTTPTransformer, I think the withColumn(getOutputCol on line 192 is moving the output column to the back
There was a problem hiding this comment.
did some surface level research and it seems if we're rearranging columns, transformSchema doesn't need a change? if that's true, we're good with the code as is
There was a problem hiding this comment.
it unfortunately will as it is ordered under the hood, you can ovverride this func too to do the switch
There was a problem hiding this comment.
changed transformSchema to move the Error column to the back
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Related Issues/PRs
A couple of bug fixes:
What changes are proposed in this pull request?
Briefly describe the changes included in this Pull Request.
How is this patch tested?
Does this PR change any dependencies?
Does this PR add a new feature? If so, have you added samples on website?
website/docs/documentationfolder.Make sure you choose the correct class
estimators/transformersand namespace.DocTablepoints to correct API link.yarn run startto make sure the website renders correctly.<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples.WebsiteSamplesTestsjob pass in the pipeline.