-
Notifications
You must be signed in to change notification settings - Fork 9.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding LLM eval for SQL generation notebook #1019
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes look good to me.
"EVALUATION_MODEL = \"gpt-4\"\n", | ||
"\n", | ||
"EVALUATION_PROMPT_TEMPLATE = \"\"\"\n", | ||
"You will be given one summary written for an article. Your task is to rate the summary on one metric.\n", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you might want to change the task description here from summary --> SQL queries
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
left one comment, lgtm overall.
This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
This PR was closed because it has been stalled for 10 days with no activity. |
Summary
Briefly describe the changes and the goal of this PR. Make sure the PR title summarizes the changes effectively.
Motivation
Why are these changes necessary? How do they improve the cookbook?
For new content
When contributing new content, read through our contribution guidelines, and mark the following action items as completed:
We will rate each of these areas on a scale from 1 to 4, and will only accept contributions that score 3 or higher on all areas. Refer to our contribution guidelines for more details.