Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding LLM eval for SQL generation notebook #1019

Merged
merged 6 commits into from
Jan 10, 2025

Conversation

colin-openai
Copy link
Collaborator

Summary

Briefly describe the changes and the goal of this PR. Make sure the PR title summarizes the changes effectively.

Motivation

Why are these changes necessary? How do they improve the cookbook?


For new content

When contributing new content, read through our contribution guidelines, and mark the following action items as completed:

  • I have added a new entry in registry.yaml (and, optionally, in authors.yaml) so that my content renders on the cookbook website.
  • I have conducted a self-review of my content based on the contribution guidelines:
    • Relevance: This content is related to building with OpenAI technologies and is useful to others.
    • Uniqueness: I have searched for related examples in the OpenAI Cookbook, and verified that my content offers new insights or unique information compared to existing documentation.
    • Spelling and Grammar: I have checked for spelling or grammatical mistakes.
    • Clarity: I have done a final read-through and verified that my submission is well-organized and easy to understand.
    • Correctness: The information I include is correct and all of my code executes successfully.
    • Completeness: I have explained everything fully, including all necessary references and citations.

We will rate each of these areas on a scale from 1 to 4, and will only accept contributions that score 3 or higher on all areas. Refer to our contribution guidelines for more details.

Copy link

@rishabhsagar rishabhsagar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me.

"EVALUATION_MODEL = \"gpt-4\"\n",
"\n",
"EVALUATION_PROMPT_TEMPLATE = \"\"\"\n",
"You will be given one summary written for an article. Your task is to rate the summary on one metric.\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might want to change the task description here from summary --> SQL queries

Copy link
Collaborator

@shyamal-anadkat shyamal-anadkat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left one comment, lgtm overall.

Copy link

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions bot added the Stale label Apr 22, 2024
Copy link

github-actions bot commented May 3, 2024

This PR was closed because it has been stalled for 10 days with no activity.

@github-actions github-actions bot closed this May 3, 2024
@colin-openai colin-openai reopened this Dec 20, 2024
@github-actions github-actions bot removed the Stale label Dec 21, 2024
@lspacagna-oai lspacagna-oai merged commit 2cca3a3 into main Jan 10, 2025
@lspacagna-oai lspacagna-oai deleted the colin/testing_sql_generation branch January 10, 2025 12:42
@lspacagna-oai lspacagna-oai restored the colin/testing_sql_generation branch January 10, 2025 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants