Merge pull request #2 from svilupp/js/add-templates

Add Template Functionality

.gitignore

@@ -5,5 +5,5 @@
 /docs/Manifest.toml
 /docs/build/
 
-/.DS_Store # macOS folder metadata
-/.vscode
+**/.DS_Store
+**/.vscode

CHANGELOG.md

@@ -0,0 +1,9 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+### Added
+- Add support for prompt templates with `AITemplate` struct. Search for suitable templates with `aitemplates("query string")` and then simply use them with `aigenerate(AITemplate(:TemplateABC); variableX = "some value") -> AIMessage` or use a dispatch on the template name as a `Symbol`, eg, `aigenerate(:TemplateABC; variableX = "some value") -> AIMessage`. Templates are saved as JSON files in the folder `templates/`. If you add new templates, you can reload them with `load_templates!()` (notice the exclamation mark to override the existing `TEMPLATE_STORE`).

Project.toml

@@ -7,11 +7,15 @@ version = "0.2.0-DEV"
 HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
 JSON3 = "0f8b85d8-7281-11e9-16c2-39a750bddbf1"
 OpenAI = "e9f21f70-7185-4079-aca2-91159181367c"
+PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
 
 [compat]
+Aqua = "0.7"
 HTTP = "1"
 JSON3 = "1"
 OpenAI = "0.8.7"
+PrecompileTools = "1"
+Test = "<0.0.1, 1"
 julia = "1.9,1.10"
 
 [extras]

README.md

@@ -36,7 +36,7 @@ ai"What is the capital of \$(country)?"
 # AIMessage("The capital of Spain is Madrid.")
 ```
 
-Pro tip: Use after-string-flags to select the model to be called, eg, `ai"What is the capital of France?"gpt4`. Great for those extra hard questions!
+Pro tip: Use after-string-flags to select the model to be called, eg, `ai"What is the capital of France?"gpt4` (use `gpt4t` for the new GPT-4 Turbo model). Great for those extra hard questions!
 
 For more complex prompt templates, you can use handlebars-style templating and provide variables as keyword arguments:
 
@@ -50,14 +50,18 @@ Pro tip: Use `asyncmap` to run multiple AI-powered tasks concurrently.
 
 Pro tip: If you use slow models (like GPT-4), you can use async version of `@ai_str` -> `@aai_str` to avoid blocking the REPL, eg, `aai"Say hi but slowly!"gpt4`
 
+For more practical examples, see the `examples/` folder and the [Advanced Examples](#advanced-examples) section below.
+
 ## Table of Contents
 
 - [PromptingTools.jl: "Your Daily Dose of AI Efficiency."](#promptingtoolsjl-your-daily-dose-of-ai-efficiency)
   - [Quick Start with `@ai_str` and Easy Templating](#quick-start-with-ai_str-and-easy-templating)
   - [Table of Contents](#table-of-contents)
   - [Why PromptingTools.jl](#why-promptingtoolsjl)
   - [Advanced Examples](#advanced-examples)
+    - [Seamless Integration Into Your Workflow](#seamless-integration-into-your-workflow)
     - [Advanced Prompts / Conversations](#advanced-prompts--conversations)
+    - [Templated Prompts](#templated-prompts)
     - [Asynchronous Execution](#asynchronous-execution)
     - [Model Aliases](#model-aliases)
     - [Embeddings](#embeddings)
@@ -90,11 +94,32 @@ Some features:
 
 ## Advanced Examples
 
-TODO:
+TODOs:
+
+- [ ] Add more practical examples (with DataFrames!)
+- [ ] Add mini tasks with structured extraction
+- [ ] Add an example of how to build a RAG app in 50 lines
+
+### Seamless Integration Into Your Workflow
+Google search is great, but it's a context switch. You often have to open a few pages and read through the discussion to find the answer you need. Same with the ChatGPT website.
+
+Imagine you are in VSCode, editing your `.gitignore` file. How do I ignore a file in all subfolders again?
 
-[ ] Add more practical examples (DataFrames!)
-[ ] Add mini tasks with structured extraction
-[ ] Add an example of how to build a RAG app in 50 lines
+All you need to do is to type:
+`aai"What to write in .gitignore to ignore file XYZ in any folder or subfolder?"`
+
+With `aai""` (as opposed to `ai""`), we make a non-blocking call to the LLM to not prevent you from continuing your work. When the answer is ready, we log it from the background:
+
+> [ Info: Tokens: 102 @ Cost: $0.0002 in 2.7 seconds
+> ┌ Info: AIMessage> To ignore a file called "XYZ" in any folder or subfolder, you can add the following line to your .gitignore file:
+> │ 
+> │ ```
+> │ **/XYZ
+> │ ```
+> │ 
+> └ This pattern uses the double asterisk (`**`) to match any folder or subfolder, and then specifies the name of the file you want to ignore.
+
+You probably saved 3-5 minutes on this task and probably another 5-10 minutes, because of the context switch/distraction you avoided. It's a small win, but it adds up quickly.
 
 ### Advanced Prompts / Conversations
 
@@ -126,6 +151,59 @@ aigenerate(new_conversation; object = "old iPhone")
 ```
 > AIMessage("Hmm, possess an old iPhone, I do not. But experience with attachments, I have. Detachment, I learned. True power and freedom, it brings...")
 
+### Templated Prompts
+
+With LLMs, the quality / robustness of your results depends on the quality of your prompts. But writing prompts is hard! That's why we offer a templating system to save you time and effort.
+
+To use a specific template (eg, `` to ask a Julia language):
+```julia
+msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
+```
+
+The above is equivalent to a more verbose version that explicitly uses the dispatch on `AITemplate`:
+```julia
+msg = aigenerate(AITemplate(:JuliaExpertAsk); ask = "How do I add packages?")
+```
+
+Find available templates with `aitemplates`:
+```julia
+tmps = aitemplates("JuliaExpertAsk")
+# Will surface one specific template
+# 1-element Vector{AITemplateMetadata}:
+# PromptingTools.AITemplateMetadata
+#   name: Symbol JuliaExpertAsk
+#   description: String "For asking questions about Julia language. Placeholders: `ask`"
+#   version: String "1"
+#   wordcount: Int64 237
+#   variables: Array{Symbol}((1,))
+#   system_preview: String "You are a world-class Julia language programmer with the knowledge of the latest syntax. Your commun"
+#   user_preview: String "# Question\n\n{{ask}}"
+#   source: String ""
+```
+The above gives you a good idea of what the template is about, what placeholders are available, and how much it would cost to use it (=wordcount).
+
+Search for all Julia-related templates:
+```julia
+tmps = aitemplates("Julia")
+# 2-element Vector{AITemplateMetadata}... -> more to come later!
+```
+
+If you are on VSCode, you can leverage nice tabular display with `vscodedisplay`:
+```julia
+using DataFrames
+tmps = aitemplates("Julia") |> DataFrame |> vscodedisplay
+```
+
+I have my selected template, how do I use it? Just use the "name" in `aigenerate` or `aiclassify` 
+ like you see in the first example!
+
+You can inspect any template by "rendering" it (this is what the LLM will see):
+```julia
+julia> AITemplate(:JudgeIsItTrue) |> PromptingTools.render
+```
+
+See more examples in the [examples/](examples/) folder.
+
 ### Asynchronous Execution
 
 You can leverage `asyncmap` to run multiple AI-powered tasks concurrently, improving performance for batch operations. 
@@ -183,14 +261,16 @@ aiclassify("Is two plus two four?")
 System prompts and higher-quality models can be used for more complex tasks, including knowing when to defer to a human:
 
 ```julia
-aiclassify(:IsStatementTrue; statement = "Is two plus three a vegetable on Mars?", model = "gpt4") 
+aiclassify(:JudgeIsItTrue; it = "Is two plus three a vegetable on Mars?", model = "gpt4t") 
 # unknown
 ```
 
-In the above example, we used a prompt template `:IsStatementTrue`, which automatically expands into the following system prompt (and a separate user prompt): 
+In the above example, we used a prompt template `:JudgeIsItTrue`, which automatically expands into the following system prompt (and a separate user prompt): 
 
 > "You are an impartial AI judge evaluating whether the provided statement is \"true\" or \"false\". Answer \"unknown\" if you cannot decide."
 
+For more information on templates, see the [Templated Prompts](#templated-prompts) section.
+
 ### Data Extraction
 
 !!! Experimental

examples/working_with_aitemplates.jl

@@ -0,0 +1,73 @@
+# This file contains examples of how to work with AITemplate(s).
+
+using PromptingTools
+const PT = PromptingTools
+
+# LLM responses are only as good as the prompts you give them. However, great prompts take long time to write -- AITemplate are a way to re-use great prompts!
+#
+# AITemplates are just a collection of templated prompts (ie, set of "messages" that have placeholders like {{question}})
+# 
+# They are saved as JSON files in the `templates` directory.
+# They are automatically loaded on package import, but you can always force a re-load with `PT.load_templates!()`
+PT.load_templates!();
+
+# You can (create them) and use them for any ai* function instead of a prompt:
+# Let's use a template called :JuliaExpertAsk
+# alternatively, you can use `AITemplate(:JuliaExpertAsk)` for cleaner dispatch
+msg = aigenerate(:JuliaExpertAsk; ask = "How do I add packages?")
+# ... some response from GPT3.5
+#
+# You can see that it had a placeholder for the actual question (`ask`) that we provided as a keyword argument. 
+# We did not have to write any system prompt for personas, tone, etc. -- it was all provided by the template!
+#
+# How to know which templates are available? You can search for them with `aitemplates()`:
+# You can search by Symbol (only for partial name match), String (partial match on name or description), or Regex (more fields)
+tmps = aitemplates("JuliaExpertAsk")
+# Outputs a list of available templates that match the search -- there is just one in this case:
+#
+# 1-element Vector{AITemplateMetadata}:
+# PromptingTools.AITemplateMetadata
+#   name: Symbol JuliaExpertAsk
+#   description: String "For asking questions about Julia language. Placeholders: `ask`"
+#   version: String "1"
+#   wordcount: Int64 237
+#   variables: Array{Symbol}((1,))
+#   system_preview: String "You are a world-class Julia language programmer with the knowledge of the latest syntax. Your commun"
+#   user_preview: String "# Question\n\n{{ask}}"
+#   source: String ""
+#
+# You see not just the description, but also a preview of the actual prompts, placeholders available, and the length (to gauge how much it would cost).
+#
+# If you use VSCode, you can display them in a nice scrollable table with `vscodedisplay`:
+using DataFrames
+DataFrame(tmp) |> vscodedisplay
+#
+#
+# You can also just `render` the template to see the underlying mesages:
+msgs = PT.render(AITemplate(:JuliaExpertAsk))
+#
+# 2-element Vector{PromptingTools.AbstractChatMessage}:
+#  PromptingTools.SystemMessage("You are a world-class Julia language programmer with the knowledge of the latest syntax. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.")
+#  PromptingTools.UserMessage("# Question\n\n{{ask}}")
+#
+# Now, you know exactly what's in the template! 
+#
+# If you want to modify it, simply change it and save it as a new file with `save_template` (see the docs `?save_template` for more details).
+# 
+# Let's adjust the previous template to be more specific to a data analysis question:
+tpl = [PT.SystemMessage("You are a world-class Julia language programmer with the knowledge of the latest syntax. You're also a senior Data Scientist and proficient in data analysis in Julia. Your communication is brief and concise. You're precise and answer only when you're confident in the high quality of your answer.")
+    PT.UserMessage("# Question\n\n{{ask}}")]
+# Templates are saved in the `templates` directory of the package. Name of the file will become the template name (eg, call `:JuliaDataExpertAsk`)
+filename = joinpath(pkgdir(PromptingTools),
+    "templates",
+    "persona-task",
+    "JuliaDataExpertAsk.json")
+PT.save_template(filename,
+    tpl;
+    description = "For asking data analysis questions in Julia language. Placeholders: `ask`")
+rm(filename) # cleanup if we don't like it
+# 
+# When you create a new template, remember to re-load the templates with `load_templates!()` so that it's available for use.
+PT.load_templates!();
+#
+# !!! If you have some good templates (or suggestions for the existing ones), please consider sharing them with the community by opening a PR to the `templates` directory!

src/PromptingTools.jl

@@ -2,7 +2,9 @@ module PromptingTools
 
 using OpenAI
 using JSON3
+using JSON3: StructTypes
 using HTTP
+using PrecompileTools
 
 # GLOBALS
 const MODEL_CHAT = "gpt-3.5-turbo"
@@ -20,7 +22,7 @@ const MODEL_ALIASES = Dict("gpt3" => "gpt-3.5-turbo",
     "gpt4t" => "gpt-4-1106-preview", # 4t is for "4 turbo"
     "gpt3t" => "gpt-3.5-turbo-1106", # 3t is for "3 turbo"
     "ada" => "text-embedding-ada-002")
-# below is defined in llm_interace.jl !
+# the below default is defined in llm_interace.jl !
 # const PROMPT_SCHEMA = OpenAISchema()
 
 include("utils.jl")
@@ -34,11 +36,25 @@ export AIMessage
 # export UserMessage, SystemMessage, DataMessage # for debugging only
 include("messages.jl")
 
+export aitemplates, AITemplate
+include("templates.jl")
+
+const TEMPLATE_STORE = Dict{Symbol, Any}()
+const TEMPLATE_METADATA = Vector{AITemplateMetadata}()
+
 ## Individual interfaces
 include("llm_openai.jl")
 
 ## Convenience utils
 export @ai_str, @aai_str
 include("macros.jl")
 
+function __init__()
+    # Load templates
+    load_templates!()
 end
+
+# Enable precompilation to reduce start time
+@compile_workload include("precompilation.jl")
+
+end # module PromptingTools

src/llm_interface.jl

@@ -40,7 +40,9 @@ end
 
 abstract type AbstractChatMLSchema <: AbstractPromptSchema end
 """
-ChatMLSchema is used by many open-source chatbots, by OpenAI models under the hood and by several models and inferfaces (eg, Ollama, vLLM)
+ChatMLSchema is used by many open-source chatbots, by OpenAI models (under the hood) and by several models and inferfaces (eg, Ollama, vLLM)
+
+You can explore it on [tiktokenizer](https://tiktokenizer.vercel.app/)
 
 It uses the following conversation structure:
 ```
@@ -54,11 +56,23 @@ It uses the following conversation structure:
 """
 struct ChatMLSchema <: AbstractChatMLSchema end
 
-## Dispatch into defaults
+abstract type AbstractManagedSchema <: AbstractPromptSchema end
+
+"""
+Ollama by default manages different models and their associated prompt schemas when you pass `system_prompt` and `prompt` fields to the API.
+
+Warning: It works only for 1 system message and 1 user message, so anything more than that has to be rejected.
+
+If you need to pass more messagese / longer conversational history, you can use define the model-specific schema directly and pass your Ollama requests with `raw=true`, 
+ which disables and templating and schema management by Ollama.
+"""
+struct OllamaManagedSchema <: AbstractManagedSchema end
+
+## Dispatch into default schema
 const PROMPT_SCHEMA = OpenAISchema()
 
 aigenerate(prompt; kwargs...) = aigenerate(PROMPT_SCHEMA, prompt; kwargs...)
 function aiembed(doc_or_docs, args...; kwargs...)
     aiembed(PROMPT_SCHEMA, doc_or_docs, args...; kwargs...)
 end
-aiclassify(prompt; kwargs...) = aiclassify(PROMPT_SCHEMA, prompt; kwargs...)
+aiclassify(prompt; kwargs...) = aiclassify(PROMPT_SCHEMA, prompt; kwargs...)