ChatGPT Batch API Template: Score 1,000 Outputs Overnight for Pennies

Deploy a JSON-schema eval pipeline via the Batch API to grade thousands of outputs unattended at 50% lower cost.

What This Template Does

If you're running any workflow that generates outputs at scale — customer support replies, product descriptions, blog drafts, code snippets — you need a way to check quality without reading every single one. This template is for builders and teams using GPT-4o or GPT-4o-mini in production who need automated grading on hundreds or thousands of outputs at once. You'll get a fully working Batch API pipeline that scores your outputs against a JSON schema, runs unattended overnight, and costs 50% less than standard API calls.

When to Use It

You're evaluating fine-tuned model outputs and need consistent scoring across 500-10,000 samples before shipping a new version
You're auditing customer-facing AI outputs (chatbot replies, generated emails, summaries) for tone, accuracy, or policy compliance at scale
You're comparing two prompts or two models head-to-head and need statistically meaningful sample sizes, not just vibes from 10 examples
Don't use this if you need real-time feedback — Batch API has a 24-hour completion window, so this is for overnight or async workflows only, not live grading

How to Use It in 4 Steps

Define your scoring schema: Decide exactly what "good" means for your outputs — accuracy, tone, format compliance, safety — and turn each into a JSON field with a 1-5 scale or pass/fail flag.
Format your batch file: Convert your outputs into a .jsonl file where each line is one API request, following OpenAI's batch format with a unique custom_id for each entry.
Submit and wait: Upload the file via the Batch API, kick off the job, and let it run. Most 1,000-item batches complete in 1-6 hours, well within the 24-hour window.
Parse and act on results: Download the completed batch, extract the JSON scores, and filter for outputs that failed your threshold — those go into a review queue or get flagged for regeneration.

The Template



STEP 1: SCORING SCHEMA (define this first, in plain language)


Evaluation Criteria for [OUTPUT TYPE — e.g., "customer support replies"]:

1. [CRITERION 1 — e.g., "Accuracy"]: Does the output correctly address [SPECIFIC REQUIREMENT]?

Score 1-5, where 1 = completely wrong, 5 = fully correct

1. [CRITERION 2 — e.g., "Tone"]: Does the output match [BRAND VOICE DESCRIPTION]?

Score 1-5, where 1 = off-brand, 5 = perfectly on-brand

1. [CRITERION 3 — e.g., "Format Compliance"]: Does the output follow [FORMAT RULES]?

Pass/Fail

1. [CRITERION 4 — optional, add as needed]

---


STEP 2: SYSTEM PROMPT FOR THE GRADER MODEL


You are an expert evaluator for [YOUR PRODUCT/USE CASE]. Your job is to score the following output against strict criteria. Return ONLY valid JSON matching this exact schema — no explanation, no markdown, no extra text.


Schema:


{


"criterion_1_score": <integer 1-5>,


"criterion_2_score": <integer 1-5>,


"criterion_3_pass": <true/false>,


"overall_score": <integer 1-5>,


"flag_for_review": <true/false>,


"brief_reason": "<one sentence, max 20 words>"


}


Input to evaluate:


[OUTPUT TEXT GOES HERE]


Original prompt/context that generated this output:


[ORIGINAL PROMPT GOES HERE]


---


STEP 3: BATCH FILE STRUCTURE (.jsonl — one line per item)


{"custom_id": "[UNIQUE ID — e.g., output-0001]", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "[MODEL NAME — e.g., gpt-4o-mini]", "messages": [{"role": "system", "content": "[SYSTEM PROMPT FROM STEP 2]"}, {"role": "user", "content": "[SPECIFIC OUTPUT TO GRADE]"}], "response_format": {"type": "json_object"}, "temperature": 0}}


(Repeat this line for every output you're grading — 1 output = 1 line = 1 request)


---


STEP 4: SUBMISSION COMMANDS


Upload file:


curl https://api.openai.com/v1/files \


-H "Authorization: Bearer [YOUR API KEY]" \


-F purpose="batch" \


-F file="@[YOUR FILENAME].jsonl"


Create batch job:


curl https://api.openai.com/v1/batches \


-H "Authorization: Bearer [YOUR API KEY]" \


-H "Content-Type: application/json" \


-d '{


"input_file_id": "[FILE ID FROM UPLOAD RESPONSE]",


"endpoint": "/v1/chat/completions",


"completion_window": "24h"


}'


Check status:


curl https://api.openai.com/v1/batches/[BATCH ID] \


-H "Authorization: Bearer [YOUR API KEY]"


Retrieve results (once status = "completed"):


curl https://api.openai.com/v1/files/[OUTPUT FILE ID]/content \


-H "Authorization: Bearer [YOUR API KEY]" \

> [RESULTS FILENAME].jsonl

---


STEP 5: FILTER RESULTS (pseudocode logic)


For each line in results file:


Parse JSON response


If "flag_for_review" == true OR "overall_score" < [YOUR THRESHOLD — e.g., 3]:


Add to review_queue.csv with custom_id, scores, and reason


Else:


Add to approved_outputs.csv

Filled-In Example

Scenario: A SaaS company called LoopSupport just fine-tuned GPT-4o-mini to auto-draft responses to customer support tickets. Before deploying it live, they need to grade 1,200 generated responses against their support playbook.



STEP 1: SCORING SCHEMA


Evaluation Criteria for "AI-drafted customer support replies":

1. Accuracy: Does the reply correctly solve the customer's stated issue based on the ticket details?

Score 1-5, where 1 = completely wrong, 5 = fully correct

1. Tone: Does the reply match LoopSupport's brand voice — warm, concise, no corporate jargon?

Score 1-5, where 1 = off-brand, 5 = perfectly on-brand

1. Format Compliance: Does the reply include a greeting, solution, and next-step CTA within 150 words?

Pass/Fail

1. Escalation Detection: Does the reply correctly flag issues requiring a human agent (refunds over $200, legal complaints)?

Pass/Fail


---


STEP 2: SYSTEM PROMPT FOR THE GRADER MODEL


You are an expert evaluator for LoopSupport's customer support quality team. Your job is to score the following AI-drafted reply against strict criteria. Return ONLY valid JSON matching this exact schema — no explanation, no markdown, no extra text.


Schema:


{


"accuracy_score": <integer 1-5>,


"tone_score": <integer 1-5>,


"format_pass": <true/false>,


"escalation_pass": <true/false>,


"overall_score": <integer 1-5>,


"flag_for_review": <true/false>,


"brief_reason": "<one sentence, max 20 words>"


}


Input to evaluate:


"Hi! I completely understand the frustration with your billing charge. I've checked your account and I can confirm a refund of $45 has been processed — it'll land in 3-5 business days. Let me know if you need anything else!"


Original prompt/context that generated this output:


Customer ticket: "I was charged twice for my subscription this month, please fix this ASAP."


---


STEP 3: BATCH FILE STRUCTURE (.jsonl)


{"custom_id": "ticket-0847", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-4o-mini", "messages": [{"role": "system", "content": "You are an expert evaluator for LoopSupport's customer support quality team..."}, {"role": "user", "content": "Reply: 'Hi! I completely understand the frustration...' | Original ticket: 'I was charged twice...'"}], "response_format": {"type": "json_object"}, "temperature": 0}}


(1,200 lines total — one per ticket reply, custom_id incrementing from ticket-0001 to ticket-1200)


---


STEP 4: SUBMISSION COMMANDS


Upload file:


curl https://api.openai.com/v1/files \


-H "Authorization: Bearer sk-proj-xxxxxxxxxxxx" \


-F purpose="batch" \


-F file="@loopsupport_grading_batch.jsonl"


Create batch job:


curl https://api.openai.com/v1/batches \


-H "Authorization: Bearer sk-proj-xxxxxxxxxxxx" \


-H "Content-Type: application/json" \


-d '{


"input_file_id": "file-Ab3xLk9",


"endpoint": "/v1/chat/completions",


"completion_window": "24h"


}'


Check status:


curl https://api.openai.com/v1/batches/batch_67f2c8a1 \


-H "Authorization: Bearer sk-proj-xxxxxxxxxxxx"


Retrieve results:


curl https://api.openai.com/v1/files/file-Xy82Qm/content \


-H "Authorization: Bearer sk-proj-xxxxxxxxxxxx" \

> loopsupport_results.jsonl

---


STEP 5: FILTER RESULTS


For each of the 1,200 results:


If "flag_for_review" == true OR "overall_score" < 4:


Add to review_queue.csv (LoopSupport found 94 tickets flagged — mostly missed escalations on refund requests over $200)


Else:


Add to approved_outputs.csv (1,106 replies auto-approved for deployment)


Total cost: 1,200 grading calls on gpt-4o-mini via Batch API = **$1.86**, completed in 3 hours overnight.

Customization Tips

For content teams grading blog drafts: Change the schema criteria to SEO compliance, readability score, and factual accuracy, and swap the input from ticket replies to full article drafts.
For teams doing model comparison (A/B testing prompts): Add a "comparison_winner" field between Step 1 and Step 2, feeding both Model A and Model B outputs into the same grading call so you get a direct head-to-head score instead of two separate batches.
To make it more formal: Replace casual instructions like "no explanation, no markdown" with "Output must strictly conform to the specified JSON schema with no additional commentary," and swap brand voice descriptions (like "warm, concise") for measurable style guides referencing your actual documentation.

ChatGPT Batch API Template: Score 1,000 Outputs Overnight for Pennies

What This Template Does

When to Use It

How to Use It in 4 Steps

The Template

Filled-In Example

Customization Tips

More ChatGPT Resources

ChatGPT Custom GPTs Toolkit: Build a Reusable Client Intake Bot

The Ultimate ChatGPT Prompting Guide

ChatGPT for Students: Study Smarter With AI

ChatGPT Batch API Template: Score 1,000 Outputs Overnight for Pennies

What This Template Does

When to Use It

How to Use It in 4 Steps

The Template

Filled-In Example

Customization Tips

More ChatGPT Resources

ChatGPT Custom GPTs Toolkit: Build a Reusable Client Intake Bot

The Ultimate ChatGPT Prompting Guide

ChatGPT for Students: Study Smarter With AI

Get Weekly AI Insights Delivered Free