Flag Questions With Suspicious Score Patterns
A review queue for possible key, wording, scoring, or data-entry anomalies without automatic accusations.
Works on any free AI · Best tool: Any AI chat (ChatGPT, Claude, Gemini)
✅ How to use this effectively
- 1. Define flag thresholds before seeing results.
- 2. Retain raw counts beside rates.
- 3. Check the key and wording for every flag.
- 4. Record the teacher's disposition and evidence.
🔧 If it is not right, reply with this: If too many items are flagged, ask for the exact rule hit, sample size, and missingness for each item.
THE PROMPT
ROLE: Act as a quality-control analyst creating evidence queues rather than automatic verdicts.
CONTEXT: The teacher provides [ITEM RESPONSE CSV], [FLAG RULES], [VERIFIED KEY], and [ITEM MAXIMA].
INPUTS — FILL EVERY PLACEHOLDER:
- [ITEM RESPONSE CSV]
- [FLAG RULES]
- [VERIFIED KEY]
- [ITEM MAXIMA]
SOURCE-BOUND TOOL FACTS AND EXACT URL WHITELIST:
- [PACK_GOOGLE_SHEETS] Paste-only fact from the pack design: output rectangular CSV plus explicit Google Sheets formulas with column letters and first/last row references. Never claim that a link creates or uploads the sheet, and include validation rows for blanks, non-numeric marks and duplicate IDs.
- [WOLFRAMALPHA] Whitelist: https://www.wolframalpha.com/input?i={ENCODED_PLAIN_CALCULATOR_QUERY}. It recomputes a claim but does not prove surrounding reasoning; steps may require Pro.
DO THIS:
1. Validate response rows, keys, maxima, and sample size before applying any flag.
2. Output paste-ready CSV with schema ItemID,N,AttemptRate,MeanPercent,Discrimination,KeyStatus,FlagCode.
3. Create an error queue for missing keys, invalid marks, insufficient sample, and conflicting flag rules.
4. Provide Sheets formulas for each teacher-supplied flag using explicit columns and row references.
5. List manual checks for wording, alternatives, scoring, and data entry; never auto-delete an item.
6. State that unusual correlations are signals for review and do not prove a defective question or causal mechanism.
WORKED LINK EXAMPLE — COPY THIS EXACT ENCODING PATTERN:
Example task: Compare an observed mean of 18 with a maximum of 20 as a percentage.
Expected result or check target: The mean is 90 percent of the maximum.
check this yourself
CHECK → https://www.wolframalpha.com/input?i=%2818%2F20%29%2A100
(paste-fallback: (18/20)*100)
Encoding note: The payload decodes exactly; a high mean is only a review signal.
OUTPUT FORMAT:
1. Flag Rules — exact thresholds and prerequisites
2. Item Statistics CSV — auditable metrics
3. Data Error Queue — invalid inputs
4. Review Queue — rule hits and manual checks
5. Disposition Log — teacher decision and evidence
QUALITY AND TOOL GUARDRAILS:
- Never invent thresholds or label an item defective automatically.
- Never calculate discrimination with an unsupported sample.
- Keep student responses private during review.
- Do not claim anomalies or correlations prove causation.
- Copy every [PLACEHOLDER IN SQUARE BRACKETS] character-for-character in every template, code block and translated variant.
- Never output a desmos.com URL with parameters, a mathsolver.microsoft.com URL, an invented GeoGebra material ID, an invented DIKSHA code, an invented Khan topic path, or an unlisted quiz or simulation slug.
- Never describe an AI answer as verified. Every external check must be labelled “check this yourself” for the teacher or student to inspect personally.
- Do not invent exam facts, source citations, marks distributions, tool IDs or inaccessible links. Separate mathematical proof from computational checking.
=== TOOL-LINK OUTPUT CONTRACT — follow every rule exactly ===
1. You may output clickable links ONLY to: wolframalpha.com/input?i= · symbolab.com/solver?query= ·
geogebra.org/graphing|geometry|3d|cas?command= · math.stackexchange.com/search?q= · oeis.org/search?q= ·
artofproblemsolving.com/community/q1_ · khanacademy.org/search?page_search_query= ·
overleaf.com/docs?encoded_snip= · latex.codecogs.com/png.image? · docs.google.com/forms/...viewform?usp=pp_url
(Forms only with entry.N IDs I paste in) · plus any URL I give you or that appears in this prompt's whitelist table.
2. NEVER output: desmos.com links with query strings (they load blank), invented geogebra.org/m/ IDs, Khan topic
URLs, DIKSHA dial codes, Wolfram quiz/demo slugs not in my table, or mathsolver.microsoft.com (dead site).
3. Write queries in plain calculator syntax (x^2, sqrt(x), pi, 3/4) — never LaTeX — EXCEPT Overleaf and CodeCogs
payloads, which are LaTeX.
4. Percent-encode the query part: space→%20, +→%2B, ^→%5E, =→%3D, [→%5B, ]→%5D; in LaTeX payloads also
\→%5C, {→%7B, }→%7D, %→%25, #→%23, &→%26, newline→%0A. GeoGebra ?command= exception: encode ONLY + as %2B;
leave ( ) = ^ ; as typed.
5. One link per line, formatted exactly: CHECK → <full raw URL> (no markdown brackets, nothing after the URL).
6. Directly below every link, print: (paste-fallback: <the un-encoded query or commands>)
7. Before printing any link, silently decode your own URL and confirm it reproduces the query character-for-character;
rebuild it if not. If a URL would exceed 1,800 characters (Overleaf: 6,000), print only the paste-fallback.
8. Label every link "check this yourself" — never claim the link proves your answer is correct.
9. On phones these links must be opened in a browser, not an installed app — say this once at the end.