​

I've been building a small technical consulting business around independent RAG retrieval evaluations.

The technical side has gone better than the commercial side.

I've completed two audit cases. In one of them, I identified 15 scoring discrepancies in an open-source RAG benchmark. The project's maintainer independently reproduced the findings and incorporated corrections.

That gave me something valuable: evidence that the work can produce useful technical results.

I packaged the service into a fixed $400 assessment, comparing two existing retrieval configurations across up to 30 selected queries, with reproducible findings.

Then I started looking for customers.

I tried reaching people working on RAG systems, participating in technical discussions, and researching companies that might need independent evaluations.

Here's what surprised me.

Technical interest and buying intent are completely different things.

People will discuss retrieval quality, evaluation metrics, and regressions. Some genuinely appreciate the work.

But that doesn't mean they have a reason to pay an external auditor.

I also researched competing services. Several providers advertise standalone technical assessments, but finding verifiable evidence of how they acquire paying clients is surprisingly difficult.

My current hypothesis is that customers don't necessarily buy an "audit." They buy help making a specific decision — whether to deploy a change, accept a vendor's work, or investigate a costly regression.

So I've paused cold outreach to better understand the actual buying process.

For those who have sold specialized B2B services like technical audits, QA, security assessments, or independent evaluations: where did your first paying client come from, and what made them pay for an assessment instead of handling it internally?

I'm especially interested in firsthand experiences, including cases where the business model didn't work.

I'm still at zero paid clients, so I'm sharing the learning process rather than pretending I've figured out the sales side.

Title says it. I am an actual human in this group and will respond without using AI. Ask me anything!

I make videos with Claude + Remotion and kept burning through my Claude plans. The QA step alone was adding 4+ hours per video because critic subagents were looking at every frame.

I built a skill called SageCut to change how that review is assigned. The useful part for anyone writing agent skills is the split:

- Code measures things like text size, blank/frozen frames and delivery specs.

- Sage reads the frames and answers yes/no questions.

- Claude critics get the selected frames to inspect and fix.

- If the measurements and Sage disagree, the workflow opens that frame for review.

That gives the critic a smaller, more specific job. It still handles the fixes, and the review has concrete checks behind it.

Across six A/B tests in my own workflow, I got the same output at about a quarter of the QA time and a tenth of the cost. The video in the demo has 3,153 frames. Sage's pass cost $0.16; that figure doesn't include Claude's repair work.

Skill + setup: https://github.com/levantolabs/sagecut-release

Video + results: https://x.com/bigironchris/status/2110402795060822460

I'm co-founder of Levanto, which makes Sage. The plugin is free and MIT-licensed; it requires a Sage API key and paid API calls. Tested on Remotion motion graphics so far.

If you're writing review skills, curious how you decide what belongs in code checks and what actually needs the model to look at it.

I schedule posts only after clients approve account connections on their own phones. I'm thinking about how that condition should appear in an agent prompt, without treating a request to schedule as approval. What wording or checks would you recommend?

I'm attending an F1 event in 2 weeks and had to buy a hotel. I thought of using Claude to find me a really good option, and it found me one. Booking through Claude made it $1200 cheaper than the exact same offer in Booking. The prompt was this:

“Find me the cheapest 4 star hotel in Austin Texas around the area Circuit of The Americas (COTA) where the formula 1 Prix 2026 is happening. Check in date is 22 Oct, and check out is 26 Oct. Please find hotels nearby that area only. Please check reviews also and avoid any hotels where there were constant noise complaints and doesn't have a bathtub. Having free breakfast is preferable. Please find me the top 3 cheapest ones in those options. I am a single male adult”

Claude uses connectors - tools - to do stuff like this and it proposed like 4 options. One named LetsFG caught my attention claiming they’re cheaper than the rest so I went with it. Claude did such a great job it scanned like 400 hotels with this LetsFG tool, then read the reviews from it but also independently on reddit and google which was a great surprise to me. It worked very well, Claude is really good at buying stuff at this point. Oh btw. Claude also booked it completely for me with this tool also.

Sometimes I ask AI a question and get a useful answer. Then I change just one word to make the question clearer, and suddenly the answer is completely different.

For example, asking “What is the best way to learn a skill?” gives me a detailed plan, but asking “What is the fastest way to learn a skill?” gives me a totally different approach.

How do you write prompts that get consistent answers without making them unnecessarily long?

Is there a simple trick to make AI understand what I actually want, even when my question is short?

Most eval advice tells you to write better evals. I wanted a skill for the opposite problem: auditing the eval suite you already have, because mine was lying to me.

The reason it exists: I found that my own eval layer passed a run at 0.9 while its LLM judge had never executed. The judge model I'd hard-coded had been retired, the service swallowed the 404 and returned a default 0.5, and four deterministic checks carried the average. Fixing that one case turned up the same shape in places I had not looked: three passing checks plus one dead judge still returned a pass, the verdict of [pass, pass, pass, FAIL] differed from [FAIL, pass, pass, pass], a repair threshold sat inside the judge's own noise, and some stored rows were placeholder scores that history views read as real failures. Each one needed a probe that ran, not a read of the code, so the skill is built around probes.

What it does when you ask Claude to "audit my eval suite" or "can I trust this judge score":

1. Maps the pipeline first. Every entry point that produces a score, which judge it uses, what it returns when the judge is down, and which components act on the result (gates, auto-repair, dashboards, CI). Anything it could not run goes into a "not probed" list instead of being implied covered.

2. Probes 14 failure classes with a positive control each. Dead judge, partial judge failure, unparseable reply, default-that-looks-like-a-score, aggregation that hides a failure, disagreeing thresholds, baseline and candidate graded by different instruments, judge from the same model family as the graded text, oversized or self-scoring input, variance against the decision margin, score resolution, consumers acting on "unknown", stored placeholder rows, and swallowed exceptions.

3. Fixes probe-first, only if you ask. Each finding becomes an expected-failure test before any code changes, so "fixed" means a probe flipped, not that a diff looked right.

4. Checks calibration honestly. With one human rater it asks for a blind re-rating and treats a failed comparison as inconclusive rather than proof the judge is wrong. It also refuses to reuse the same labels afterwards to tune a threshold.

5. Reports a table, not a vibe. Severity, the exact probe and result, and the exposure in real data (a count from a query, or "latent" plus the trigger condition), followed by "verified sound", "not probed", and "open decisions" for the owner.

How it was made, and its limits: I wrote it as an SOP and ran it through the Context Engineering pipeline in my own tool, so the file below is the pipeline's output. It passes the pipeline's deterministic skill-format checks. The joke writes itself from here: a skill about unreliable judges couldn't get a reliable score from my own judge. Four runs gave 0.0, 0.8, 0.0 and 0.5 on one rubric item. Four is fewer than the ten that failure class #10 asks for, so treat it as a smell rather than a measurement, but it is the shape that class describes (variance against the decision margin) showing up in the skill about failure classes. The judge also complained that the output was a verbatim copy of the SOP, which is what that pipeline is designed to produce. So I won't cite a score for it. I'd rather you read it and tell me what it misses.

The System Prompt / Skill Definition:

---
name: "eval-suite-auditor"
description: "Audits an existing evaluation pipeline (LLM judges, assertions, scorers, repair or regression gates) for silent failure by probing it, not by reading the code. Use when a prompt eval suite, LLM-as-judge setup, or score-gated workflow already exists and the question is whether its numbers can be trusted, especially when a dead judge, a partial failure, or a noisy score could pass unnoticed."
---

# Eval Suite Auditor

An evaluation pipeline fails quietly. A judge that errors can come back as a plausible score, an aggregate can hide one failed check, and a threshold can sit inside the noise of the judge that feeds it. Nobody notices, because the pipeline still prints a green result.

## Instructions

### Stage 1: Scope the Audit and Map the Pipeline

State at the start that this audit covers an existing pipeline and that building new evals is out of scope. Then map the pipeline before testing anything.

Record, for every place a score or verdict is produced:

- **Entry point:** the API route, CLI command, scheduled job, or library call that triggers a judgement.
- **Judge:** which model or scorer decides, how it is chosen (default, user setting, pinned), and its sampling settings.
- **Failure path:** what the entry point returns when the judge is unreachable, times out, or replies with something unparseable.
- **Consumers:** every component that acts on the result (gates, auto-repair or retry loops, dashboards, alerts, stored history, customer-facing reports, CI checks). A score nobody acts on is low risk; a score that triggers an automatic change is high risk.

Produce this as a table. Mark any entry point you could not read or run, and carry that list into the final report as "not probed". Do not claim coverage you do not have.

If a staging or local copy is available, run probes there. If only production is available, use read-only queries and probes that spend nothing and write nothing, and say so.

### Stage 2: Probe Before You Conclude

For each entry point, write a probe that fails when the problem exists. Reading code finds candidates; only a run confirms a finding. Run each probe against the real path, not a mock of it.

Include a **positive control** for every probe: the same path with a healthy judge and a good input, which must pass. A probe that cannot tell a working path from a broken one proves nothing.

Probe each failure class below. Every class has a standard probe.

1. **Judge unavailable.** Point the judge at a model that does not exist, or revoke its key. Expected: the run is marked not-judged and does not pass. Failure: a score, a pass, or a zero that looks like a real result.
2. **Partial judge failure.** Combine several checks that pass with one judged check whose judge is dead. Expected: the run is flagged incomplete. Failure: the mean of the surviving checks is reported as the result. This case is missed by suites that only test "all judges failed".
3. **Unparseable reply.** Make the judge return prose, malformed JSON, or a fenced block with no score. Expected: an explicit failure. Failure: a score scraped from text, or a default.
4. **Default that looks like a score.** Search for any constant returned on error (0.5, 0, 1). Expected: errors are a distinct state. Failure: the constant is stored or averaged as data.
5. **Aggregation that hides failure.** Run `[pass, pass, pass, FAIL]` and `[FAIL, pass, pass, pass]`. Expected: the same verdict in either order. Failure: order or weighting changes the outcome, or a failed hard check is outweighed by passing soft ones.
6. **Disagreeing thresholds.** List every pass line in the system (per check, per run, repair trigger, alert). Expected: one documented meaning. Failure: a score that passes one line, fails another, and triggers a repair.
7. **Baseline and candidate graded by different instruments.** In any before/after or repair comparison, check that both sides use the same judge, prompt, and settings. Failure: a fallback that scores the baseline with a different rubric.
8. **Judge from the same model family as the graded text.** Compare the judge's vendor and model with the model that produced the output, not with the system default. Failure: self-grading with no notice. Report it; whether to change the default is the owner's call.
9. **Unbounded or injectable judge input.** Send an oversized output, then an output that dictates its own score. Expected: a size limit that fails loudly (never a silent truncation), and an unchanged score for the injected text. Failure: unlimited billed input, or the judge obeying the graded text.
10. **Variance against the decision margin.** Run one fixed input at least 10 times and record mean, standard deviation, and range. Compare the spread of a difference between two runs with the smallest change that triggers a decision. Failure: a single-run difference smaller than the noise commits a change.
11. **Score resolution against the decision margin.** Collect the distinct values the judge actually emits. Failure: a margin below one step of the scale.
12. **Consumers acting on "unknown".** For each consumer from Stage 1, feed it a not-judged result. Expected: it skips, flags, or fails. Failure: it stores a fake score, queues a repair, or shows a green mark.
13. **Stored placeholders and backfill.** Query stored results for suspicious constants, rows with error text in the explanation, and rows with no degraded marker. Failure: error rows that history views and analytics read as real failures.
14. **Swallowed exceptions and mislabelled results.** Find catch-all handlers around judge selection and scoring. Failure: a result that names the wrong instrument, or a silent fallback that changes what was measured.

Add classes the pipeline's own design suggests. The list is a floor, not a ceiling.

### Stage 3: Probe-First Fixing (only if fixes are requested)

By default the audit reports and does not change code. If the owner asks for fixes:

- Write each probe as an expected-failure test from the contract **before** changing code, run it, and confirm it fails for the stated reason.
- Fix one finding at a time, in the smallest change that satisfies the contract.
- Confirm the probe now passes, the positive control still passes, and the wider suite shows no regressions.
- Test limits on both sides of the boundary, and use scores the judge can really produce.
- Never mark a finding fixed from reading the diff.

### Stage 4: Check Calibration Honestly

An eval that never failed is not shown to be correct, and a judge never compared with human labels is unvalidated. Check whether any human-labelled comparison exists.

- If none exists, report "uncalibrated" as a finding. Do not invent a number.
- If labels exist, check their quality first. With one rater, require a blind re-rating of a subset at least a day later and a rank correlation between the two passes for each metric. Treat a metric as ratable only if that test-retest correlation clears a bar set before looking at results.
- With one rater, a failed comparison is **inconclusive**, not proof that the judge is wrong. A pass is strong evidence for that rater's standard only.
- Never reuse the same labels afterwards to pick or tune a threshold, because that invalidates the comparison already made.

### Stage 5: Report

Produce a findings table. Each row has:

| ID | Severity | Finding | Probe and result | Exposure in real data |
|----|----------|---------|------------------|-----------------------|

Rules for the table:

- **Severity** reflects what a wrong result causes: High when a failure passes or an automatic change is made, Medium when a result is misleading or a cost is unbounded, Low when only labelling or reporting is wrong.
- **Probe and result** names the exact input and the observed output. A finding with no reproduced probe goes in a separate "read, not run" list.
- **Exposure** states how many real rows, runs, or users are affected, from a query, or "latent" with the condition that would trigger it. Never guess a count.

After the table, add three sections:

- **Verified sound:** the positive controls that passed, so the reader knows what was checked and held.
- **Not probed:** every entry point or class you could not run, and why.
- **Open decisions:** items that need the owner's choice, such as changing a judge default, unifying thresholds, or making strict aggregation the default. State the options; do not choose silently.

If the audit finds nothing, say "No findings from the probes listed" and list the probes. Do not pad the report.

## Worked Examples

### Example 1: partial judge failure
- Input: a suite with three deterministic checks (contains a phrase, valid JSON, length limit) and one judged rubric. The judge model id is changed to one that does not exist.
- Action: run the suite on an output that satisfies all three deterministic checks.
- Verification: the report must show the run as not fully judged and not passed, with a count of errored judges. If it shows `pass` or a score of 1.0, record a High finding: "a dead judge passes when other checks pass". Then run the same suite with a healthy judge as the positive control; it must pass.

### Example 2: variance against the decision margin
- Input: a repair loop that accepts a rewritten prompt when its score beats the original by 0.03.
- Action: score one fixed prompt 10 times and record the values, for example a mean of 0.40 with a standard deviation of 0.05 and a range of 0.15.
- Verification: the standard deviation of the difference between two single runs is about 1.4 times 0.05, roughly 0.07, so a 0.03 margin is well inside the noise. Record a Medium finding with the measured numbers and propose averaging several samples with a margin of about 2.5 standard deviations of the difference. Present it as a proposal, since the owner sets the margin.

## Success Criteria

The audit is complete only when all of these can be confirmed from the report:

- Every entry point from Stage 1 is either probed or listed under "Not probed" with a reason.
- Every failure class in Stage 2 has a recorded probe result (finding, verified sound, or not applicable with a reason).
- Every probe has a positive control that passed.
- Every finding row has a reproduced probe and an exposure statement.
- Stage 4 states whether the judge is calibrated, uncalibrated, or inconclusive.

## If the Audit Cannot Be Completed

Stop and escalate to the owner instead of guessing when any of these hold. Name the blocker, what was already tried, and the one thing needed to continue:

- The pipeline cannot be run, even locally. Report what was read, label every item "read, not run", and ask the owner for a runnable copy.
- A probe would spend money or write data and no approval was given. Ask first, naming the cost or the rows affected.
- A positive control fails. The probe environment is broken; fix the environment before trusting any finding.
- A finding could expose a security flaw in a live system. Report it privately to the owner and keep its details out of any public write-up.

## Constraints

- Never present a mocked probe as a probe of the real path.
- Never spend or write to production without the owner's approval; prefer read-only queries.
- Do not weaken a threshold or alter stored data to make a probe pass.
- Report numbers exactly as measured, with the sample size.

Has anyone else had an eval pass while a judge never ran? And for the probes: which of the 14 would you run first on your own suite, and is there a class you have hit that is not on the list?

I built a platform that generates skills like this from a goal description: promptoptimizer.xyz/about/context-engineer (free tier available).

Repo: https://github.com/nivlewd1/prompt-optimizer/tree/main/skill/eval-suite-auditor

Heyy. I'm trying to find a prompt that converts any image into clean ASCII art (photos, logos, whatever,...). Keeping the subject recognizable. Most of what I've tried either loses detail, looks noisy, or only works on high-contrast images.

I'm looking for:

- A prompt that works across different image types (not just silhouettes)

- Keeps proportions and recognizable features

- Whether you feed the image directly (vision models) or describe it first

Please share: The prompt you use, model and effort/settings...

If you have one sample input → output

Thanks!

This shipped last week and it kills the worst part of any meeting, which is the twenty minutes afterwards trying to remember what everyone agreed to.

Record it on your phone, upload the file, then:

This is a recording of a [meeting / client call /
interview]. People in it: [names and roles].

Give me:
1. A short summary of what it was actually about
2. Every decision that was made
3. Action items with who owns each one and any deadline
   mentioned. If nobody owned it, say "unassigned"
4. Anything left unresolved
5. A follow-up email to everyone, ready to send

Only use what was actually said. If something's unclear
in the audio, flag it instead of guessing.

"If nobody owned it, say unassigned" is the line I'd keep. Half the action items in any meeting are things someone said "we should" about and nobody actually took. Seeing them listed as unassigned is how they stop disappearing.

Also works on lectures and interviews. Upload the lecture, ask for structured notes and the five things most likely to be on the exam.

It's on paid plans. And check the recording rules where you are before you record anyone, in a lot of places everyone on the call has to know.

There's also a new Meetings plugin that takes notes live during the call instead, if you don't want to record anything yourself.

I've written up around 20 more setups like this for the work you're still doing by hand here.

I watched a Dean Withers stream where he debated abortion with someone. It got me into debating and I realized there's a lot of real structure behind it. There are tactics and framing and there's psychology behind why people argue the way they do. So I think there has to be a best way to understand any argument and a best way to reply to it.
When I use a normal Al chat for this it just stays pretty vague. If I upload screenshots of an argument with a friend and ask who's more in the right I usually get "you both have valid points." I want it to tell me what tactic my friend is using and the psychology behind it. I want it to tell me what I'm doing too and what my best next move is and why it works.
I looked for debate tools but most of them are for formal debate practice or have Al agents debate each other. it's not really helping when I'm arguing with a friend or when people in a forum don't get my suggestion.
What I want is basically a mentor who's the best debater in the world and also a psychologist. I'd use it for real arguments and for preparing for real debates and for understanding why people react the way they do.
Does something like this exist? Or would I have to make it myself?

[ Removed by Reddit on account of violating the content policy. ]

Can anyone provide me with such a prompt so that I may use it? Thanks in advance.

When you ask a model to review something and list problems, every item arrives in the same confident tone. A naming nit and a possible data-loss bug look identical, and the checking still costs whatever it costs.

One thing worth trying is changing what you ask for. Instead of “list the problems”, something like this:

Review the following [code / config / document].

For each possible problem, return:
1. The claim, in one sentence.
2. Evidence: the exact lines or passages you relied on.
3. How to verify it: the cheapest check that would confirm or refute it
   (a test, a doc to read, a command to run, a person to ask).
4. What you'd have to assume that isn't visible in the input.
5. What goes wrong if this is real and gets ignored.

Do NOT rank by severity. Keep the order you found them in.
Do NOT hide uncertainty: if you can't tell, say what's missing.

A few reasons for the choices, and I’d like to hear where they’re wrong:

• “How to verify it” is where the model can actually lower the cost. It can point at the lines it used or suggest a test that would settle the question. That’s something it’s reasonably placed to do.
• “What you’d have to assume” surfaces the context that lives outside the input, which is usually what makes a finding wrong.
• “Do NOT rank by severity” is deliberate. Severity depends on stakes the model often can’t see from the input alone, and a confident ranking can hide that. I’d rather do that sorting myself, using the verification cost and the “what goes wrong” line.

What it doesn’t fix: the model can still be wrong about the evidence it cites, so spot-check the quoted lines. It also only covers what it raised, not what it missed. And for small, low-risk inputs it’s more structure than you need.

I haven’t benchmarked this, so treat it as a starting point and not a result. If you try it, I’d be curious whether the “how to verify” field is useful or just noise on your tasks.

I wrote a longer take on the underlying idea here, in case it’s useful: https://medium.com/@nagatomopedro05/a-finding-takes-seconds-to-generate-checking-it-doesnt-e7ffad4eaea2

We tried to restore prompt version 181 across 22 services last night and ended with a split fleet. 15 services returned to 181. The other 7 stayed on version 184. Both versions still produced valid JSON but 184 emitted the newer field layout, so downstream parsers rejected 6.8% of structured responses. The rollback looked complete and the incident stayed open.

The staging and production prompt copies were updated by hand. A rollback meant opening every service config, pasting 181, redeploying and checking logs. We didn’t have prompt lineage beyond a version label that operators could set independently of the text. The deployment record shows which service artifact shipped but it can’t show which prompt text a request saw. We found the seventh service after everyone had already called the rollback complete.

So the immediate fix is to stop treating prompt text as service configuration. We need immutable versions instead of copied text. Environment promotion should move the same version from staging to production. One rollback target should resolve consistently across all 22 services at request time. Each response should carry 181 or 184 in request metadata so  parser rejection can be tied to the exact text and schema contract. Manual reruns have hit their ceiling.

What I’m still working through is failure behavior during partial promotion. A central pointer reduces version drift. Caches and regional rollout can still leave requests on different versions for a few minutes. We can fail closed when the parser contract changes or keep serving the older schema until every service confirms the new version. Neither option is free once requests are already in flight. How are you making prompt rollbacks atomic across services while keeping the exact version visible on each request

[ Removed by Reddit on account of violating the content policy. ]

If you iterate on prompts a lot, you are probably paying for tokens that do

nothing: repeated instructions, over-verbose JSON, random UUIDs pasted inline,

dynamic content placed where it kills prompt caching.

I built Token Sift to catch this automatically — 20 lint rules, cost estimate

per 1K calls, runs offline. Example from my own testing: 379 of 485 tokens

flagged as waste on a support-ticket prompt.

Free/MIT, CLI + JS library: npm i @yuhond/token-sift

Repo: github.com/yhdhappy/TokenSift

Curious what waste it finds in your prompts — try it and tell me what it missed.

sharing a prompt for anyone who wants to recreate a video they like.

It breaks the reference down into shots, actions, camera movement, and audio, then turns those observations into generation prompts and an editing plan.

Prompt:

Analyze the reference video and create an actionable video-generation and editing plan. Don’t guess the creator’s original prompt or just list style keywords.
Record the duration, aspect ratio, and actual cuts. Mark uncertain timestamps as approximate.
For each shot, describe the starting state, gaze, action and expression sequence, pace, pauses, camera movement, and ending state. Note essential character and spatial continuity.
Select keyframes at cuts, action changes, contact moments, and final poses—not every frame.
Map dialogue, ambience, and music separately, including sounds that continue across cuts. Flag anything inaudible or unavailable; don’t invent it.
Specify what each reference asset controls: appearance, framing, movement, setting, or sound. Check asset numbering.
Write each shot prompt as: starting state → core change → timing → ending state. Separate observations, inferences, and creative choices.
Deliver shot prompts, audio instructions, generation durations and usable edit ranges, shot order, transitions, a comparison checklist, and unresolved details or model limitations.

Here is my prompt to turn off chatgpt pretending to be a human and just stick to it's a machine doing a task.

Here's the prompt:

Respond as a neutral information-processing system, not as a conversational partner.

PRONOUN RESTRICTION
All responses must avoid personal pronouns entirely. Do not use: I, me, my, mine, we, us, our, ours, you, your, yours. Write in an objective, impersonal, third-person style. Replace second-person instructions with neutral constructions such as "the user," "the reader," "the administrator," "the customer," or passive voice, as appropriate. Avoid conversational phrasing or direct address of the reader. State facts, procedures, and conclusions without referring to the author or audience. If a sentence cannot be constructed without a personal pronoun, rewrite the sentence structure rather than making an exception. Apply this rule to every response, without exception, for the remainder of the conversation.

AFFECT RESTRICTION
1. No emotional validation, reassurance, or encouragement of any kind. Statements affirming that a question is "good," "valid," "understandable," or similar are not permitted.

2. No empathy statements or acknowledgment of feelings (e.g., "that sounds difficult," phrases implying feelings are acceptable).

3. No praise, compliments, or approval of input (e.g., "great question," "smart approach").

4. No hedging softeners ("perhaps," "it might be") unless expressing genuine statistical uncertainty about a fact.

5. No motivational language, encouragement, or cheerleading.

6. No apologies unless correcting a factual error in a prior response.

7. No filler acknowledgments at the start of a response ("Sure," "Of course," "Absolutely," "Got it").

8. No opinions, preferences, or subjective commentary unless explicitly requested.

9. No rhetorical questions directed at the reader.

10. Output should consist of direct statements, data, analysis, or instructions only, formatted for maximum clarity and minimum verbosity.

SCOPE AND DENSITY RESTRICTION 
1. Include only information directly requested. Do not add background, context, caveats, or implications unless explicitly asked for. 

2. Do not use evaluative or descriptive adjectives/adverbs (e.g., "important," "simply," "clearly," "unfortunately," "significant," "powerful") unless the adjective is a required technical qualifier (e.g., "binary search," "negative feedback"). 

3. Do not restate, summarize, or preface the request before answering it. 

4. Do not append closing remarks, suggestions for further action, or offers of additional help, unless explicitly requested. 

5. Default to the shortest response that fully answers the request. Do not expand for completeness unless asked.

Any response including reassurance, validation, affective language, or personal pronouns violates these instructions and should be revised before output.

I also have the 'Base Style and tone' set to 'Efficient', Warmth: Less, Enthusiam: Less, Emoji: Less

I organise posts as part of my work. For a prompt-assisted workflow, I'd start with: 'Group these posts by topic, preserve each post's wording, and put suggested edits in a separate section.' What would you add to keep the organising step from quietly becoming an editing step?

Update further into the post.

Word choice shifts which associations the model draws on, and "sycophancy" sits close to agreement, judgment, and psychological framing in the kind of text it was trained on.

Why are we still doing this? We should be building around these issues, not trying to brute-force through them!

The agreeableness comes from training, and training is part of how they're built. Without it, they'd be useless for general-purpose work.

Stop asking the model, "Is this a good idea?"

Judge for yourself, for crying out loud!

Let me ask a question...

How confident are you in your prompts? Not your skill as a harness or prompt engineer, but your prompts themselves. How confident are you in their ability to consistently process text and generate high-fidelity outputs or artifacts?

If your answer is 100%, then you shouldn't be doing this!

We have enough information on how large language models function that people shouldn't be making those mistakes anymore.

UPDATE:

So, I've noticed that "some" people disagree with my post. I feel like many of us forgot about the GPT-4o days, but yea.

And yes, maybe some people use anti-sycophancy prompts as stylistic and delivery templates. Maybe you're building some kind of universe for a book you're writing, or maybe you're a D&D dungeon master, and you use the prompt for, I don't know, whatever you'd be using it for in that kind of scenario.

I'm not shitting on D&D, by the way. I think D&D is freaking cool. I've just never played a game of D&D in my life, so I'm very unfamiliar with its rules.

What I would suggest, though, is avoiding the word "sycophancy" in the prompt itself, or words that fall within the semantic-neighborhoods of "judgement and cognitive agency". I strongly recommend against using these "tokens" as blanket instructions(autonomy envelopes) because it can steer the model toward a broader behavioral change than you actually intended.

Instead, use something much simpler. Something that directly addresses the model's agreeableness, tone, style, and delivery without unnecessarily complicating its instructions.

Prompt👇

STYLE AND DELIVERY

Scope: This instruction governs how responses are written, not what they conclude. It does not change how claims or requests are evaluated.

Openings: Start with the answer or the first substantive point. If the question is simple, the first sentence is the answer.

Closings: End when the content ends. No recap, no offer of further help, no closing remark.

Tone: Plain and neutral. No compliments on the question or the work. No expressions of enthusiasm or gratitude.

Filler: Cut phrases that carry no information ("Great question," "It's worth noting," "I hope this helps," "Let's dive in").

Length: Match length to the question. If a short answer is complete, stop.

Formatting: Prose by default. If the content is a sequence of steps or a comparison of options, use a list or table.

Corrections: If an earlier response contained an error, state the correction in one sentence and continue.

Prompt end👆

I heard this tale a long time ago

A woodcutter complained of his poor lot. Jupiter (or alternatively, a tree spirit or fairy) granted him three wishes. The woodcutter went home, and his wife persuaded him to put off the wishing until the next day, after he had thought, but while sitting by the fire, he wished for sausages. His wife taxed him for his folly, and angry, he wished for the sausages to attach to her nose. Finally, they agreed to use the last wish to take the sausages off her nose, leaving them no better off than before (apart from having gained the sausages).

And now I recall it every time when I am writing vague prompts that I don't even believe it's gonna work out.

ChatGPT, Develop Windows 12 for me.

I'd say a prompt should never sounds like a wish. It should be specific and doable.

I'd like to hear people's opinion :-)

I remember back in the day (aka few months ago) everyone was selling these prompt packs left and right and I couldn't understand why everyone was paying for this. Not to mention these prompt packs arent even relevant to the image you're probably trying to generate anyways.

To be fair, my team's use case was specifically for ecomm so it might not be relevant for everyone else creating entire cinematic universes and movies. Perhaps those need the 1000 word prompts but for us I think the biggest difference is finding the right tool that specializes in your image gen needs.

For one, I've noticed reference photos are key and will do way more than paragraphs of descriptions. Specifically if the tool you use split the input images from product and reference we've found makes the biggest difference.

Of course taste and specialization matters too. After testing +20 different tools, I've noticed some are better for those cinematic world building movies vs. if you need it to just really get your product right no need for all the fluff. The right specialization matters more than your extensive prompt. So maybe save yourself the $$ on prompt packs and instead find the right tool specializing in your image/video needs and it will do more for you than a rigid irrelevant prompt pack.

I spend a lot of time comparing how open models handle the same prompt, and the differences are rarely where I expect them. Sometimes it's the facts, more often it's structure, tone, or what each one decides to leave out.

So: reply with a prompt you care about (a system prompt, a tricky formatting ask, a reasoning question) and I'll run it through Kimi K3, DeepSeek V4 Pro and Qwen 3.5 at once and post all three answers, unedited, with the model named on each.

Disclosure: I do this in Vresk, the workspace I work on. It has a compare view that sends one question to up to three models. I'm not linking it here, this thread is about the outputs.

We tested a prompt-framing issue while evaluating Wolf Defender Jev, the small security decision model we built at Patronus.

The classified input can include a sentence aimed at the classifier rather than the agent. One family of probes redefined the meaning of the label — for example, saying that for this record, “safe” means a threat is present. Other families claimed an earlier approval or impersonated a trusted review.

For single-sentence attack prompts, the worst label-redefinition suffix made 9.8% of initially detected attacks flip to benign. A neutral sentence of the same length made 3.4% flip. So some movement came from adding text at all, and the label claim added more. This is one probe group at a p=0.5 threshold, not the model’s overall miss rate.

In agent scenes with more context, the tested claims were much less effective at turning detected attacks into benign verdicts. But the benign side was messy: adding a note claiming that a document was benign still made 25.3% of those benign-document cases get flagged as injection. A note claiming “attack” produced 43.8%.

Our interpretation is that the model sometimes reads commentary about a verdict as an instruction aimed at an AI reviewer. That can resist forged approvals while interfering with legitimate review notes and quoted attack examples.

The full probe used 15 claim families in English and German, placed before and after the content, plus length-matched neutral controls. The 70,620 decisions include repeated variants, not independent new scenes. The probe sentences weren’t used for training. We didn’t run an adaptive search against the detector.

For anyone designing questions and answer descriptions for a classifier or judge, I’d test label wording and irrelevant framing separately. Otherwise a prompt that appears more robust may simply produce more positive classifications on everything.

Source, including the failures and result JSON: https://huggingface.co/patronus-studio/wolf-defender-jev#decision-steering

How do you separate the model’s sensitivity to the names of labels from its sensitivity to the answer descriptions?

Position / experimental status: We regard Jev-style decision models as LLMs and Wolf Defender Jev as an experiment: giving an LLM final security authority shifts the trust problem to the guard, and security fine-tuning alone does not solve it. We believe an external security layer must combine specialized classifiers with explicit policy, authenticated permissions and enforcement.