Problem interview: the problem prompt pack
I build Settl, an expense-splitting app, with AI. Fast building makes it easy to mistake activity for evidence. This pack helps you learn from specific past behaviour instead of asking people to design or predict a future product, before another week disappears into work that looked sensible at the time.
Why this trap costs more than it looks
The tempting assumption is that users can accurately specify the feature they will adopt later. It feels reasonable because visible work is emotionally satisfying. A new screen, a longer feature list, or a positive answer gives you something concrete to point at. The weaker evidence stays hidden because it is less exciting and often more uncomfortable.
The cost is not limited to the first build. Every decision creates maintenance, explanations, support, and more decisions around it. The earlier choice becomes harder to question because you have already spent time on it. A small evidence check before building is cheaper than defending the work after it exists.
What useful evidence looks like
Look for behaviour that existed before your solution was suggested:
- The story includes a recent specific event. This shows the problem is already changing behaviour.
- A workaround already exists. Repetition separates a real pattern from a strange afternoon.
- The problem created a visible cost or delay. A visible cost tells you why solving it might matter now.
Do not demand perfect proof. Early products rarely have it. You need enough evidence to make the next decision smaller and more honest. One specific recent story is usually worth more than ten broad opinions because you can inspect what actually happened.
Start with an evidence inventory
Gather the material you already have: user messages, notes, product screens, support conversations, usage summaries, and the assumptions behind the idea. Then paste this:
I am deciding how to learn from specific past behaviour instead of asking people to design or predict a future product.
Review the material I paste below.
Separate it into four groups:
1. Observed behaviour that already happened.
2. Direct statements with a specific example.
3. Opinions or compliments with no behaviour behind them.
4. Assumptions I am making without evidence.
Do not suggest solutions yet.
Quote the exact evidence for every item.
End with the three biggest evidence gaps.
This first pass is intentionally boring. Its job is to prevent the AI from filling empty spaces with plausible advice. If the material contains no evidence, that is a useful result. You now know the next step is learning, not polishing.
Run the smallest honest test
The test for this decision is to ask for the last real incident and follow it step by step, including the workaround and the consequence. Keep the test small enough to run this week. You are not trying to prove the whole business. You are trying to expose one assumption while changing as little as possible.
Design a small test for this decision: ask for the last real incident and follow it step by step, including the workaround and the consequence.
The assumption is: users can accurately specify the feature they will adopt later.
The test must take no more than seven days.
It must observe real behaviour, not ask for a prediction.
Give me: the participant, task, evidence to capture, pass condition,
failure condition, and the decision each result would trigger.
Do not propose a larger feature or a marketing campaign.
Decide the pass and failure conditions before seeing the result. Otherwise a hopeful founder can explain almost any outcome as good news. A useful test is allowed to disappoint you. That is how it saves time.
Ask questions that produce a timeline
Use questions that return to a real moment. Avoid asking whether someone likes the idea or might use it later. This prompt turns a vague conversation into something you can inspect:
Create five interview questions about this problem.
Every question must ask about a specific past event.
Cover: what happened, what they did first, the workaround,
what the problem cost, and what happened afterward.
Do not ask "would you use", "do you like", or "what features do you want".
Add one follow-up question under each question.
During the conversation, follow the timeline instead of rushing to your next prepared question. Ask to see the note, message, spreadsheet, or screen when that is appropriate. Specific artefacts reduce the pressure on memory and make contradictions easier to notice without turning the conversation into an interrogation.
Read the result without rescuing it
After the test, write down what happened before explaining why. Builders are good at rescuing a preferred idea with context: the participant was unusual, the timing was bad, or the test was too small. Some of that may be true, but record the behaviour first. Interpretation comes second.
Review these test notes as a skeptical product partner.
Create two columns: what happened, and my interpretation of it.
Flag any interpretation that is not supported by an observation.
Then give the strongest case for continuing and the strongest case for stopping.
Name one missing fact that would most change the decision.
Make the decision smaller
Build around repeated past behaviour, not enthusiasm for a hypothetical feature. You do not need a permanent answer. Choose the smallest next move that matches the evidence. That might be building one narrow path, rewriting a sentence, watching another session, charging a real price, or removing something that creates noise.
Use this decision note so the next week does not reopen the same argument:
- Decision: What are we doing now?
- Evidence: Which observed behaviours support it?
- Risk: What could still make this wrong?
- Next check: What will we inspect, and when?
- Not doing: Which attractive option is deliberately waiting?
A written "not doing" line matters. Good ideas return wearing new clothes. Recording the decision keeps the team from confusing repetition with new evidence.
Run this on your codebase
Paste your relevant product notes, user feedback, screens, and current plan after this prompt:
Help me learn from specific past behaviour instead of asking people to design or predict a future product.
My current assumption is: users can accurately specify the feature they will adopt later.
1. List the observed evidence and quote its source.
2. Separate behaviour, statements, opinions, and assumptions.
3. Name the weakest assumption carrying the most risk.
4. Design a seven-day test that will ask for the last real incident and follow it step by step, including the workaround and the consequence.
5. Set a pass condition and failure condition before the test runs.
6. Give me five past-behaviour interview questions.
7. End with a one-page decision note: decision, evidence, risk,
next check, and what we are deliberately not doing.
Do not invent customer facts. Mark missing evidence clearly.
Download the runnable pack
This pack turns the guide into a repeatable dry-run. It includes a local Node runner, an importable n8n workflow, Docker Compose, deterministic sample data, and GitHub Actions checks. Start with the sample input, inspect the proposed output, and keep a person in the loop before acting on it.