Methodology & data notes

This entire case study — the population, the 7,487 insight events, regression coefficients, and 125 simulated qualitative excerpts — is a synthetic dataset built for a portfolio piece. Nothing here describes real people, real wearable data, or a validated clinical model.

Design of the data

• 1,000 synthetic users, ~7.5 insight events each, repeated events within users.
• 100 users carry a simulated Crohn's diagnosis — intentionally oversampled for subgroup analysis. Their 10% share of this sample is a study-design choice, not a claim about real-world prevalence.
• 125 simulated qualitative excerpts were coded against a predefined 8-code framework in a separate AI-assisted analysis pass. I reviewed the resulting themes, contradictions, and negative cases before incorporating them into the product interpretation.

The model

• A lightweight, interpretable logistic decision-support model — not a production predictive system. Standard errors are user-clustered to account for repeated events per user.
• Evaluated on a user-level train/test split (no user appears in both sets): held-out AUC ≈ 0.73, Brier score ≈ 0.20.
• Performance falls substantially if ability_to_act is removed — the single largest driver of what this model can predict.
• ability_to_act is a hypothetical contextual variable used to explore whether a user's current capacity to act could improve delivery decisions. This case study does not establish how a real product would observe or infer that context.

Primary outcome

Action taken within 4 hours of receiving an insight. Exact time-to-action is preserved only as a secondary descriptive measure (median ≈ 62 min among those who acted) — it is not modeled or optimized for.

Decision Simulator's predicted probability

An illustrative combination of the primary adjusted model (ability to act, notification burden, baseline motivation, wearable signal, Crohn's status) with the message-design sensitivity model and the Crohn's×activity interaction term, evaluated at the levels you select. It is one transparent calculation, not a single validated predictive model — and per the decision rules, this number is the last input consulted, never the first.

Why I Designed 3 Evidence Views

The same evidence should not be presented the same way to every audience. I designed three ways to engage with it, moving from synthesis to exploration to application.

1 — Executive Summary: Understand the decision.
A one-page view for leaders who need the key findings, implications, and recommendations without working through the underlying analysis.

2 — Visual Dashboard: Explore the evidence.
Interactive visualizations make patterns, subgroup differences, and mixed-method findings easier to understand without requiring statistical expertise. Technical details remain available for readers who want to go deeper.

3 — Decision Simulator: Apply the evidence.
A hands-on model-assisted experience shows how changing user context can change the recommended product action: surface, modify, delay, or suppress.

The goal was not to present the same analysis three times. It was to translate the same evidence into the level of detail and interaction different stakeholders may need.

AI-assisted, human-directed analysis

I used AI throughout the workflow to accelerate the work—not to make the analytical decisions for me.

AI accelerated

• Synthetic data generation within predefined research constraints
• Quantitative analysis and statistical QA
• Qualitative coding and synthesis support
• Exploration of alternative interpretations
• Dashboard and simulator implementation in Claude Code

I directed

• Framing the product question and defining the outcome
• Designing the dataset, subgroup strategy, and analysis plan
• Choosing which statistical methods and sensitivity checks were appropriate
• Interpreting quantitative and qualitative evidence together
• Deciding which findings were strong enough to communicate
• Translating the evidence into product recommendations and future tests

A key part of the process was challenging the output rather than accepting it at face value. For example, I removed findings that were statistically valid but difficult to defend or explain, rejected a notification-volume result that was too weak to carry an executive recommendation, replaced a more dramatic trust statistic with a cleaner and more transparent comparison, and revised the message-design analysis when I realized “paired” messages were confounded with actionable content.

I also deliberately looked for null findings, contradictory qualitative perspectives, and subgroup differences rather than forcing the data to support the original hypotheses.

AI helped me move faster. Human judgment determined what the evidence meant, what could be responsibly claimed, and what the product should do next.