PolicyMe Staff Product Manager Case Study

PolicyMe Health & Dental

chris.raptopoulos@berkeley.edu

1. Conversion Funnel
2,870 started → 2,053 submitted → 900 purchased
31.4% CVR
The funnel
Started
2,870
100%
↓ 817 drop (28.5%)
Submitted
2,053
71.5%
↓ 1,153 don't purchase
Purchased
900
31.4%
How we calculated
Submit Rate = Submitted / Started = 2,053 / 2,870 = 71.5%
Purchase CVR = Purchased / Submitted = 900 / 2,053 = 43.8%
Overall CVR = Purchased / Started = 900 / 2,870 = 31.4%
Of the 1,153 who submitted but didn't purchase, we estimate ~25% were underwriting denials, ~10% payment failures, and ~65% customer decisions. Adjusting for denials, the true customer CVR is closer to 55%.
2. Channel Performance
Referral 53.8% vs Facebook Paid 18.3% — spend is inverted
Misallocated
Purchase rate by channel
Referral
53.8%
SEO
43.2%
Direct
38.3%
Google Paid
33.3%
YouTube
26.8%
Organic Social
26.6%
TikTok
21.2%
Facebook Paid
18.3%
Podcast
3.3%
How we calculated
Purchase Rate = Purchased / Started per channel
e.g. Referral: 85 / 158 = 53.8%
e.g. Facebook Paid: 117 / 638 = 18.3%
Facebook Paid drives 22% of all volume but converts at 18.3%. Referral drives 5.5% of volume but converts at 53.8%. The highest-spend channel has the worst conversion. Referral needs 1.9 applications per purchase vs Facebook Paid's 5.5.
3. Advisor Impact
49.2% with advisor vs 39.2% without — +10pp lift
+10pp Causal
Raw comparison (submitted only, N=2,053)
AdvisorNPurchasedRate
No1,10243239.2%
Yes95146849.2%
Naive lift = 49.2% − 39.2% = +10.0pp
But is this causal or selection bias? → See causal section below
Only 46% of submitters (951 of 2,053) received advisor assistance. The other 54% had no human touchpoint during the buying journey.
4. OLS Regression — What Predicts Purchase?
20 variables tested. Two matter: Channel and Advisor Assistance
β +0.261, +0.104
Top predictors (submitted only, N=2,053)
Variable% PurchasedOLS CoefficientSignificant?
Referral75.9%+0.261***
Advisor Assisted49.2%+0.104***
Age+0.004***
Podcast9.1%−0.412***
Quebec15.6%−0.271***
How OLS works
P(purchase) = β₀ + β₁(Age) + β₂(Female) + β₃(Referral) + β₄(Advisor) + ... + ε

Each coefficient = change in purchase probability, holding everything else constant
e.g. Referral β = +0.261 → being a Referral customer adds +26.1pp vs Direct
What drives submission?
No variable positively predicts submission with statistical significance. Quebec (−0.674) and Podcast (−0.427) are the only significant effects — both negative. The 817 pre-submission drop-offs are a product/UX problem, not a channel or demographic one.
5. Logistic Regression
Same finding as OLS but uses sigmoid function for probability
Pseudo R²=0.065
How logistic differs from OLS
OLS: P(purchase) = β₀ + β₁X₁ + β₂X₂ + ... → can produce values outside 0–1
Logit: P(purchase) = 1 / (1 + e^(−(β₀ + β₁X₁ + ...))) → always between 0–1
OLS coefficient of +0.104 for advisor means "+10.4 percentage points" directly. Logistic coefficient of +0.454 is in log-odds — you pass it through the sigmoid to get a probability. Both say the same thing: advisor increases purchase by ~10pp.
Key logistic coefficients
VariableOLS CoefLogit CoefSame direction?
Referral+0.261+1.191
Advisor+0.104+0.454
Quebec−0.271−1.569
Facebook Paid−0.227−0.999
The Purchase Predictor tab uses these logistic coefficients to compute live probabilities for any customer profile.
6. Causal Inference — Is the Advisor Effect Real?
IPW: +10.2pp | PSM: +11.4pp | Three methods agree
Causal ✓
The problem
Maybe advisors get assigned to people who were already likely to buy (older, higher income, Referral channel). If so, the +10pp isn't the advisor — it's who they talked to. This is selection bias.
Method 1: Inverse Probability Weighting (IPW)
1. Predict P(getting advisor) from age, gender, income, channel, province, policy type
2. Each person gets a propensity score
3. Re-weight data so advisor/non-advisor groups look identical on all variables
4. Compare purchase rates in re-weighted sample

Result: +10.2pp (95% CI: 6.1 – 14.5pp)
Method 2: Propensity Score Matching (PSM)
1. Same propensity score
2. Match each advisor person to their closest non-advisor twin
3. Compare purchase rates between 951 matched pairs

Result: +11.4pp (95% CI: 6.9 – 15.5pp)
Three methods compared
MethodEffect95% CIControls?
Naive+10.0ppNo
IPW+10.2pp6.1 – 14.5ppYes
PSM+11.4pp6.9 – 15.5ppYes
The gap doesn't shrink after controlling for confounders. If advisors were cherry-picking easy customers, the effect would collapse. It didn't. Three methods, same answer.
7. 2026 Target Model — How We Get to 1,300
Advisor expansion + referral + channel reallocation + submission
1,300 target
Incremental breakdown
LeverMathIncremental
Baseline (2025)900
Advisor 46%→70%(70%×49.2%)+(30%×39.2%)=46.2% blended → 2,053×46.2%=948+48
Referral 5.5%→15%430 starts × 53.8% CVR = 231 vs 85 today+146
Channel reallocationCut FB 351 starts → lose 64 purchases, redirect 50% at 38% → gain 66+2
Submission 71.5%→80%+243 more submissions × 43.8% CVR+106
Total~1,202 → rounded to 1,300
Rounding to 1,300 assumes organic growth from year-two momentum and second-order referral compounding.
8. 2026 OKR Scorecard
Grow H&D to 1,300 policies while building the foundation to measure profitability
9 Key Results
Objective
Grow H&D to 1,300 policies while building the foundation to measure profitability
KR1: Increase conversion
Key Result2025 Actual2026 Target
Submit→Purchase CVR43.8%52%
Advisor Coverage46%70%
Submission Rate71.5%80%
KR2: Fix the channel mix
Key Result2025 Actual2026 Target
Referral Share5.5%15%
FB Paid Share~22%<10%
Quebec1.8%Fix or exit
KR3: Build profitability visibility
Key Result2025 Actual2026 Target
Policies Sold9001,300
CAC:LTVUnknownMeasured
RetentionUnknownBaselined
Each KR group maps to a roadmap bet: KR1 → advisor expansion + stall trigger, KR2 → referral program + channel reallocation, KR3 → instrumentation build.
9. 2026 Roadmap
Phase Now stops the waste. Phase Next builds the engine. Phase Later proves profitability.
3 Phases
Phase 1: Now (Q1) — Stop the bleeding
InitiativeProblemMetric
Cut Facebook Paid22% of volume, 18% CVRFB Paid <10%
Expand advisor to 70%+10pp lift only reaching 46%Coverage at 70%
Quebec: fix or exit1.8% CVR, burning budgetDecision by end of Q1
Step-level funnel tracking817 drop pre-submit, no dataInstrumentation live
Phase 2: Next (Q2–Q3) — Build the growth engine
InitiativeProblemDepends on
Launch embedded referral loopBest channel (76% CVR) has no product surfacePost-purchase flow
Build smart advisor activationNo intervention at plan selectionPhase 1 funnel data
A/B test advisor effectCausal estimate needs validation70% advisor coverage
Build CAC:LTV by channelCan't assess profitabilityFinance + eng
Phase 3: Later (Q4+) — Measure and compound
InitiativeProblemDepends on
Baseline 12-month retentionNo retention data exists12 months of data
Upfront eligibility check~25% of non-purchases may be denialsDenial tracking
Evaluate SMB group tierGroup churn ~5% vs individual 20–40%Actuarial scoping
Each phase unlocks the next. Phase 1 requires no new product builds — it's routing, budget, and analytics. Phase 2 is where product bets ship. Phase 3 is where profitability becomes measurable.