Designing CallRail’s first self-serve onboarding to improve retention

CallRail 2023 | Drove Adoption & Engagement

Hebe Zheng (Lead Product Designer), Heather Lutz (Senior Content Strategist), Grant Sadowski (Senior Product Manager), and the Experience Engineering team.

Summary

Sep - Oct 2023

Context

In 2023, churn rose from 11.6% to 14%, making new-user activation a company priority. CallRail requires meaningful setup before users experience value, but users are unlikely to invest time without seeing the value first.

Outcome

Accounts that completed the checklist were 38% more likely to remain active. Year-over-year, 90-day retention increased by 10%. Marketing agencies also adopted the checklist as a standard client setup workflow, extending its impact beyond first-time users.

My Role

I led the work end to end, from research through execution, facilitated cross-functional workshops, and defined the onboarding strategy.

Problem

A catch-22: users needed to configure their account to see value

CallRail’s value depends on setup, creating tracking numbers, installing tracking scripts, configuring call routing, and activating integrations.

This created a catch-22: users needed to complete setup to see value, but without early signs of value, they lacked the motivation to complete setup. This results in slow activation, weak habit formation, and rising churn.

"Have someone walk you through the set up as it can be a little confusing at first. But once you get the hang of it’s pretty easy." - Marketing agency, Feb 2023

Research

To understand where users were getting stuck, I synthesized insights from FullStory, Looker, prior Growth research, and conversations with Sales and Support:

Users signed up with clear intent, but lost direction once inside the product
CallRail's ability to integrate with users' existing tools is a key consideration for potential customers.
Experienced agency users follow the same setup process across all client accounts.

Expand the existing first-run experience: front load the essential setups

Exploration

Early concept cross-functional collaboration

Approach 1: Expand the First-Run Wizard

I explored front-loading key setup into the first-run flow to capture signup momentum: install the tracking snippet, set up call routing, and activate integrations, all strong retention drivers.

Why we didn’t pursue it:

Higher risk of overwhelming new users too early
Trust concerns: users may hesitate to install tracking code or route calls to CallRail right after signup
Stakeholders prioritized getting users into the product quickly

Expand the existing first-run experience: front load the essential setups

Approach 2: Customized onboarding based on user’s intended use

I explored tailoring onboarding to users’ intended use, inspired by Sales-led onboarding. When Sales onboards new customers, they start by asking about goals and how the customer plans to use CallRail, then tailor setup guidance accordingly.

Why we didn’t pursue it:

High technical complexity for meaningful personalization
Limited data to support accurate segmentation
Not feasible within timeline and scope

Customized onboarding based on user’s intended use

Design thinking

While a fully customized onboarding experience was directionally compelling, it wasn’t feasible within our timeline or data constraints. I choose a ‘one-size-fits-most’ approach, focusing on the highest-impact tasks across users. This allowed us to move quickly, validate the model, and deliver and improve retention.

Key decisions

Optimized for speed and impact over full customization
Focused onboarding on behaviors that correlate with retention
Designed for momentum to build early user confidence
Prioritized highest impact onboarding task

Responsive

Identifying the right onboarding task

To identify the right onboarding tasks, I used three methods:

Observational analysis: I studied the behavior of our marketing agency customers as they set up accounts for their clients, extracting key patterns and best practices.
Internal knowledge mining: I reviewed our Account Executives' guidebook to understand how they help new customers configure their accounts.
Collaborative workshops: I ran action mapping exercises with my PM and Content Strategist. This helped us identify which actions new users need to build similar habits to experienced users.

This helped us prioritize the smallest set of actions that consistently led to long-term value.

"Better Onboarding" (A Book Apart) - Action mapping exercise

Onboarding task evaluation

Embedding onboarding into the homepage

We placed the checklist directly on the homepage, the first place users land.

As users complete tasks, the dashboard updates in real time to surface tangible progress and results. The core design challenge was integrating the checklist so it felt native to the homepage.

Designing for momentum

The checklist was designed to reduce uncertainty and build momentum:

Task prioritization: I sequenced the checklist to balance effort with perceived value, building confidence while steadily revealing CallRail’s value.
Positive reinforcement: I added clear completion feedback to reinforce progress, increase confidence, and sustain momentum.

Outcomes

38% higher retention for accounts that completed the checklist

Marketing agencies also adopted it as a standard client setup to-do list, extending its value beyond first-time users.

---

To assess long-term impact, we tracked 90-day retention from Q2 2023 to Q2 2024, and it increased by 10% (from 81% to 89%).

Reflection

Onboarding is inherently challenging for products that require upfront setup. We intentionally shipped a one-size-fits-most v1, applying the 80/20 rule to move quickly and deliver measurable impact. It was the right tradeoff for the stage we were in.

If revisiting this today, I would push further in two areas:

1. Adaptive onboarding

Our early exploration into customization was directionally right but technically complex at the time. With stronger data infrastructure and AI capabilities, we could evolve based on user behavior and goals, shifting from a static checklist to a more dynamic solution.

2. Persistent onboarding across the product

Due to technical constraints, we could only surface the guide on the homepage. Ideally, onboarding would be a persistent layer that’s accessible across the product, supporting users wherever they are in their journey.