Led key design experiment optimization through A/B testing and data analysis, establishing a replicable success framework that delivers long-term value to the company.
Key result
+9x increase in conversion rates (from <1% to 9%)
+32% YoY revenue growth.
+44% MoM revenue growth
Role & Timeline & Company
UI/UX Designer
2024/01-2024/04
MixerBox
My main contribution
Led A/B testing to optimize the subscription screen, experimenting with over 50 variables.
Focused on a holistic approach, going beyond paywall optimization to proactively improve the onboarding process.
Broke through growth stagnation by addressing both subscription and onboarding flows.
Established a replicable success template, delivering long-term value to the company.
Challenge
Business→
Low Conversion Rate
Current process conversion rate is below 1%, far below market average, resulting in low revenue.
Previous high was under 3%.
Stakeholders→
Revenue Impact
This process's revenue is highly correlated with company's overall revenue. C-level executives are highly focused on this and it needs improvement.
MVP
Hypothesis
Prompting users earlier in the app flow increases the likelihood of subscription.
MVP
Test the existing onboarding process with a paywall added for new users.
Testing
Start with the most simple version for testing.
Key Changes
Altered Flow Order
The sequence of steps in the process has been modified.
Reduced Subscription Steps
The number of steps required to subscribe has been decreased.
Expanded Acquisition Funnel
The process for attracting and acquiring new customers has been broadened.
Result
92% of users who completed the onboarding process (3k/day).
Conversion rate directly doubled.
The MVP validated that the strategy is on the right track.
Next, we will continuously test and optimize based on 2 different entry points.
A/B Test
How We Did It:
Rapid Iteration (Daily)
We conducted iterations at an extremely fast pace, testing at least three variations simultaneously.
Successful variations were then used as the new control group for further optimization.
User base: 1000/day
Variation Collection
We maintained a repository of potential variations to test.
A/B Test Insight 1
Title：Text 「AI」 → 「ChatGPT」
Compared to the broad term "AI," the popular keyword "ChatGPT" is more accessible and helps users connect with the latest and most effective impression.
Interestingly, we experimented with paywalls featuring "ChatGPT" versus those without it.
A/B Test Insight 2
Short Content is Better than Long Content.
Instead of overwhelming users with excessive information, concise and well-structured content improves decision-making efficiency.
By observing user behavior during the onboarding process, we found that users instinctively tap with their thumbs rather than scrolling up and down. Presenting all key information within a single screen is more effective than displaying excessive content, ensuring a smoother and quicker decision-making process.
Anchoring Effect: Two Pricing Options Work Better Than One or Three
When comparing pricing plans, two options perform better than one or three.
Three pricing options require too much time for quick decision-making during onboarding, while a single option lacks a reference point, making it unclear whether the price is high or low.
To improve clarity, pricing details should be presented with a direct comparison using the same unit for evaluation.
A/B Test Insight 3
In AI apps, paywalls are often designed with a dark background. To validate its impact, we conducted an experiment comparing a dark-themed and light-themed paywall.
Results showed that maintaining visual consistency between the onboarding experience and the paywall significantly improved user flow. A seamless transition encouraged users to continue clicking through without friction, leading to a smoother conversion process.
Second Challenge
Research to Define Design Direction
I proposed that testing the Paywall alone was not enough. Instead, we should take a more holistic approach by optimizing the entire Onboarding process leading up to the Paywall.
Onboarding Ideation
Design concept Ａ
Function-Oriented Design
Focus on Gradual Disclosure, Micro-interactions, and Clarity
Gradual Content Display for Better Engagement:
Advocates for a staged reveal of features.
Leveraging Micro-Interactions for Visual Guidance:
Emphasizes subtle animations.
Clear and Simple Copywriting:
Prioritizes easy-to-understand language.
Design concept Ｂ
Personalized immersive design
Focus on Personalization, Interaction, and Data
Personalized Results Trigger Engagement:
Emphasizes tailoring the experience.
Increased Interaction Leads to Greater Retention:
Focuses on user involvement.
Quantitative Data Captures Attention:
Highlights the power of numbers.
How to Make Design Decisions
Decision Criteria: Take small steps, iterate quickly for rapid optimization.
Based on decision criteria and engineering resources, we decided to try A first.
Final Design
Enhancing Clarity with Dynamic AI Showcase
1
Progressive Content Display
AI Can Answer Any Question
Real-life examples demonstrate the speed and power of AI in providing answers to a wide range of questions.
Use Cases for AI Tools
AI tools can be applied in various contexts, whether for work, learning, or entertainment.
Highlighting the Most Popular Features
AI-Generated Images: According to backend data, AI-generated images are the most frequently used feature by users, aside from general Q&A.
2
Onboarding to Paywall Consistency
Using dynamic elements makes it easier for users to understand the benefits that AI features can bring to them, compared to static elements.
3
Why keep the permission notification? To increase user retention.
In the original data, retaining the permission notification effectively boosts user retention.
Old
Too much text information overwhelms users.
New
Use animations to showcase AI's intelligence and power.
Focus on Core Features to Build User Anticipation
Old
Remove outdated product goals.
New
Directly showcase the most popular AI features to create anticipation before entering the app.
Result
9%
Conversion Rate
x18
Avg Daily Revenue
32%
YoY Growth
44%
MoM Growth
Quickly testing hypotheses with an MVP helps minimize risk and identify effective solutions faster.
This approach ensures resources are spent efficiently on ideas with real potential.
A/B Testing for Continuous Optimization
Implementing A/B testing enables rapid iteration, allowing us to fine-tune the user experience and steadily increase conversion rates through data-driven decisions.
Onboarding Optimization from Research to Execution
Led the optimization of the onboarding experience, conducting in-depth research and rigorous testing. The improvements successfully boosted engagement and revenue.
Creating a Replicable Success Framework
Developed a scalable and repeatable success model, ensuring long-term business value and sustained growth for the company.
Business Need
Utilize MVP and continuous A/B testing to achieve the team's monthly revenue goals.
Enhance product conversion rates to achieve over 40% monthly revenue growth. This strategy effectively shortened the development cycle and significantly improved product market fit.
Team Goal
Proactively conduct data analysis and design experiments to lead the project smoothly.
Use a data-driven approach to successfully guide the project from the concept phase to market release.
When the team faces bottlenecks, actively guide them to find the next steps through data analysis and design experiments.
Develop effective solutions. This approach not only increased the project's success rate but also enhanced the team's problem-solving capabilities and collaboration efficiency.
Design a successful template that becomes a model for other teams.
Through multiple iterations and user testing, create an efficient and innovative product version. This version achieved significant market success, improving user satisfaction and business metrics.
