Lilu has been exploring how to best evolve our product line. After in-depth market analysis and customer research we narrowed it down to a handful of ideas, that we wanted to test quantitatively. We set out to benchmark different metrics of the new product concepts against our debut product.
A fake door test was conducted with conversion-optimized landing pages for products, using identical ads targeted at a 1% lookalike audience based on an existing customer database. The campaign, leveraging Meta’s A/B experiment feature, ensured unique ad exposure per audience member. Visitors could click a "learn more" button and sign up with their email.
Key metrics collected included
cost per click
conversion rates for both button clicks and email sign-ups
acquisition cost (ad spend per email sign-up).
To minimize variation among the product concepts due to design, we used the same wireframe for the different landing pages. We included some of the most important blocks we’ve seen that drive conversion including adding very compelling hero imagery, clear CTAs, testimonials borrowed from the Lilu Massage Bra (Essential product line) and thorough yet concise product feature descriptions.
We opted to test the new product variations under a faux brand to avoid biases introduced by brand recognition.
I came up with a new name, logo and color palette for the faux brand.
Depicted below we have the Essential Massager (the Lilu Massage Bra), and 2 examples of the variations we tested. A wearable massage pump, and a massaging flange.
Results: In a comparison, Product A outperformed Product B in all categories, with Product A's Customer Acquisition Cost (CAC) being approximately 40% lower than that of Product B. This seems significant on its own, but gains context when compared to another fake door test involving the current product, the Lilu Massage Bra (Essential), both Product A and Product B demonstrated a substantial improvement in CAC over the Lilu Massage Bra.