IntentGrid vs Optimizely Web Experimentation

Optimizely Web Experimentation answers “which version of this page performs better on average?” You create variants, in a visual editor or in code, split visitors between them and measure the outcome until the result is statistically significant. It is aimed at enterprise teams running an experimentation programme, and it produces evidence.

IntentGrid answers “what should this page say to the visitor reading it right now?” It reads the context of the visit (traffic source, campaign, approximate city, device, language, local time) and rewrites the headlines, body copy and button labels of the page you already have. There is no test to reach significance, which is also why it produces no evidence.

This page describes Optimizely only at the level of what it is for, because that is stable. Optimizely also sells other products, including personalization; feature lists and prices change, so check Optimizely's own site for the current ones.

More than most new sites get. The number depends on your current conversion rate and on the smallest improvement you care about detecting, and low conversion rates and small improvements both push it up sharply.

A worked example, using the standard two-proportion calculation at 95% confidence and 80% power: at a 2% conversion rate, detecting an improvement to 2.4% (a 20% relative lift) needs roughly 21,000 visitors per variant, so about 42,000 for a simple two-way test. At a few hundred visitors a week, that is a very long wait for one answer.

That is not a flaw in Optimizely. It is statistics, and it applies to every A/B testing tool. It is also the honest reason per-visitor rewriting exists: it does not wait for significance. The price is that it produces no evidence at all. Put your own numbers into any sample size calculator before you choose.

• You have the traffic for tests to reach significance in weeks rather than months.
• You need evidence, a measured result you can defend, rather than a page that feels better matched.
• You want to test more than words: layout, flows, pricing presentation, whole page designs.
• You run experiments as a team, with a process and people whose job it is.

• Your traffic is too small for a test to finish in any reasonable time.
• Your visitors arrive from very different places, and a single winning page would still be written for an average visitor who does not exist.
• You are one person or a small team, with no one to build and run experiments.

Be clear about what you give up. IntentGrid is in early access, the part that works today is a sandboxed demo, and it measures nothing, so you will not know from IntentGrid alone whether it helped.

If what you want is personalization by predefined audiences rather than testing, the category to look at is segment-based platforms, which are compared in the Mutiny alternative page.

If you have little traffic and one main channel, you may not need a tool yet: rewrite the headline for that channel yourself and watch what happens to sign-ups. It will not be statistically clean, and it costs nothing.