We’ve integrated Heap and VWO, our A/B testing platform, in order to get a more comprehensive and detailed look into A/B testing data. We’d sometimes lose cross-device data or multiple session tracking in VWO, maybe due to the length of our conversion funnel. So Heap fills in those gaps and is actually where we do most of our performance analysis and statistical significance checks.
What’s been most beneficial is to be able to dive into A/B test results based on all the customer attributes we have in Heap. We've had A/B tests where we found that they perform really well for organic traffic, but actually hurt paid traffic, or other tests where desktop and mobile are performing differently, or we can look at the customer source, or if they’re visiting different pages pre-conversion funnel, etc. etc. - a lot of deeper cuts. It's been super useful for us! Something that I struggled with in the past was that we could only group by two attributes, but I know that was just changed, so I think we can now group by up to four attributes, which is very exciting!
Curious to hear how others use Heap and VWO or other platforms to explore A/B testing data!
