Despite people discussing the topic in 2001, the issue of content flicker still exists. And the more we speak to various businesses, the more we realize that this is still a problem in our industry. A bit of the page loads in, you get a bit of white flash, and then the user is presented with content from the experiment that they’ve fallen into. Just today – we were talking to a large European airliner as part of a Pre-Sales engagement, and one of their gripes with their existing provider was this very problem. The more you pay attention, the more you see it – everywhere. But is it an acceptable compromise for being able to run ABn/MVT testing projects?
Let’s get this bit out of the way so we’re all on the same page – this comes in various names, including Flash of Original Content, Content Flickering, or Flash of Unstyled Content. It happens because most mechanisms for project delivery are asynchronous, meaning that they do not block the page whilst waiting for an experiment to download and run. Instead, they allow the page to continue to load whilst waiting for information to come back.
Does it matter?
While there is clearly a mixed opinion in the industry, two points should be addressed.
The first is a simple principle of User Experience. As humans, we’re able to recognize images that we’ve seen for just 13 miliseconds. If you see images, layouts, or even worse prices/discounts that quickly change, this becomes nothing short of a failure for the company. If things become less favourable for you as a user, you’re likely to leave and possibly never return.
The second is the notion of the Hawthorne Effect, Pygmalion or Placebo. In short, participants of studies have been noticed to behave differently when it is clear that they’re being studied, or when they feel there is some form of expectation on them. In this context, users who know that they are part of an experiment may well behave differently to those who don’t. Even personally, I’ll close a browser if I see Content Flickering despite understanding its need for businesses. If a user understands that Tests are usually constructed to help make a business more money, you certainly wouldn’t want to be a part of that.
So, does it matter? Absolutely. Users having knowledge of being in a Test could invalidate your results. How long have you been testing for with this issue? And most importantly, how many business decisions have been made based on that data? It’s a dangerous line to tread.
Having crawled the various help-websites available, I’ve come across many supposed solutions to this problem. A few of these are as follows:
– Speed up your website
– Compress the code being delivered
– Order your variation code to match the order of elements on your page
– Consolidate variation code into as short a format as possible
– Consider using Vanilla JS instead of jQuery (as it’s faster)
None of these are correct. They imply that you, as a website administrator, are the cause of the problem. Whereas, in reality, your Testing solution should be held accountable for dealing with this problem.
Solutions and People are the greatest tools to overcome this solution, and both are just as important as each other.
You need a solution that is architected to ensure that Flash of Original Content is avoidable in its entirety. The challenge in finding one is that solutions typically either don’t support flickerless rendering, or don’t feel as though it’s even worth discussing. However, considering the above, it’s clearly worth doing some research before committing.
Just as importantly, you need people who can walk you through the whole processes – from Tag Integration through to building tests well. It’s not something that comes naturally to most people, including developers who build websites the traditional way. There’s definitely no substitute for an experienced professional to help make the whole Testing process meaningful.