Putting a F__k Off Dalek on the BBC Homepage isn’t big or clever
A 2005 Dalek-themed experiment with a one-off re-design of the BBC homepage taught me a lot about gathering feedback from users of a live mainstream media website.
When I moved from my old blog to my new home here at martinbelam.com, I avoided doing a full content migration, but I knew that I would eventually start republishing some of the key talks I’ve given and articles I’ve written over the years. The death of Ray Cusick, who designed the original Daleks, prompted me to dig this essay out as the first.
It is a version of a talk that I gave at the first OpenTech conference in London in 2005, discussing the feedback the BBC had received to a redesign of their homepage to coincide with the season finale of the recently revived Doctor Who. At the time I was “Senior Development Producer” for the page, which was effectively a product management role for the software systems that under-pinned it.
So, set your TARDIS controls for 2005, as we head off to find out why “Putting a F__k Off Dalek on the BBC Homepage isn’t big or clever”…
A version of this article was first published on currybetdotnet on July 24, 2005.
If you’ve never worked in the BBC’s New Media department it is difficult to know what a “Development Producer” does. In effect we are the “Babel Fish” of the department. In one ear we receive the editorial aspiration that “Wouldn’t it be great if we could do some coverage of Glastonbury on the homepage with the news from the festival updating as it happens” and we translate it so that the technical team hear a functional spec and a set of deliverables.
Likewise, we hear “The switcher logic relies on a module in Apache 2, and as you know, one of the server farms hasn’t migrated from Apache 1.3 yet so we can’t guarantee this system will work all the time depending on which server the user ends up on” and we translate it into “We are going to launch three weeks late.”
One of the areas of bbc.co.uk that I oversee this translation for is the BBC’s homepage at bbc.co.uk.
Now as a child, watching Tom Baker and K9 save the universe every Saturday evening in Doctor Who, I used to think “Wouldn’t it be great if I grew up and worked at the BBC.” So I’m sure you can imagine my excitement earlier this year when the editorial aspiration whispered into my Babel Fish ears was “We’d like to do something spectacular on the homepage for the climactic final episode of Doctor Who.”
It has been difficult to miss the return of Doctor Who in the UK. It has been one of the television success stories of the year. It has been one of those very rare things — a hit programme from the BBC that has attracted both critical and popular acclaim, and scored great family orientated ratings on a Saturday evening. It reduced ITV to showing Beverly Hills Cop against the final episode — a movie made in 1984 when Peter Davison was still Doctor Who in the ‘classic’ series.
For the season finale we produced a one-off Dalek version of the homepage. In order to accommodate the large Dalek image we had to make some changes to other areas of the page. We reduced the granularity of the directory section, so that it consisted of the 12 major categories titles, but lost the 3 sub-category links belonging to each section. We simplified the Television and Radio panels, removing the TV Pick of the Day, and links to the BBC’s individual radio networks. We also reduced the amount of local information provided on the page, retaining only the personalised weather.
This was the first significant change to the content and layout of the BBC homepage for around two years, as our last redesign, in May 2004, consisted mostly of aesthetic changes to the branding of the page.
As we were producing the page at reasonably short notice we didn’t have time to user-test any prototype designs, which is how we would normally work. Instead we decided we would try and capture feedback from the BBC’s audience from the page itself when it was live. Within the team we found the change very impactful — but it was hard for us to judge whether that was just because we were pleased to be working free from the constraint of the regular layout, and whether it would have the same impact on the audience.
So to find out we did two things — starting a thread on our bbc.co.uk Points of View message board asking for feedback, and putting a link at the bottom of the revised homepage taking users through to a page asking for their views.
The message board approach wasn’t very successful — we only got a couple of replies — the first of which was actually a complaint about the message board system itself:
“We’ve spent the entire week being told to hold on because we’re all about to be transferred to a new (it seems inferior) message board system and it still isn’t working. Indeed, bits of the old system have also stopped working.”
The appeal for email was very successful — perhaps too successful — with around 1,500 people mailing us in the space of 20 hours to say what they thought about the page. Still we had promised to read every one, and so whilst myself and various other members of the team dipped into the mailbox to read a sample, one diligent colleague read through and analysed all of the mail.
She found that just under half of the emails we received (about 650) said they disliked it, or preferred the usual page. On the other hand 39%, around 530 people, mailed to say they liked or preferred the new design. Around 9% gave mixed views or expressed no opinion, and lastly 5% of the mail was unrelated to the homepage.
You simply can’t please all the people all of the time, and we received a lot of contradictory feedback. Some people were annoyed that we had reduced the number of links on the page, and so had removed their regular routes to certain content:
“Half the links are gone — I have to go through more than one page to find them now.”
“If I were new to the site I think I’d find it a bit bare and may not look further to find the true wealth of content.”
“I look for the BBC7 link each day and it appears to be missing.”
“I don’t really think it’s in the public interest to have the home page based around an huge BBC advert.”
Equally, others were pleased that there was less on the page, using words and phrases like “the clarity is astounding”, “crisp”, “simple” and “much less clutter”.
The size of the page also elicited a lot of comment – but for everyone complaining that the fixed width of 770 pixels looked ridiculous on their large monitors, we seemed to get another complaint that on their smaller monitor set-up the page was too large and required a lot of scrolling past the Dalek to get to the content.
“The big white space at the top of the page was a bit like a building with the ground floor missing.”
“Why can’t you centre the body to avoid the imbalanced white strip on right hand side.”
We made two definite mistakes. Realising that people might prefer the regular version we included a link to return the user to the standard page layout. However, we didn’t set a cookie to store that preference, so users received the Dalek again if they browsed away from the homepage and then returned. This annoyed quite a few users, and if we were to produce a page as drastically different again I would want to ensure that the preference was recorded.
We also realised that we probably didn’t make it clear enough during the feedback process that this was only a temporary one-off special. A few people said “I will never use the BBC as my home page again” — I certainly hope that they will have a look again and see that it wasn’t a permanent switch.
What was very interesting for us was that without having to commission and pay for a new usability study, we got a sample-size of 1,500 people commenting on all sorts of aspects of the way they use the BBC homepage. There were anecdotal comments “I like to check it for the news and weather before I go out”, comments about using the homepage as a starting point for specific journeys, and lots of praise for the way the site is usually laid out.
“One of the great success stories of the BBC is the website — a model of clarity and elegant design.”
“As the world leaders in web design its good to see new indicatives.”
“The present homepage is clear and represents the BBC well. It is respected for its design across the net.”
The negative reactions, however, were often strongly negative — some accused us of the dreaded double ‘d’ words — “Dumbing Down”.
“Is there nothing the Beeb won’t dumb down? Please give us the old one back.”
“Page now lacks the detailled [sic] information on BBC services. The old one allowed ‘one click access’ to the main BBC services. Dumbing down?”
“The new home page is too much. The graphics are overpowering. Information is no longer the focus. Is the BBC intent on dumbing down to the lowest common denominator?”
The colour wasn’t always popular either – we picked a set of browns to compliment the gold casing of the 2005-style Daleks, but the page was used shortly after the BBC had changed the style of the weather maps used on television, which depict the UK landmass in brown.
“Not content with turning Britain brown in the weather you’ve turned the homepage brown as well?”
“Why the awful mucky brown colour?”
“I think brown is a real no-no — it sends out a negative, boring non-sharp message.”
Some of the feedback concentrated on the subject matter rather than the page layout, both for and against…
“Is the BBC about no more than Daleks these days?”
“Love the design — nice to see the BBC backing the best thing around at present!”
“This is the BBC website not the Doctor Who website.”
And someone emailed in a very Christopher Eccleston-ish “Fantastic!”
It was just as well we were not giving out prizes for originality in the feedback — 14 people used variations on “If it ain’t broke, don’t fix it.”
I think there are really two points that I’d like to draw out from this that are more generally informative than just a story about how the BBC once put a massive Dalek on the homepage.
The first is about how mainstream users deal with change. The OpenTech conference is all about innovation and change — about changing the way we do things on the web and on other devices, about changing the relationship between content providers like the BBC and developers, and about changing our interaction models.
By virtue of being at OpenTech, or being interested enough to read an article like this, you are in a small sub-section of the UK’s internet super-users. It is very hard to keep that in the forefront of your mind. Change worries people. Change in a familiar setting can be very disruptive. People won’t like change unless you bring them along with you.
One user mailed us to say:
“Where’s my old BBC home page…why does this feverish need for change penetrate every damned area of everything…did anyone actually ask for change — or did someone decide we needed it?”
If any of the developments you see at OpenTech are to gain a mainstream audience, they will need to overcome the inertia against change in the mainstream internet audience.
Secondly, I’d like to point out how useful it has been to do this kind of testing on our live user-base. We got a much wider sample of views than we would have been able to afford otherwise — even the BBC’s new media budget doesn’t stretch to user-testing focus groups over 1,000 strong.
Nimbly testing in public isn’t something that the BBC has historically been good at. We very rarely label anything a ‘beta’, and when we did, iCan was in a public beta stage for a couple of years before it finally blossomed into the “Action Network” a few weeks back.
Tim O’Reilly at EtCon recently talked about the advantages that services like Flickr have by being in a seemingly perpetual beta state, over the difficulties of long drawn-out releases like Microsoft’s Longhorn experience.
The key though isn’t just to slap a beta gif underneath your logo and then hope people will be more forgiving of a service, but to genuinely attempt to learn from the feedback you get from the audience. And iterate, and iterate, and iterate again.
Since we did the Dalek page we have done two similar ones — one for the Africa Calling Live8 concert at the Eden Project in Cornwall, and one to remember the victims of the London bombings.
The Live8 page was up for three-and-a-half days, yet only generated about 50 negative comments compared to the 650 the Doctor Who page received in just a few hours. Why? Because we learnt from the feedback and adjusted the design to eliminate the commonest complaints — the banner area was reduced, and with less space taken up by the image we could restore all of the content that we had removed, eliminating the need to provide a ‘return to the normal layout’ option.
So, although people are resistant to change in a familiar place, you can gain a great advantage by user-testing ‘in the wild’. However, without that feedback loop in place though, and a willingness to adapt your beta again and again, you will not get the maximum value out of the test.
With thanks to the people who helped to make the page possible — Janet Copeland, Gemma Garmeson, Andrew Webb, Robert Isset and Nick Holmes, and to Alice Gardiner who analysed the feedback. And especially to Ray Cusick (1928 – 2013), who designed the Daleks who have given me so much enjoyment over the years.
A version of this article was first published on currybetdotnet on July 24, 2005.
In December 2005, my friend Andrew Webb, who was picture editor on the BBC homepage when we did the Dalek special, contributed a load of material to the Doctor Who Cuttings Archive website about the production of the special page. Sadly the site no longer exists online, but I do have one image salvaged from a blog post I wrote about it, showcasing one of the earlier designs. At the time I wrote:
“The first time I saw it there was a split-second in my brain between the ‘Fucking hell that is awesome!’ excitement and the ‘Oh, we’ll never get away with doing that’ realism sinking in.”
And in researching the re-publishing of this blog post, I was reminded that we had broken the BBC homepage when trying to take down the Dalek, which was blogged by Chris McEvoy. He took screengrabs of the broken homepage, and the smaller Dalek promo slot that followed it. Ever the diligent producer, I’d actually left a comment on his blog post.
Get blog posts like this via my email. Sign up here.