Update as of primetime on Sunday, September 7th: ESPN's fantasy site and app are down. Looks like it's not just Yahoo that has the occasional technical problem.
Yahoo gets the blocking and tackling down
Just like me, millions of Americans tuned into the first full weekend of the NFL season. In this case, "like me" means hunched over various mobile devices while refreshing multiple fantasy football leagues.
Today, the smooth performance of Yahoo's Fantasy Football app made it easy to see how matchups were shaking out. Contrast this to the first few weeks of the 2013 season when crippling stability problems plagued the iOS Yahoo Fantasy Football app, leading to threads on Reddit like:
2013 wasn't the first time there were issues as a notable November crash in 2012 caused major problems, leading to media coverage.
Most telling was this quote from last year by Reddit user Ctmarlin:
"Every year yahoo, which is a well capitalized tech company, cannot get their shit together. Usually it's stat tracker, but now it's the app."
I set out to identify the effect of Yahoo's previous app experience issues on its fantasy sports audience and what Yahoo had done to ensure a smooth first weekend this year. My theory was that basic technical problems were causing people to steer away from using Yahoo for leagues, but the company had probably approached 2014 in a different way to avoid repeating the same mistakes.
Not an audience willing to mess around
Why spend time on the app (and customer) experience of fantasy football? Because fantasy football is big business and customer experience is a major factor in league choice. The Fantasy Sports Trade Association says 41 million people in the US and Canada played fantasy sports last year and 69% said football was their favorite. These people are also valuable due to high household income - typically people with the highest expectations for customer experience. Most importantly, they spend a LOT of time on fantasy sports, more than a full work day (8.67 hrs/week).
The most committed are also a very mobile audience with Nielsen saying 10 million people accessed ESPN or Yahoo in September 2013 and opening the app an average of once a day.
Studies have shown the value of basic stability in apps with consumers unlikely to try again after a failure, but fantasy sports players are committed people spending massive time on this hobby so may not be normal. FSTA research on how the "daily" subset fantasy football players choose their site indicates Yahoo app experience problems may have helped ESPN become a bigger fantasy player. The top two reasons are directly related to software performing as expected:
How do Daily Games players choose which website to play?
- 48% say they trust the site
- 41% say [the site] is the easiest to use
Was there an impact to dropping the customer experience ball?
All of this was bad news for Yahoo because consistency is cited by research firms like Forrester as an important driver of customer experience satisfaction.
At the same time, ESPN, the other big player, ramped up focus on fantasy football. Every year, it creates more shows, articles and podcasts providing mountains of tips, lists and projections. In 2012, it launched spots and a toolkit aimed at commissioners in fantasy football, causing spikes in searches and press attention.
The chart below shows that in September 2014, ESPN eclipsed Yahoo in "fantasy football searches" and erased the gap that existed a few years before. That 2012 Yahoo snafu? It's visible in an uncharacteristic mid-season search spike.
Another study in 2012 at Fantasy Sports Portal looked at site usage for its 2,000 subscribers and found a 5 percentage point decline of Yahoo's share and a small 1.3 percentage point increase in ESPN's share from 2011 to 2012. This indicates the Yahoo advantage has been eroding for some time - whether caused by customer experience or something else.
Similar to Google, Nielsen's numbers say Yahoo sites were still a key news source for fantasy football players in 2013, but ESPN wasn't far behind.
All of this suggests that the advantage Yahoo Fantasy Football once held is now gone. However, app customer experience was probably overshadowed as a factor by ESPN's constant focus on fantasy football and increasing influence on the sports news landscape. Basically, the ESPN fantasy ecosystem had a bigger impact than Yahoo's iOS struggles. Looking at overall audience trends, it's likely Yahoo missed out on some of the growth it could have achieved, but it probably still grew due the overall growth in players.
How did Yahoo achieve a smooth 2014 opening weekend?
While not the only thing, nailing the app for the 2014 season seemed to be critical for Yahoo to stop further erosion in the market for players. Yahoo couldn't afford to slip again on the basics.
I took a look at the update history for last year to see if I could learn a few things:
- It doesn't seem like they were quite ready last year. The core of last years' app launched in July 2013 and probably wasn't quite ready for primetime as the release cycle got dramatically faster while the team made a number of changes throughout August and September. The end of this probably coincides with the app being stabilized.
- Focusing resources in a single app. Whether because of the 2012 and 2013 snafus or something else, all Yahoo fantasy sports apps were combined in February 2014. From a player perspective, this is fantastic as many play more than one sport. From Yahoo's perspective, this probably let the team work from a common, stable core. However, it must have been a massive undertaking to ensure good user experience. For example, do fantasy baseball players want the same thing as fantasy football players?
- Phased rollouts in 2014. It's hard to tell, but it looks like there were several big updates in preparation for 2014 vs one in preparation for 2013. In 2014, a commissioner tools rollout happened in early August, then a bug fixing push in mid August, then a design push in late August. Hard to tell how complex each of these were, but each looks more substantial than the single push in late July of 2013.
The impact of customer experience and Yahoo iOS app changes
It seems likely Yahoo's iOS struggles have been a factor in declining/stagnant market share in fantasy sports, but the bigger story is ESPN's share of the sports media marketing and growing fantasy ecosystem. However, Yahoo did what it could with its iOS app and met player expectations in 2014 with a (so far) smooth opening weekend likely brought about by a shift to a unified app strategy.