EE, Vodafone and Three all achieved a “good” ratings score in P3s annual network test, making this year’s test the best industry performance to date. O2 scored a “satisfactory” rating.
The tests by the consulting, engineering and testing services company P3 communications use field-testing in 13 UK cities, a range of towns and 2,000 miles of roads. This years results show the gap between the networks and between geographical locations narrowing for both voice and data.
Commenting on this year’s results, P3 communications CEO Hakan Ekmen said, ‘Our tests are good news for the UK’s mobile customers, because for the first time three networks are in our ‘good’ category, even though this year’s tests were the toughest yet.’
He continued, ‘It is gratifying that over the three years we have been testing we can see substantial improvements in the overall quality of service, which is a clear indication of how our benchmarks are helping to drive improvements in the networks.’
How they performed
See below for a rundown of individual network performances and key trends.
Vodafone was the most improved network, going from 67% in 2015 to 80% in 2016, a 19.4% year on year increase. The network also won the London test category, scoring 548 out of 675 and doing especially well in call quality. The network also came joint top of the leaderboard with EE, as both scored 803 points out of 1,000.
The operator did best in voice categories, leading the field in city call success ratios and also showing the quickest call setup times in both towns and on the 2,000 miles of roads.
Commenting on the report, Vodafone UK chief executive Nick Jeffery said, ‘We're building the UK's strongest network designed, site by site, to meet all of our customers' needs, wherever they are. These results are a welcome validation that our massive investment programme is making a real difference for our customers.’
Though its rivals have caught up in many areas, EE maintained its lead in data transmission, including city wins in file uploads, downloads and uninterrupted streaming, and road drive test wins in web page success ratio, file downloads, file uploads and video streams without interruptions.
The wins in the data on roads tests are crucial for EE, due to the government’s decision to move its emergency services communications from Tetra to EE’s 4G network – making the UK the only country in the world to use a commercial 4G network for critical communications. Therefore, any drops in coverage could have serious ramifications for public safety.
However, the were some weakiness with EE coming last in every voice test category including cities drive test, cities walktest, towns drivetest and roads drivetest.
Three finished third overall, scoring best on small town and minor road voice coverage though losing its 2015 voice category win to both O2 and Vodafone.
On data, Three showed improvement over last year but suffered in certain key metrics – especially outside of cities with the lowest web page load, file download, file upload success ratios in towns. In nearly all data categories measured, Three finishes third, only saved from last place by a much lower performance by O2.
Commenting on the results, Three’s chief technology officer Bryn Jones stated, ‘From a customer experience perspective the P3 results demonstrate that the Three network provides a reliable service and that their experience is getting better each year. We know that reliability is the most valued part of a mobile service and that is why we focus our investment in this area. We will continue to invest in 4G Supervoice next year, a service that has banished many blackspots across the country since it was launched last year. We anticipate this new technology will be measured in the next P3 survey, which we are confident will show a step change in overall performance.”
Though finishing last overall by a considerable distance, O2 can take some comfort in achieving a considerable increase in its 2016 score compared to last year, going from 64% to 75% - a 17.2% increase.
The Slough based operator did win in a few voice categories, winning the London voice tests and coming second overall in voice nationally.
Where O2 really suffers is data, where in some categories like file download kbit/s on roads, towns and cities where at best O2 manages half the speed of the category leader, and at worst fewer than 10% of the category leader.
Commenting on the results, an O2 spokesperson stated, ‘At O2, we focus on improving coverage and capacity as data demand grows – that’s something we are doing daily. What’s really important is that we deliver on our customers’ expectations so we design, build and run our network based their feedback by constantly monitoring and analysing their experience to enhance our network for customers. As we continue to improve and expand our network through our five-year £3bn plus modernisation program, additional feedback like this allows us to further improve areas of our network and ensure our customers continue to have the best possible experience with O2.’
Every network pledged to the government and to the public to reach 90% mobile network geographic coverage by 2017 and to increase 3G and 4G geographic coverage to 85%. Furthermore, EE has previously announced its goal of 95% coverage by 2020.
The British Infrastructure Group – a crossbench panel of MPs focused on UK infrastructure, just last week queried how likely the operators are to meet these targets. Despite assurances by both Ofcom and the networks, this report shows there’s a while to go.
Though the call and data success rates shown by P3’s network testing meet these targets, this is unlikely to be reflective of the entire country, with most of the notorious rural not spots such as north & west wales, Cornwall, Norfolk, Lake District, The peak district and North Scotland not included in the test.
O2’s bottom ranking in the 2016 P3 test contrasts with their legendarily low churn rate, suggesting that perhaps investing in coverage isn’t always the best way to grow your customer base. Meanwhile EE and BT’s q3 results showed a decline in pre-pay customers while O2 grew theirs in the same period.
Despite having the lowest market share of the four networks and one of the lowest scores on mobile data, Three claims to carry 40% of the UK’s data traffic. All this shows that perhaps the correlation between network investment and customer loyalty is more fickle than many outside the industry would think.
