Reviewing a VPN can be tricky. There's a lot to consider, an array of factors and features to weigh up, and everyone has their own view on what's important, and what really isn't.
In this feature we're going to explain the key issues we consider, how we go about evaluating and testing them, and how we ultimately decide that a service like ExpressVPN or NordVPN comes out on top.
And it's worth noting that what we've laid out here may be just a fraction of what might potentially happen in our actual reviews. If we're curious about exactly what a VPN app is doing – maybe we're wondering if it might be malicious, for instance – we might run its executable past a security scanner, examine the code of a browser extension or decompile and browse a .NET client. But then, going into that sort of detail would require an entire book to cover properly!
Our approach to testing VPNs
In basic terms, we do the geeky stuff so that you don't have to. But to understand more deeply how we evaluate and test VPNs, just keep reading.
Definitive verdicts are hard to find. It doesn't matter how often or how carefully anyone measures connection speeds from the UK to Germany on a Windows laptop – this won't tell you very much if you're interested in connecting from Australia to New York on your Android mobile.
What we can do is check out a lot of different aspects of the service, and tell you more about them. You may not be interested in all of these – perhaps you only ever use the same location, for instance, so couldn't care less about how the server list is organized, or whether there's a favorites system – but hopefully we'll cover enough ground to give you a feel for what a provider is like.
Collecting features
The process of evaluating a VPN begins at its website, by collecting details on the service and its features.
Network size matters, though not as much as providers sometimes claim (as long as a VPN has the locations you need, it doesn't matter whether there are 20 or 2,000 more). As well as the numbers, we check to see how widely dispersed the locations are, whether they include countries you won't always get with the competition, or if they leave gaps in coverage elsewhere.
It's equally important to understand the services supported by each server. Do they all support P2P, streaming, WireGuard, OpenVPN, and everything else offered by the provider? This isn't always easy to find out, but it's worth making the effort - that list of 300 locations may look much less impressive if you realize you can only use P2P with three of them. (We don't take these P2P figures on trust, either. We'll connect to at least three test locations, and confirm that we can access torrents as usual.)
A good VPN should offer custom desktop and mobile VPN apps, as they'll be the easiest way to use the service. Be careful when you're checking this on a website, though. Providers often display a long list of platforms, but although some of these will lead to downloads, others might just point you to a tutorial explaining how you can set up the service manually. Follow each link to find out for sure.
Identifying supported VPN protocols tells you a lot about the service. If it covers the key standards – WireGuard, OpenVPN, L2TP/IPSec, IKEv2 – there's a good chance you'll be able to use it on most devices and platforms (routers, game consoles and so on), even if they're not directly supported by provider apps.
We pay close attention to any details of how each protocol is implemented, too, including the specifics of any encryption and authentication methods which might be employed. These aren't always visible upfront, but there's usually information tucked away in the support site.
Does the VPN have any interesting bonus features? Common examples are ad, tracker and malicious URL blocking; the provider's own DNS system; VPN over Tor; split tunneling, allowing you to choose which applications use the VPN, and which don't; and maybe Bitcoin support for payments, improving your anonymity when you sign up.
Checking out the price helps us to understand whether the service is good value, but there's more to that than looking at a single headline figure. Some apparently cheap VPNs don't offer their lowest price unless you sign up for several years, so it's important to look at the range of plans and prices on offer.
If collecting all this data sounds like it could be a lengthy and tedious process, you're right; it is indeed. But it's not just about finding figures and feature lists. Just the experience of looking for the relevant information will give you a good feel for what a VPN is like.
A dubious provider might be short on detail, for instance, with a poorly organized website making it hard to find what you need. Information might be inconsistent or outdated, and you'll sometimes see a little marketing trickery, such as unrealistic speed or website unblocking promises which you suspect the company can't deliver.
But a professional provider will have a well-designed website which makes all the most important information visible upfront, with plenty of technical detail tucked away if you want it. Overselling will be kept to a minimum, and honesty and transparency is likely to be the order of the day.
Privacy
Assessing the level of privacy offered by any VPN is difficult, as you're relying on the provider to honestly tell you what they're doing and how the service works. Still, there's often enough information to point you in the right direction.
This starts with the technical data you collected earlier on the protocols supported, the encryption and authentication. WireGuard and OpenVPN support is best, IKEv2 not far behind, L2TP/IPSec is acceptable, but the outdated and insecure PPTP is best avoided entirely.
We might look for AES-256 data encryption, RSA-2048 or 4096 to cover handshaking, and Perfect Forward Secrecy to generate new keys for each session – but the truth is that VPNs don't always fully spell out what they're doing. Whereas some go above and beyond the call of duty, like the way that NordVPN offers extra security smarts like Double VPN and Onion support.
It's sometimes necessary to browse the support site for clues, or you can download sample OpenVPN configuration files to give you an idea of how the service works. If the service has live chat, try it out; the agents can usually answer pre-sales queries, so you shouldn't have to sign up to get advice.
App features are key. Quality VPNs use their own DNS servers and implement DNS leak protection (IPv4 and IPv6) to reduce the chance of your internet activities becoming visible to others. You'll also want to have a kill switch to automatically block internet access if the VPN connection drops.
Beware: just because a service has 'kill switch' on a website feature list doesn't mean that this capability will be available on every platform. Mobile apps often have fewer features and functions than their desktop siblings, for instance, so we check each one to fully understand what it does.
To confirm that a VPN is properly hiding your IP address, connect to any VPN location and point your browser at ipleak.net. If you see your real IP address, or an address owned by your ISP, there's a problem which you need to investigate further. Results can vary according to browser setup, so repeat this in every browser and on every device you use.
We assess kill switches on Windows VPN clients by using various low-level tricks to forcibly close our VPN connection, then monitor how long the internet remains accessible, and whether our regular external IP is visible to the outside world.
It's a technical business, but our How to test your VPN kill switch guide has simple checks you can make yourself.
At a minimum, just turn off your router off for 15 seconds, back on again, and watch your app. Does it recognise it's disconnected, does it warn you, does it reconnect when the router is back? A good VPN app handles unexpected events like this and keeps you informed; a bad app might not even realise you're now offline.
Logging
VPN providers, particularly free services, are regularly accused of selling their users browsing history. Most will try to fight back, typically by yelling 'No logging!' on the front page of their website, but years of review experience has told us this isn't always true. That's why we always look a little deeper.
The privacy policy can tell you a lot about a provider, even before you read it. 5,000 words of poorly-formatted, jargon-packed legalese is a bad sign, and so is a 100-word document which tells you almost nothing. What we're looking for is a clearly formatted, well-organized and readable document which allows anyone to find the key details they need.
It's important to keep this in perspective. However well-presented a privacy policy might be, there's no guarantee that anything said in the document is actually true. It's all based on trust.
Checking out the small print can highlight logging that a VPN is admitting takes place, though. And if nothing else, it can give you clues about how honest and open a provider might be.
We've previously found dubious free VPNs who have copied and pasted someone else's privacy policy, changed any company names in the text and pretended it's theirs, for instance. That's lazy, incompetent, and fraudulent, really – more than enough to tell you that this isn't a provider you should use.
Security audits
VPN providers know there's a lack of trust in the business, but many are trying to counter this by putting themselves through independent security audits.
The idea is that a provider invites an expert team to look in-depth at security or privacy issues with its systems, and report back with their findings.
Any VPN which puts itself through this process deserves real credit for having the courage to expose its inner workings to the world. But not all audits are the same, and their true value depends on a range of factors.
What is the scope of the audit, for instance – what did the auditors try to look at? If it's a tiny aspect of the service, like the browser extensions, that's not going to mean very much. Inspecting the full range of apps is more useful, and the best audits will go even further. TunnelBear's security audit includes its infrastructure and even the website, while NordVPN called in PricewaterhouseCoopers to complete an independent review of its no-logs policy.
Check out what the audit was designed to do, too. Sometimes audits simply check that, say, the service follows the no-logging rules in the privacy policy. That's good, but others go further, perhaps putting apps through in-depth testing to identify bugs and privacy flaws.
What level of access did the auditors have to the service? Being able to test an app is good, but having access to the source code, and real VPN servers, is much, much better.
Is the final report available, in full, to the public? Ideally anyone should be able to look at it. Some reports are only available to paying customers – not so convenient, but we can live with that. A few aren't publicly available at all, though, which leaves you having to trust the VPN's own summary of the verdict. (Ironic, considering audits are supposed to be about reducing this need for trust.)
Finally, how long ago did the audit take place? A quality VPN is always extending and improving its systems, so a report from two years ago won't have nearly as much value today. We look for providers that not only put all their services up for inspection, but firms that commit to do it next year, too.
Performance
We measure VPN performance using Windows 10 systems in two locations: a US home, and a Windows 365 cloud PC hosted in the UK.
Each uses wired Ethernet, avoiding Wi-Fi issues, to access a 1Gbps internet connection. That's much faster than most devices and homes, but gives us the space to see exactly what each service can do.
A test session begins by checking our normal internet speeds (that's with the VPN off.) We use at least three speed test sites or services, from a selection including SpeedTest's website and command line app, SpeedOf.me, nPerf and Netflix's Fast.com. We run at least five tests at each site, and take the median value as our average.
Next, we use the VPN app to connect to its nearest server using the best available protocol (WireGuard, OpenVPN or a custom protocol of its own). After that, we visit the same speed test sites we used earlier, again running at least five checks, and calculating our new median speeds.
VPN apps often support several protocols, and we can't be completely sure which will deliver the best speeds. So after measuring an app's WireGuard speeds, say, we'll disconnect the VPN, reconnect using OpenVPN and run all the tests again.
This takes a while and involves many tests (an absolute minimum of 45), but we're not finished yet. We run all these tests in both morning and evening sessions, just in case our first checks are affected by temporary network, test environment or other issues.
While we're trying to get a representative idea of VPN performance, there's a limit to what these results can tell us. They're giving us a snapshot idea of how the VPN performs from our test locations in the US and UK, but if you're elsewhere in the world, connecting to another server, and six months later, you may well see different results.
Unblocking
Many VPNs boast that they are the best streaming VPN – claiming they can help you get around geographically-based blocking and enable access to content which you might not otherwise be able to see. Sounds good, but is it true? To assess this, we start by looking at the claims the provider makes.
A good provider will state clearly which services it claims to make accessible (Netflix, Hulu, and so forth), and maybe which it can't. The support site might include articles which explain what you can do if a site remains blocked, and invite you to contact the support team directly if you need help. If you can't access Netflix via ExpressVPN, for instance, you're able to open a live chat session with the support team, ask them which server to use, and get a recommendation within a couple of minutes.
Lesser VPNs might not have any troubleshooting articles, and they'll be less likely to commit to helping you access any particular website. Instead of saying 'if you're blocked, contact us for help', they'll give more of an impression that 'if you're blocked, tough, we can't promise anything, that's just the way it is'.
The most dubious VPNs will look like they're making big promises, perhaps claiming they 'allow you to access ANY website', or displaying Netflix and other logos to imply that they'll be available, without specifically saying that. But dig deeper and you'll either find no background information at all, or just a disclaimer saying they can't guarantee anything.
To test this ourselves, we attempt to access and stream exclusive regional Netflix titles from a number of servers worldwide, Amazon Prime Video and Disney+ content from multiple US VPN servers, and BBC iPlayer from all available UK servers. The results are logged, and if they're entirely negative and the provider suggests they shouldn't be (the company claims we can access Netflix, and our checks say otherwise), and recommends asking for help, we get in touch with the VPN's support team to see what they can offer.
Even at this late stage, how an organization responds can be crucial. If the provider can recommend another working server, tell us that it's aware there's an issue and that it is taking action, the firm will earn bonus points. If not, and if the company doesn't live up to promises made on the website, then it will be penalized.
Apps
A VPN can have a vast network of fast and reliable servers, but that's not much help if its apps make the service difficult to access or use. That's why we place a lot of importance on a provider's software, its features and usability.
We expect a service to provide its own Android VPN apps and others for Windows, Mac and iOS. Support for additional platforms isn't as important, but does give us valuable information about the provider. The majority of users won't care if their VPN has a Linux client, for instance, but its existence alone tells us the provider has resources, real VPN experience, and is doing its best to reach and help the maximum possible audience.
Some services also offer browser extensions. These are more limited than apps, as most only protect browser traffic, and not other programs running on your device. But there are exceptions: ExpressVPN's Chrome VPN extension can communicate with and control the native ExpressVPN client on a PC.
App usability is key. It's also very subjective, but we try to take the most important factors into consideration.
The opening interface should make it easy to find the server you need, or reconnect to commonly used locations. A simple alphabetically sorted list, with a search box and a favorites system, could do the trick. Apps where locations are sorted in an apparently random order, or which force you to make multiple clicks to get connected, normally get marked down.
It's vital that a user knows when the VPN is protecting them, so the app should display notifications to show when it's connecting, connected, or when the connection drops. Not everyone wants this level of interaction, though, so the very best apps will also give you the option to customize the notifications you see.
Many apps include an 'automatic' or 'fastest' selection mode, where in theory they choose the best server for you. This doesn't always work as we expect, though (despite testing from the UK, one VPN decided our fastest server was in India), so we check this carefully to confirm the results make sense.
We've already described how we measure peak download speeds, but they're not the only measure of how a VPN performs. That's why we might also check connection times, benchmark DNS queries, browse websites, look at video streaming performance, and generally evaluate the real-world experience of using each VPN.
Settings matter, too. On desktops in particular, we want to see a kill switch, ensuring internet access will be blocked if the connection drops. An app which can automatically connect when you access new networks is valuable, as you won't have to remember to do it yourself. The best apps add extra configuration touches, such as always connecting when you access a known untrusted Wi-Fi network (a local coffee shop), but not stepping in with networks you know are safe (home, work.)
Additional geeky touches, such as the ability to launch custom scripts when you connect or disconnect, are welcome, and can be hugely powerful. But we don't place too much weight on these in scoring terms, as the vast majority of users will never pay them any attention at all.
Support
VPNs can run into all kinds of hard-to-diagnose issues, and it's important that every provider offers fast, accurate and reliable support.
The website should include plenty of helpful setup, usage and troubleshooting guides. The installation section needs tutorials for multiple platforms, for instance, and should cover as many operating system versions as possible. A generic Windows setup guide is okay, for instance, but the interface has changed over the years, so separate guides for Windows 7, 8 and 10/11 is better.
Troubleshooting advice should be thorough, and demonstrate genuine expertise. We might search any web knowledgebase using common keywords, such as 'speed', then browse the results. Poor providers might not give any useful hits. A better service will return articles on topics like improving performance, though with only basic information which contains nothing you couldn't figure out already ('connect to a closer server', 'try again later').
But the best providers will expand every idea to give you more than you expected ('here's how to use the client's displayed ping times to find a closer server'), and leave even the most experienced users with a better understanding of the problem than they had before.
A web knowledgebase can never tell you everything you need to know, so it's important that you're able to get real, live, human support. Ideally this should be available 24/7, with live chat as an option, but don't rule out email or ticket support. This has its advantages – you're able to think more about what you say before you say it, and attach logs or screenshots to help explain the situation – and the best providers respond to questions in under an hour.
We try out the live chat or email support of each VPN provider with at least one sample question, just to get a feel for how it performs. As with many other aspects of a review, this kind of one-off snapshot can't tell us everything we might want to know about a provider's hand-holding abilities.
We've found it often gives us interesting information about the service, though – an ultra-fast response here, a very unhelpful one there – and it all helps to build a more detailed and accurate picture of how good (or otherwise) a VPN might be.
