How to get to the front page of Google

Sure, you know about search engine optimization - increasing your site’s ranking in search engines. You might not be an expert, but you have a good understanding of the basics: title tags, clean URLs, text-based design and so on. 

Even if you’re more of an expert than most, it’s often what you don’t know about SEO that will hurt you. Many webmasters found this out the hard way in 2011, when Google’s Panda update was released. 

Many of these same webmasters were hit again little more than a year later, when the search giant’s Penguin update followed – targeting low-quality, spammy link-building tactics. Pigeon and Payday followed, again targeting spam, and in late 2015 a machine-learning AI system known as RankBrain was introduced to interpret searches and provide results that may not include the words searched for but are still relevant. 

• These are the best online marketing services
• Here are the best SEO tools of 2021
• We have shortlisted the best onpage SEO tools

Rising from the ashes

With all these changes, many SEO techniques turned out to be less than optimal. Sites were penalized; traffic lost, and so-called ‘SEO experts’ fired. Since then there’s been a new tone in the SEO industry. Not all of us were creating low-quality content and links; but for those who were, these two updates were a wake-up call. And for those of us who have always been focused on high-quality, relevant content and links, it was something like redemption. Our sites soared while so many fell. This is the new normal for SEO.

Yes, there are still some who call themselves SEOs but focus on manipulative tactics with short-term revenue goals; yet there are also many who are part of a large and growing industry of specialists in a highly complex discipline that requires marketing, technical, research and communication skills. 

So how big is SEO? Believe it or not, it’s bigger in the minds of Google searchers than web design. Once considered a subset of web design, searches for SEO now eclipse those of web design worldwide. The total projected value of the North American search marketing industry (SEO and paid search) in 2013 was $26.8 billion, according to industry trade organization SEMPO, and 13 percent of companies have SEO budgets of half a million to more than three million a year (up from 8 percent in 2011). 

As budgets increase, there’s more to lose, and many companies have become more risk-averse – forgoing the shady tactics they may have pursued in the past. In cutting out the garbage, we start to see what SEO is good for (and has always been good for): connecting relevant content with relevant searchers, and making content discoverable through accessibility and marketing. For those of you who still think of SEOs as greasy algorithm-chasers in cheap suits or parents’ basements, consider the new reality.

Engines are not enemies

Ten years ago, there were many people who were a little nervous about what we were doing. Nothing was against the Google guidelines… because there were no such guidelines in existence. 

At that time, there were a few books, but SEO was largely something that was spoken of covertly, and certainly never to search engines, which, it was thought, would likely think of it as manipulation. Today we know better.

Google and Bing have both published extensive webmaster guidelines, and Google has even published a guide to SEO for beginners . In August 2011, Matt Cutts, Google’s head of webspam, released a video statement saying that Google does not consider SEO by itself to be spam. This sentiment now appears in Google’s definition of search engine optimization, in which it says: “Many SEOs and other agencies and consultants provide useful services for website owners.” 

Still, because of a few spammers who call themselves SEOs, SEOs in general have the reputation of being charlatans, and have been portrayed as such on television shows such as The Good Wife and Dexter. “SEO has got a bad rap, and it’s due to questionable SEO practitioners who perpetuate the ‘snake oil’ stereotype by making customers believe there’s some magic ‘black box’ that ‘tricks’ the search engines,” says Gord Hotchkiss, digital strategies consultant for Mediative and columnist for Search Insider. 

Hotchkiss, and the other experts involved in this article, explain that SEO is simply about getting relevant content indexed, and making sure it’s visible to search engines. All of the veteran SEOs who we spoke to understand why SEO still has the reputation in some circles of being snake oil. But they insist that it has, now, become much more mainstream and credible. 

Rand Fishkin, founder of SEO software company Moz, discusses his favorite reasons for SEO being something other than snake oil, including that, “SEOmoz has more than two million monthly visits, nearly all from web marketers looking to learn more about the practice. And our software, which bills monthly, has more than 18,000 subscribers. If SEO were just   snake oil, I suspect folks would stop paying.” 

SEO, in its legitimate form, is now a more accepted part of the web design process, and in many organizations is finally getting a seat at the table when it comes to designing professional, search engine-friendly websites. 

A process, not a project

In our experience, there are many instances in which the SEO team is brought in after the website has been completed, and told to magically make it search engine friendly. This is far from ideal. The really competitive sites understand this, and integrate SEO into every stage of the planning process, from information architecture to content strategy to design, development, launch and post-launch. 

A lot of web designers and developers are hesitant about integrating SEO further into the process, because doing so effectively produces extra work. But the rewards can be great, reminds Vanessa Fox, founder and CEO of Nine by Blue and author of Marketing in the Age of Google. “Organisations are losing 1) tremendous insight into their customers and potential customers if they don’t take advantage of the free search data that’s available from the millions of searches we do each day; 2) the opportunity to reach a significantly larger audience through being visible in search results.” 

Would you put the Mona Lisa in a closet? Would you spend hours cooking beef wellington, and as it emerges perfect from the oven throw it in the bin? Then why on earth would you build a website without considering how it will be found? There’s another reason for making SEO a priority in the web design process, advises Hotchkiss. 

“It forces you to create a better website! Good SEO optimization should be baked into your information architecture. It will force you to think about common content themes. It requires you to consider how all digital assets (such as videos and user-generated content) will be integrated into the overall user experience. 

It helps eliminate user experience dead ends such as gratuitous Flash interfaces and, my personal pet peeve, content locked in PDFs. It extends your perception of your online footprint beyond the bounds of your website, including things like social media. It will also instill a healthy rigor when it comes to thinking about how your site links together. Good SEO practices mean a better user experience.” 

From our experience, more organizations than ever are learning these lessons, and are no longer thinking of SEO as a project, but as an ongoing process that ensures a website will be as visible in search as possible. This is good for web design because it gives it a larger audience, but also good for business. 

It's not all links: emerging SEO trends

While many commentators have claimed that SEO is dead since it began around 1997, the truth is that it doesn’t die; it evolves with the search engines. While SEO is constantly evolving, at the moment it seems focused on mobility, utility, the audience and automation, among other things. 

One of these trends is the dissolving distinction between SEO, user experience and content strategy. In one Webmaster Tools YouTube video, Matt Cutts even suggested that those looking to change the name might consider “searcher experience optimization” to differentiate from the ‘snake oil’ variety of SEO. 

Some, such as Vanessa Fox, have suggested that SEO need not proceed as a separate activity from UX and content strategy: “I think that both disciplines should incorporate best practices from search rather than thinking of it as something tacked on later,” she says. “Particularly, the data available from search is extremely valuable. Also, understanding that many visitors begin with a major  search engine and that any page of the website can therefore become the homepage of the site can shift how we look at both page design and content.” 

At the same time, Vanessa Fox – and all of the other SEOs that we spoke to – recognize that content strategy and usability, while essential for reputable SEO, need technical and other elements from SEO to be useful as a way of getting incremental search engine traffic. “When SEO is done the right way, usability and content is a huge part of the plan,” opines Eric Enge, founder and CEO of Massachusetts-based Stone Temple Consulting and co-author of The Art of SEO.

“This is something that the snake-oil SEO people don’t worry about. For long-term success as a web publisher, the use must come first. However, for success as a business, you need more.” With this concentration on content strategy and usability comes a focus on the audience as well. 

For Hotchkiss, this is a shift from word-matching to utility, and follows the search engines’ own evolution. “Today, good  SEO is about making sure that when a prospect uses a word (or words) to search for something, you match that as best as possible,” he says. “But in the future, SEO will be about ensuring that when your prospect wants something, you deliver it. It may not be content. It may be a movie ticket, hotel booking, restaurant reservation or a downloaded TV show.”

A change in analysis

Delivering on this promise frequently requires a new type of analysis. In the past, marketers have done keyword research to uncover keywords as proxies for user intent. In Marketing in the Age of Google, Vanessa Fox describes the process of creating searcher personas that get beyond simple keyword matching and search volume exercises.

And still others, such as Hearst’s Core Audience, try to understand the characteristics of audiences, including but not limited to the keywords they use. For some businesses, mobility will not change user intent. For example, news is not going to be rewritten for a separate platform, as Karen McGrane and other adaptive content advocates frequently point out. However, for some businesses it does; and if marketers want to get the most traffic and conversions from the mobile platform (in other words, be optimized for that) the devil is in the detail, and understanding the differences between mobile and desktop audiences is key. 

There are technical considerations as well, which make a type of SEO geared towards these differences – what is commonly called mobile SEO – what it is. Many people in the SEO world, as with their counterparts in the design world, believe that responsive design is the answer to these differences – and Google stated a preference for responsive design in June 2013. However, responsive design is not always best for the user, and Google wouldn’t prefer it in those cases. 

If mobile and desktop search behavior is significantly different, Google supports using dynamic serving or switchboard tags as well. Cindy Krum, CEO of Denver-based mobile marketing agency Mobile Moxie and author of Mobile Marketing: Finding Your Customers Wherever They Are agrees that responsive design is one of many solutions for mobile search engine optimization. 

“I have been recommending a mixed solution for most of my clients,” she says, “Leveraging responsive design when it makes  sense, and special mobile-only landing pages when  keywords or use-cases cannot be appropriately addressed with a responsive design approach.” 

Tools of the trade

Another big shift in SEO has been the introduction of more automation to the process. SEO software has been around as long as SEO has, but the sophistication and breadth of tools has increased in the last year or two. 

Now there are tools to help you optimize the long-tail through semantic relevance (BloomReach), reporting tools (Conductor and BrightEdge), link building tools (Ontolo, Ahrefs, Moz link explorer and Majestic SEO) and more, all geared towards automating various aspects of the search engine optimization process. 

When this happens, inevitably someone in the press will claim that the tool will allow you to fire your redundant SEO; but none of the SEO experts or software providers we’ve spoken to agree. “Great SEO starts with human beings who are creative, tenacious, and empathetic to the needs of searchers,” says Fishkin. “No software can ever automate those processes.” 

There are other trends in search engine optimization that are important as well, among them the integration of social signals into ranking algorithms, and SEO that is not just text-based, but about understanding and optimizing images and videos – perhaps eventually for wearable technology purposes as that sector grows. 

All in all, legitimate SEO has evolved with the search engines – and it continues to do so. As we’ve said, understanding and applying this new information requires a new type of SEO practitioner, and a different kind of user-focused SEO.

• We've also listed the best online marketing services.