Posted by randfish
When is it smart to focus on viral-worthy content and clickbait? When is it not? To see fruitful returns from these kinds of efforts, they need to be done the right way and used in the right places. Rand discusses what kind of content investments make sense for this type of strategy and explains why it works in this week's Whiteboard Friday.
Howdy, Moz fans, and welcome to another edition of Whiteboard Friday. This week we're chatting about when and where you might use clickbait and linkbait and viral-focused content as compared to other types for your SEO-driven campaigns.
Let's say that you're a payroll software provider. Your goal is to increase traffic and conversions, and so you're considering what types of content investments you and your consultant or agency or in-house team might be making on the content front. That could be things like what we've got here:
A. Viral, news-worthy linkbait
I don't necessarily love the word "linkbait," but it still gets a lot of searches, so we're putting it in the title of the Whiteboard Friday because we practice what we preach here, baby.
B. Searcher-focused solutions
But there are other options for that, like searcher-focused solutions. So they might say, "Hey, we want to build some content around how to set up payroll as an LLC. That gets a lot of searches. We serve LLCs with our payroll solution. Let's try and target those folks. So here's how to set up payrolls in LLCs in six easy steps. There are the six steps."
Scrolling anchor button
Easy implementation, little coding needed
Jarring user experience
Multiple different forms
Requires building the form elsewhere and introduces some styling and analytics complexity to workflow
Sticky form (always there)
Keeps and design tracking within one Unbounce project
CTA fatigue, using up a lot of page space
Sticky form (appearing and disappearing)
The benefits of a sticky form, plus avoiding the CTA fatigue and large space requirement
CSS knowledge required, can only use one form
Personally, my favorite has been the Sticky Form (appearing and disappearing) option, to reduce the need to integrate external tools, but if I had to use multiple different forms I could definitely imagine using an iFrame.
Which is your favorite? Have I missed any cool solutions? Feel free to ping me in the comments.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
via Blogger Getting Around the "One Form" Problem in Unbounce
Posted by AlliBerry3
One of the advantages of working for an agency is the volume of websites we get to evaluate. The majority of clients who sign up for ongoing SEO and/or content services will receive a content audit. Similar to a technical SEO audit, the results of the content audit should drive the strategies and priorities of the next stages of content work. Without the audit, you can’t create an effective strategy because you first need to know what types of content you’ve got, what content you’re missing, and what content you’ve got too much of.
While there are many posts out there about how to perform a content audit (and I encourage you to check out these posts: How to Do a Content Audit and 5 Lessons Learned from a Content Audit), I am going to be focusing on what my common findings have been from recently conducting 15 content audits. My aim is to give you more of a framework on how you can talk to clients about their content or, if you are the client, ways you can improve your website content to keep users on the site longer and, ultimately, convert.
Mistake #1: No clear calls-to-action
I have yet to complete a content audit where creating clearer calls-to-action wasn’t a focus. The goal of a page should be obvious to any visitor (or content auditor). What is it that you want a visitor who lands on this page to do next? Many of our clients are not e-commerce, so it may feel less obvious; however, assuming you want someone to stay on your website, what’s next?
Even if answer is “I want them to visit my store,” make it easy for them. Add a prominent “Visit Our Store” button. If it’s a simple blog page, what are the next blog articles someone should read based on what they just read? Or do you have a relevant e-book you’d like them to download? You got them to the end of your post — don’t lose the visitor because they aren’t sure what to do next!
Mistake #2: A lack of content for all stages of the customer journey
One thing we often do when conducting content audits is track where in the sales funnel each page is aimed (awareness, consideration, purchase, or retention). What we sometimes find is that clients tend to have a disproportionate amount of content aimed at driving a purchase, but not enough for awareness, consideration, and retention. This isn’t always the case, particularly if they have a blog or resources hub; however, the consideration and retention stages are often overlooked. While the buyer cycle is going to be different for every product, it’s still important to have content that addresses each stage, no matter how brief the stage is.
Retention is a big deal too! It is way more cost-efficient and easier to upsell and cross-sell current customers than bring in new. Your customers are also less price-sensitive because they know your brand is worth it. You definitely want to provide content for this audience too to keep them engaged with the brand and find new uses for your products. Plus, you’ve already got their contact information, so delivering content to them is much easier than a prospect.
Here are some examples of content for each stage:
Awareness: Blog posts (explainers, how-tos, etc), e-books, educational webinars, infographics
Consideration: Product comparisons, case studies, videos
Purchase: Product pages, trial offers, demos, coupons
Retention: Blog posts (product applications, success stories, etc), newsletters, social media content
Mistake #3: Testimonials aren’t used to their full potential
There are so many pages dedicated solely to testimonials out there on the Interwebs. It’s painful. Who trusts a testimonials page over reviews on third-party sites like Yelp, Google My Business, or Tripadvisor? No one. That being said, there is a place for testimonials. It’s just not on a testimonials page.
The best way to use a testimonial is to pair it with the appropriate copy. If it’s a testimonial about how easy and fast a customer received their product, use that on a shipping page. If it’s a testimonial about how a product solved a problem they had, use it on that product page. This will enhance your copy and help to alleviate any anxieties a prospective customer has with their decision to purchase.
Testimonials can also help you improve your local relevance in search. If you have a storefront that is targeting particular cities, ask for a customer’s city and state when you gather testimonials. Then, include relevant testimonials along with their city and state on the appropriate location page(s). Even if your store is in Lakewood, Colorado, collecting testimonials from customers who live in Denver and including them on your location page will help both search engines and users recognize that Denver people shop there.
Mistake #4: Not making content locally relevant (if it matters)
If location matters to your business, you should not only use testimonials to boost your local relevance, but your content in general. Take the auto dealership industry, for example. There are over 16,000 car dealerships in the United States and they all (presumably) have websites. Many of them have very similar content because they are all trying to sell the same or similar models of cars.
The best car dealership websites, however, are creating content that matters to their local communities. People who live in Denver, for example, care about what the best cars are for driving in the mountains, whereas people in the Los Angeles area are more likely to want to know which cars get the best highway gas mileage. Having your sales team take note of common questions they get asked and addressing them in your content can go a long way toward improving local relevance and gaining loyal customers.
Mistake #5: Not talking about pricing
Many companies, B2B companies in particular, do not want to list pricing on their website. It’s understandable, especially when the honest answer to “how much does your service cost?” is “it depends.” The problem with shying away from pricing altogether, though, is that people are searching for pricing information. It’s a huge missed opportunity not to have any content related to pricing, and it annoys prospective customers who would rather know your cost range before giving you a call or submitting a form for follow up.
It’s mutually beneficial to have pricing information (or at least information on how you determine pricing) on your website because it’ll help qualify leads. If a prospect knows your price range and they still reach out for more information, they're going to be a much better lead than someone who is reaching out to get pricing information. This saves your sales team the trouble of wasting their time on bad leads.
Having pricing information on your website also helps establish trust with the prospect. If you aren’t transparent about your pricing, it looks like you charge as much as you can get away with. The more information you provide, the more trustworthy your business looks. And if all of your competitors are also hiding their pricing, you’re the first one they’ll likely reach out to.
Mistake #6: Getting lost in jargon
There are a lot of great companies out there doing great work. And more often than not, their website does not reflect it as well as it could. It isn’t uncommon for those tasked with writing web copy to be quite close to the product. What sometimes happens is jargon and technical language dominates, and the reason why a customer should care gets lost. When it comes to explaining a product or service, Joel Klettke said it best at MozCon 2017. A web page should include:
- What is the product and why should a prospect care about it?
- How will this product make the prospect’s life easier/better?
- What’s the next step? (CTA)
It’s also important to include business results, real use cases, and customer successes with the product on your website too. This establishes more trust and supports your claims about your products. Doing this will speak to your customers in a way that jargon simply will not.
Mistake #7: Page duplication from migration to HTTPS
With more sites getting an SSL certificate and moving to HTTPS, it’s more important than ever to make sure you have 301 redirects set up from the HTTP version to the HTTPS version to prevent unintentional duplication of your entire website. Duplicate content can impact search rankings as search engines struggle to decide which version of a page is more relevant to a particular search query. We’ve been seeing quite a few sites that have an entire duplicate site or some isolated pages that didn’t get redirects in place in their migrations. We also keep seeing sites that have www and non-www versions of pages without 301 redirects as well. Running regular crawls will help you stay on top of this kind of duplicate content.
Here are a couple of good resources to check out when doing an HTTPS migration:
- HTTP to HTTPS Migration: The Ultimate Stress-Free Guide
- Implementing HTTPS: Options to Consider with JR Ridley (podcast)
Mistake #8: Poor internal linking and site architecture
How content is organized on a site can be just as important as what the content is. Without proper organization, users can struggle to surf a website successfully and search engines have a difficult time determining which pages are considered most important. Making sure your most important pages are structured to be easy to find, by listing them in your navigation, for example, is a good user experience and will help those pages perform better.
Part of making important pages easy to find is through internal linking. Web content is often created on an ongoing basis, and being smart about internal linking requires taking the time to look holistically at the site and figuring out which pages make the most sense to link to and from. I keep encountering blog content that does not link back to a core page on the site. While you don’t want product to be the focus of your blog, it should be easy for a user to get to the core pages of your site if they want to do so. As you’re auditing a site, you’ll find pages that relate to one another that don’t link. Make notes of those as you go so you can better connect pages both in copy and with your calls to action.
What I find most interesting about content audits is how subjective they are. Defining what makes content good or bad is gray in a way that identifying whether or not a page has, say, a canonical tag, is not. For that reason, I have found that what content auditors focus most heavily on tend to be a reflection of the background of the person doing the audit. And the most common content mistakes I have touched on here reflect my background perfectly, which is a meld of SEO and content marketing.
So, I’m curious: what do you look for and find in your content audits? What would you add to my list?
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
via Blogger 8 Common Website Mistakes Revealed Via Content Audits
Posted by SarahBird
Yay! We’ve traversed another year around the sun. And I’m back with another Moz year-in-review post that promises to be as boring as its predecessors. Reading it feels like being locked in your tin can space capsule through lightyears of empty space. If you’re a little odd and like this kind of thing, do please continue.
Before we begin our odyssey, I invite you to check out previous reports: 2016, 2015, 2014, 2013, 2012. Transparency is a Moz core value. Putting detailed financial and customer data on the blog is one of the ways we live our values. We’re a little weird like that.
Launch to your favorite parts:
Part 1: TL;DR
Commencing countdown, engines on
What a year! 2017 was a time of doing new things differently — new teams, new goals, and new ways of operating. I’m so proud of Mozzers; they can do anything. If I’m sent to a far-off space colony, I want them with me.
In spite of (and because of!) all this change, we grew revenue and became significantly EBITDA and cash flow positive. Nice! We have a nice economic engine primed and ready to make some more big investments in 2018. Stay tuned.
These positive results were not from one single thing. Rather, iterative product and operations improvements helped improve both our top and bottom line. Plus, we made a bunch of longer-term investments that don’t show up yet in the 2017 report but will bear fruit in 2018 and beyond.
Part 2: Ch-ch-ch... Changes!
Now it's time to leave the capsule if you dare
Here’s a little more detail on some of the changes I talked about.
We launched Keywords By Site, relaunched our crawler (a major technical undertaking), sunsetted two products (content and Followerwonk), built a bunch of new developer tools and standardized on some dev frameworks, and improved our local data distribution network. Check out Adam Feldstein’s post for a lot more detail on our 2017 major product accomplishments!
We’ve got another exciting launch on the way, too. We’ve invested a ton of blood, sweat, and tears into it during 2017 and can’t wait to share it with everyone.
All of these changes support our 2016 strategy of “more wood behind fewer arrows.” We choose to focus our energy on being the best place to learn and do SEO. Our mission is to simplify SEO for everyone through software, education, and community.
For those of you worried about Followerwonk, it’s going to okay. Better than okay. Our beloved “Work Dad” Marc Mims is now the proud father of Followerwonk! Marc’s dedication to the success of Followerwonk has never wavered over the many, many years he’s been building and maintaining it. We already miss his compassion, humor, and bike stories around the Mozplex. We wish him and Followerwonk the best! We bought that product because we loved it then; we love it even now. Sadly, though, it never quite fit with our mission as well as we'd hoped.
We created new programs to help people get the SEO help that’s right for them. We completely rebuilt our SEO Learning Center with fresh educational content. There’s a brand-new SEO podcast, MozPod, for you to check out.
We also began experimenting with and are now expanding SEO training workshops delivered by experts we trust and admire. I’m so excited about this because it’s a new way for Moz to have impact; it’s personal, live, interactive, and immediate in a way that most of our SEO education work can’t be. We won’t stop doing free, scalable education. It’s core to our beliefs. But it is fun to deliver custom, live training sessions in the mix too.
Many of our accomplishments are behind the scenes, and will deliver long-term positive impact.
Our investments in retiring tech debt, improving monitoring, investing in our development platforms, and nurturing our engineering culture have resulted in the most stable and performant software in Moz history. Our hard work and ingenuity is paying off in resilient and performant software.
We’ve also rebuilt most of our customer stack: new Salesforce implementation, HubSpot launch, new internal data warehouse, new CMS (Craft), Segment.io, and more! Phew! That’s a lot! In Q1 2018, we started with Terminus for Account-Based Marketing, and partnered with third-party data vendors, like Full Contact, to supplement our data warehouse. These big changes are going to set us up really well for the years ahead. And we’ve got more internal tools launching soon!
We are on a roll with internal improvements and momentum.
Part 3: Customer metrics
You've really made the grade
We could ship and launch until our circuits go dead, but at the end of the day all our work is in service of meeting your needs.
We know you can hear us! You’re following us now more than ever before.
Part 4: Financial performance
And the stars look very different today
Check out the infographic view of our data barf.
I’m proud of what we accomplished in 2017, especially considering the incredible amount of change in strategy and team structure. More revenue while spending less = magic! Also, the economic strength we’ve built will allow us to place some nice-sized bets this year. Boom!
We made $47.4 million in GAAP revenue in 2017, an increase of 11% from 2016.
We brought our over all expenses way down in 2017. Cost of Revenue increased slightly to $11.8 million. We reduced operating expenses aggressively. Curious on what we spend on, and trends? Check out this breakdown of our major expenses (OpEx and Cost of Revenue) as a percentage of annual revenue:
We generated cash, positive EBITDA, and for the first time in recent Moz history, we were positive net income.
That’s quite a turnaround from 2016, in which we closed the year negative EBITDA of $5.7 million! We flipped EBITDA! We have adopted a cash-flow-neutral-to-positive operating philosophy right now to be ready for some future investments. We may decide to go cash flow negative again to fund further growth.
Part 5: Inside Moz HQ
The papers want to know whose shirts you wear
So, who is behind the wheel here?
We ended 2017 with roughly the same number of Mozzers as we began. It was a conscious choice to remain approximately headcount neutral in 2017; we only opened up new positions after ensuring rigorous conversations took place around the business need for the role. This discipline is hard to live under, but we like the results. We’re working smarter, and getting more rigorous in our decision-making.
Let me be clear: WE ARE HIRING! These are just 5 of our currently open positions:
- DevOps Engineer
- Sr. Platform Software Engineer
- Backend Engineer - C++
- Chief Marketing Officer
See more at our Careers page!
Here’s where we need YOU: Moz is committed to bringing more women into tech. There is a dire lack of diversity in the technology industry. This past year we added 6% more women to the company overall and 9% to engineering specifically. We must and will do better. We need more women in engineering and leadership roles here. Check out those jobs above and join the team!
Moz partners with some fantastic organizations focused on getting more women into the tech pipeline. Ada Academy, Year Up, Ignite Worldwide, and Techbridge all encourage women and girls to pursue STEM careers early in their lives. Our newest partner, Unloop, enables people who have been in prison to develop skills and succeed in careers in tech. It is our responsibility to ensure that all people have opportunity and access to participate in STEM fields.
Generosity comes in many forms. One way in which we support the generosity of Mozzers is to match charitable donations to 501c3 organizations by 150%.
We also donated our space 35 times to various organizations in the community requesting to use the Mozplex as a venue for their meetups. Check our our event brochure and take a 360 tour of the Mozplex!
Mozzers also donate a ton of time to causes they are passionate about. We also offer a very discounted price for nonprofits that we’re happy many folks take advantage of. We’re passionate about communities and helping folks.
Moz partnered with Halo Partners to provide professional coaching to all employees. 54 Mozzers received coaching. 27 Mozzers used this benefit for the first time! I’m a huge believer in coaching and training. Beginner’s mind is how we grow and become the best versions of ourselves.
Through it all, we made sure to have some fun. Moz offers a Paid Paid Vacation benefit, reimbursing employees $3k per year in vacation costs. Yes, that’s right. You get your regular pay, plus another $3k a year to spend on your trip! It’s bonkers!
Mozzers visited 6 of the 7 continents last year!
We also had 7 Mozling babies last year. Luv those babies.
Part 6: Into the future
I think my spaceship knows which way to go
2017 was a strengthening year for Moz. We went through a lot of change and made some important investments. Mozzers are dynamic, helpful, smart, and hardworking. They have a service orientation and build for the long term. The investments we made in 2017 will bear fruit in the years ahead. And we’re poised to make some ambitious moves in the coming months.
While I’m proud of what we’ve accomplished, I believe we have higher mountains still to climb. We have had triumphs and tribulations, heartbreaks and happy dances. These many years later, the SEO industry is healthy, growing, and dynamic. Many organizations are still struggling with basic web presence, let alone thoughtful SEO strategy. Moz is still teensy-tiny compared to the opportunity. I believe the opportunity for SEO expertise is vast.
I want to close on a note of gratitude.
First, a bunch of folks helped pull together the metrics for our 2017 report, and I am deeply grateful for their help. This post is kind of a bear! Thank you Jess, Felicia, Christian, Kevin, Susan, Michael, Jeremy, and anyone else who pulled data and helped get this post off the ground!
Second, thank you to this community. It’s because of you that we are here. This community would be nothing if it wasn’t for your care, attention, and feedback. We will continue to work hard to make your work lives more enjoyable and successful. We want to be your favorite resource for doing great SEO. If we’re not there yet, trust that we will keep working to be. Thank you for the opportunity to serve.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
via Blogger The Moz Year in Review 2017
Posted by BritneyMuller
Winning featured snippets is one of the best ways to get visibility on page one of Google's SERPs. It's a competitive environment, though, and there are tons of specific considerations when it comes to increasing your chances of earning that spot. Today's Whiteboard Friday, number one of an upcoming three-part series, is brought to you by Moz's resident SEO and mini-pig advocate, Britney Muller. She covers the keyword research you'll need to do, evaluating your current ranking, and recording relevant data in a spreadsheet with the help of a couple useful tools.
Hey, Moz fans, welcome to another edition of Whiteboard Friday. Today we're going over all things discovering featured snippet opportunities. So this is the first part to three videos. So this will be the discover, but we're also going to have a target and a measure video as well. So really, really excited. It's going to be a ton of fun. I'm doing it with you, so you're not going to be alone. It's going to be this cool thing we all do together.
Part 1 of 3: Discover, target, measure
So for those of you that don't know what a featured snippet is, it is one of those answer boxes at the top of search results. So let's say you do a search like, oh, I don't know, "Are teacup pigs healthy?" which they're not, super sad. I love pigs. But you'll get a featured snippet box like this that tells you like, "No, they're actually starved." It gives you all this information. So it's different than something like "People also ask..." boxes or your typical search results.
They're particularly appealing because of the voice search component. So most voice searches come from those featured snippet boxes as well as it just being really appealing in general to have that top spot.
#1 Keyword research
So this process is pretty straightforward. You're going to start with just your basic keyword research. So you're also going to focus on questions. A.J. Ghergich did this incredible study write-up, on SEMrush, about featured snippets, where he discovered that around 41% of all featured snippets come from questions, which makes sense. The how questions are really interesting because those results are most likely to result in lists.
Now, lists take both the form of numerical as well as bullets. So something to kind of keep in mind there. But what's interesting about these lists is that they tend to be a little bit easier to truncate. So if you know that Google is showing 8 results, maybe you go back to your page and you make sure that you have 10. That way it lures people in to click to see the full list. Really interesting there.
#2 Evaluate your current ranking
You also want to evaluate your current ranking for these particular keywords. You want to prioritize keywords based on ones that you rank on page one for. It tends to be much easier to grab a featured snippet or to steal one if you're also on page one.
#3 Put data into a spreadsheet
From there, we're going to put all of this data and more data into a big spreadsheet so that we can really analyze and prioritize some of these opportunities. So some of the metrics I came up with — feel free to share some ideas below — are your keyword, average monthly search volume, current featured snippet URL, that's this guy over here. What is that domain authority and page authority? You want to make note of those. Is it a featured snippet hub? This is such a cool term, that A.J. came up with in his article, that essentially coins a featured snippet URL that ranks for 10 or more featured snippet boxes. You probably won't know this right away, so this might stay blank. But once you start seeing more and more of those same URLs, you might think it's one of those hubs. It's kind of cool.
Featured snippet type. Is it a paragraph, a list, or a table? Is there any markup? Is there schema markup? What's going on, on the page in general? Just sort of scope all that out. What's your rank? This is actually supposed to be over here. So, again, you want to see if you're ranking 10 or under on a particular page, hopefully on page 1.
Then is there an image? So the featured snippet images are really interesting, because Google likes to swap them out and remove them and test them and do all this crazy stuff. I got to talk about these images and the tests that I've been doing on them on the Two Peas podcast with Brian Childs, part of his MozPod podcast series. It was super fun. I share some secrets in there, so go check it out.
Then what's the type of image? So typically, you can start to see a theme of a particular niche or industry in their featured snippet images. Maybe they're all stock photos, or maybe they're all informational type photos. Maybe they all have the same color. Really interesting to sort of keep an eye on those.
What's your desired featured snippet URL? This will typically be whatever URL is ranking. But maybe not. Make note of that.
Other notes, you can mention where Google is pulling the featured snippet from that page. I think that stuff is super interesting. You can do all sorts of fun stuff.
Research tools to use
So two primary tools to do all of this research within are Moz Keyword Explorer and SEMrush. Both have some caveats. Moz Keyword Explorer is great because you can do a bunch of keyword research and save them into lists. However, you can't do keyword research only viewing the keywords that have featured snippets. You can't do that. You have to do all the keyword research, put it into a list, and then we give you that data.
With SEMrush, it's pretty cool. You get to filter only by featured snippet keywords. So that, off the bat, is awesome.
However, once you get a keyword list put together in Keyword Explorer, not only do you get that information of whether or not there's a featured snippet, but right within your list of keywords, you have the ability to add your website and immediately see your rank for all of those particular keywords in your list, making this super, super easy.
I tried to do this with SEMrush. I know they have all of the features necessary to do so. However, it's just not as easy. You have to use a combination of their different tools within their tool. I hit a couple different limits within Keyword Analyzer, and then by the time I got to position tracking, I lost my search volume from Keyword Magic tool, which was super frustrating.
There might be a better, easier way to do that. Maybe their API are pulling some stuff a little bit differently. Feel free to comment down below. Maybe there's a better way than either of these. I don't know. You could also do it pretty manually too. You could use Google Keyword Planner and look some of this stuff up yourself.
But I hope you enjoyed this. Thank you so much for joining me on this edition of Whiteboard Friday. I look forward to seeing you all soon. Thanks.
via Blogger Discover Featured Snippet Opportunities - Whiteboard Friday
Posted by Modestos
What is a site migration?
A site migration is a term broadly used by SEO professionals to describe any event whereby a website undergoes substantial changes in areas that can significantly affect search engine visibility — typically substantial changes to the site structure, content, coding, site performance, or UX.
Google’s documentation on site migrations doesn’t cover them in great depth and downplays the fact that so often they result in significant traffic and revenue loss, which can last from a few weeks to several months — depending on the extent search engine ranking signals have been affected, as well as how long it may take the affected business to rollout a successful recovery plan.
Quick access links
Site migration examples
Site migration types
Common site migration pitfalls
Site migration process
1. Scope & planning
2. Pre-launch preparation
3. Pre-launch testing
4. Launch day actions
5. Post-launch testing
6. Performance review
Appendix: Useful tools
Site migration examples
The following section discusses how both successful and unsuccessful site migrations look and explains why it is 100% possible to come out of a site migration without suffering significant losses.
Debunking the “expected traffic drop” myth
Anyone who has been involved with a site migration has probably heard the widespread theory that it will result in de facto traffic and revenue loss. Even though this assertion holds some truth for some very specific cases (i.e. moving from an established domain to a brand new one) it shouldn’t be treated as gospel. It is entirely possible to migrate without losing any traffic or revenue; you can even enjoy significant growth right after launching a revamped website. However, this can only be achieved if every single step has been well-planned and executed.
Examples of unsuccessful site migrations
The following graph illustrates a big UK retailer’s botched site migration where the website lost 35% of its visibility two weeks after switching from HTTP to HTTPS. It took them about six months to fully recover, which must have had a significant impact on revenue from organic search. This is a typical example of a poor site migration, possibly caused by poor planning or implementation.
But recovery may not always be possible. The below visibility graph is from another big UK retailer, where the HTTP to HTTPS switchover resulted in a permanent 20% visibility loss.
In fact, it’s is entirely possible to migrate from HTTP to HTTPS without losing so much traffic and for so such a long period, aside from the first few weeks where there is high volatility as Google discovers the new URLs and updates search results.
Examples of successful site migrations
What does a successful site migration look like? This largely depends on the site migration type, the objectives, and the KPIs (more details later). But in most cases, a successful site migration shows at least one of the following characteristics:
- Minimal visibility loss during the first few weeks (short-term goal)
- Visibility growth thereafter — depending on the type of migration (long-term goal)
The following visibility report is taken from an HTTP to HTTPS site migration, which was also accompanied by significant improvements to the site’s page loading times.
The following visibility report is from a complete site overhaul, which I was fortunate to be involved with several months in advance and supported during the strategy, planning, and testing phases, all of which were equally important.
As commonly occurs on site migration projects, the launch date had to be pushed back a few times due to the risks of launching the new site prematurely and before major technical obstacles were fully addressed. But as you can see on the below visibility graph, the wait was well worth it. Organic visibility not only didn’t drop (as most would normally expect) but in fact started growing from the first week.
Visibility growth one month after the migration reached 60%, whilst organic traffic growth two months post-launch exceeded 80%.
This was a rather complex migration as the new website was re-designed and built from scratch on a new platform with an improved site taxonomy that included new landing pages, an updated URL structure, lots of redirects to preserve link equity, plus a switchover from HTTP to HTTPS.
In general, introducing too many changes at the same time can be tricky because if something goes wrong, you’ll struggle to figure out what exactly is at fault. But at the same time, leaving major changes for a later time isn’t ideal either as it will require more resources. If you know what you’re doing, making multiple positive changes at once can be very cost-effective.
Before getting into the nitty-gritty of how you can turn a complex site migration project into a success, it’s important to run through the main site migration types as well as explain the main reason so many site migrations fail.
Site migration types
There are many site migration types. It all depends on the nature of the changes that take place to the legacy website.
Google’s documentation mostly covers migrations with site location changes, which are categorised as follows:
- Site moves with URL changes
- Site moves without URL changes
Site move migrations
These typically occur when a site moves to a different URL due to any of the below:
A classic example is when migrating from HTTP to HTTPS.
Subdomain or subfolder change
Very common in international SEO where a business decides to move one or more ccTLDs into subdomains or subfolders. Another common example is where a mobile site that sits on a separate subdomain or subfolder becomes responsive and both desktop and mobile URLs are uniformed.
Domain name change
Commonly occurs when a business is rebranding and must move from one domain to another.
Top-level domain change
This is common when a business decides to launch international websites and needs to move from a ccTLD (country code top-level domain) to a gTLD (generic top-level domain) or vice versa, e.g. moving from .co.uk to .com, or moving from .com to .co.uk and so on.
Site structure changes
These are changes to the site architecture that usually affect the site’s internal linking and URL structure.
Other types of migrations
There are other types of migration which are triggered by changes to the site’s content, structure, design, or platform.
This is the case when a website is moved from one platform/CMS to another, e.g. migrating from WordPress to Magento or just upgrading to the latest platform version. Replatforming can, in some cases, also result in design and URL changes because of technical limitations that often occur when changing platforms. This is why replatforming migrations rarely result in a website that looks exactly the same as the previous one.
Major content changes such as content rewrites, content consolidation, or content pruning can have a big impact on a site’s organic search visibility, depending on the scale. These changes can often affect the site’s taxonomy, navigation, and internal linking.
Mobile setup changes
With so many options available for a site’s mobile setup moving, enabling app indexing, building an AMP site, or building a PWA website can also be considered as partial site migrations, especially when an existing mobile site is being replaced by an app, AMP, or PWA.
These are often caused by major changes to the site’s taxonomy that impact on the site navigation, internal linking and user journeys.
These can vary from major design changes in the look and feel to a complete website revamp that may also include significant media, code, and copy changes.
In addition to the above, there are several hybrid migration types that can be combined in practically any way possible. The more changes that get introduced at the same time the higher the complexity and the risks. Even though making too many changes at the same time increases the risks of something going wrong, it can be more cost-effective from a resources perspective if the migration is very well-planned and executed.
Common site migration pitfalls
Even though every site migration is different there are a few common themes behind the most typical site migration disasters, with the biggest being the following:
Some site migrations are doomed to failure way before the new site is launched. A strategy that is built upon unclear and unrealistic objectives is much less likely to bring success.
Establishing measurable objectives is essential in order to measure the impact of the migration post-launch. For most site migrations, the primary objective should be the retention of the site’s current traffic and revenue levels. In certain cases the bar could be raised higher, but in general anticipating or forecasting growth should be a secondary objective. This will help avoid creating unrealistic expectations.
Coming up with a detailed project plan as early as possible will help avoid delays along the way. Factor in additional time and resources to cope with any unforeseen circumstances that may arise. No matter how well thought out and detailed your plan is, it’s highly unlikely everything will go as expected. Be flexible with your plan and accept the fact that there will almost certainly be delays. Map out all dependencies and make all stakeholders aware of them.
Avoid planning to launch the site near your seasonal peaks, because if anything goes wrong you won’t have enough time to rectify the issues. For instance, retailers should avoid launching a site close to September/October to avoid putting the busy pre-Christmas period at risk. In this case, it would be much wiser launching during the quieter summer months.
Lack of resources
Before committing to a site migration project, estimate the time and effort required to make it a success. If your budget is limited, make a call as to whether it is worth going ahead with a migration that is likely to fail in meeting its established objectives and cause revenue loss.
As a rule of thumb, try to include a buffer of at least 20% in additional resource than you initially think the project will require. This additional buffer will later allow you to quickly address any issues as soon as they arise, without jeopardizing success. If your resources are too tight or you start cutting corners at this early stage, the site migration will be at risk.
Lack of SEO/UX consultation
When changes are taking place on a website, every single decision needs to be weighted from both a UX and SEO standpoint. For instance, removing great amounts of content or links to improve UX may damage the site’s ability to target business-critical keywords or result in crawling and indexing issues. In either case, such changes could damage the site’s organic search visibility. On the other hand, having too much text copy and few images may have a negative impact on user engagement and damage the site’s conversions.
To avoid risks, appoint experienced SEO and UX consultants so they can discuss the potential consequences of every single change with key business stakeholders who understand the business intricacies better than anyone else. The pros and cons of each option need to be weighed before making any decision.
Site migrations can span several months, require great planning and enough time for testing. Seeking professional support late is very risky because crucial steps may have been missed.
Lack of testing
In addition to a great strategy and thoughtful plan, dedicate some time and effort for thorough testing before launching the site. It’s much more preferable to delay the launch if testing has identified critical issues rather than rushing a sketchy implementation into production. It goes without saying that you should not launch a website if it hasn’t been tested by both expert SEO and UX teams.
Attention to detail is also very important. Make sure that the developers are fully aware of the risks associated with poor implementation. Educating the developers about the direct impact of their work on a site’s traffic (and therefore revenue) can make a big difference.
Slow response to bug fixing
There will always be bugs to fix once the new site goes live. However, some bugs are more important than others and may need immediate attention. For instance, launching a new site only to find that search engine spiders have trouble crawling and indexing the site’s content would require an immediate fix. A slow response to major technical obstacles can sometimes be catastrophic and take a long time to recover from.
Business stakeholders often do not anticipate site migrations to be so time-consuming and resource-heavy. It’s not uncommon for senior stakeholders to demand that the new site launch on the planned-for day, regardless of whether it’s 100% ready or not. The motto “let's launch ASAP and fix later” is a classic mistake. What most stakeholders are unaware of is that it can take just a few days for organic search visibility to tank, but recovery can take several months.
It is the responsibility of the consultant and project manager to educate clients, run them through all the different phases and scenarios, and explain what each one entails. Business stakeholders are then able to make more informed decisions and their expectations should be easier to manage.
Site migration process
The site migration process can be split into six main essential phases. They are all equally important and skipping any of the below tasks could hinder the migration’s success to varying extents.
Phase 1: Scope & Planning
Work out the project scope
Regardless of the reasons behind a site migration project, you need to be crystal clear about the objectives right from the beginning because these will help to set and manage expectations. Moving a site from HTTP to HTTPS is very different from going through a complete site overhaul, hence the two should have different objectives. In the first instance, the objective should be to retain the site’s traffic levels, whereas in the second you could potentially aim for growth.
A site migration is a great opportunity to address legacy issues. Including as many of these as possible in the project scope should be very cost-effective because addressing these issues post-launch will require significantly more resources.
However, in every case, identify the most critical aspects for the project to be successful. Identify all risks that could have a negative impact on the site’s visibility and consider which precautions to take. Ideally, prepare a few forecasting scenarios based on the different risks and growth opportunities. It goes without saying that the forecasting scenarios should be prepared by experienced site migration consultants.
Including as many stakeholders as possible at this early stage will help you acquire a deeper understanding of the biggest challenges and opportunities across divisions. Ask for feedback from your content, SEO, UX, and Analytics teams and put together a list of the biggest issues and opportunities. You then need to work out what the potential ROI of addressing each one of these would be. Finally, choose one of the available options based on your objectives and available resources, which will form your site migration strategy.
You should now be left with a prioritized list of activities which are expected to have a positive ROI, if implemented. These should then be communicated and discussed with all stakeholders, so you set realistic targets, agree on the project, scope and set the right expectations from the outset.
Prepare the project plan
Planning is equally important because site migrations can often be very complex projects that can easily span several months. During the planning phase, each task needs an owner (i.e. SEO consultant, UX consultant, content editor, web developer) and an expected delivery date. Any dependencies should be identified and included in the project plan so everyone is aware of any activities that cannot be fulfilled due to being dependent on others. For instance, the redirects cannot be tested unless the redirect mapping has been completed and the redirects have been implemented on staging.
The project plan should be shared with everyone involved as early as possible so there is enough time for discussions and clarifications. Each activity needs to be described in great detail, so that stakeholders are aware of what each task would entail. It goes without saying that flawless project management is necessary in order to organize and carry out the required activities according to the schedule.
A crucial part of the project plan is getting the anticipated launch date right. Ideally, the new site should be launched during a time when traffic is low. Again, avoid launching ahead of or during a peak period because the consequences could be devastating if things don’t go as expected. One thing to bear in mind is that as site migrations never go entirely to plan, a certain degree of flexibility will be required.
Phase 2: Pre-launch preparation
These include any activities that need to be carried out while the new site is still under development. By this point, the new site’s SEO requirements should have been captured already. You should be liaising with the designers and information architects, providing feedback on prototypes and wireframes well before the new site becomes available on a staging environment.
Review the new site’s prototypes or wireframes before development commences. Reviewing the new site’s main templates can help identify both SEO and UX issues at an early stage. For example, you may find that large portions of content have been removed from the category pages, which should be instantly flagged. Or you may discover that some high traffic-driving pages no longer appear in the main navigation. Any radical changes in the design or copy of the pages should be thoroughly reviewed for potential SEO issues.
Preparing the technical SEO specifications
Once the prototypes and wireframes have been reviewed, prepare a detailed technical SEO specification. The objective of this vital document is to capture all the essential SEO requirements developers need to be aware of before working out the project’s scope in terms of work and costs. It’s during this stage that budgets are signed off on; if the SEO requirements aren’t included, it may be impossible to include them later down the line.
The technical SEO specification needs to be very detailed, yet written in such a way that developers can easily turn the requirements into actions. This isn’t a document to explain why something needs to be implemented, but how it should be implemented.
Make sure to include specific requirements that cover at least the following areas:
- URL structure
- Meta data (including dynamically generated default values)
- Structured data
- Canonicals and meta robots directives
- Copy & headings
- Main & secondary navigation
- Internal linking (in any form)
- XML sitemap(s)
- HTML sitemap
- Hreflang (if there are international sites)
- Mobile setup (including the app, AMP, or PWA site)
- Custom 404 page
- Page loading times (for desktop & mobile)
The specification should also include areas of the CMS functionality that allows users to:
- Specify custom URLs and override default ones
- Update page titles
- Update meta descriptions
- Update any h1–h6 headings
- Add or amend the default canonical tag
- Set the meta robots attributes to index/noindex/follow/nofollow
- Add or edit the alt text of each image
- Include Open Graph fields for description, URL, image, type, sitename
- Include Twitter Open Graph fields for card, URL, title, description, image
- Bulk upload or amend redirects
- Update the robots.txt file
It is also important to make sure that when updating a particular attribute (e.g. an h1), other elements are not affected (i.e. the page title or any navigation menus).
Identifying priority pages
One of the biggest challenges with site migrations is that the success will largely depend on the quantity and quality of pages that have been migrated. Therefore, it’s very important to make sure that you focus on the pages that really matter. These are the pages that have been driving traffic to the legacy site, pages that have accrued links, pages that convert well, etc.
In order to do this, you need to:
- Crawl the legacy site
- Identify all indexable pages
- Identify top performing pages
How to crawl the legacy site
Crawl the old website so that you have a copy of all URLs, page titles, meta data, headers, redirects, broken links etc. Regardless of the crawler application of choice (see Appendix), make sure that the crawl isn’t too restrictive. Pay close attention to the crawler’s settings before crawling the legacy site and consider whether you should:
- Ignore robots.txt (in case any vital parts are accidentally blocked)
- Follow internal “nofollow” links (so the crawler reaches more pages)
- Crawl all subdomains (depending on scope)
- Crawl outside start folder (depending on scope)
- Change the user agent to Googlebot (desktop)
- Change the user agent to Googlebot (smartphone)
Pro tip: Keep a copy of the old site’s crawl data (in a file or on the cloud) for several months after the migration has been completed, just in case you ever need any of the old site’s data once the new site has gone live.
How to identify the indexable pages
Once the crawl is complete, work on identifying the legacy site’s indexed pages. These are any HTML pages with the following characteristics:
- Return a 200 server response
- Either do not have a canonical tag or have a self-referring canonical URL
- Do not have a meta robots noindex
- Aren’t excluded from the robots.txt file
- Are internally linked from other pages (non-orphan pages)
The indexable pages are the only pages that have the potential to drive traffic to the site and therefore need to be prioritized for the purposes of your site migration. These are the pages worth optimizing (if they will exist on the new site) or redirecting (if they won’t exist on the new site).
How to identify the top performing pages
Once you’ve identified all indexable pages, you may have to carry out more work, especially if the legacy site consists of a large number of pages and optimizing or redirecting all of them is impossible due to time, resource, or technical constraints.
If this is the case, you should identify the legacy site’s top performing pages. This will help with the prioritization of the pages to focus on during the later stages.
It’s recommended to prepare a spreadsheet that includes the below fields:
- Legacy URL (include only the indexable ones from the craw data)
- Organic visits during the last 12 months (Analytics)
- Revenue, conversions, and conversion rate during the last 12 months (Analytics)
- Pageviews during the last 12 months (Analytics)
- Number of clicks from the last 90 days (Search Console)
- Top linked pages (Majestic SEO/Ahrefs)
With the above information in one place, it’s now much easier to identify your most important pages: the ones that generate organic visits, convert well, contribute to revenue, have a good number of referring domains linking to them, etc. These are the pages that you must focus on for a successful site migration.
The top performing pages should ideally also exist on the new site. If for any reason they don’t, they should be redirected to the most relevant page so that users requesting them do not land on 404 pages and the link equity they previously had remains on the site. If any of these pages cease to exist and aren’t properly redirected, your site’s rankings and traffic will negatively be affected.
Once the launch of the new website is getting close, you should benchmark the legacy site’s performance. Benchmarking is essential, not only to compare the new site’s performance with the previous one but also to help diagnose which areas underperform on the new site and to quickly address them.
Keywords rank tracking
If you don’t track the site’s rankings frequently, you should do so just before the new site goes live. Otherwise, you will later struggle figuring out whether the migration has gone smoothly or where exactly things went wrong. Don’t leave this to the last minute in case something goes awry — a week in advance would be the ideal time.
Spend some time working out which keywords are most representative of the site’s organic search visibility and track them across desktop and mobile. Because monitoring thousands of head, mid-, and long-tail keyword combinations is usually unrealistic, the bare minimum you should monitor are keywords that are driving traffic to the site (keywords ranking on page one) and have decent search volume (head/mid-tail focus)
If you do get traffic from both brand and non-brand keywords, you should also decide which type of keywords to focus on more from a tracking POV. In general, non-brand keywords tend to be more competitive and volatile. For most sites it would make sense to focus mostly on these.
Don’t forget to track rankings across desktop and mobile. This will make it much easier to diagnose problems post-launch should there be performance issues on one device type. If you receive a high volume of traffic from more than one country, consider rank tracking keywords in other markets, too, because visibility and rankings can vary significantly from country to country.
The new site’s page loading times can have a big impact on both traffic and sales. Several studies have shown that the longer a page takes to load, the higher the bounce rate. Unless the old site’s page loading times and site performance scores have been recorded, it will be very difficult to attribute any traffic or revenue loss to site performance related issues once the new site has gone live.
It’s recommended that you review all major page types using Google’s PageSpeed Insights and Lighthouse tools. You could use summary tables like the ones below to benchmark some of the most important performance metrics, which will be useful for comparisons once the new site goes live.
Old site crawl data
A few days before the new site replaces the old one, run a final crawl of the old site. Doing so could later prove invaluable, should there be any optimization issues on the new site. A final crawl will allow you to save vital information about the old site’s page titles, meta descriptions, h1–h6 headings, server status, canonical tags, noindex/nofollow pages, inlinks/outlinks, level, etc. Having all this information available could save you a lot of trouble if, say, the new site isn’t well optimized or suffers from technical misconfiguration issues. Try also to save a copy of the old site’s robots.txt and XML sitemaps in case you need these later.
Search Console data
Also consider exporting as much of the old site’s Search Console data as possible. These are only available for 90 days, and chances are that once the new site goes live the old site’s Search Console data will disappear sooner or later. Data worth exporting includes:
- Search analytics queries & pages
- Crawl errors
- Blocked resources
- Mobile usability issues
- URL parameters
- Structured data errors
- Links to your site
- Internal links
- Index status
The redirects implementation is one of the most crucial activities during a site migration. If the legacy site’s URLs cease to exist and aren’t correctly redirected, the website’s rankings and visibility will simply tank.
Why are redirects important in site migrations?
Redirects are extremely important because they help both search engines and users find pages that may no longer exist, have been renamed, or moved to another location. From an SEO point of view, redirects help search engines discover and index a site’s new URLs quicker but also understand how the old site’s pages are associated with the new site’s pages. This association will allow for ranking signals to pass from the old pages to the new ones, so rankings are retained without being negatively affected.
What happens when redirects aren’t correctly implemented?
When redirects are poorly implemented, the consequences can be catastrophic. Users will either land on Not Found pages (404s) or irrelevant pages that do not meet the user intent. In either case, the site’s bounce and conversion rates will be negatively affected. The consequences for search engines can be equally catastrophic: they’ll be unable to associate the old site’s pages with those on the new site if the URLs aren’t identical. Ranking signals won’t be passed over from the old to the new site, which will result in ranking drops and organic search visibility loss. In addition, it will take search engines longer to discover and index the new site’s pages.
When the URLs between the old and new version of the site are different, use 301 (permanent) redirects. These will tell search engines to index the new URLs as well as forward any ranking signals from the old URLs to the new ones. Therefore, you must use 301 redirects if your site moves to/from another domain/subdomain, if you switch from HTTP to HTTPS, or if the site or parts of it have been restructured. Despite some of Google’s claims that 302 redirects pass PageRank, indexing the new URLs would be slower and ranking signals could take much longer to be passed on from the old to the new page.
302 (temporary) redirects should only be used in situations where a redirect does not need to live permanently and therefore indexing the new URL isn’t a priority. With 302 redirects, search engines will initially be reluctant to index the content of the redirect destination URL and pass any ranking signals to it. However, if the temporary redirects remain for a long period of time without being removed or updated, they could end up behaving similarly to permanent (301) redirects. Use 302 redirects when a redirect is likely to require updating or removal in the near future, as well as for any country-, language-, or device-specific redirects.
If you’d like to find out more about how Google deals with the different types of redirects, please refer to John Mueller’s post.
Redirect mapping process
If you are lucky enough to work on a migration that doesn’t involve URL changes, you could skip this section. Otherwise, read on to find out why any legacy pages that won’t be available on the same URL after the migration should be redirected.
The redirect mapping file is a spreadsheet that includes the following two columns:
- Legacy site URL –> a page’s URL on the old site.
- New site URL –> a page’s URL on the new site.
When mapping (redirecting) a page from the old to the new site, always try mapping it to the most relevant corresponding page. In cases where a relevant page doesn’t exist, avoid redirecting the page to the homepage. First and foremost, redirecting users to irrelevant pages results in a very poor user experience. Google has stated that redirecting pages “en masse” to irrelevant pages will be treated as soft 404s and because of this won’t be passing any SEO value. If you can’t find an equivalent page on the new site, try mapping it to its parent category page.
Once the mapping is complete, the file will need to be sent to the development team to create the redirects, so that these can be tested before launching the new site. The implementation of redirects is another part in the site migration cycle where things can often go wrong.
Increasing efficiencies during the redirect mapping process
Redirect mapping requires great attention to detail and needs to be carried out by experienced SEOs. The URL mapping on small sites could in theory be done by manually mapping each URL of the legacy site to a URL on the new site. But on large sites that consist of thousands or even hundreds of thousands of pages, manually mapping every single URL is practically impossible and automation needs to be introduced. Relying on certain common attributes between the legacy and new site can be a massive time-saver. Such attributes may include the page titles, H1 headings, or other unique page identifiers such as product codes, SKUs etc. Make sure the attributes you rely on for the redirect mapping are unique and not repeated across several pages; otherwise, you will end up with incorrect mapping.
Pro tip: Make sure the URL structure of the new site is 100% finalized on staging before you start working on the redirect mapping. There’s nothing riskier than mapping URLs that will be updated before the new site goes live. When URLs are updated after the redirect mapping is completed, you may have to deal with undesired situations upon launch, such as broken redirects, redirect chains, and redirect loops. A content-freeze should be placed on the old site well in advance of the migration date, so there is a cut-off point for new content being published on the old site. This will make sure that no pages will be missed from the redirect mapping and guarantee that all pages on the old site get redirected.
Don’t forget the legacy redirects!
You should get hold of the old site’s existing redirects to ensure they’re considered when preparing the redirect mapping for the new site. Unless you do this, it’s likely that the site’s current redirect file will get overwritten by the new one on the launch date. If this happens, all legacy redirects that were previously in place will cease to exist and the site may lose a decent amount of link equity, the extent of which will largely depend on the site’s volume of legacy redirects. For instance, a site that has undergone a few migrations in the past should have a good number of legacy redirects in place that you don’t want getting lost.
Ideally, preserve as many of the legacy redirects as possible, making sure these won’t cause any issues when combined with the new site’s redirects. It’s strongly recommended to eliminate any potential redirect chains at this early stage, which can easily be done by checking whether the same URL appears both as a “Legacy URL” and “New site URL” in the redirect mapping spreadsheet. If this is the case, you will need to update the “New site URL” accordingly.
URL A redirects to URL B (legacy redirect)
URL B redirects to URL C (new redirect)
Which results in the following redirect chain:
URL A –> URL B –> URL C
To eliminate this, amend the existing legacy redirect and create a new one so that:
URL A redirects to URL C (amended legacy redirect)
URL B redirects to URL C (new redirect)
Pro tip: Check your redirect mapping spreadsheet for redirect loops. These occur when the “Legacy URL” is identical to the “new site URL.” Redirect loops need to be removed because they result in infinitely loading pages that are inaccessible to users and search engines. Redirect loops must be eliminated because they are instant traffic, conversion, and ranking killers!
Implement blanket redirect rules to avoid duplicate content
It’s strongly recommended to try working out redirect rules that cover as many URL requests as possible. Implementing redirect rules on a web server is much more efficient than relying on numerous one-to-one redirects. If your redirect mapping document consists of a very large number of redirects that need to be implemented as one-to-one redirect rules, site performance could be negatively affected. In any case, double check with the development team the maximum number of redirects the web server can handle without issues.
In any case, there are some standard redirect rules that should be in place to avoid generating duplicate content issues:
- URL case: All URLs containing upper-case characters should be 301 redirected to all lower-case URLs, e.g. https://www.website.com/Page/ should be automatically redirecting to https://www.website.com/page/
- Host: For instance, all non-www URLs should be 301 redirected to their www equivalent, e.g. https://website.com/page/ should be redirected to https://www.website.com/page/
- Protocol: On a secure website, requests for HTTP URLs should be redirected to the equivalent HTTPS URL, e.g. http://www.website.com/page/ should automatically redirect to https://www.website.com/page/
- Trailing slash: For instance, any URLs not containing a trailing slash should redirect to a version with a trailing slash, e.g. http://www.website.com/page should redirect to http://www.website.com/page/
Even if some of these standard redirect rules exist on the legacy website, do not assume they’ll necessarily exist on the new site unless they’re explicitly requested.
Avoid internal redirects
Try updating the site’s internal links so they don’t trigger internal redirects. Even though search engines can follow internal redirects, these are not recommended because they add additional latency to page loading times and could also have a negative impact on search engine crawl time.
Don’t forget your image files
If the site’s images have moved to a new location, Google recommends redirecting the old image URLs to the new image URLs to help Google discover and index the new images quicker. If it’s not easy to redirect all images, aim to redirect at least those image URLs that have accrued backlinks.
Phase 3: Pre-launch testing
The earlier you can start testing, the better. Certain things need to be fully implemented to be tested, but others don’t. For example, user journey issues could be identified from as early as the prototypes or wireframes design. Content-related issues between the old and new site or content inconsistencies (e.g. between the desktop and mobile site) could also be identified at an early stage. But the more technical components should only be tested once fully implemented — things like redirects, canonical tags, or XML sitemaps. The earlier issues get identified, the more likely it is that they’ll be addressed before launching the new site. Identifying certain types of issues at a later stage isn’t cost effective, would require more resources, and cause significant delays. Poor testing and not allowing the time required to thoroughly test all building blocks that can affect SEO and UX performance can have disastrous consequences soon after the new site has gone live.
Making sure search engines cannot access the staging/test site
Before making the new site available on a staging/testing environment, take some precautions that search engines do not index it. There are a few different ways to do this, each with different pros and cons.
Site available to specific IPs (most recommended)
Making the test site available only to specific (whitelisted) IP addresses is a very effective way to prevent search engines from crawling it. Anyone trying to access the test site’s URL won’t be able to see any content unless their IP has been whitelisted. The main advantage is that whitelisted users could easily access and crawl the site without any issues. The only downside is that third-party web-based tools (such as Google’s tools) cannot be used because of the IP restrictions.
Password protecting the staging/test site is another way to keep search engine crawlers away, but this solution has two main downsides. Depending on the implementation, it may not be possible to crawl and test a password-protected website if the crawler application doesn’t make it past the login screen. The other downside: password-protected websites that use forms for authentication can be crawled using third-party applications, but there is a risk of causing severe and unexpected issues. This is because the crawler clicks on every link on a page (when you’re logged in) and could easily end up clicking on links that create or remove pages, install/uninstall plugins, etc.
Adding the following lines of code to the test site’s robots.txt file will prevent search engines from crawling the test site’s pages.
User-agent: * Disallow: /
One downside of this method is that even though the content that appears on the test server won’t get indexed, the disallowed URLs may appear on Google’s search results. Another downside is that if the above robots.txt file moves into the live site, it will cause severe de-indexing issues. This is something I’ve encountered numerous times and for this reason I wouldn’t recommend using this method to block search engines.
User journey review
If the site has been redesigned or restructured, chances are that the user journeys will be affected to some extent. Reviewing the user journeys as early as possible and well before the new site launches is difficult due to the lack of user data. However, an experienced UX professional will be able to flag any concerns that could have a negative impact on the site’s conversion rate. Because A/B testing at this stage is hardly ever possible, it might be worth carrying out some user testing and try to get some feedback from real users. Unfortunately, user experience issues can be some of the harder ones to address because they may require sitewide changes that take a lot of time and effort.
On full site overhauls, not all UX decisions can always be backed up by data and many decisions will have to be based on best practice, past experience, and “gut feeling,” hence getting UX/CRO experts involved as early as possible could pay dividends later.
Site architecture review
A site migration is often a great opportunity to improve the site architecture. In other words, you have a great chance to reorganize your keyword targeted content and maximize its search traffic potential. Carrying out extensive keyword research will help identify the best possible category and subcategory pages so that users and search engines can get to any page on the site within a few clicks — the fewer the better, so you don’t end up with a very deep taxonomy.
Identifying new keywords with decent traffic potential and mapping them into new landing pages can make a big difference to the site’s organic traffic levels. On the other hand, enhancing the site architecture needs to be done thoughtfully. Itt could cause problems if, say, important pages move deeper into the new site architecture or there are too many similar pages optimized for the same keywords. Some of the most successful site migrations are the ones that allocate significant resources to enhance the site architecture.
Meta data & copy review
Make sure that the site’s page titles, meta descriptions, headings, and copy have been transferred from the old to the new site without issues. If you’ve created any new pages, make sure these are optimized and don’t target keywords that have already been targeted by other pages. If you’re re-platforming, be aware that the new platform may have different default values when new pages are being created. Launching the new site without properly optimized page titles or any kind of missing copy will have an immediate negative impact on your site’s rankings and traffic. Do not forget to review whether any user-generated content (i.e. user reviews, comments) has also been uploaded.
Internal linking review
Internal links are the backbone of a website. No matter how well optimized and structured the site’s copy is, it won’t be sufficient to succeed unless it’s supported by a flawless internal linking scheme. Internal links must be reviewed throughout the entire site, including links found in:
- Main & secondary navigation
- Header & footer links
- Body content links
- Pagination links
- Horizontal links (related articles, similar products, etc)
- Vertical links (e.g. breadcrumb navigation)
- Cross-site links (e.g. links across international sites)
A series of technical checks must be carried out to make sure the new site’s technical setup is sound and to avoid coming across major technical glitches after the new site has gone live.
Robots.txt file review
Prepare the new site’s robots.txt file on the staging environment. This way you can test it for errors or omissions and avoid experiencing search engine crawl issues when the new site goes live. A classic mistake in site migrations is when the robots.txt file prevents search engine access using the following directive:
If this gets accidentally carried over into the live site (and it often does), it will prevent search engines from crawling the site. And when search engines cannot crawl an indexed page, the keywords associated with the page will get demoted in the search results and eventually the page will get de-indexed.
But if the robots.txt file on staging is populated with the new site’s robots.txt directives, this mishap could be avoided.
When preparing the new site’s robots.txt file, make sure that:
- It doesn’t block search engine access to pages that are intended to get indexed.
- The legacy site’s robots.txt file content has been reviewed and carried over if necessary.
- It references the new XML sitemaps(s) rather than any legacy ones that no longer exist.
Canonical tags review
Review the site’s canonical tags. Look for pages that either do not have a canonical tag or have a canonical tag that is pointing to another URL and question whether this is intended. Don’t forget to crawl the canonical tags to find out whether they return a 200 server response. If they don’t you will need to update them to eliminate any 3xx, 4xx, or 5xx server responses. You should also look for pages that have a canonical tag pointing to another URL combined with a noindex directive, because these two are conflicting signals and you;’ll need to eliminate one of them.
Meta robots review
Once you’ve crawled the staging site, look for pages with the meta robots properties set to “noindex” or “nofollow.” If this is the case, review each one of them to make sure this is intentional and remove the “noindex” or “nofollow” directive if it isn’t.
XML sitemaps review
Prepare two different types of sitemaps: one that contains all the new site’s indexable pages, and another that includes all the old site’s indexable pages. The former will help make Google aware of the new site’s indexable URLs. The latter will help Google become aware of the redirects that are in place and the fact that some of the indexed URLs have moved to new locations, so that it can discover them and update search results quicker.
You should check each XML sitemap to make sure that:
- It validates without issues
- It is encoded as UTF-8
- It does not contain more than 50,000 rows
- Its size does not exceed 50MBs when uncompressed
If there are more than 50K rows or the file size exceeds 50MB, you must break the sitemap down into smaller ones. This prevents the server from becoming overloaded if Google requests the sitemap too frequently.
In addition, you must crawl each XML sitemap to make sure it only includes indexable URLs. Any non-indexable URLs should be excluded from the XML sitemaps, such as:
- 3xx, 4xx, and 5xx pages (e.g. redirected, not found pages, bad requests, etc)
- Soft 404s. These are pages with no content that return a 200 server response, instead of a 404.
- Canonicalized pages (apart from self-referring canonical URLs)
- Pages with a meta robots noindex directive
<!DOCTYPE html> <html><head> <meta name="robots" content="noindex" /> (…) </head> <body>(…)</body> </html>
- Pages with a noindex X-Robots-Tag in the HTTP header
HTTP/1.1 200 OK Date: Tue, 10 Nov 2017 17:12:43 GMT (…) X-Robots-Tag: noindex (…)
- Pages blocked from the robots.txt file
Building clean XML sitemaps can help monitor the true indexing levels of the new site once it goes live. If you don’t, it will be very difficult to spot any indexing issues.
Pro tip: Download and open each XML sitemap in Excel to get a detailed overview of any additional attributes, such as hreflang or image attributes.
HTML sitemap review
Depending on the size and type of site that is being migrated, having an HTML sitemap can in certain cases be beneficial. An HTML sitemap that consists of URLs that aren’t linked from the site’s main navigation can significantly boost page discovery and indexing. However, avoid generating an HTML sitemap that includes too many URLs. If you do need to include thousands of URLs, consider building a segmented HTML sitemap.
The number of nested sitemaps as well as the maximum number of URLs you should include in each sitemap depends on the site’s authority. The more authoritative a website, the higher the number of nested sitemaps and URLs it could get away with.
For example, the NYTimes.com HTML sitemap consists of three levels, where each one includes over 1,000 URLs per sitemap. These nested HTML sitemaps aid search engine crawlers in discovering articles published since 1851 that otherwise would be difficult to discover and index, as not all of them would have been internally linked.
Structured data review
Errors in the structured data markup need to be identified early so there’s time to fix them before the new site goes live. Ideally, you should test every single page template (rather than every single page) using Google’s Structured Data Testing tool.
Be sure to check the markup on both the desktop and mobile pages, especially if the mobile website isn’t responsive.
The tool will only report any existing errors but not omissions. For example, if your product page template does not include the Product structured data schema, the tool won’t report any errors. So, in addition to checking for errors you should also make sure that each page template includes the appropriate structured data markup for its content type.
Again, the Fetch and Render tool can help diagnose this type of issue that, if left unresolved, could have a significant negative impact.
Mobile site SEO review
Assets blocking review
Mobile-first index review
In order to avoid any issues associated with Google’s mobile-first index, thoroughly review the mobile website and make there aren’t any inconsistencies between the desktop and mobile sites in the following areas:
- Page titles
- Meta descriptions
- Canonical tags
- Meta robots attributes (i.e. noindex, nofollow)
- Internal links
- Structured data
A responsive website should serve the same content, links, and markup across devices, and the above SEO attributes should be identical across the desktop and mobile websites.
In addition to the above, you must carry out a few further technical checks depending on the mobile site’s set up.
Responsive site review
A responsive website must serve all devices the same HTML code, which is adjusted (via the use of CSS) depending on the screen size.
To signal browsers that a page is responsive, a meta="viewport" tag should be in place within the <head> of each HTML page.
<meta name="viewport" content="width=device-width, initial-scale=1.0">
If the meta viewport tag is missing, font sizes may appear in an inconsistent manner, which may cause Google to treat the page as not mobile-friendly.
Separate mobile URLs review
If the mobile website uses separate URLs from desktop, make sure that:
- Each desktop page has a tag pointing to the corresponding mobile URL.
- Each mobile page has a rel="canonical" tag pointing to the corresponding desktop URL.
- When desktop URLs are requested on mobile devices, they’re redirected to the respective mobile URL.
- Redirects work across all mobile devices, including Android, iPhone, and Windows phones.
- There aren’t any irrelevant cross-links between the desktop and mobile pages. This means that internal links on found on a desktop page should only link to desktop pages and those found on a mobile page should only link to other mobile pages.
- The mobile URLs return a 200 server response.
Dynamic serving review
Dynamic serving websites serve different code to each device, but on the same URL.
On dynamic serving websites, review whether the vary HTTP header has been correctly set up. This is necessary because dynamic serving websites alter the HTML for mobile user agents and the vary HTTP header helps Googlebot discover the mobile content.
Regardless of the mobile site set-up (responsive, separate URLs or dynamic serving), review the pages using a mobile user-agent and make sure that:
- The viewport has been set correctly. Using a fixed width viewport across devices will cause mobile usability issues.
- The font size isn’t too small.
- Touch elements (i.e. buttons, links) aren’t too close.
- There aren’t any intrusive interstitials, such as Ads, mailing list sign-up forms, App Download pop-ups etc. To avoid any issues, you should use either use a small HTML or image banner.
- Mobile pages aren’t too slow to load (see next section).
Google’s mobile-friendly test tool can help diagnose most of the above issues:
AMP site review
If there is an AMP website and a desktop version of the site is available, make sure that:
- Each non-AMP page (i.e. desktop, mobile) has a tag pointing to the corresponding AMP URL.
- Each AMP page has a rel="canonical" tag pointing to the corresponding desktop page.
- Any AMP page that does not have a corresponding desktop URL has a self-referring canonical tag.
You should also make sure that the AMPs are valid. This can be tested using Google’s AMP Test Tool.
Mixed content errors
Mixed content occurs when a page that’s loaded over a secure HTTPS connection requests assets over insecure HTTP connections. Most browsers either block dangerous HTTP requests or just display warnings that hinder the user experience.
There are many ways to identify mixed content errors, including the use of crawler applications, Google’s Lighthouse, etc.
Image assets review
Google crawls images less frequently than HTML pages. If migrating a site’s images from one location to another (e.g. from your domain to a CDN), there are ways to aid Google in discovering the migrated images quicker. Building an image XML sitemap will help, but you also need to make sure that Googlebot can reach the site’s images when crawling the site. The tricky part with image indexing is that both the web page where an image appears on as well as the image file itself have to get indexed.
Site performance review
Analytics tracking review
Make sure that analytics tracking is properly set up. This review should ideally be carried out by specialist analytics consultants who will look beyond the implementation of the tracking code. Make sure that Goals and Events are properly set up, e-commerce tracking is implemented, enhanced e-commerce tracking is enabled, etc. There’s nothing more frustrating than having no analytics data after your new site is launched.
Testing the redirects before the new site goes live is critical and can save you a lot of trouble later. There are many ways to check the redirects on a staging/test server, but the bottom line is that you should not launch the new website without having tested the redirects.
Once the redirects become available on the staging/testing environment, crawl the entire list of redirects and check for the following issues:
- Redirect loops (a URL that infinitely redirects to itself)
- Redirects with a 4xx or 5xx server response.
- Redirect chains (a URL that redirects to another URL, which in turn redirects to another URL, etc).
- Canonical URLs that return a 4xx or 5xx server response.
- Canonical loops (page A has a canonical pointing to page B, which has a canonical pointing to page A).
- Canonical chains (a canonical that points to another page that has a canonical pointing to another page, etc).
- Protocol/host inconsistencies e.g. URLs are redirected to both HTTP and HTTPS URLs or www and non-www URLs.
- Leading/trailing whitespace characters. Use trim() in Excel to eliminate them.
- Invalid characters in URLs.
Pro tip: Make sure one of the old site’s URLs redirects to the correct URL on the new site. At this stage, because the new site doesn’t exist yet, you can only test whether the redirect destination URL is the intended one, but it’s definitely worth it. The fact that a URL redirects does not mean it redirects to the right page.
Phase 4: Launch day activities
When the site is down...
While the new site is replacing the old one, chances are that the live site is going to be temporarily down. The downtime should be kept to a minimum, but while this happens the web server should respond to any URL request with a 503 (service unavailable) server response. This will tell search engine crawlers that the site is temporarily down for maintenance so they come back to crawl the site later.
If the site is down for too long without serving a 503 server response and search engines crawl the website, organic search visibility will be negatively affected and recovery won’t be instant once the site is back up. In addition, while the website is temporarily down it should also serve an informative holding page notifying users that the website is temporarily down for maintenance.
Technical spot checks
As soon as the new site has gone live, take a quick look at:
- The robots.txt file to make sure search engines are not blocked from crawling
- Top pages redirects (e.g. do requests for the old site’s top pages redirect correctly?)
- Top pages canonical tags
- Top pages server responses
- Noindex/nofollow directives, in case they are unintentional
The spot checks need to be carried out across both the mobile and desktop sites, unless the site is fully responsive.
Search Console actions
The following activities should take place as soon as the new website has gone live:
- Test & upload the XML sitemap(s)
- Set the Preferred location of the domain (www or non-www)
- Set the International targeting (if applicable)
- Configure the URL parameters to tackle early any potential duplicate content issues.
- Upload the Disavow file (if applicable)
- Use the Change of Address tool (if switching domains)
Pro tip: Use the “Fetch as Google” feature for each different type of page (e.g. the homepage, a category, a subcategory, a product page) to make sure Googlebot can render the pages without any issues. Review any reported blocked resources and do not forget to use Fetch and Render for desktop and mobile, especially if the mobile website isn’t responsive.
Phase 5: Post-launch review
Once the new site has gone live, a new round of in-depth checks should be carried out. These are largely the same ones as those mentioned in the “Phase 3: Pre-launch Testing” section.
However, the main difference during this phase is that you now have access to a lot more data and tools. Don’t underestimate the amount of effort you’ll need to put in during this phase, because any issues you encounter now directly impacts the site’s performance in the SERPs. On the other hand, the sooner an issue gets identified, the quicker it will get resolved.
In addition to repeating the same testing tasks that were outlined in the Phase 3 section, in certain areas things can be tested more thoroughly, accurately, and in greater detail. You can now take full advantage of the Search Console features.
Check crawl stats and server logs
Keep an eye on the crawl stats available in the Search Console, to make sure Google is crawling the new site’s pages. In general, when Googlebot comes across new pages it tends to accelerate the average number of pages it crawls per day. But if you can’t spot a spike around the time of the launch date, something may be negatively affecting Googlebot’s ability to crawl the site.
Reviewing the server log files is by far the most effective way to spot any crawl issues or inefficiencies. Tools like Botify and On Crawl can be extremely useful because they combine crawls with server log data and can highlight pages search engines do not crawl, pages that are not linked to internally (orphan pages), low-value pages that are heavily internally linked, and a lot more.
Review crawl errors regularly
Keep an eye on the reported crawl errors, ideally daily during the first few weeks. Downloading these errors daily, crawling the reported URLs, and taking the necessary actions (i.e. implement additional 301 redirects, fix soft 404 errors) will aid a quicker recovery. It’s highly unlikely you will need to redirect every single 404 that is reported, but you should add redirects for the most important ones.
Pro tip: In Google Analytics you can easily find out which are the most commonly requested 404 URLs and fix these first!
Other useful Search Console features
Other Search Console features worth checking include the Blocked Resources, Structured Data errors, Mobile Usability errors, HTML Improvements, and International Targeting (to check for hreflang reported errors).
Pro tip: Keep a close eye on the URL parameters in case they’re causing duplicate content issues. If this is the case, consider taking some urgent remedial action.
Measuring site speed
Once the new site is live, measure site speed to make sure the site’s pages are loading fast enough on both desktop and mobile devices. With site speed being a ranking signal across devices and becauseslow pages lose users and customers, comparing the new site’s speed with the old site’s is extremely important. If the new site’s page loading times appear to be higher you should take some immediate action, otherwise your site’s traffic and conversions will almost certainly take a hit.
Evaluating speed using Google’s tools
Two tools that can help with this are Google’s Lighthouse and Pagespeed Insights.
ThePagespeed Insights Tool measures page performance on both mobile and desktop devices and shows real-world page speed data based on user data Google collects from Chrome. It also checks to see if a page has applied common performance best practices and provides an optimization score. The tool includes the following main categories:
- Speed score: Categorizes a page as Fast, Average, or Slow using two metrics: The First Contentful Paint (FCP) and DOM Content Loaded (DCL). A page is considered fast if both metrics are in the top one-third of their category.
- Optimization score: Categorizes a page as Good, Medium, or Low based on performance headroom.
- Page load distributions: Categorizes a page as Fast (fastest third), Average (middle third), or Slow (bottom third) by comparing against all FCP and DCL events in the Chrome User Experience Report.
- Page stats: Can indicate if the page might be faster if the developer modifies the appearance and functionality of the page.
- Optimization suggestions: A list of best practices that could be applied to a page.
Google’s Lighthouse is very handy for mobile performance, accessibility, and Progressive Web Apps audits. It provides various useful metrics that can be used to measure page performance on mobile devices, such as:
- First Meaningful Paint that measures when the primary content of a page is visible.
- Time to Interactive is the point at which the page is ready for a user to interact with.
- Speed Index measures shows how quickly a page are visibly populated
Both tools provide recommendations to help improve any reported site performance issues.
You can also use this Google tool to get a rough estimate on the percentage of users you may be losing from your mobile site’s pages due to slow page loading times.
The same tool also provides an industry comparison so you get an idea of how far you are from the top performing sites in your industry.
Measuring speed from real users
Once the site has gone live, you can start evaluating site speed based on the users visiting your site. If you have Google Analytics, you can easily compare the new site’s average load time with the previous one.
In addition, if you have access to a Real User Monitoring tool such as Pingdom, you can evaluate site speed based on the users visiting your website. The below map illustrates how different visitors experience very different loading times depending on their geographic location. In the below example, the page loading times appear to be satisfactory to visitors from the UK, US, and Germany, but to users residing in other countries they are much higher.
Phase 6: Measuring site migration performance
When to measure
Has the site migration been successful? This is the million-dollar question everyone involved would like to know the answer to as soon as the new site goes live. In reality, the longer you wait the clearer the answer becomes, as visibility during the first few weeks or even months can be very volatile depending on the size and authority of your site. For smaller sites, a 4–6 week period should be sufficient before comparing the new site’s visibility with the old site’s. For large websites you may have to wait for at least 2–3 months before measuring.
In addition, if the new site is significantly different from the previous one, users will need some time to get used to the new look and feel and acclimatize themselves with the new taxonomy, user journeys, etc. Such changes initially have a significant negative impact on the site’s conversion rate, which should improve after a few weeks as returning visitors are getting more and more used to the new site. In any case, making data-driven conclusions about the new site’s UX can be risky.
But these are just general rules of thumb and need to be taken into consideration along with other factors. For instance, if a few days or weeks after the new site launch significant additional changes were made (e.g. to address a technical issue), the migration’s evaluation should be pushed further back.
How to measure
Performance measurement is very important and even though business stakeholders would only be interested to hear about the revenue and traffic impact, there are a whole lot of other metrics you should pay attention to. For example, there can be several reasons for revenue going down following a site migration, including seasonal trends, lower brand interest, UX issues that have significantly lowered the site’s conversion rate, poor mobile performance, poor page loading times, etc. So, in addition to the organic traffic and revenue figures, also pay attention to the following:
- Desktop & mobile visibility (from SearchMetrics, SEMrush, Sistrix)
- Desktop and mobile rankings (from any reliable rank tracking tool)
- User engagement (bounce rate, average time on page)
- Sessions per page type (i.e. are the category pages driving as many sessions as before?)
- Conversion rate per page type (i.e. are the product pages converting the same way as before?)
- Conversion rate by device (i.e. has the desktop/mobile conversion rate increased/decreased since launching the new site?)
Reviewing the below could also be very handy, especially from a technical troubleshooting perspective:
- Number of indexed pages (Search Console)
- Submitted vs indexed pages in XML sitemaps (Search Console)
- Pages receiving at least one visit (analytics)
- Site speed (PageSpeed Insights, Lighthouse, Google Analytics)
It’s only after you’ve looked into all the above areas that you could safely conclude whether your migration has been successful or not.
Good luck and if you need any consultation or assistance with your site migration, please get in touch!
Appendix: Useful tools
- Screaming Frog: The SEO Swiss army knife, ideal for crawling small- and medium-sized websites.
- Sitebulb: Very intuitive crawler application with a neat user interface, nicely organized reports, and many useful data visualizations.
- Deep Crawl: Cloud-based crawler with the ability to crawl staging sites and make crawl comparisons. Allows for comparisons between different crawls and copes well with large websites.
- Botify: Another powerful cloud-based crawler supported by exceptional server log file analysis capabilities that can be very insightful in terms of understanding how search engines crawl the site.
- On-Crawl: Crawler and server log analyzer for enterprise SEO audits with many handy features to identify crawl budget, content quality, and performance issues.
Handy Chrome add-ons
- User agent switcher: Switch between different user agents including Googlebot, mobile, and other agents.
- Ayima Redirect Path: A great header and redirect checker.
- SEO Meta in 1 click: An on-page meta attributes, headers, and links inspector.
- Scraper: An easy way to scrape website data into a spreadsheet.
Site monitoring tools
- Uptime Robot: Free website uptime monitoring.
- Robotto: Free robots.txt monitoring tool.
- Pingdom tools: Monitors site uptime and page speed from real users (RUM service)
- SEO Radar: Monitors all critical SEO elements and fires alerts when these change.
Site performance tools
- PageSpeed Insights: Measures page performance for mobile and desktop devices. It checks to see if a page has applied common performance best practices and provides a score, which ranges from 0 to 100 points.
- Lighthouse: Handy Chrome extension for performance, accessibility, Progressive Web Apps audits. Can also be run from the command line, or as a Node module.
- Webpagetest.org: Very detailed page tests from various locations, connections, and devices, including detailed waterfall charts.
Structured data testing tools
- Google’s structured data testing tool & Google’s structured data testing tool Chrome extension
- Bing’s markup validator
- Yandex structured data testing tool
- Google’s rich results testing tool
Mobile testing tools
Backlink data sources
via Blogger The Website Migration Guide: SEO Strategy & Process
Posted by andrewchoco
Picture this scenario. You’re a new employee that has just been brought in to a struggling marketing department (or an agency brought on to help recover lost numbers). You get access to Google Analytics, and see something like this:
This can generate two types of emotional response: excitement or fear (or both). The steady decline in organic traffic excites you because you have so many tactics and ideas that you think can save this company from spiraling downward out of control. But there’s also the fear that these tactics wont be enough to correct the course.
Regardless of whether these new tactics would work or not, it’s important to understand the history of the account and determine not only what is happening, but why.
The company may have an idea of why the traffic is declining (i.e. competitors have come in and made ranking for keywords much harder, or they did a website redesign and have never recovered).
Essentially, this boils down to two things: 1) either you’re struggling with organic optimization, or 2) something was off with your tracking in Google Analytics, has since been corrected, and hasn’t been caught.
In this article, I’ll go over an audit I did for one of my clients to help determine if the decline we saw in organic traffic was due to actual poor SEO performance, an influx in competitors, tracking issues, or a combination of these things.
I’ll be breaking it down into five different areas of investigation:
Keyword ranking differences from 2015–2017
- Did the keywords we were ranking for in 2015 change drastically in 2017? Did we lose rankings and therefore lose organic traffic?
Top organic landing pages from 2015–2017
- Are the top ranking organic landing pages the same currently as they were in 2015? Are we missing any pages due to a website redesign?
- Did something happen to the site speed / bounce rate / page views etc.
SEMrush/Moz keyword, traffic, and domain authority data
- Looking at the SEMrush organic traffic cost metric as well as Moz metrics like Domain Authority and competitors.
- Did our conversion numbers stay consistent throughout the traffic drop? Or did the conversions drop in correlation with the traffic drop?
By the end of this post, my goal is that you’ll be able to replicate this audit to determine exactly what’s causing your organic traffic decline and how to get back on the right track.
Let’s dive in!
Keyword ranking differences from 2015–2017
This was my initial starting point for my audit. I started with this specifically because the most obvious answer, for a decline in traffic is a decline in keyword rankings.
I wanted to look at what keywords we were ranking for in 2015 to see if we significantly dropped in the rankings or if the search volume had dropped. If the company you’re auditing has had a long-running Moz account, start by looking at the keyword rankings from the initial start of the decline, compared to current keyword rankings.
I exported keyword data from both SEMrush and Moz, and looked specifically at the ranking changes of core keywords.
March was a particularly strong month across the board, so I narrowed it down and exported the keyword rankings in:
- March 2015
- March 2016
- March 2017
- December 2017 (so I could get the most current rankings)
Once the keywords were exported, I went in and highlighted in red the keywords that we were ranking for in 2015 (and driving traffic from) that we were no longer ranking for in 2017. I also highlighted in yellow the keywords we were ranking for in 2015 that were still ranking in 2017.
One thing that immediately stood out: in 2015, this company was ranking for five keywords, including the word “free.” They have since changed their offering, so it made sense that in 2017, we weren’t ranking for those keywords.
After removing the free queries, we pulled the “core” keywords to look at their differences.
March 2015 core keywords:
- Appointment scheduling software: position 9
- Online appointment scheduling: position 11
- Online appointment scheduling: position 9
- Online scheduling software: position 9
- Online scheduler: position 9
- Online scheduling: position 13
December 2017 core keywords:
- Appointment scheduler: position 11
- Appointment scheduling software: position 10
- Online schedule: position 6
- Online appointment scheduler: position 11
- Online appointment scheduling: position 12
- Online scheduling software: position 12
- Online scheduling tool: position 10
- Online scheduling: position 15
- SaaS appointment scheduling: position 2
There were no particular red flags here. While some of the keywords have moved down 1–2 spots, we had new ones jump up. These small changes in movement didn’t explain the nearly 30–40% drop in organic traffic. I checked this off my list and moved on to organic landing pages.
Top organic landing page changes
Since the dive into keyword rankings didn’t provide the answer for the decline in traffic, the next thing I looked at were the organic landing pages. I knew this client had switched over CMS systems in early 2017, and had done a few small redesign projects the past three years.
After exporting our organic landing pages for 2015, 2016, and 2017, we compared the top ten (by organic sessions) and got the following results.
2015 top organic landing pages:
2016 top organic landing pages:
2017 top organic landing pages:
Because of their redesign, you can see that the subfolders changed between 2015/2016 to 2017. What really got my attention, however, is the /get-started page. In 2015/2016, the Get Started page accounted for nearly 16% of all organic traffic. In 2017, the Get Started page was nowhere to be found.
If you run into this problem and notice there are pages missing from your current top organic pages, a great way to uncover why is to use the Wayback Machine. It's a great tool that allows you to see what a web page looked like in the past.
When we looked at the /get-started URL in the Wayback Machine, we noticed something pretty interesting:
In 2015, their /get-started page also acted as their login page. When people were searching on Google for “[Company Name] login,” this page was ranking, bringing in a significant amount of organic traffic.
Their current setup sends logins to a subdomain that doesn’t have a GA code (as it’s strictly used as a portal to the actual application).
That helped explain some of the organic traffic loss, but knowing that this client had gone through a few website redesigns, I wanted to make sure that all redirects were done properly. Regardless of whether or not your traffic has changed, if you’ve recently done a website redesign where you’re changing URLs, it’s smart to look at your top organic landing pages from before the redesign and double check to make sure they’re redirecting to the correct pages.
While this helped explain some of the traffic loss, the next thing we looked at was the on-page metrics to see if we could spot any obvious tracking issues.
Comparing on-page engagement metrics
Looking at the keyword rankings and organic landing pages provided a little bit of insight into the organic traffic loss, but it was nothing definitive. Because of this, I moved to the on-page metrics for further clarity. As a disclaimer, when I talk about on-page metrics, I’m talking about bounce rate, page views, average page views per session, and time on site.
Looking at the same top organic pages, I compared the on-page engagement metrics.
2015 on-page metrics:
2016 on-page metrics:
2017 on-page metrics:
While the overall engagement metrics changed slightly, the biggest and most interesting discrepancy I saw was in the bounce rates for the home page and Get Started page.
According to a number of different studies (like this one, this one, or even this one), the average bounce rate for a B2B site is around 40–60%. Seeing the home page with a bounce rate under 20% was definitely a red flag.
This led me to look into some other metrics as well. I compared key metrics between 2015 and 2017, and was utterly confused by the findings:
Looking at the organic sessions (overall), we saw a decrease of around 80,000 sessions, or 27.93%.
Looking at the organic users (overall) we saw a similar number, with a decrease of around 38,000 users, or 25%.
When we looked at page views, however, we saw a much more drastic drop:
For the entire site, we saw a 50% decrease in pageviews, or a decrease of nearly 400,000 page views.
This didn’t make much sense, because even if we had those extra 38,000 users, and each user averaged roughly 2.49 pages per session (looking above), that would only account for, at most, 100,000 more page views. This left 300,000 page views unaccounted for.
This led me to believe that there was definitely some sort of tracking issue. The high number of page views and low bounce rate made me suspect that some users were being double counted.
However, to confirm these assumptions, I took a look at some external data sources.
Using SEMrush and Moz data to exclude user error
If you have a feeling that your tracking was messed up in previous years, a good way to confirm or deny this hypothesis is to check external sources like Moz and SEMrush.
Unfortunately, this particular client was fairly new, so as a result, their Moz campaign data wasn’t around during the high organic traffic times in 2015. However, if it was, a good place to start would be looking at the search visibility metric (as long as the primary keywords have stayed the same). If this metric has changed drastically over the years, it’s a good indicator that your organic rankings have slipped quite a bit.
Another good thing to look at is domain authority and core page authority. If your site has had a few redesigns, moved URLs, or anything like that, it’s important to make sure that the domain authority has carried over. It’s also important to look at the page authorities of your core pages. If these are much lower than when they were before the organic traffic slide, there’s a good chance your redirects weren’t done properly, and the page authority isn’t being carried over through those new domains.
If, like me, you don’t have Moz data that dates back far enough, a good thing to check is the organic traffic cost in SEMrush.
Organic traffic cost can change because of a few reasons:
- Your site is ranking for more valuable keywords, making the organic traffic cost rise.
- More competitors have entered the space, making the keywords you were ranking for more expensive to bid on.
Usually it’s a combination of both of these.
If our organic traffic really was steadily decreasing for the past 2 years, we’d likely see a similar trendline looking at our organic traffic cost. However, that’s not what we saw.
In March of 2015, the organic traffic cost of my client’s site was $14,300.
In March of 2016, the organic traffic cost was $22,200
In December of 2017, the organic traffic cost spiked all the way up to $69,200. According to SEMrush, we also saw increases in keywords and traffic.
Looking at all of this external data re-affirmed the assumption that something must have been off with our tracking.
However, as a final check, I went back to internal metrics to see if the conversion data had decreased at a similar rate as the organic traffic.
Analyzing and comparing conversion metrics
This seemed like a natural final step into uncovering the mystery in this traffic drop. After all, it’s not organic traffic that's going to profit your business (although it’s a key component). The big revenue driver is goal completions and form fills.
This was a fairly simple procedure. I went into Google Analytics to compare goal completion numbers and goal completion conversion rates over the past three years.
If your company is like my client’s, there’s a good chance you’re taking advantage of the maximum 20 goal completions that can be simultaneously tracked in Analytics. However, to make things easier and more consistent (since goal completions can change), I looked at only buyer intent conversions. In this case it was Enterprise, Business, and Personal edition form fills, as well as Contact Us form fills.
If you’re doing this on your own site, I would recommend doing the same thing. Gated content goal completions usually have a natural shelf life, and this natural slowdown in goal completions can skew the data. I’d look at the most important conversion on your site (usually a contact us or a demo form) and go strictly off those numbers.
For my client, you can see those goal completion numbers below:
Goal completion name
Goal completion name
This was pretty interesting. Although there was clearly fluctuation in the goal completions and conversion rates, there were no differences that made sense with our nearly 40,000 user drop from 2015 to 2016 to 2017.
All of these findings further confirmed that we were chasing an inaccurate goal. In fact, we spent the first three months working together to try and get back a 40% loss that, quite frankly, was never even there in the first place.
Tying everything together and final thoughts
For this particular case, we had to go down all five of these roads in order to reach the conclusion that we did: Our tracking was off in the past.
However, this may not be the case for your company or your clients. You may start by looking at keyword rankings, and realize that you’re no longer ranking on the first page for ten of your core keywords. If that’s the case, you quickly discovered your issue, and your game plan should be investing in your core pages to help get them ranking again for these core keywords.
If your goal completions are way down (by a similar percentage as your traffic), that’s also a good clue that your declining traffic numbers are correct.
If you’ve looked at all of these metrics and still can’t seem to figure out the reasoning for the decrease and you’re blindly trying tactics and struggling to crawl your way back up, this is a great checklist to go through to confirm the ominous question of tracking issue or optimization issue.
If you’re having a similar issue as me, I’m hoping this post helps you get to the root of the problem quickly, and gets you one step closer to create realistic organic traffic goals for the future!
via Blogger Declining Organic Traffic? How to Tell if it’s a Tracking or Optimization Issue
Posted by randfish
Pouring money into a paid ad campaign that's destined to fail isn't a sound growth strategy. Time and again, companies breaking into online ads don't see success due to the same issue: they aren't known to their audiences. There's no trust, no recognition, and so the cost per click remains high and rising.
In this edition of Whiteboard Friday, Rand identifies the cycle many brands get trapped in and outlines a solution to make those paid ad campaigns worth the dollars you put behind them.
Howdy, Moz fans, and welcome to another edition of Whiteboard Friday. This week we're chatting about the number one reason so many paid ad campaigns, especially from new companies and companies with new products or new ventures that they're going into, new markets and audiences they're addressing, fail. They just fall apart. I see this scenario play out so many times, especially in the startup and entrepreneurial world but, to be honest, across the marketing landscape.
Here's how it usually goes. You've got your CEO or your CMO or your business owner and they're like, "Hey, we have this great new product. Let's spread the word." So they talk to a marketer. It could be a contractor. It could be an agency. It could be someone in-house.
The marketer is like, "Okay, yeah, I'll buy some ads online, help us get the word out there and get us some traffic and conversions."
Then a few months later, you basically get this. "How's that paid ad campaign going?" "Well, not so good. I have bad news."
Almost always, this is the result of a cycle that looks like this. You have a new company's campaign. The campaign is to sell something or get exposure for something, to try and drive visits back to a web page or a website, several pages on the site and then get conversions out of it. So you buy Facebook ads, Instagram ads, maybe LinkedIn and Twitter. You probably use the Google Display Network. You're probably using AdWords. All of these sources are trying to drive traffic to your web page and then get a conversion that turns into money.
Now, what happens is that these get a high cost per click. They start out with a high cost per click because it's a new campaign. So none of these platforms have experience with your campaign or your company. So you're naturally going to get a higher-than-normal cost per click until you prove to them that you get high engagement, at which point they bring the cost per click down. But instead of proving to them you get high engagement, you end up getting low engagement, low click-through rate, low conversion rate. People don't make it here. They don't make it there. Why is that?
Why does this happen?
Well, before we address that, let's talk about what happens here. When these are low, when you have a low engagement rate on the platform itself, when no one engages with your Facebook ads, no one engages with your Instagram ads, when no one clicks on your AdWords ad, when no one clicks on your display ads, the cost to show to more people goes up, and, as a result, these campaigns are much harder to make profitable and they're shown to far fewer people.
So your exposure to the audience you want to reach is smaller and the cost to reach each next person and to drive each next action goes up. This, fundamentally, is because...
- The audience that you're trying to reach hasn't heard of you before. They don't know who you are.
- They don't know, trust, or like you or your company product, they don't click. They don't click. They don't buy. They don't share. They don't like.
They don't do all the engagement things that would drive this high cost per click down, and, because of that, your campaigns suffer and struggle.
I see so many marketers who think like this, who say yes to new company campaigns that start with an advertising-first approach. I want to be clear, there are some exceptions to the rule. I have seen some brand new companies that fit a certain mold do very well with Instagram advertising for certain types of products that appeal to that audience and don't need a previously existing brand association. I've seen some players in the Google AdWords market do okay with this, some local businesses, some folks in areas where people don't expect to have knowledge and awareness of a brand already in the space where they're trying to discover them.
So it's not the case always that this fails, but very often, often enough that I'm calling this the number one reason I see paid ads fail.
There's only one solution and it's not pretty. The solution is...
You have to get known to your audience before you pour money into advertising.
Meaning you need to invest in organic channels — content or SEO or press and PR or sponsorships or events, what have you, anything that can get your brand name and the names of your product out there.
Brand advertising, in fact, can work for this. So television brand advertising, folks have noticed that TV brand advertising often drives the cost per click down and drives engagement and click-through rates up, because people have heard of you and they know who you are. Magazine and offline advertising works like this. Sometimes even display advertising can work this way.
The second option is to...
Advertise primarily or exclusively to an audience that already has experience with you.
The way you can do this is through systems like Google's retargeting and remarketing platforms. You can do the same thing with Facebook, through custom audiences of email addresses that you upload, same thing with Instagram, same thing with Twitter. You can target people who specifically only follow the accounts that you already own and control. Through these, you can get better engagement, better click-through rate, better conversion rate and drive down that cost per click and reach a broader audience.
But if you don't do these things first, a lot of times these types of investments fall flat on their face, and a lot of marketers, to be honest, and agencies and consultants lose their jobs as a result. I don't want that to happen to you. So invest in these first or find the niches where advertising can work for a first-time product. You're going to be a lot happier.
All right, everyone. Look forward to your comments. We'll see you again next week for another edition of Whiteboard Friday. Take care.
via Blogger The #1 Reason Paid Ads (On Search, Social, and Display) Fail - Whiteboard Friday
Posted by Daniel_Marks
Almost every consultant or in-house SEO will be asked at some point to investigate an organic traffic drop. I’ve investigated quite a few, so I thought I’d share some steps I’ve found helpful when doing so.
Is it just normal noise?
Before you sound the alarm and get lost down a rabbit hole, you should make sure that the drop you’re seeing is actually real. This involves answering two questions:
A.) Do you trust the data?
This might seem trivial, but at least a quarter of the traffic drops I’ve seen were simply due to data problems.
The best way to check on this is to sense-check other metrics that might be impacted by data problems. Does anything else look funky? If you have a data engineering team, are they aware of any data issues? Are you flat-out missing data for certain days or page types or devices, etc.? Thankfully, data problems will usually make themselves pretty obvious once you start turning over a few rocks.
B.) Is this just normal variance?
Metrics go up and down all the time for no discernible reason. One way to quantify this is to use your historical standard deviation for SEO traffic.
For example, you could plot your weekly SEO traffic for the past 12 months and calculate the standard deviation (using the STDEV function on Google Sheets or Excel makes this very easy) to figure out if a drop in weekly traffic is abnormal. You’d expect about 16% of weeks to be one standard deviation below your weekly average just by sheer luck. You could therefore set a one-standard-deviation threshold before investigating traffic drops, for example (but you should adjust this threshold to whatever is appropriate for your business). You can also look at the standard deviation for your year-over-year or week-over-week SEO traffic if that’s where you’re seeing the drop (i.e. plot your % change in YoY SEO traffic by week for the past 12 months and calculate the standard deviation).
Let’s assume you’ve decided this is indeed a real traffic drop. Now what? I’d recommend trying to answer the eleven questions below, at least one of them will usually identify the culprit.
Questions to ask yourself when facing an organic traffic drop
1. Was there a recent Google algorithm update?
If there was an algorithm update, do you have any reason to suspect you’d be impacted? It can sometimes be difficult to understand the exact nature of a Google update, but it’s worth tracking down any information you can to make sure your site isn’t at risk of being hit.
2. Is the drop specific to any segment?
One of the more useful practices whenever you’re looking at aggregated data (such as a site’s overall search traffic) is to segment the data until you find something interesting. In this case, we’d be looking for a segment that has dropped in traffic much more than any other. This is often the first step in tracking down the root cause of the issue. The two segments I’ve found most useful in diagnosing SEO traffic drops specifically:
- Device type (mobile vs. desktop vs. tablet)
- Page type (product pages vs. category pages vs. blog posts vs. homepage etc.)
But there will likely be plenty of other segments that might make sense to look at for your business (for example, product category).
3. Are you being penalized?
This is unlikely, but it’s also usually pretty quick to disprove. Look at Search Console for any messages related to penalties and search for your brand name on Google. If you’re not showing up, then you might be penalized.
4. Did the drop coincide with a major site change?
5. Did you lose ranking share to a competitor?
There are a bunch of tools that can tell you if you’ve lost rankings to a competitor:
If you've lost rankings, it’s worth investigating the specific keywords that you’ve lost and figuring out if there’s a trend. Did your competitors launch a new page type? Did they add content to their pages? Do they have more internal links pointing to these pages than you do?
It could also just be a new competitor that’s entered the scene.
6. Did it coincide with a rise in direct or dark traffic?
If so, make sure you haven’t changed how you’re classifying this traffic on your end. Otherwise, you might simply be re-classifying organic traffic as direct or dark traffic.
7. Has there been a change to the search engine results pages you care about?
You can either use Moz’s SERP features report, or manually look at the SERPs you care about to figure out if their design has materially changed. It’s possible that Google is now answering many of your relevant queries directly in search results, put an image carousel on them, added a local pack, etc. — all of which would likely decrease your organic search traffic.
8. Is the drop specific to branded or unbranded traffic?
If you have historical Search Console data, you can look at number of branded clicks vs. unbranded clicks over time. You could also look at this data through AdWords if you spend on paid search. Another simple proxy to branded traffic is homepage traffic (for most sites, the majority of homepage traffic will be branded). If the drop is specific to branded search then it’s probably a brand problem, not an SEO problem.
9. Did a bunch of pages drop out of the index?
Search Console’s Index Status Report will make it clear if you suddenly have way fewer URLs being indexed. If this is the case, you might be accidentally disallowing or noindexing URLs (through robots.txt, meta tags on the page, or HTTP headers).
10. Did your number of referring domains and/or links drop?
It’s possible that a large number of your backlinks have been removed or are no longer accessible for whatever reason.
Ahrefs can be a quick way to determine if you’ve lost backlinks and also offers very handy reports for your lost backlinks or referring domains that will allow you to identify why you might have lost these links.
11. Is SEM cannibalizing SEO traffic?
It’s possible that your paid search team has recently ramped up their spend and that this is eating into your SEO traffic. You should be able to check on this pretty quickly by plotting your SEM vs. SEO traffic. If it’s not obvious after doing this whether it’s a factor, then it can be worth pausing your SEM campaigns for specific landing pages and seeing if SEO traffic rebounds for those pages.
To be clear, some level of cannibalization between SEM and SEO is inevitable, but it’s still worth understanding how much of your traffic is being cannibalized and whether the incremental clicks your SEM campaigns are driving outweigh the loss in SEO traffic (in my experience they usually do outweigh the loss in SEO traffic, but still worth checking!).
That’s all I’ve got — hopefully at least one of these questions will lead you to the root cause of an organic search traffic drop. Are there any other questions that you’ve found particularly helpful for diagnosing traffic drops? Let me know in the comments.
via Blogger How to Diagnose SEO Traffic Drops: 11 Questions to Answer
business credit cards
Find the best card offers for your business and apply today. Explore & choose small business credit cards based on their benefits & rewards.