Search Engine Optimisation can refer to so many different types of actions. It can be about building links, growing authority, getting citations, or dealing with a thousand other important ranking factors. It can also be about fixing problems.
For me, this is where Screaming Frog comes in as the top tool in any SEO’s arsenal. This oddly named product can be used to scan your website and then hunt down every little SEO issue that your site has – including broken links, duplicate content, missing metadata, problems with canonicalisation, hreflang, and so much more.
If you’ve ever worried that your site is not quite right, then Screaming Frog is for you. While I especially recommend it for big and/or old sites (as problems are more likely to happen and go unnoticed), Screaming Frog is also great for small sites – as it is free to use for the first 500 pages you scan.
So because it’s free for small sites (or just to get started with), and because it is incredibly powerful and comprehensive I highly recommend Screaming Frog.
[Please note: I am not being paid to endorse this product, I just like it]
Fixing Link Rot
Link Rot is a problem the whole internet faces. Over time websites change. They re-organise pages or sometimes they get switched off altogether. This means that links leading to those websites stop working. Many websites redirect pages to their new home when they move, but if a link is really old it might have been around through several re-organisations (and therefore several redirects) meaning that a link click can take a very long time to load as it pings around.
Whether a link is broken or just redirecting too many times this reduces confidence in the website the link is on. If that link is meant to lead to another page within the same website, that loss of confidence grows exponentially. I mean, who trusts a broken site?
Screaming Frog can help you with this. When you scan your site, Screaming Frog will tell you the response code of every link on your site (whether the link is internal or external), meaning you can quickly and easily fix all those little problems.
Top Tip
Find out which sites link to yours the most (in Google Search Console or Bing Webmaster Tools), then scan their site and find broken or redirecting links to your site. Then use this as a nice outreach opportunity.
While finding broken links is not a capability unique to Screaming Frog, it is a very important capability nonetheless. It should also be noted that broken links are not an SEO ranking factor (as far as I am aware) however, links are how Google et al crawl the internet. This means fixing them not only makes your site more respectable it also makes the internet work better as a whole.
Incredibly Powerful and Comprehensive Reporting
Usually, when I recommend a product I like to call out features that I especially like. That’s not really the case with Screaming Frog as there is just so much to it. Which features will be useful to you depends on how your site is set up and what its problems are.
I’ve used it on small sites to comprehensively make sure the metadata was all unique and properly formatted, and I’ve used it on gigantic sites to hunt down the hundreds of small problems that have accumulated over the years. For sites that are in multiple languages, the hreflang features are invaluable. For a while when it seemed like AMP was going to be the future, I was obsessed with the AMP analysis that it runs.
According to Screaming Frog, there are currently 48 categories it crawls and reports on. There are also 6 built-in integrations helping you to take the reporting further (Google Analytics, Google Search Console, PageSpeed Insights, Majestic, Ahrefs, & Moz if you’re interested). There is just so much built into this tiny program that you won’t know what is useful to you until you scan your website.
More than that, this program is still being updated and improved. And because it’s so old there are countless articles out there telling you how to get the most out of it by using the reports in different ways. It is immense, both in size and usefulness.
This is the short list of features. THE SHORT LIST!
Reasonable Pricing (Free for small sites)
As I’ve mentioned, Screaming Frog is free for the first 500 pages it scans. I don’t know how it chooses those pages exactly, but even if you have a site with over 500 pages I highly recommend downloading it and scanning your site. Once you do this, you’ll immediately see why this product can be useful to you. It’s also not unlikely that even with a random sampling of 500 pages you’ll find enough things to fix to make the exercise entirely worthwhile.
If you have a site under 500 pages, then download this and fix everything immediately. Even if you just have ten pages, it’s worth doing (especially as it’s free!).
I don’t know anyone who has used the free version and not wanted to graduate to the paid version. It’s currently £149 a year, and honestly, I would be shocked if you didn’t get £149 worth of value out of it. If you don’t think you are though, you can always drop down to the free version again and keep scanning those 500 pages and see what pops up!
Downsides
The main downside is psychological – it’s a program that you actually need to download to use, and it’s called Screaming Frog. In this day and age of web apps, downloading something onto your computer with such a suspiciously strange name can be a mental barrier to overcome – especially if you have to get your IT department to allow it. Honestly, though it’s not dodgy. Google it and you’ll find that top SEOs have been talking about Screaming Frog for years.
The other downside for me is just that there isn’t an “all problems” tab. I’d like to be able to click on any URL, and then just have all the issues listed for me. Right now they are split into tabs, which is helpful in one way, but not my preference.
In Conclusion
If you are looking for your very first professional SEO tool, then start with Screaming Frog. You’ll come for the free version, but I’m certain you’ll stay for the paid version.
If you’re a site owner who always has a gnawing feeling in your gut that things on your site are broken (or at least not quite right), then Screaming Frog can solve that too. It will help you find those problems and give you peace of mind.
And oh with all this fawning I didn’t even mention the most obvious thing of all – Screaming Frog will make your SEO easier and more effective.