What is Screaming Frog?
Screaming Frog is an endlessly useful SEO tool. It is a big timesaver and makes technical SEO life so much easier. It allows you to quickly identify issues your website might have.
Screaming Frog is an SEO spider. It works using a unique blend of data, technology, and creativity to help companies grow their business online. It is helping the community with Spider conundrums. It is a bot that crawls all over your site and looks for problems.
It easily scans a website and exports the Meta titles and descriptions for each page and helps you to know “Which of your pages are missing meta descriptions?” It also helps you to know other information such as “How many broken links are on your website right now?”, “Do any of your URLs have a missing H1 tag?” and “How many broken images?”
We use it for technical SEO audits, checks, and QA, it easily helps identify any errors or missing information that needs to be updated for SEO purposes especially. The features of screaming frog are as follow:
Screaming Frog Features
You could manually audit your website by visiting each page, but it would take forever to check the technical aspects of a site launch manually.
It improves efficiency in terms of completing SEO audits as you could let an SEO tool like Screaming Frog handle that tedious process for you.
It is well suited for anyone who deals with SEO at an organization as Screaming Frog is incredibly easy to use!
Screaming Frog generates a report so you can find what is wrong and take corrective action. It is also well suited for large websites such as e-commerce sites.
The SEO tool does have very advanced and technical features also. However, it is not a cloud-based solution. By using Screaming Frog you can see how discoverable the pages are on your website.
You don’t have to use the advanced paid version of Screaming Frog and still make a lot out of it. Screaming Frog is a simple software that you have to download and install on your PC. It is especially useful on web platforms that don’t have their sitemap generator.
Screaming Frog runs on Windows, macOS, and Ubuntu and it is the ease of use for beginner and advanced SEOs. Screaming Frog is a necessary SEO tool if you aren’t buying into one of the larger SaaS systems like Moz.
This tool comes with new functionality if you opt for the paid version. But you might be disappointed that you have to install something just to check for errors on your website if you have a habit of using cloud-based tools.
Screaming Frog pricing
The Screaming Frog SEO Spider is free to start.
Paid Version will cost £149.00 Per Year if you want additional features.
Screaming Frog is a British company so you’ll have to pay in pounds. Licenses are per user. 2 users will require 2 individual licenses.
Be sure to check the exchange rate before you purchase.
To understand better whether you need the paid version or free one, we have listed the benefits of both free and paid versions as follow:
Benefits of free version
Additional benefits with the paid version
For us, even the free version of Screaming Frog is quite useful, as Screaming Frog helps with all our day to day SEO related tasks.
Currently, Screaming Frog has restricted the number of pages you can crawl for free that is 500 links if you’re using the free version.
With the paid version it also allows integration with Google Analytics and Search Console (WMT) along with unlimited crawling. It is such a powerful tool and with the level of deep and detailed information that it provides, that it is almost impossible to go without it.
However, you won’t be able to integrate with popular tools like Google Analytics, Google Search Console, and Link Metrics with the free version. Screaming Frog is awesome for its ability to crawl huge sites very quickly.
Most of the features discussed here are only available with the licensed version of the software, with Screaming Frog and Google Sheets you can build a fairly comprehensive overview of Meta, alt tags, image size, titles, H tags, etc. in a very short amount of time.
You can also start by taking a look at the HTTP status codes under the ‘Response Codes’ tab to quickly see which URLs on your site are returning anything other than a good (200) response. The most important HTTP status codes are:
The typical “bad” response is the 404 code (not found), check all the redirects (301 and 302) and verify that you’ve used the correct version.
However, the report will also show you redirect (3XX) and server error (5XX) response codes, remember that 301 are permanent redirects, so they pass authority to the new page.
There are several modifiers for the crawler which makes it so powerful. Screaming Frog is first and foremost a crawler. A solid crawler with a lot of options that can connect to quite a few external services to pull in additional data and assist with a complete picture of a website and its technical issues.
Once you have installed it, you can use it to visit pages on your website, it allows one to see the core SEO elements of a site very quickly.
The paid version is a bit pricey but for what it does it’s worth it, the unlimited access from a paid plan is essential for any mid-size or enterprise sites.
If you have a website with thousands of pages, you need it for getting a feel for how well a site has been maintained as far as core on-site factors and being able to pull that data to build spreadsheets.
Screaming Frog supports pretty much any kind of proxy you want to use. You can use it for pre-launch and post-launch crawls as well.
You might need to increase the amount of memory that the spider uses. It allows us to quickly see if there are any underlying issues with their web pages, it is very fast and highly customizable. You can also switch to database mode.
Screaming Frog is great when you know what you are looking for. We use the filtering extraction feature very often.
The tool will store your crawl data on disk instead of in memory. But if you don’t understand the technical result you get as the outcome of it such as Meta information, status codes, redirects, canonical, and other technical aspects of a website, then it is not the tool for you.
It is a powerful tool without being too flashy. You don’t have to crawl your entire site, and you can specify a subdomain or folder to crawl. It won’t have a popup that guides you. It just gives you the data in an organized form.
The interface is simple, yet provides a powerful web auditing. If you want the tool to crawl only pages under blog you can configure it to do that, so that you can do the research you need to optimize a website.
Results can be easily exported into XLS and CSV. You can also give Screaming Frog a list of URLs to crawl. If you are newer to SEO you either need to learn what all of these things are or use another tool that will put the results into layman’s terms.
Analysis can also be made via their interface, which is handy when you don’t want to export or format. If certain pages are disallowed, don’t expect the tool to crawl them.
This way we immediately see how a website is perceived in the eyes of Google, what pages are properly indexed or not, title and Meta tags, and even Google Analytics data. The right tools designed to find problems can make the SEO audit of a website so much easier.
If you’ve found that Screaming Frog crashes when crawling a large site, this may be due to high memory issues with your device.
Download and install the Screaming Frog SEO tool. It is available for Windows, Mac, and Ubuntu and it is completely free.
You might have broken links on your site. Screaming Frog will find them. The internal tab gives you all the details about the pages on your website. You will need to go back to that spider configuration menu, to put a throttle on it and keep it from crashing.
If you want to crawl additional subdomains you need to check the Crawl All Subdomains box under Configuration > Spider. Each line in the table represents a separate URL. Under Advanced, check “pause on high memory usage.”
The crawl process can take a couple of minutes depending on the size of your website. For each URL, you will see important info such as the status H1 contents, H2 contents, code returned, page title, meta description, and word count.
Pick a limit for the number of requests it can make per second to rate limit your crawling, go to the speed submenu in the configuration menu. Check the progress bar in the top right corner to see when it is done. It is the best starting point if you are new to the tool.
To avoid bot-capture blocking, use proxies with your crawling. The URL tab includes data related to the requested URLs. You can analyze all the URLs of a website via the URL tab. It includes info about each URL, such as its content, status code returned, and hash value.
If you’ve configured your site properly, it should hit all the pages. The UX can seem a bit confusing at first, but once you begin to use it, the learning curve is quick.
It is easy data export into CSV for documentation or graphical manipulation. It crawls the same way that Google crawls your site. It follows links.
If you are doing SEO audits, then I think Screaming Frog is the best first-line tool for looking at a site. It has a super-simple interface and navigation that isn’t bloated with design complexity.
It will find links on your menu, header, footer, and within your content. We have not found another tool that is capable of doing what Screaming Frog can in a very short amount of time. It has a very simple interface that makes it easy to use and figure out.
This website contains affiliate links to some products and services. We may receive a commission for purchases made through these links at no extra cost to you.
