Help & Support Tutorials

Smartcrawl – SEO Health

Smartcrawl – SEO Health

3.3 SEO Health

The SEO Health module runs a full SEO scan of your site, and provides a comprehensive report of how well your site is optimized for search engines and social media.

Click the New Test button to run a new SEO Audit. You can choose to apply Desktop or Mobile SEO audit simulation.

3.3.1 SEO Audits

Here you can find the audit results from the latest SEO Audits scan. We recommend addressing as many of the identified issues as possible to ensure your site is SEO friendly.

Click the down arrow next to any listed audit to see suggestions on how to improve your SEO.

SEO audit report in SmartCrawl

Let’s go over each audit so you have a better understanding of what they mean for your site.

Link text is the clickable word or phrase in a hyperlink. When link text clearly conveys a hyperlink’s target, both users and search engines can more easily understand your content and how it relates to other pages.

Learn more about how descriptive link text can affect SEO from web.dev.

<title> Element

Having a <title> element on every page helps all your users:

  • Search engine users rely on the title to determine whether a page is relevant to their search.
  • The title also gives users of screen readers and other assistive technologies an overview of the page. The title is the first text that an assistive technology announces.

Learn more about how title elements can affect SEO from web.dev.

Meta Description

The <meta name=”description”> element provides a summary of a page’s content that search engines include in search results. A high-quality, unique meta description makes your page appear more relevant and can increase your search traffic.

Learn more about how meta description can affect SEO from web.dev.

Hreflang

Many sites provide different versions of a page based on a user’s language or region. Hreflang links tell search engines the URLs for all the versions of a page so that they can display the correct version for each language or region.

Learn more about how hreflang links can affect SEO from web.dev.

Rel=canonical

When multiple pages have similar content, search engines consider them duplicate versions of the same page. For example, desktop and mobile versions of a product page are often considered duplicates.

Search engines select one of the pages as the canonical, or primary, version and crawl that one more. Valid canonical links let you tell search engines which version of a page to crawl and display to users in search results.

See also Working with Canonicals in SmartCrawl below.

Learn more about how canonical links can affect SEO from web.dev.

[alt] Attributes

Informative elements should aim for short, descriptive alternate text. Decorative elements can be ignored with an empty alt attribute.

Learn more about how image alt attributes can affect SEO from web.dev.

HTTP Status Code

Servers provide a three-digit HTTP status code for each resource request they receive. Status codes in the 400s and 500s indicate that there’s an error with the requested resource. If a search engine encounters a status code error when it’s crawling a web page, it may not index that page properly.

Learn more about how image HTTP status codes can affect SEO from web.dev.

Crawlable Pages

Search engines can only show pages in their search results if those pages don’t explicitly block indexing by search engine crawlers. Some HTTP headers and meta tags tell crawlers that a page shouldn’t be indexed.

Only block indexing for content that you don’t want to appear in search results.

Learn more about how index blocking can affect SEO from web.dev.

Robots.txt

The robots.txt file tells search engines which of your site’s pages they can crawl. An invalid robots.txt configuration can cause two types of problems:

  • It can keep search engines from crawling public pages, causing your content to show up less often in search results.
  • It can cause search engines to crawl pages you may not want shown in search results.

robots.txt not getting updated

If your site is hosted with WPMU DEV, the temporary domain (e.g. xxxxxx.tempurl.host) has a default robots.txt file that cannot be overridden by any physical or virtual robots.txt file until you add a custom domain and set it as the primary domain. This means you won’t be able to modify the robots.txt file of your temporary domain. However, once you’ve added a custom domain and set it as the primary domain, your custom robots.txt file will become functional.

Also, the default robots.txt file of the staging sites can’t be overridden as the file rules are configured at the server level to prevent crawlers from indexing our hosted staging sites.

Learn more about how robots.txt can affect SEO from web.dev.

Browser Plugins

Search engines often can’t index content that relies on browser plugins, such as Java or Flash. That means browser plugin-based content doesn’t show up in search results.

Also, most mobile devices don’t support browser plugins, which creates frustrating experiences for mobile users.

Learn more about how browser plugins can affect SEO from web.dev.

Google can follow links only if they are an <a> tag with an href attribute. Links that use other formats won’t be followed by Google’s crawlers. Google cannot follow links without an href, or links created by script events.

Learn more about how href attributes can affect SEO from Google Search Central.

Meta Viewport

Many search engines rank pages based on how mobile-friendly they are. Without a viewport meta tag, mobile devices render pages at typical desktop screen widths and then scale the pages down, making them difficult to read.

Setting the viewport meta tag lets you control the width and scaling of the viewport so that it’s sized correctly on all devices.

Learn more about how viewport meta tags can affect SEO from web.dev.

Structured Data

Search engines use structured data to understand what kind of content is on your page. For example, you can tell search engines that your page is an article, a job posting, or an FAQ.

Marking up your content with structured data makes it more likely that it will be included in rich search results. For example, content marked up as an article might appear in a list of top stories relevant to something the user searched for.

Learn more about how structured data can affect SEO from web.dev.

3.3.2 Reporting

SEO scans can be run automatically and on a recurring basis. To scheduled automatic scans, toggle on Send scheduled performance reports and select your desired email Recipients.

Enter SEO test report recipients

Then set the Schedule you’d like the regular tests to be run on, and select whether reports should only be sent if the score drops below a certain percentage.

Set SEO reporting schedule

Finally, select whether reports should be generated for Desktop or Mobile only, or for Both device types.

Select device types for SEO reports

Remember to click Save Settings when you’re done.

3.3.3 SEO Settings

SmartCrawl runs audits against your site’s homepage and will generate reports focused on improving SEO results. You can choose to display SEO test results for either Desktop or Mobile devices in the Dashboard.
Choose to run Desktop or Mobile SEO audits

3.3.4 Troubleshooting SEO Health Issues

SEO Audit Returns a Score of 0

If your SEO Audit score is 0, it may be because the page is very large and the audit tool simply times out.

SmartCrawl’s audit tool uses Google’s Lighthouse engine, and receives its results from Lighthouse via our API.

While Lighthouse by itself can scan content well over 2Mb in size, its implementation in SmartCrawl can time out if the page is 1.8Mb or greater.

The total HTML size of an average homepage would be around 500-600k or less.

You can check the total size of your page by right-clicking on it and selecting the View page source option. Then copy the entire code and paste into a tool like this one.

If it’s over 1.8Mb, you may want to check your active plugins to see if one or more of them may be contributing to the large page size, and disable any you don’t really need.

If you cannot discard any such plugins, you can always use the Lighthouse engine directly in Chrome browser, or try Google’s own tool at web.dev/measure

SEO Audit says “We were not able to get results for your site”

This can be caused by Bot Fight Mode enabled at Cloudflare, which blocks the Google Lighthouse API.

So if you are routing your domain through Cloudflare with Bot Fight Mode enabled and are facing this issue, try allowlisting the Google ASN number (AS15169) under Security > WAF > Tools.

Allowlist Google ASN number at Cloudflare