Help & Support Tutorials

Smartcrawl Pro – Setup & Configuration

Smartcrawl Pro – Setup & Configuration

This guide explains how to use SmartCrawl’s SEO features to drive more traffic to your site or network. Use the Index on the left to quickly locate usage guidance on specific features.

If you haven’t installed SmartCrawl yet, then you should visit the SmartCrawl Pro page where you can explore the plugin’s many features, and where WPMU DEV members can install SmartCrawl Pro directly to any connected site.

INCLUDED WITH EVERY WPMU DEV HOSTED SITE

Every WPMU DEV hosted site comes fully loaded with Pro versions of the following plugins automatically installed for you: SmushHummingbirdDefenderSmartCrawl, and Forminator. Not hosting with us yet? Check out our Hosting page to see if we might be a good fit for your projects.

Streamline SmartCrawl configuration in your Hub

Staying ahead of your competition just got a whole lot easier. SmartCrawl’s powerful suite of SEO features are fully integrated with the Hub, so you can manage all the SEO needs of all your connected sites right from one location. See our blog post, Easily Configure Your WordPress SEO with SmartCrawl and The Hub, for all the details.

Introducing Google Lighthouse

Google Lighthouse, the industry standard for testing website performance, is now built right into both SmartCrawl and The Hub. See SEO Health below for configuration details, or read our blog post on SmartCrawl’s Lighthouse integration for a complete overview of the feature.

The Quick Setup Wizard popup will appear on the SmartCrawl Dashboard screen the first time it is accessed after activating the plugin.

All the options are turned ON by default, but you can turn OFF the ones you don’t need. Don’t worry as you can always turn them On/Off later from Settings/General Settings.

After you’ve chosen the features that are right for you, click the Get Started button at the bottom of the module. This will activate enabled features, open your SmartCrawl Dashboard, and start your first SEO test under the SEO Audit module.

The dashboard gives you an overview of your site SEO configuration, is where you can see your setup, and can be used to quickly jump to your desired module.

On top of the module, you can see your site’s Current SEO Score and the number of SEO and sitemap issues, if any.

The SEO Audits module will present you with the most recent SEO issues you might want to fix or look into.

SEO dashboard widget

You can run a full checkup right from the SmartCrawl Dashboard screen by clicking the + Run Test button.

Once the scan is complete you can review your report by clicking the View Report button or by going to SmartCrawl > SEO Health.

For more info – jump to the SEO Health section.

The Titles & Meta module controls how your website’s pages, posts, and custom post types appear in search engines.

Titles & Meta in SmartCrawl dashboard

For more info – jump to the Title & Meta section.

The Sitemap module will let you see and quickly access your sitemap configuration.

Sitemap in SmartCrawl dashboard

For more info – jump to the Sitemap section.

The Advanced Tools module lets you set up fine details of SEO including internal linking, redirections, breadcrumbs, and Moz analysis.

For more info – jump to the Advanced Tools section.

The Social module lets you control how your website appears when shared on social platforms like Facebook and Twitter.

For more info – jump to the Social section.

The Content Analysis module lets you see recommended improvements to your content for the best chance of ranking high in search engines, as well as improving the readability for the average person.

Content analysis in SmartCrawl dashboard

The Emails & Report module allows you to manage your email notifications and reports for SEO Audits and Sitemap Crawler reports.

Emails and report widget on the dashboard

For more info, jump to the SEO Audits or Sitemap sections.

The Schema module indicates whether schema has been activated in SmartCrawl (Default Markup) and displays any additional active/inactive custom Schema Types that you have configured. The module includes handy buttons to jump straight to the configuration screen, or add new customizable schema types.

Schema in SmartCrawl dashboard

For more info – jump to the Schema section.

The SEO Health module runs a full SEO scan of your site, and provides a comprehensive report of how well your site is optimized for search engines and social media.

Click the New Test button to run a new SEO Audit. You can choose to apply Desktop or Mobile SEO audit simulation.

3.3.1 SEO Audits

Link to chapter 3

Here you can find the audit results from the latest SEO Audits scan. We recommend addressing as many of the identified issues as possible to ensure your site is SEO friendly.

Click the down arrow next to any listed audit to see suggestions on how to improve your SEO.

SEO audit report in SmartCrawl

Let’s go over each audit so you have a better understanding of what they mean for your site.

Link text is the clickable word or phrase in a hyperlink. When link text clearly conveys a hyperlink’s target, both users and search engines can more easily understand your content and how it relates to other pages.

Learn more about how descriptive link text can affect SEO from web.dev.

<title> ElementCopy heading anchor to clipboard

Having a <title> element on every page helps all your users:

  • Search engine users rely on the title to determine whether a page is relevant to their search.
  • The title also gives users of screen readers and other assistive technologies an overview of the page. The title is the first text that an assistive technology announces.

Learn more about how title elements can affect SEO from web.dev.

Meta DescriptionCopy heading anchor to clipboard

The <meta name=”description”> element provides a summary of a page’s content that search engines include in search results. A high-quality, unique meta description makes your page appear more relevant and can increase your search traffic.

Learn more about how meta description can affect SEO from web.dev.

HreflangCopy heading anchor to clipboard

Many sites provide different versions of a page based on a user’s language or region. Hreflang links tell search engines the URLs for all the versions of a page so that they can display the correct version for each language or region.

Learn more about how hreflang links can affect SEO from web.dev.

Rel=canonicalCopy heading anchor to clipboard

When multiple pages have similar content, search engines consider them duplicate versions of the same page. For example, desktop and mobile versions of a product page are often considered duplicates.

Search engines select one of the pages as the canonical, or primary, version and crawl that one more. Valid canonical links let you tell search engines which version of a page to crawl and display to users in search results.

See also Working with Canonicals in SmartCrawl below.

Learn more about how canonical links can affect SEO from web.dev.

[alt] AttributesCopy heading anchor to clipboard

Informative elements should aim for short, descriptive alternate text. Decorative elements can be ignored with an empty alt attribute.

Learn more about how image alt attributes can affect SEO from web.dev.

HTTP Status CodeCopy heading anchor to clipboard

Servers provide a three-digit HTTP status code for each resource request they receive. Status codes in the 400s and 500s indicate that there’s an error with the requested resource. If a search engine encounters a status code error when it’s crawling a web page, it may not index that page properly.

Learn more about how image HTTP status codes can affect SEO from web.dev.

Crawlable PagesCopy heading anchor to clipboard

Search engines can only show pages in their search results if those pages don’t explicitly block indexing by search engine crawlers. Some HTTP headers and meta tags tell crawlers that a page shouldn’t be indexed.

Only block indexing for content that you don’t want to appear in search results.

Learn more about how index blocking can affect SEO from web.dev.

Robots.txtCopy heading anchor to clipboard

The robots.txt file tells search engines which of your site’s pages they can crawl. An invalid robots.txt configuration can cause two types of problems:

  • It can keep search engines from crawling public pages, causing your content to show up less often in search results.
  • It can cause search engines to crawl pages you may not want shown in search results.

robots.txt not getting updated

If your site is hosted with WPMU DEV, the temporary domain (e.g. xxxxxx.tempurl.host) has a default robots.txt file that cannot be overridden by any physical or virtual robots.txt file until you add a custom domain and set it as the primary domain. This means you won’t be able to modify the robots.txt file of your temporary domain. However, once you’ve added a custom domain and set it as the primary domain, your custom robots.txt file will become functional.

Also, the default robots.txt file of the staging sites can’t be overridden as the file rules are configured at the server level to prevent crawlers from indexing our hosted staging sites.

Learn more about how robots.txt can affect SEO from web.dev.

Browser PluginsCopy heading anchor to clipboard

Search engines often can’t index content that relies on browser plugins, such as Java or Flash. That means browser plugin-based content doesn’t show up in search results.

Also, most mobile devices don’t support browser plugins, which creates frustrating experiences for mobile users.

Learn more about how browser plugins can affect SEO from web.dev.

Google can follow links only if they are an <a> tag with an href attribute. Links that use other formats won’t be followed by Google’s crawlers. Google cannot follow links without an href, or links created by script events.

Learn more about how href attributes can affect SEO from Google Search Central.

Meta ViewportCopy heading anchor to clipboard

Many search engines rank pages based on how mobile-friendly they are. Without a viewport meta tag, mobile devices render pages at typical desktop screen widths and then scale the pages down, making them difficult to read.

Setting the viewport meta tag lets you control the width and scaling of the viewport so that it’s sized correctly on all devices.

Learn more about how viewport meta tags can affect SEO from web.dev.

Structured DataCopy heading anchor to clipboard

Search engines use structured data to understand what kind of content is on your page. For example, you can tell search engines that your page is an article, a job posting, or an FAQ.

Marking up your content with structured data makes it more likely that it will be included in rich search results. For example, content marked up as an article might appear in a list of top stories relevant to something the user searched for.

Learn more about how structured data can affect SEO from web.dev.

3.3.2 Reporting

Link to chapter 3

SEO scans can be run automatically and on a recurring basis. To scheduled automatic scans, toggle on Send scheduled performance reports and select your desired email Recipients.

Enter SEO test report recipients

Then set the Schedule you’d like the regular tests to be run on, and select whether reports should only be sent if the score drops below a certain percentage.

Set SEO reporting schedule

Finally, select whether reports should be generated for Desktop or Mobile only, or for Both device types.

Select device types for SEO reports

Remember to click Save Settings when you’re done.

3.3.3 SEO Settings

Link to chapter 3

SmartCrawl runs audits against your site’s homepage and will generate reports focused on improving SEO results. You can choose to display SEO test results for either Desktop or Mobile devices in the Dashboard.
Choose to run Desktop or Mobile SEO audits

3.3.4 Troubleshooting SEO Health Issues

Link to chapter 3

SEO Audit Returns a Score of 0Copy heading anchor to clipboard

If your SEO Audit score is 0, it may be because the page is very large and the audit tool simply times out.

SmartCrawl’s audit tool uses Google’s Lighthouse engine, and receives its results from Lighthouse via our API.

While Lighthouse by itself can scan content well over 2Mb in size, its implementation in SmartCrawl can time out if the page is 1.8Mb or greater.

The total HTML size of an average homepage would be around 500-600k or less.

You can check the total size of your page by right-clicking on it and selecting the View page source option. Then copy the entire code and paste into a tool like this one.

If it’s over 1.8Mb, you may want to check your active plugins to see if one or more of them may be contributing to the large page size, and disable any you don’t really need.

If you cannot discard any such plugins, you can always use the Lighthouse engine directly in Chrome browser, or try Google’s own tool at web.dev/measure

SEO Audit says “We were not able to get results for your site”Copy heading anchor to clipboard

This can be caused by Bot Fight Mode enabled at Cloudflare, which blocks the Google Lighthouse API.

So if you are routing your domain through Cloudflare with Bot Fight Mode enabled and are facing this issue, try allowlisting the Google ASN number (AS15169) under Security > WAF > Tools.

Allowlist Google ASN number at Cloudflare

The Titles & Meta section enables you to control how your website’s pages, posts and custom post types appear in search engines like Google and Bing.

Each post type has its own settings so you can fine-tune things exactly as you need them.

3.4.1 Home Page

Link to chapter 4

This section allows you to setup how your homepage title and meta will appear in Google Search.

Search engines read the title and description for each element of your site. So it’s important to take the time to configure these settings.

You can override the global meta you set here for individual posts/pages (more on this under In-Post Analysis), and fine-tune them but first, let’s take a look at what you can do with your global settings.

Homepage title and meta settings in SmartCrawl

At the top of this module, you can see how will your site be presented in the search results and this will change as you modify the settings (so you can see the preview).

Note that this is all related to your HomePage and that you can modify settings for each page separately (see In-Post Analysis).

TitleCopy heading anchor to clipboard

This is where you can enter exactly what you want search engines to prominently display on results pages. This is what identifies your site in the results. You can insert one of the available macros (by clicking on the Insert Dynamic macro button) or you can just type in your desired description.

DescriptionCopy heading anchor to clipboard

This is probably the most important setting here, as it is the most likely to be indexed by search engines. As described above, you can use one of the available macros or you can type in your custom description.

OpenGraphCopy heading anchor to clipboard

OpenGraph allows you to enhance how your site will be presented when making a post on Facebook and linking your page (Home Page).

Homepage OpenGraph meta settings in SmartCrawl

When you setup your desired layout and Save the Settings, you can see how it will look at https://developers.facebook.com/tools/debug/. This enables you troubleshoot any problems before linking your site (like if the image is too small or too big).

Homepage Facebook preview in SmartCrawl

Note that if you embed several images Facebook will allow you to choose which one to present as your post image so you can give each post a fresh look.

Using WooCommerce?Copy heading anchor to clipboard

Please see the note below in the Post Types chapter if WooCommerce is active on your site, and you want all available OpenGraph tags to be automatically added to your individual product pages.

TwitterCopy heading anchor to clipboard

This option enables you to enhance how your site will be presented when linking your page (Home Page) on Twitter.

Homepage Twitter meta settings in SmartCrawl

As explained before, you can use available macros or you can insert custom text. This module also allows you to setup a Featured image/images (these images will be available to use if the post or page being shared doesn’t contain any images).

When you setup your desired layout and click Save Settings you can see how your Twitter card will look at https://cards-dev.twitter.com/validator. This will also help you troubleshoot any potential problems before linking your site (like if the image is too small or too big).

Homepage Twitter card preview in SmarCrawl

Note that if you upload a new image, Twitter will override all the previous posts with the latest image.

IndexingCopy heading anchor to clipboard

Indexing enables you to configure how you want your website to appear in search results.

Homepage indexing options in SmartCrawl

The available features are as follows:

  • Index this website – This is particularly useful to prevent indexing duplicate content on your website (which highly impacts your website ratings).
  • Follow this website – By default, search engine crawlers follow each link on your site until the site is crawled. When disabled, this can be used to stop them from following dynamic URLs that lead to the same/similar content on your site, or if you do not wish to share your link to external URLs.
  • Apply to all pages except the first – Enable this option to exclude applying the indexing settings to the first page, but apply them to all subsequent pages. This setting is available for the homepage, taxonomies, and archives as those types can be paginated, but not individual post types.

Just make sure to click Save Settings when you’re done. 🙂

3.4.2 Post Types

Link to chapter 4

The Post Types section allows you to setup the search result appearance of your Posts, Pages, Media and custom post types.

Each post type on your site will be listed alphabetically. Just click on the ones you want to edit their title & meta

Set SEO title & meta for each post type on a site

For more info on how to set up Post types, refer to the Home Page section as it follows the same rules and principles.

See the Video Schema chapter below for additional guidance on ensuring your embedded videos have all the needed schema to pass Google’s Rich Results Test.

Note that if WooCommerce is active on your site, and you want all available og: tags automatically added to your individual product pages, you’ll want to enable OpenGraph here for the Product post type. Then go to WooCommerce SEO and toggle on the Enable Product Open Graph option.

3.4.3 Taxonomies

Link to chapter 4

Taxonomies section will allow you to set up the search result appearance for your post categories, tags and custom taxonomies. For more info, refer to the Home Page section as it follows the same rules and principles.

3.4.4 Archives

Link to chapter 4

The Archives section allows you to setup the search result appearance for your Archive pages. For more info refer to the Home Page section as it follows the same rules and principles.

Author Archive: If you are the only author of the content of your website, Google may see your author archives as duplicate content to your Blog Homepage. If this is the case we recommend disabling author archives.

Note: The default meta description for the Author Archive is the Author bio but this can easily be edited at any point.

author bio in meta description

Date Archive: Google may see your date archives as duplicate content to your Blog Homepage. For this reason, we recommend disabling date archives.

3.4.5 Settings (Title & Meta)

Link to chapter 4

SeparatorCopy heading anchor to clipboard

The separator refers to the break between variables which you can use by referencing the %%sep%% tag. You can choose a preset or make your own.

Title & Meta settings

For example, this is your default appearance:

Default meta separator in SmartCrawl

But if you like to stand out from the crowd and use something like ~ for your separator your appearance will look something like:

Custom meta separator in SmartCrawl

Character LengthsCopy heading anchor to clipboard

By default we recommend best practice characters lengths for your meta titles and descriptions. However, you can adjust these settings to suit your own requirements.

The character lengths that are set here affect the SEO recommendations for individual posts & pages. For more info, see the In-Post Analysis section.

Title & meta character lengths

DeactivateCopy heading anchor to clipboard

Click the Deactivate button if you don’t need the Title & Meta features on your site.

Deactivate Title & Meta feature in SmartCrawl

3.4.6 BuddyPress

Link to chapter 4

If you have BuddyPress enabled on your site, you will see an additional section where you can customize the title & meta for BuddyPress profiles. For more info, refer to the Home Page section as it follows the same rules and principles.

3.4.7 Available Macros

Link to chapter 4

There are numerous macros available to use in the title & meta of your various post_types. To view them all in your admin, click the Browse Macros button at the top of the Title & Meta screen.

SmartCrawl browse macros button in wp-admin

For your convenience, here’s the full list of available macros with their descriptions:

General macrosCopy heading anchor to clipboard

  • %%sep%% – Separator
  • %%sitename%% – Site’s name
  • %%sitedesc%% – Site’s tagline / description
  • %%page%% – Current page number (i.e. page 2 of 4)
  • %%pagetotal%% – Current page total
  • %%pagenumber%% – Current page number
  • %%spell_pagenumber%% – Current page number, spelled out as numeral in English
  • %%spell_pagetotal%% – Current page total, spelled out as numeral in English
  • %%spell_page%% – Current page number, spelled out as numeral in English
  • %%currenttime%% – Current time
  • %%currentdate%% – Current date
  • %%currentmonth%% – Current month
  • %%currentyear%% – Current year

Post macrosCopy heading anchor to clipboard

  • %%id%% – Post/page ID
  • %%title%% – Title of the post/page
  • %%excerpt%% – Post/page excerpt (or auto-generated if it does not exist)
  • %%excerpt_only%% – Post/page excerpt (without auto-generation)
  • %%modified%% – Post/page modified time
  • %%date%% – Date of the post/page
  • %%name%% – Post/page author’s ‘nicename’
  • %%userid%% – Post/page author’s userid
  • %%caption%% – Attachment caption
  • %%category%% – Post categories (comma separated)
  • %%tag%% – Current tag/tags

Taxonomy archives macrosCopy heading anchor to clipboard

  • %%id%% – Term ID
  • %%term_title%% – Term name
  • %%term_description%% – Term description
  • %%category%% – Category name
  • %%category_description%% – Category description
  • %%tag%% – Tag name
  • %%tag_description%% – Tag description

Author archives macrosCopy heading anchor to clipboard

  • %%name%% – Author’s ‘nicename’
  • %%userid%% – Author’s userid

Date archives macrosCopy heading anchor to clipboard

  • %%date%% – Date of the archive

Custom post type archives macrosCopy heading anchor to clipboard

  • %%pt_plural%% – Post type label plural
  • %%pt_single%% – Post type label singular

Search macrosCopy heading anchor to clipboard

  • %%searchphrase%% – Current search phrase

BuddyPress profiles macrosCopy heading anchor to clipboard

  • %%bp_user_username%% – BuddyPress username
  • %%bp_user_full_name%% – BuddyPress user’s full name

BuddyPress groups macrosCopy heading anchor to clipboard

  • %%bp_group_name%% – BuddyPress group name
  • %%bp_group_description%% – BuddyPress group description
  • %%id%% – Post/page ID
  • %%title%% – Post/page title
  • %%modified%% – Post/page modified time
  • %%date%% – Post/page date
  • %%name%% – Post/page author’s ‘nicename’
  • %%userid%% – Post/page author’s userid
  • %%category%% – Post categories
  • %%tag%% – Post tag
  • %%sep%% – Separator
  • %%sitename%% – Site name
  • %%sitedesc%% – Site description or tagline
  • %%currenttime%% – Current time
  • %%currentdate%% – Current date
  • %%currentmonth%% – Current month
  • %%currentyear%% – Current year
  • %%page%% – Displays page number in the format (Page ’current page no’ of ‘total’)
  • %%pagetotal%% – Current page total
  • %%pagenumber%% – Current page number
  • %%spell_pagenumber%% – Displays page number in the format (Page ’current page no in words’ of ‘total page number in words’)
  • %%spell_pagetotal%% – Current page total in words
  • %%spell_page%% – Current page number in words
  • %%searchphrase%% – Search keyword

Dynamic macrosCopy heading anchor to clipboard

In addition to all the above, the following dynamic macros can be used for cases where none of the above are a good fit. Note these ones do not appear in the list when you click the Browse Macros button.

  • %%ct_TAXONOMY%% – Replace TAXONOMY with the term you need.
  • %%ct_desc_TAXONOMY%% – Replace TAXONOMY with the term you need.
  • %%cf_META_KEY%% – Replace META_KEY with the post meta value you need.

On a singular post, the TAXONOMY macros are replaced by the title/desc of the first taxonomy term associated with the post. On the taxonomy archive they are replaced by the title/desc of the current term.

For example, dynamic macros can be used to get a WooCommerce product category to appear in the SEO Title (and browser tab) of corresponding single product pages. In this case, you’d want to use the %%ct_TAXONOMY%% macro, and replace TAXONOMY with product_cat so the macro becomes %%ct_product_cat%%.

Note that if you have multiple categories assigned to a product, only the first one will be fetched and displayed.

3.5 Primary Category

Copy chapter anchor to clipboard

If your post(s) or WooCommerce product(s) have more than one category enabled, then you can set one of those as a primary category.

In the post editor, selecting more than one category or subcategory will display the Primary Category drop-down. Select the category/subcategory you want to set as the primary category using the drop-down.

Similarly for WooCommerce products, selecting more than one category or subcategory will display a Make Primary link next to each category and subcategory. Click the link of any category/subcategory to set it as the primary category.

You can disable the primary category feature, by adding the following code to the wp-config.php file

define( 'SMARTCRAWL_DISABLE_PRIMARY_TERMS', true );

Primary category in post breadcrumbs

If the breadcrumbs feature is enabled, the chosen primary category will be displayed as the category/subcategory in the posts’ breadcrumbs. And, the chosen primary category will be displayed in the post or product URL if the post or product permalink structure has a %%category%% or product-category tag.

Schema markup is code that you put on your website to help search engines return more informative results for users. SmartCrawl has full support for default schema types and makes it easy for you to let search engines know what your data means, not just what it says.

The settings you configure here will be included as JSON LD data into the head section that you can see when viewing a page’s source code. It would appear in a tag as follows:

<script type="application/ld+json">{"@context":"https:\/\/schema.org","@graph":[{ ... }]</script>

Everything between the opening [{ and closing }] brackets in that tag would be the schema output resulting from your configuration here, and that’s what search engines will use to populate the rich snippets in results pages giving your visitors much more relevant information.

Before diving in below though, please take note of some important information about images in your schema:

  • For your company logo or profile image to appear in rich snippets, Google requires an image that is at least 112x112px.
  • Featured images should be at least 1200px wide, and article & product images should be at least 400px wide.
  • Images in your schema can be in any of the following formats: BMP, GIF, JPEG, PNG, WebP, or SVG.
  • All images should respect at least one of the following aspect ratios: 16×9, 4×3, or 1×1.

3.6.1 General

Link to chapter 6

Website DetailsCopy heading anchor to clipboard

This section enables you to specify the name and logo to use for your site in search results.

  • Website Name – Add the site name you would like to appear in the schema markup.
  • Website Logo – Specify the image to use for your website’s logo.

schema-general

Person or OrganizationCopy heading anchor to clipboard

Personal DetailsCopy heading anchor to clipboard

Here, you’ll want to specify whether your site represents a Person or an Organization. This information will be used in Google’s Knowledge Graph Card; the block you can see on the right side of the search results.

If you select Person, you’ll first fill out your personal information including your Name, Job Title, Bio and a Portrait Photo.

schema-person

Personal BrandCopy heading anchor to clipboard

Enter a brand name if it is different from your personal name. Then add a brand logo to be used in Google search results and in the Knowledge Graph.

schema-person-brand

Personal contactCopy heading anchor to clipboard

Finally, add your contact information to enhance search results. This includes your Phone Number and the page on your site where visitors can find your Contact form.

schema-person-contact

If you select Organization, you’ll first fill out your organization’s information including its Type, Name, Description and Logo.

schema-organization

Note that if you wish to use Local Business markup for your organization, you can do that by creating a Local Business schema type.

Then add your organization’s contact information including the Contact Type, Phone Number and the page where visitors can find your Contact form.

schema-organization-contact

Social AccountsCopy heading anchor to clipboard

In the Social Accounts section, you’ll want to specify as many social accounts as you can so search engines can associate them with you or your business. Currently supported social accounts include Twitter, Facebook, Instagram, LinkedIn, Pinterest and YouTube.

Enable this option to include your site’s WordPress searchbox directly in the main result on search results pages so users can search your site right from there.

schema-sitelinks-search

Output PageCopy heading anchor to clipboard

Select the page that should include your base person or organization schema. It is recommended that this be the page that most reflects information related to your brand/company.

schema-output

3.6.2 Advanced

Link to chapter 6

This section of the schema settings enables you to configure several more advanced options for your markup.

Special PagesCopy heading anchor to clipboard

Here, you can identify which pages are used on your site as the About page, the Contact page, as well as which navigation menu gives your visitors the most general overview of your website.

advanced special pages

Structured DataCopy heading anchor to clipboard

Enable WpHeader and WpFooter – Enable this setting to add markup for the header and footer sections which contain general information about your website like the copyright year, site name, and site tagline.

Enable Comments – Enabling this setting will add comments that a page has received nested in the output markup using Comment as a value.

schema-advanced-structured-data

Default ImageCopy heading anchor to clipboard

Upload a fallback image here to use in any post that does not yet have a featured image, or where the featured image may be missing.

schema advanced default image

AuthorCopy heading anchor to clipboard

Enable the Author URL and/or the Gravatar ImageObject options to include the post author and Gravatar URLs in the schema markup for every post.

schema-advanced-author

ArchivesCopy heading anchor to clipboard

Here, you can disable any option in this section if you do not want schema markup to be included for any of the specified post types. You can also specify whether your site contains only blog posts or various types of content.

schema-advanced-archives

Media ObjectsCopy heading anchor to clipboard

Select whether to include media objects in your schema markup or not.

  • AudioObject – Enable this option to include markup for any audio elements embedded in your posts or pages.
  • VideoObject – Enable this to include markup for any video elements included in your posts and pages. If you use YouTube videos on your site, you can connect directly to the YouTube API to fetch all the needed info automatically.

schema-advanced-media

Enabling the YouTube APICopy heading anchor to clipboard

If you use YouTube videos on your site and want all the needed schema to be fetched automatically, you’ll need to create a Google project and enable the YouTube API in an app in that project.

To get that done, go to your Google Developers Console and click Create Project.

Create a project in Google developer console

Give your project a Name and click Create.

Name your Google project

Then go to APIs & Services > Dashboard.

Google APIs & Services Dashboard

Be sure you’re viewing the project you just created. Then click Enable APIs and Services.

Enable APIs and Services in Google project

Locate and select the YouTube Data API, then click the Enable button.

Enable YouTube Data API in Google project

Next, go to APIs & Services > OAuth Consent Screen.

OAuth Consent Screen in Google project

On that screen, select External as the User Type, and click Create.

Set Google project as external

That will direct you to the Edit App Registration screen. Enter a Name for your app and select the User support email address of the same Google account you’re currently logged-into Google with.

Enter name and email for Google app

Ignore the App Domain section on that screen, and scroll down to the Authorized Domains section.

Click the + Add Domain button and enter your site’s domain name (without the protocol or trailing slash).

Then enter a contact email address (can be the same as the one you selected above).

Click Save and Continue.

Enter domain and email for Google app

Ignore everything on the Scopes and Test Users screens, and just scroll down and click Save and Continue on both of those.

Then, on the final Summary screen, scroll down and click Back to Dashboard.

That will bring you back to the OAuth Consent Screen. Click the Publish App button there, then click Confirm in the modal that pops open.

Publish app in Google Console

Now that you’ve created your app, you just need to get the API Key you need to enter in SmartCrawl.

Go to APIs & Services > Credentials.

Google console APIs & Services Credentials

Click the Create Credentials button, then select API Key.

Create API key in Google project

Copy the API Key from the modal that pops open.

Copy API key from Google project

Paste that key in the Access Code field in the Connect with YouTube section of your SmartCrawl Schema settings. Then click the Authorize button.

Paste API key in SmartCrawl settings

You’ll see a notice pop up at the top of your screen to confirm that the API key is valid.

Valid API key entered in SmartCrawl

Finally, click Save Settings at the bottom of your SmartCrawl Schema settings screen to complete the setup.

Save settings in SmartCrawl

3.6.3 Types Builder

Link to chapter 6

By default, SmartCrawl generates general markup on key WordPress pages. The Types Builder feature allows you to further tailor this to meet your needs and configure different schema types that you can apply to certain posts, pages, and taxonomies.

SmartCrawl’s Schema

The schema feature in SmartCrawl offers several tools to get you completely set up for search engines. Check out our blog on SmartCrawl’s Schema for a quick run-through of the feature. And see How to Use Advanced Features in SmartCrawl’s Schema Types Builder for some useful tips when building your schema types.

This is particularly helpful for customizing the information that is assigned to different types of content so that search engines can display that information in rich snippets.

The next few sections will guide you through the following concepts:

  • Add new schema type – Instructions on how to add a new schema type to the type builder.
  • Configure schema types – Information on how to configure your schema types and how to apply locations (conditions). This affects which schema types are assigned to your different kinds of content.
  • Types – A guide to the available properties for each schema type.
  • Mapping property values – How to map properties to custom post meta for directory-type sites, like jobs boards or recipe collections.

Add new schema typeCopy heading anchor to clipboard

You can add a new type by clicking Add New Type.

Select which type you want and click Continue.

Add new schema type in SmartCrawl

You can rename your new type to whatever will help you to distinguish it from your other schema types, or you can leave it as the default name and click Continue.


Depending on the schema type you are creating, you may be presented with the option to then select a subtype. You can skip this step to use the generic type instead.

Select subtype for new schema type in SmartCrawl

Then create and add at least one location rule for your schema type, and click + Add. More info on location rules can be found in Configure schema types below.

Add location rule to new schema type in SmartCrawl

Configure schema typesCopy heading anchor to clipboard

Once you have added schema types, you can configure them to suit your needs. You can easily go back and rename a schema type by clicking on the gear icon and selecting Rename, or you can duplicate a schema type for easier configuration of a new one by clicking Duplicate. You can also delete the schema type by clicking Delete.


If needed, you can Disable a schema type at any time without losing its configuration by toggling off the slider. Re-enable at any time by toggling it back on again.

Click the arrow to drop down the row and reveal the full set of available configuration options for each chosen schema type.


When configuring any schema type, you will need to determine the location of that type. This is essentially a set of conditions that you must set and any page, post, or taxonomy that matches the set of conditions will be assigned the information for that particular schema type.

schema type location

Each condition, except for Show Globally and Homepage, is made up of two parts. These two parts include two different dropdown menus. The first dropdown will select the general location of the condition and currently, the available options are:

  • Page
  • Post Type
  • Show Globally
  • Homepage
  • Post Author Role
  • Post
  • Post Category
  • Post Tag
  • Media
  • Page Template
  • Custom post types
  • Post format

first location for schema type

The custom post types will be visible as additional items when the relevant plugins are active. So if, for instance, your site has WooCommerce installed and active, your first dropdown will also include Product and Product Type in addition to the default options listed above.

Once you have selected your general location, you can refine the application by specifying the location within the category of whatever your first choice was. This is controlled by the second dropdown menu and is dependent on the first dropdown menu.

For each first dropdown (shown in bold), these are the corresponding second dropdowns:

  • Page – Search for a page by typing the name and selecting the correct page.
  • Post Type – Choose either Post, Page, or Media from the list.
  • Show Globally (only one dropdown menu) – This option does not have a corresponding second dropdown because the condition itself is applied globally on your site and does not require further specification.
  • Homepage (only one dropdown menu) – This option does not have a corresponding second dropdown menu because the condition itself only applies to the homepage and does not require any refinement.
  • Post Author Role – Select either the Administrator, Editor, Author, Contributor, or Subscriber role for this author condition.
  • Post – Search for a post by typing in the name and then select the particular post that you want the condition to include.
  • Post Category – Search for a post category by typing the name and choosing the category that you want.
  • Post Tag – Type the name of a post tag and select the tag that you want.
  • Media – Search for the specific media that you want for the condition.
  • Page Template – Choose the template that you want from the dropdown.
  • Product – Search for the desired product.
  • Product Type – Choose either a VariableSimple, Grouped, or External Product.
  • Post format – Choose from among the available WordPress post formats.

In the example below, the Page condition has been used.

second location for schema type

An important thing to remember when creating your conditions is that they function with AND, OR and NOT logic. This means that conditions can be stacked to either work in union with each other, they can be made to overlap, and they can work as an inclusion or an exclusion.

Equal

By default, each condition stands as an inclusion to the schema type and is indicated by the equal sign; meaning that each added condition represents a case where the schema type will be applied. To change this to an exclusion, simply click on the equal sign to turn it into an unequal sign. When a condition is set to an exclusion (unequal sign), it means that the schema type will not be applied to that case.

equal sign for schema type location

AND

To add a new condition with AND logic, click AND.

add AND for schema type location

This means that the schema type will only be applied when all the conditions that have been added with AND logic are true. As an example, if you want to apply a schema type to posts in a specific post category but only to the ones that also include a specific tag, you could add a Post Category condition AND Post Tag as shown below.

AND condition example

All conditions that have been added with AND will appear together within the same section, which is separated by a horizontal line.

AND condition division

OR

To add a new rule with OR logic, click Add Rule (OR).

add OR rule for schema type

This will add a case for the schema type as an addition to any other conditions applied. For example, if you want the schema type to be applied to all content that falls under a certain category or that includes a specific tag, you can create a Post Category condition and a Post Tag condition separated by the Or condition, as shown below. Conditions that have been added with OR will be separated from each other with the word “Or”.

Or condition in schema locations

Delete

Remove conditions by hovering on the right-hand side of the row and clicking the red arrow.

delete condition for schema type location

TypesCopy heading anchor to clipboard

SmartCrawl includes support for the following schema types:

You can also generate a custom schema using the Custom Type.

New schema types

See our blog on New Schema Markup For SmartCrawl to get up to speed with the latest additions to Schema Types.

The different schema types are made up of a set of configurable properties. You can add and remove properties to suit your needs and customize the data assigned to each property. Each type will offer different property options. To delete properties, hover over the right-hand section of the property row and click the trashcan icon that appears.

delete property

This will trigger a confirmation modal to pop up. Click Delete to proceed or click Cancel to exit without deleting the property.

delete property confirmation

Made changes to your property that you want to revert? No problem, you can always click the Reset Properties button at the bottom of the panel which will return your property to its default setup. This would undo all changes that you have made in the property settings.

If you have deleted a property by mistake, you can easily re-add it, as well as add new properties, by clicking Add Property.

reset and add new properties

If any properties required by Google are not yet included in the configuration of any of your Schema Types, you’ll see the following alert at the top of the Schema screen. A yellow alert icon will also appear next to each type that is missing required properties.

Missing required properties notice

The missing required types will be highlighted with a blue asterisk (*) in the list of available properties for any Type when you click the + Add Property button, and the same notice as above will appear there as well.

Missing required properties notice

ArticleCopy heading anchor to clipboard

When adding an Article type, you’ll have the option to select either a Blog Posting or News Article subtype. You can skip this step to use the generic Article type instead.

Select subtype for new Article type in SmartCrawl

The Article schema type is pre-configured so that you can easily adjust each property to meet your individual needs – no need to start from scratch. Most properties require both a source and a value, which will differ depending on the property type. What we mean by this is that properties like the Headline and Name will offer a text value option whereas the URL property will offer a permalink value field. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • Headline
  • Name
  • Description
  • URL
  • Thumbnail URL
  • Date Modified
  • Date Published
  • Author
  • Publisher
  • Images
  • Article Body
  • Alternative Headline

article properties

Note that you can have multiple Images properties; simply click the plus icon next to the trashcan icon when hovering over the row.

add new images property

ProductCopy heading anchor to clipboard

This schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The available properties are as follows:

  • Name
  • Description
  • SKU
  • Images
  • Brand or Organization
  • Reviews
  • Aggregate Rating
  • Offers or Aggregate Offer
  • GTIN
  • GTIN-8
  • GTIN-12
  • GTIN-13
  • GTIN-14
  • MPN

product properties

You can add multiple Images and Reviews properties; simply click the plus icon next to the trashcan icon when hovering over the row.

Add images or reviews properties to Product schema

The Brand property can be switched to an Organization property by clicking the arrows icon in its row. You also switch Offers to Aggregate Offer by clicking the arrows icon in its row.

Offers are typically associated with singular items/ products, whereas an Aggregate Offer can be used with variable products or when a single product has multiple offers associated with it. If we use price as an example, the Offers property would be associated with a product that shows only one price and the Aggregate Offer property would be associated with a product that shows multiple prices.

Switch property types in SmartCrawl Schema

HowToCopy heading anchor to clipboard

Deprecated results type

Please note that Google has deprecated the HowTo schema results type as of September 14, 2023 and it no longer appears in rich search results. While you can remove this structured data from your site, there’s no need to proactively do so. Structured data that’s not being used does not cause problems for Search, but also has no visible effects in Google Search. See this Google help article for details.

The HowTo schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • Name
  • Description
  • Total Time
  • Images
  • Supplies
  • Tools
  • Estimated Cost
  • Steps

schema-types-howto

Note that you can have multiple Images, Supplies, Tools, and Steps properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

schema-types-howto-images

EventCopy heading anchor to clipboard

The Event schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • Name
  • Description
  • Start Date
  • End Date
  • Event Attendance Mode
  • Event Status
  • Images
  • Location
  • Organizer
  • Performers
  • Aggregate Offer

schema-types-event

Note that you can have multiple Images and Performers properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

schema-types-event-images

The Aggregate Offer can be switched to an Offers property by clicking Switch Type. See the Product schema type for more information on Aggregate Offer and Offers properties.

FAQPageCopy heading anchor to clipboard

Modified results type

Please note that Google has modified the FAQPage schema results type as of August 8, 2023. Rich search results for this type will only be shown for well-known, authoritative government and health websites. For all other sites, this rich result will no longer be shown regularly. See this Google help article for details.

The FAQPage schema type offers a single property that requires both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default property is:

  • Questions

You can have multiple Questions properties; simply click the plus icon next to the trashcan icon when hovering over the row.

schema-types-faqpage

WooCommerce ProductCopy heading anchor to clipboard

The WooCommerce Product schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property.

Singular Schema

When a WooCommerce Product schema is used, it will replace the WooCommerce default schema to avoid duplicate product schema on the same product pages. This is to ensure that the Rich Results remain properly intact.

When adding a WooCommerce Product type, you’ll have the option to select either a Simple Product or Variable Product subtype. You can skip this step to use the generic WooCommerce Product type instead.

Select subtype for new WooCommerce Product type in SmartCrawl

The default properties are as follows:

  • Name
  • Description
  • SKU
  • Images
  • Brand or Organization
  • Reviews or WooCommerce Reviews
  • Aggregate Rating
  • Offers or Aggregate Offer
  • GTIN
  • GTIN-8
  • GTIN-12
  • GTIN-13
  • GTIN-14
  • MPN

wooproduct properties

You can add multiple Images, Reviews and Offers properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

Add images, reviews or offers properties to WooCommerce Product schema

By clicking the arrows icon in the corresponding row, Reviews can be switched to WooCommerce Reviews (which are repeated for each product review), Brand can be switched to Organization, and Offers can be switched to Aggregate Offer.

Switch property types in SmartCrawl Schema

See the Product schema type for more information on Aggregate Offer and Offers properties.

Local BusinessCopy heading anchor to clipboard

When adding a Local Business schema type, you’ll have the opportunity to specify the kind of business you want to configure. Select the business subtype that most closely matches your business, or skip this step to use the generic type.

Select subtype for new Local Business schema type in SmartCrawl

As previously noted in the Add new schema type section of this document, many of the Local Business subtypes will have an additional step where you can further refine the business type. For example, the Food Establishment subtype will give you the opportunity to specify your business as any of the following (skip this step to use the generic type):

  • Bakery
  • Brewery
  • Distillery
  • Ice Cream Shop
  • Winery
  • Bar or Pub
  • Cafe or Coffee Shop
  • Fast Food Restaurant
  • Restaurant

Food Establishment schema subtypes

Note that when you create a Local Business type, a notice will pop up on your screen to remind you that you can easily duplicate it if you have a business with multiple locations and want to include schema for each location.

Notice about Duplicate type option

Once you have configured things for the first location, click the Duplicate option under the gear icon as noted in Configure schema types. Then edit the properties of the duplicate for the next location.

The Local Business schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • @id
  • Name
  • Logo
  • URL
  • Price Range
  • Telephone
  • Currencies Accepted
  • Payment Accepted
  • Address
  • Images
  • Aggregate Rating
  • Geo Coordinates
  • Opening Hours

Local Business schema default properties

Note that the @id property is a globally unique ID of the specific business location in the form of a URL. The ID should be stable and unchanging over time. Google Search treats the URL as an opaque string and it does not have to be a working link. If the business has multiple locations, make sure the @id is unique for each location.

Some Food Establishment subtypes, like Restaurant, will also include additional properties enabling you to further enhance the information users see when your business appears in their search results.

  • Accepts Reservations
  • Menu URL
  • Serves Cuisine

Local Business Restaurant schema additional properties

For Currencies Accepted and Payment Accepted sources, you can add multiple values by clicking in the corresponding fields and selecting the values needed.

Local Business schema payment and currency properties

You can have multiple Address, Images and Opening Hours properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

Local Business schema additional properties

If you wish to include Geo Coordinates for your local business type, you can get the needed coordinates easily by following the directions in this Google help article.

Local Business schema geo-coordinates properties

RecipeCopy heading anchor to clipboard

The Recipe schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • Name
  • Date Published
  • Description
  • Recipe Category
  • Recipe Cuisine
  • Keywords
  • Prep Time
  • Cook Time
  • Total Time
  • Calories Per Serving
  • Recipe Yield
  • Images
  • Ingredients
  • Instructions
  • Author
  • Aggregate Rating
  • Reviews
  • Video

Recipe schema type default properties

Note that you can have multiple ImagesIngredientsInstructions, and Reviews properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

Recipe schema type default properties

You can switch Instructions to Instruction HowTo Steps and Author to Author Organization by clicking the arrows icon. You can switch back at any point by clicking the same arrows icon.

Recipe schema type default properties

Job PostingCopy heading anchor to clipboard

The Job Posting schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • Title
  • Description
  • Date Posted
  • Valid Through
  • Employment Type
  • Job Location Type
  • Education Level
  • Month of Experience
  • Experience In Place Of Education
  • Hiring Organization
  • Job Locations
  • Applicant Location Requirements
  • Base Salary
  • Identifier

Job Posting schema type default properties

Note that you can have multiple Job Locations, and Applicant Location Requirements properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

Job Posting schema type default properties

MovieCopy heading anchor to clipboard

The Movie schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • Name
  • Release Date
  • Images
  • Director

Note that you can have multiple Image properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

Movie schema type default properties

CourseCopy heading anchor to clipboard

The Course schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • Name
  • Description
  • Course Code
  • Number of Credits
  • Provider

Course schema type default properties

BookCopy heading anchor to clipboard

The Book schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • @id
  • Name
  • URL
  • Authors
  • Same As
  • Editions

Note that the @id property is a globally unique ID for the book in URL format. It must be unique to your organization. The ID must be stable and not change over time. URL format is suggested though not required. It doesn’t have to be a working link. The domain used for the @id value must be owned by your organization.

You can have multiple AuthorsSame As, and Editions properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

Book schema type default properties

Software ApplicationCopy heading anchor to clipboard

When adding a Software Application schema type, you’ll have the opportunity to specify the kind of application you want to configure. Select the application subtype that most closely matches your application, or skip this step to use the generic type.

Select subtype for Software Application schema type in SmartCrawl

The Software Application schema type offers a set of properties that require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property. The default properties are as follows:

  • Name
  • Description
  • URL
  • Application Category
  • Operating System
  • Screenshots
  • Price
  • Aggregate Rating
  • Reviews

Software Application schema type default properties

Note that you can have multiple Screenshots and Reviews properties; simply click the plus icon next to the trashcan icon when hovering over the corresponding row.

Software Application schema type default properties

Custom TypeCopy heading anchor to clipboard

Building Your Own Schema Type

For a quick and easy guide to creating your own custom schema, see How to Use SmartCrawl’s Free Custom Schema Type Builder on the blog.

This schema type enables you to build your own custom schema and add custom properties. The set of properties that you add require both a source and a value. The source refers to the general location of the data and the value refers to the specific piece of information that will be assigned to that property.

Once you have added the custom type, you will see the @type property. In the corresponding Value field, enter the name of the schema type you are building—for example, JobPosting. You can find a list of all valid schema types here.

Add the schema type value

To add a property, click the Add Property button. You can choose between SimpleNested, or Collection property structures.

Simple StructureCopy heading anchor to clipboard

When adding a Simple property, enter the property name in the Property field, add a descriptive name for it in the Label field, then click the Add button. For example, if we want to add the employer overview property, the Property field value will be employerOverview, and the label will be Employer Overview.

Adding a simple property

IMPORTANT

Make sure to add a valid schema property name in the property field for the schema to work correctly. You can find all schema property names on schema.org.

We also highly recommend checking Google’s structured data documentation to see which properties are required and which are recommended.

Nested StructureCopy heading anchor to clipboard

The Nested property structure allows you to add a main property that groups related properties together. For example, the JobPosting baseSalary property groups the currencymaxValue, and minValue properties.

When adding a nested property, enter the property name in the Property field, add a descriptive name for it in the Label field, enter its schema type in the Type field, then click the Add button.

Adding a nested property

To add a nested property, click the Add Property button under the main property that you created.

Adding a child property

In the Add Property modal window, enter the property name and label, then click the Add button.

Adding child property name and label

This is how the nested property will look like in your custom schema.
Nested property in custom schema

Repeat the same steps to add more properties to your custom schema. Note that the nested property can have a nested or collection structure.

Collection StructureCopy heading anchor to clipboard

The Collection structure is similar to the Nested structure. The only difference is that the Collection structure allows you to add multiple instances of the property. For example, you can add review as the main property that nests the itemReviewedreviewBody, and reviewRating properties. You can have multiple Review properties by clicking the plus icon next to the trashcan icon when hovering over the row.

Duplicating a collection property

Once you are done adding the properties, make sure to set each property’s Source and Value.

Setting the properties source and value fields

Remember to click Save Settings when you’re done.

Adding Image PropertiesCopy heading anchor to clipboard

When adding an image property to a custom schema type, you have many options to use as the source. Among them, you’ll find Custom Image and Custom Image URL.

Both options would add the same image data to all instances of your custom schema type as both require you to upload a single image as the value. However, they do not output the same data.

If you need only the image URL in your schema output, select Custom Image URL as the source. Even though you upload an image, only the image URL will be included in the schema.

You can also use the Custom Text option for this, and simply enter the URL of the image you want to use.

See under the Structure tab in Example 1 here: https://schema.org/image#eg-0011

If you want all image data, including EXIF, to appear in the image schema output, select Custom Image instead. See under the Structure tab in Example 2 here: https://schema.org/ImageObject#eg-0404

Note that if the Location of your custom type is a Post Type, and you want the data of the featured image attached to each post to be included in your schema output, you would select Post Data as the source. Then use either the Featured Image or Featured Image URL options as the value. The differences in their output is similar to the above, but would include data for the image attached to each post in the schema output.

Mapping Property Values to Custom Post MetaCopy heading anchor to clipboard

Regardless of the schema type, most properties that you add will require both a source and a value that is mapped to that source. Both of those have defaults, but they can be customized per your specific needs by mapping the values to any available post meta data.

Jobs directory exampleCopy heading anchor to clipboard

As an example, let’s say you’re creating a Jobs Directory with offers from various employers using the built-in Job Posting schema type.

Job Posting schema properties

The numerous properties included in this type can be left as they are if you’re only posting a single job offer on your site; simply select or enter the property value needed for each source.

But if you’re creating a directory, then you’d want to automatically pull that data from the individual job postings, so each one will have distinct schema values.

Every property in every schema type has the option to use Post Meta as the source for any value. You can select the meta key of any existing post meta, or create your own custom fields and map the values to those field keys.

The image below illustrates an example where custom fields have been created using the popular Advanced Custom Fields plugin, and assigned to a Job custom post type.

Custom fields created for Job Posting schema

The Job Posting schema type properties are then mapped using Post Meta as the sources, and the names of the ACF custom fields as the values.

Custom fields in post meta for Job Posting schema

Those fields are then available in each Job posting where the post author enters the relevant information.

Custom fields in post for Job Posting schema

Once the job posting is published, the custom values entered for the property sources can be verified by clicking the Test Schema button on the published post.

Rich results for Job Posting schema

Cooking site exampleCopy heading anchor to clipboard

Another example could be a cooking site where each recipe has several ingredients and preparation steps.

You cannot simply list all the ingredients or steps as comma-separated strings in single custom fields, and map them to single Ingredients or Instructions properties. Doing that would output the whole string as a single value, and it would not pass the Rich Results test.

So, as above, you would create as many custom fields as needed for each ingredient or preparation step, and assign those custom fields to the post-type being used for your recipes. They could be named simply ingredient-1, ingredient-2, ingredient-3, step-1, step-2, step-3, and so on. Go ahead and create more custom fields than you might need for any one recipe; any empty values will simply not appear in rich results.

Then add the same number of property instances as you have custom fields, and map each instance to a specific custom field.

Custom fields in post meta for Recipe schema

Then when a recipe post is authored, the ingredients and steps are entered into the respective custom fields in the post, and the corresponding values for each would appear in the schema as distinct properties & values.

Rich results for Recipe schema

3.6.4 Settings (Schema)

Link to chapter 6

Test SchemaCopy heading anchor to clipboard

SmartCrawl has a handy button in the adminbar on all frontend pages so you can easily test the schema output of any page using Google’s test tools.

SmartCrawl Test Schema button enabled

You can disable this option if you don’t want or don’t need that tool.

Enable test button in SmartCrawl Schema options

DeactivateCopy heading anchor to clipboard

If, for any reason, you do not wish to include any schema markup in any of your site’s content, click this button to deactivate the feature and remove it entirely.

schema-advanced-deactivate

3.6.5 Video Schema

Link to chapter 6

This chapter provides guidance to help ensure self-hosted videos embedded in your content include all VideoObject schema and pass Google’s Rich Results Test.

You’ll first want to ensure that you’re using the Gutenberg video block to add your videos, and not the old video embed shortcode used in the Classic Editor. All you need to add for each video is a description and a poster image.

Add a Poster ImageCopy heading anchor to clipboard

When you click on the video block in the editor, you’ll see an option to add a poster image. Simply click on the button under Poster image and upload or select the image you want to use.

Add a poster image to an embedded video

Add a DescriptionCopy heading anchor to clipboard

To get a description added to your video, you’ll first want to add a caption to it. You can add the caption directly in the editor when embedding the video if you want it to also display under the video on your page. Or click the Edit link for the video in your media library and add the caption there instead if you don’t necessarily want it to show on the frontend.

Adding a caption to a video in the media library

Then you just need to tell SmartCrawl to use the video caption for the description.

Go to SmartCrawl > Title & Meta > Post Types > Media. In the field for the Description, enter just the following macro: %%caption%%

Now when you check source code for your page, you’ll see all the VideoObject schema for your embedded video, including the description and poster image you just added, and can double-check that it passes Google’s Rich Results Test.

Video schema in Google Rich Results Test

Still Using the Classic Editor?Copy heading anchor to clipboard

If you are still using the Classic Editor on your site, you can always use Gutenberg’s wp-block-video code when editing your post in Text mode. That block looks like this:

<figure class="wp-block-video"><video controls poster="https://yoursite.tld/wp-content/uploads/path-to-your-poster-image.png" src="https://yoursite.tld/wp-content/uploads/path-to-your-video.mp4"></video></figure>

Replace the URL for the poster attribute with the full URL to the image you want to use as the video poster image.

Replace the src URL with the full URL to your video from your media library.

This section enables you to control and optimize how your website content appears when shared on social platforms like Facebook and Twitter.

3.7.1 OpenGraph

Link to chapter 7

The OpenGraph section allows you to totally disable/enable OpenGraph for your site.

OpenGraph support in SmartCrawl

Don’t forget to click Save Settings once you are done.

3.7.2 Twitter Cards

Link to chapter 7

Here you can set your Twitter Card. This controls how your content looks when it is shared on Twitter. You can choose No Image to show it as a Summary Card:

twitter cards no image preview

Or you can choose Image to set it to display a large image:

twitter cards image preview

As stated before, to setup your HomePage Twitter cards read the Home Page section.

3.7.3 Pinterest verification

Link to chapter 7

Here, you can verify your website with Pinterest to attribute your website when your content is pinned.

Go to: https://www.pinterest.com/settings#claimWebsite, enter your website URL in the designated line and press Claim website.

Claim your site at Pinterest

This will present you with a pop-up window:

Claim options at Pinterest

Select “Add HTML Tag” here, copy the presented tag and paste it in the appropriate field on your site within Social/Pinterest verification.

Once you do, press Save Settings. This will add the appropriate meta tag in the <head> of your site, and you will be presented with something like:

Pinterest verified

After verification, navigate to the Pinterest tab, click Next and then Submit.

Submit for review at Pinterest

Don’t forget to save the settings on Pinterest as the last step (most of the time your site will be connected instantly and you will receive an email immediately).

Save settings at Pinterest

3.7.4 Settings (Social)

Link to chapter 7

Click the Deactivate button if you don’t need the Social features on your site.

Deactivate Social feature in SmartCrawl

Automatically generate detailed sitemaps to tell search engines what content you want them to crawl and index.

Google XML Sitemaps vs SmartCrawl Sitemaps

Curious to see how SmartCrawl’s Sitemap feature stacks up against the competition? Check out our blog post on Google XML Sitemaps vs SmartCrawl Sitemaps.

3.8.1 General Sitemap

Link to chapter 8

Here you can see basic information and any issues SmartCrawl found during Sitemap scan. You can also turn off the automated sitemap feature, run a new crawl or view your sitemap (by clicking on the link). Your sitemap is located in public_html/wp-content/uploads or public_html/wp-content/uploads/sites/number for each subsite separately on a Multisite.

If you would prefer to customize the native WordPress core sitemap, you can do so by clicking the Switch button. Note that if you switch to WordPress core sitemap, SmartCrawl sitemap will be disabled. It is also worth keeping in mind that using the SmartCrawl map comes with many advantages, such as:

  • It is cached for performance.
  • It includes images from post content.
  • The ability to add styling to the sitemap.
  • The possibility of automatically updating the sitemap.
  • The option to auto-notify search engines.

switch to wp core button

You will be asked to confirm the switch. Click Switch to continue with the switch to WordPress core sitemap or click Cancel to exit without switching.

confirm switch to wp core

You can switch back to SmartCrawl sitemap at any point by clicking the Switch button again.

switch to smartcrawl sitemap button

Below, you can set up what you would like your sitemap to include (everything is enabled by default) and you can even enter extra URLs, in the Inclusions field, to manually add ones that are not located in your sitemap.

include groups to sitemap or individual urls

Pro Tip

If you disable all content types here and don’t manually include any URLs, your sitemap will effectively be empty and will simply return a 404 page not found error. See Additional Troubleshooting Options (Sitemap) for details.

You can also explicitly exclude URLs or post IDs from your Sitemap in the Exclusions section.

Exclude URLs or post IDs from SmartCrawl sitemap

Don’t forget to click Save Settings once you are done 🙂

3.8.2 News Sitemap

Link to chapter 8

If your site is publishing newsworthy content, you may want to enable and generate a Google News Sitemap to ensure that your news articles and posts published in the last 48 hours show up in Google News.

Why use a News Sitemap?

For more insight into News Sitemaps and how they can help boost your content, see how to Make Headlines on Google News with SmartCrawl’s Free Google News Sitemap on the blog.

To enable the Google News Sitemap, click Enable.

Enable Google news sitemap

Enter your Google News publication name, matching the name as it appears on news.google.com.

Set Google news publication name

Select the Post Types that should be included in your Google News Sitemap. To include specific items or groups, expand the post type first.

Select post types to include in Google news sitemap

After creating your News Sitemap, submit it through Google’s Search Console to tell Google where to find it on your site.

Automatic NewsArticle SchemaCopy heading anchor to clipboard

Any posts that you add to your News Sitemap that do not already have any specific schema types mapped to them will be automatically switched to the NewsArticle schema type, with the corresponding schema added to the post.

However, if you have already assigned a schema type to certain posts that you include in your News Sitemap, or have excluded certain categories from the News Sitemap, those posts will not be switched to the NewsArticle type.

NOTE

Your News Sitemap will only include content published in the last 48 hours. If no content has been published within the last 48 hours, then any request for your sitemap will return an HTTP 404 error.

3.8.3 Multilingual Sitemaps with WPML

Link to chapter 8

The WPML plugin offers different formatting options for your language URLs. These options are:

  • Different languages in directories
  • The language name as a parameter
  • A different domain per language

When using the different languages in directories or the language name added as a parameter language URL option, SmartCrawl creates a single sitemap that includes your default and secondary languages site pages.

On the other hand, when using the different domain per language option, SmartCrawl creates a sitemap for each language (domain) you have on your site. You can view the secondary languages’ sitemaps by going to languagedomain.com/sitemap.xml or languagedomain.com/?wds_sitemap=1&wds_sitemap_type=index if you are using the Plain permalink structure on your site.

3.8.4 Crawler

Link to chapter 8

The Crawler will detect issues and URLs that are not on your sitemap. It will go through your site, starting at the homepage, and follow any and all links it can find, up to a maximum of 500 distinct URLs. Note that this is simply a reporting feature, it does not affect sitemap creation in any way.

Click New Crawl in the top right-hand corner to start a new sitemap crawl.

start new crawl

A progress bar will be visible during the crawl to keep you in the loop of the duration.


Once the crawl has finished, it will display any issues and URLs that are not on your sitemap. We recommend fixing them to ensure you aren’t penalized by search engines – but if you want to ignore any of the warnings you can.

Url-crawler-missing-from-sitemap

You can Add or Ignore presented URLs here or bulk Ignore/Add them all by pressing the corresponding buttons.

Note that once a scan has completed, whether initiated from either SmartCrawl or the site’s SEO tab in The Hub, you must wait at least 1 hour before running a new scan.

3.8.5 Reporting (Sitemap)

Link to chapter 8

Enable the Run regular URL crawls option to have SmartCrawl automatically crawl your URLs daily, weekly or monthly and send an email report to your inbox. You can add as many email recipients as you need by clicking the Add Recipient button and filling in the popup form for each one.

sitemap-reporting

3.8.6 Settings (Sitemap)

Link to chapter 8

Sitemap StructureCopy heading anchor to clipboard

Your sitemap will be automatically split into multiple sitemaps; one per post type. Each one will contain up to 2000 URLs, but you can change the maximum number of items that should be included in each one here.

Set the number of links per post-type sitemap

If the total number of URLs of any post type exceeds the number you set here, additional sitemap pages will be created and numbered automatically, following this format:
https://example.com/post-sitemap1.xml
https://example.com/post-sitemap2.xml

Include imagesCopy heading anchor to clipboard

Here you can include images within your sitemap. For this to properly function make sure to add titles and captions that clearly describe your images.

Include image items in SmartCrawl sitemap

Note that plugin memory consumption will considerably increase if you enable this. How much depends on how much image content you have, as well as your server configuration & capabilities.

Auto-Notify Search EnginesCopy heading anchor to clipboard

Auto Notify does exactly that – auto notifies search engines (specifically Google and Bing) that your sitemap has changed. You can choose between the following two modes:

  • Default – You don’t need to do anything; search engines will automatically be notified when your sitemap changes.
  • Manual – This allows you to manually trigger notifying search engines. Click Notify Search Engines whenever you want them to be notified that your sitemap has changed.

auto-notify search engines

Style SitemapCopy heading anchor to clipboard

Enabling the Style sitemap option will make your sitemap easier to read (for human eyes).

Add stylesheet to SmartCrawl sitemap

For example, this is a sitemap before enabling it (you can access your sitemap by going to yourdomain.com/sitemap.xml or http://yourdomain.com/?wds_sitemap=1&wds_sitemap_type=index if you are using the Plain permalink structure on your site)

SmartCrawl sitemap without stylesheet

And the same sitemap after:

Automatic Sitemap UpdatesCopy heading anchor to clipboard

You can choose whether you want to automatically update your sitemap when you publish new pages, posts, post types, or taxonomies. The three available modes are:

  • Default – You don’t need to do anything; your sitemap will automatically be updated.
  • Manual – This allows you to manually trigger a sitemap update. Click Update Sitemap whenever you want to update your sitemap after publishing new pages, posts, post types, or taxonomies.
  • Scheduled – This allows you to schedule your sitemap updates. Sitemap updates can be scheduled on an hourly, daily, or weekly basis. For daily updates, you can set the time of the day and for weekly updates, you can set the day of the week at which the update should be triggered.

Troubleshoot SitemapCopy heading anchor to clipboard

This tool automatically detects and resolves common sitemap problems. It also provides suggestions to fix issues caused by sitemap conflicts that you need to fix manually. Click the Troubleshoot button to open the Troubleshoot Sitemap modal window.

Troubleshoot Sitemap button

Click the Start button.

Troubleshoot sitemap Start button

A progress bar will be visible during the scan for sitemap issues. This process usually takes a few seconds.

When the discovered issues are automatically fixed, or no issues were detected after scanning the sitemaps for conflicts and issues, a green checkmark will appear at the top of the modal window.

Green checkmark appears when issues are resolved automatically or no issues found

red cross-mark will be displayed on the top of the modal window if any issue is detected. You will see one of the issues below that requires fixing manually:

    • Plugin Conflict – This issue indicates that you are using another plugin that generates sitemaps, causing conflicts with SmartCrawl sitemaps. Click the Go To The Plugins Screen button to deactivate the plugin causing the issue.

Troubleshoot sitemap plugin conflict

    • File Conflict – This error is displayed when you have a physical file on your server conflicting with SmartCrawl sitemaps. To fix the issue, delete the file causing the error.

Troubleshoot sitemap file conflict

    • Permalink Problem – This error shows when pretty permalinks don’t work for your sitemaps. The troubleshooter first tries to fetch the sitemap through the pretty permalink. If it does not get valid XML, it then checks the plain version of the sitemap URL. If that works, it assumes that pretty permalinks are not working. You need to manually include some rewrite rules to your server’s configuration files, or exclude the sitemap URL from caching to fix this issue. For instructions, see the Additional Troubleshooting Options (Sitemap) section.

Troubleshoot sitemap permalink problem

    • Incorrect Permalink Settings – This error indicates that you are using a permalink structure that is preventing SmartCrawl sitemaps from working properly. To fix this issue, click on the Go To Permalink Settings to change the permalink structure.

Troubleshoot sitemap incorrect permalink settings

Once you have resolved any of the sitemap issues and conflicts that may occur, click the Check Again button to ensure that the issue is resolved and there are no other pending issues.

3.8.7 Additional Troubleshooting Options (Sitemap)

Link to chapter 8

Sitemap 404Copy heading anchor to clipboard

No ContentCopy heading anchor to clipboard

If your site only has a homepage with no other content, the sitemap.xml will return a 404 error as it is effectively empty.

To resolve that, simply add a / character in the Inclusions box and the sitemap should load as expected.

If your site does have content other than just the homepage, check to ensure that at least one content type is active in the Include section.

NGINXCopy heading anchor to clipboard

If your site is running on a NGINX server and your sitemap URLs are returning 404, try adding the following rewrite rules to your nginx.conf file:

rewrite ^/sitemap.xml$ /index.php?wds_sitemap=1&wds_sitemap_type=index last;
rewrite ^/([^/]+?)-sitemap([0-9]+)?.xml$ /index.php?wds_sitemap=1&wds_sitemap_type=$1&wds_sitemap_page=$2 last;

Remember to resave permalinks after making this change.

Sitemap CachedCopy heading anchor to clipboard

If you are using Page Caching in Hummingbird, ensure that the following is present in the URL Strings section of the Page Caching Exclusions:

sitemap[^\/.]*\.xml

Crawler User AgentCopy heading anchor to clipboard

If you ever need to allowlist the user agent used by SmartCrawl’s Crawler feature in your robots.txt file or a firewall, it is called: SmartCrawl SEO Audit 1.0

No Meaningful ResultsCopy heading anchor to clipboard

Make sure that your site is not blocked somehow. For example, the robots.txt or a firewall could be blocking SmartCrawl.

As noted above, the Crawler user agent is: SmartCrawl SEO Audit 1.0

Also make sure WPMU DEV IP addresses are allowlisted. You’ll find our IPs at WPMU DEV IP Addresses

URLS Can’t be FoundCopy heading anchor to clipboard

Recrawling a site doesn’t have any effect on the sitemap itself. The crawler is an external service that just checks your sitemap by looking at it from the outside as Google would.

However, if you get a message like “xxx URLs can’t be found“, try recrawling the site as there may have been some connection hiccup the first time around.

Crawler Timeout or Crawl StuckCopy heading anchor to clipboard

This can happen when the external servers responsible for crawling your website are facing issues.

Please contact support so they can try resetting the crawler using our backend tools.

3.8.8 Crawling Your Site in Google Search Console

Link to chapter 8

Getting your site or specific URLs crawled and indexed in Google Search Console is a relatively simple process, but it can take anywhere from a few days to weeks according to Google’s own help documentation.

This chapter aims to make getting that done a little easier for you.

First, get your property created and verifiedCopy heading anchor to clipboard

The first thing you need to do is add your site as a new property if you haven’t already done that.

Log into Google Search Console and, in the Search property box, click the + Add Property option.

Add a property in Google Search Console

In the modal window that pops open, you’ll be prompted to select one of two ways to verify that you own the domain you want to use:

  • Verify the domain using a DNS record
  • Verify the domain using alternate methods like a meta tag

As you are using SmartCrawl, let’s do it the easy way and use the 2nd option.

Enter the full URL to your site’s homepage, including the https:// part, in the URL prefix box. Then click Continue.

Add a doman in Google Search Console

In the next modal window that pops open, click the HTML tag option to expand it, and click the Copy button to copy the meta tag you need to verify ownership of your site.

Copy meta tag from Google Search Console

Now, log into your site admin in a new tab, and go to SmartCrawl > Settings. Scroll down to the Search Engines section, and paste the meta tag you just copied into the Google Verification field there. Then click Save Settings.

Paste meta tag from Google Search Console

Next, back in Google Search Console, click the Verify button where you copied the HTML tag to verify your site.

Verify meta tag in Google Search Console

You should see a nice Ownership Verified success message. Click the Go To Property link and proceed to the next step.

Property verified in Google Search Console

Add your sitemap URL to get your site crawledCopy heading anchor to clipboard

Click on the Sitemaps option in the left navigation.

Add a sitemap in Google Search Console

In the Add a new sitemap field, enter the slug of your sitemap’s index page, and click the Submit button.

  • If you are using SmartCrawl’s sitemap, enter sitemap.xml
  • If you had switched to use WP core sitemaps, enter wp-sitemap.xml

Add a sitemap URL in Google Search Console

Once the sitemap URL has been processed, you can click on it to view the individual sitemaps included in it.

New sitemap processed in Google Search Console

Request indexing for new or changed pagesCopy heading anchor to clipboard

Occasionally, there may be URLs of pages that are in your sitemap, but that are not showing up in search results. Or perhaps content has been updated, but those changes aren’t reflected in search results.

Fortunately, there’s a handy tool in Google Search Console to get that fixed.

Enter any site URL in the Inspect any URL box at the top of the screen to get a detailed report of how Google sees that URL.

If the page has already been indexed, you’ll see a green checkmark and confirmation that the URL is on Google. Expand any sections in the URL Inspection report to get as much detail as you need about how Google sees your page.

URL confirmed indexed by Google

If the URL has not been indexed, click on the Coverage section to gain insight into why it’s not.

URL confirmed not indexed by Google

If the data of an existing URL appears outdated, perhaps after you’ve edited your content, or if you need to index a missing URL, you can do that by clicking the Request Indexing link.

Request indexing of a page in Google Search Console

You’ll then see a confirmation that Indexing has been requested. Note once again that this process can take some days or weeks to complete.

Indexing of a page requested in Google Search Console

For more information about how these things work in Google Search Console, see Google’s help article here.

Advanced tools focus on the finer details of SEO including internal linking, redirections and Moz analysis.

3.9.1 Automatic Linking

Link to chapter 9

This section lets you set specific keywords to always link to content on your site, or even a different site altogether.

For example, you could set your site to link to https://wordpress.org/news/ any time you type ‘WordPress news’. Without this plugin, you would have to manually create these links each time you write the text in your pages and posts – this can save you a bunch of time.

Start by enabling the module:

Post typesCopy heading anchor to clipboard

Here you can select the post types for which you want to enable auto-linking and post types or taxonomies that can be linked to.

Automatic linking feature

The Insert Links option enables you to select exactly which post type(s) the plugin should automatically insert links in. Every post type active on your site will be available for keyword linking here.

The Link To option lets you specify the post types & taxonomies that can be automatically linked to. For example, if you have a “Really Cool Stuff” category on your site, the plugin could automatically link to that category archive any time it finds “really cool stuff” on your site.

The Custom Links enables you to really tweak things. If there are any keywords or key phrases that you want to automatically link to specific URLs, you can enter them here.

Auto-link custom keywords in SmartCrawl

Click the Add Link button. This will pop open a ‘Add Custom Keywords’ modal. Enter the custom keyword(s) in the Keyword Group field (separated by commas) and the URL you want the keywords to link to in the Link URL field.

Configure custom keywords to auto-link in SmartCrawl

Click Add to add the entered custom keywords and links. Click the gear icon to the right of any custom keywords list item to reveal additional options to edit or remove the added custom keywords.

Edit custom keywords to auto-link in SmartCrawl

ExclusionsCopy heading anchor to clipboard

The Exclusions setting enables you to enter any keywords that the plugin should not use for links. This can be very handy if you notice certain words or phrases are linking to places you don’t want them to or if there are certain areas of your site you don’t necessarily want to be linked to (the “Uncategorized” category for example).

Advanced links exclusion

Excluded keywordsCopy heading anchor to clipboard

Enter the keyword(s) you want to exclude from auto-linking in the input field (separated by commas).

Excluded posts/pages/mediaCopy heading anchor to clipboard

You can also specify Posts, Pages, or Media files you want to exclude from auto linking by pressing the Add Exclusion button.

This will pop open a ‘Add Exclusion’ modal where you can select specific posts, pages or media files to be excluded from auto-linking. Click Add to add the selected posts/pages/media to the exclusions.

All the added exclusions are listed here. Click the Trash icon to the right of any added exclusion to remove it.

Remove excluded posts from auto-linking

Min LengthsCopy heading anchor to clipboard

Sets the minimum length for your post types and taxonomies. Just enter the desired number and all posts and/or taxonomies with titles/length shorter than that number will be ignored.

Set minimum length for auto-links in SmartCrawl

Max LimitsCopy heading anchor to clipboard

Max Limits enables you to setup a maximum number of links per single post or you can limit a total number per keyword group that will appear on your site.

Set maximum number of auto-links per post or keyword in SmartCrawl

Per Keyword Group example: If you want a keyword to link to mysite.com/smartcrawl page and you set this to 10, it will link up to 10 times in total.

Optional SettingsCopy heading anchor to clipboard

These options enable you to set up everything down to the smallest detail. Let’s go over each one.

Optional auto-linking settings in SmartCrawl

Sometimes you want to have your autolinks point to empty taxonomies. You can use this for a number of reasons:

  • Affiliate links (nofollow, noindex).
  • Deep linking into your archive content  (best use).
  • Cross-linking to partner sites (with caution).

For example, you have a Category “Cats” but you still did not write anything about Cats – this will help you set links for future posts.

Prevent Linking in Heading TagsCopy heading anchor to clipboard

This will prevent linking in heading tags. No worries, h1, h2, h3 and h4 tags are exempted by default, so this setting only affects h5 and h6 headings.

Process Only Single Posts And PagesCopy heading anchor to clipboard

Enabling this option will ensure that autolinking does not occur in places like archives, post excerpts or search results pages.

Process RSS FeedsCopy heading anchor to clipboard

Process RSS Feeds will ensure that links are automatically included in your RSS feed.

Case Sensitive MatchingCopy heading anchor to clipboard

Case Sensitive Matching will ensure that links are automatically created only if uppercase and lowercase spelling is an exact match.

Prevent Duplicate Links ensures that only the first occurrence of any matched text in any post will be linked.

This ensures that the target=“_blank” attribute is added to all automatic links so they open in new tabs when clicked.

By default, search engine crawlers follow each link on your site until all the site is crawled. This can be used to stop them from following dynamic URLs that lead to the same/similar content on your site or if you do not wish to share your link juice to external URLs.

Prevent linking on no-index pagesCopy heading anchor to clipboard

Enable this option to prevent automatic linking on pages that are set to no-index.

Prevent linking on image captionsCopy heading anchor to clipboard

Enable this to prevent links from being added in image captions.

Prevent caching for autolinked contentCopy heading anchor to clipboard

Enable this option to prevent object cache conflicts with some page builder plugins and themes when automatic linking is enabled.

As always, once you’re done – press the SAVE SETTINGS button.

In The Post EditorCopy heading anchor to clipboard

You can also manipulate the behavior of individual links in the Gutenberg post editor by adding rel attributes.

individual link settings

The following attributes can be applied to specific links:

  • Open in new tab – Marks the link to open in a new tab.
  • Sponsored link – Marks the link as a paid placement or an advertisement.
  • User generated content – It is recommended to mark user-generated content with this rel attribute. This is generally content that has been posted by users, like comments or forum posts.
  • Nofollow – Use this when you don’t want the linked page to be associated with your site or you’d prefer it if Google didn’t crawl the linked page.

FiltersCopy heading anchor to clipboard

By default, the maximum number of automatic links that the plugin will create in all post_types combined is limited to 2000.

You can override that default and set a custom limit by adding this filter to your child-theme’s functions.php file or a mu-plugin, and adjusting the value to what you need:

define( 'SMARTCRAWL_AUTOLINKS_GET_POSTS_LIMIT', 3000 );

3.9.2 URL Redirection

Link to chapter 9

Redirection enables you to forward one URL to another. It’s a handy way of sending both users and search engines to a different URL and allows you to preserve your search engine rankings for a particular page.

URL redirection in SmartCrawl

Use the dynamic Search field to locate specific redirects that you’ve already set up. Just start typing and the search will display all matches as you type.

To Edit or Remove an existing redirect, click the gear icon in its row and select the appropriate option.

Redirection is also a useful way to preserve the “link juice” of out-of-date content by redirecting old pages or posts to new ones with new information.

If you trash a published post or page, a notice will appear at the top of the All Posts/Pages screen to remind you to create a redirect for that old URL. Copy the slug shown in the notice, then click the Add redirection link to be taken to the Add a redirect screen. Note that a future update of the plugin will ensure the slug gets automatically added for you so you won’t need to copy it.

Notice to redirect a trashed post or page

Add a redirectCopy heading anchor to clipboard

To add a new redirect, click the Add Redirect button at the top of the screen to open the options modal.

Redirect TabCopy heading anchor to clipboard

Add a redirect in SmartCrawl

  • Redirect From – Enter the absolute or relative URL that you want to redirect from.
  • Redirect To – Enter the destination you want to redirect to. This can be a manually entered absolute or relative URL, or you can start typing the title of the page, post or custom post type you want, and select it from the results found.
  • Redirect Type – Select the type of redirect you want to apply if different from your Default Redirection Type.
  • Location-based Rules – This option enables you to selectively redirect users based on their geographical location.
Location-based RulesCopy heading anchor to clipboard

Enable this feature if you wish to add rules to redirect users based on their geographical locations. Note that this feature requires you to sign up for a MaxMind account. See Location-based Rules in the Settings chapter below for instructions on getting your Maxmind key.

Enable location-based URL redirection in SmartCrawl

Toggle on the option to Disable default Redirect To if you wish to redirect users based on their location only.

Then Click the + Add Rule button to add a new location rule.

Add a location-based URL redirection in SmartCrawl

  • Rule – Select whether you wish to redirect users who are From or are Not From the countries you specify.
  • Countries – Select the country or countries this rule should apply to.
  • Redirect To – Enter the URL where users who match this rule should be redirect to.

You can add more than one location rule for a redirect, and their precedence will follow the top-to-bottom order. The rule added first has higher precedence over the rule next to it.

If you add one or more conflicting location rules, a notice will appear to remind you so you can adjust.

Notice of conflicting location-based URL redirection rules in SmartCrawl

To delete an existing rule, click on the Trash icon next to it.

Advanced TabCopy heading anchor to clipboard

Advanced URL redirection options in SmartCrawl

Here you can add a label for the redirect and also choose how the redirect URLs should be treated.

  • Label – Add an optional label to your redirect if you have long URLs or several similar ones so you can easily tell them apart.
  • Regular Expressions – Select if the redirect from/to URLs should be treated as plain text or regular expressions. If your URL contains a regex pattern, be sure to select the Regex option. See About Regex Redirects below for more.

About Regex RedirectsCopy heading anchor to clipboard

SmartCrawl supports regex patterns in your redirects. Regex is short for regular expression, which is a handy shorthand that can be used to match numerous URLs using a single string of text.

IMPORTANT

Regex redirects must use absolute URLs, like https://yoursite.tld/cats/(.+). They will not work with relative URLs, like /cats/(.+)

To illustrate how powerful regex can be, let’s say you wanted to catch all URLs containing the distinct word cats anywhere in the URL, and redirect all of them to a /cats-new/ post instead. For example:
yoursite.tld/cats
yoursite.tld/old-cats
yoursite.tld/cats-here
yoursite.tld/old-cats/here
yoursite.tld/even-more/old-cats
yoursite.tld/still/more/cats/here/too

You could create a separate redirect for each one, but if you have dozens or hundreds of such URLs, that can get quite tedious. The easy alternative is to “catch” all of them with a regex pattern like so:

yoursite\.tld\/(?!cats-new\/|cats-new$).*\bcats\b.*

This pattern would match all of the above URLs. But it would not match any of the following URLs, as the letter combination c-a-t-s in each one is actually part of a different word.

yoursite.tld/ducats
yoursite.tld/catsup
yoursite.tld/scats

It would also not match the URL you’re redirecting to, which also contains the word cats, as that would cause an endless redirect loop:

yoursite.tld/cats-new

Here’s a quick breakdown of how the above regex works.

The first part of the expression checks if the URL is exactly where you want to redirect all your stuff (in our example, that’s /cats-new/) and if it is, do not redirect. That little bit of magic happens with a negative lookahead – ?! – to exclude that particular pattern.

(?!cats-new\/|cats-new$)

The second part of the expression says to match any combination of characters (the .* tokens) before and/or after the explicit word cats (indicated by enclosing the word in \b tokens).

.*\bcats\b.*

Note that regex is also supported in the target or destination URL that you’re redirecting to.

For example, let’s say you just changed a post category on your site from cats to felines, but have a ton of old cats links that your users have shared on social media.

You could create a redirect to catch all those inbound links, like so:
https://yoursite.tld/cats/(.+)

Where anything after cats in the URL is identified as a capture group by enclosing it in parentheses. The (.+) will match any number of any characters, so should catch all old links to posts in that old category.

Then redirect them to the corresponding new category URLs, like so:
https://yoursite.tld/felines/$1

The $1 token will match whatever is in the capture group in the old URLs.

If you’re new to regex and want to get a feel for the basics, see this excellent article: Regular Expressions for Non-Programmers.

We suggest using a handy tool like this one to help you build & validate your regex patterns: https://regex101.com/

Bulk OptionsCopy heading anchor to clipboard

You can also bulk delete or bulk update multiple redirects that you’ve already set up. To bulk update multiple redirects, select them from your list and click the Update button.

Bulk update URL redirects in SmartCrawl

In the modal window that pops open, enable any or all options corresponding to the values you wish to update.

  • Redirect URL – The new URL where all selected items will be redirected.
  • Redirect Type – The redirect type to apply to all selected items.
  • Location-based Rules – The location-based rules to apply to all selected items.

Bulk update URL redirects in SmartCrawl

The available bulk options function exactly as when adding a new redirect. See Add a Redirect above for details.

Note however that you cannot apply regex rules when bulk updating existing redirects. To apply regex rules, you would need to edit each redirect individually.

Import & ExportCopy heading anchor to clipboard

This feature enables you to export a JSON file with all your redirects, including location rules, and import it later on other sites where the same relative URLs are used.

Note that CSV files of redirects without location rules that were created in prior versions of the plugin can still be imported. But any new imports & exports with location rules must be in JSON format.

Importing and exporting URL redirects

To export your redirects, click the Export button at the top of the screen to download a file of all redirects from your site.

To import redirects into a site, click the Import button. In the modal window that pops open, click the Upload button and navigate to the file on your computer that you wish to import, then click Import.

Importing URL redirects

PaginationCopy heading anchor to clipboard

If you have more than 10 redirects set up, a pagination option will appear where you can select to display 10, 20, 50 or 100 redirects per page.

Pagination of URL redirections in SmartCrawl

Delete AllCopy heading anchor to clipboard

If you wish to delete all your redirects in a single click, click the Delete All button at top-right. A modal window will appear prompting you to confirm the action.

Delete all URL redirections in SmartCrawl

SettingsCopy heading anchor to clipboard

URL Redirection Settings in SmartCrawl

Redirect AttachmentsCopy heading anchor to clipboard

Did you know that each time you upload a file to the WordPress Media Library and choose Link to: Attachment Page, WordPress creates a separate media attachment page for every single file? This page contains nothing except the media content and has its own generated URL.

In most cases, this page isn’t particularly beneficial, which is why you might want to redirect WordPress attachments to their original files.

Separate media pages may work for photographers and graphic designers, as they help to create galleries but, for an average WordPress user, it makes sense to redirect WordPress attachments to their original files (improving SEO in the process).

Let’s say you create a post and add three images as Link to: Attachment Page to it. WordPress then automatically creates four URLs, three for the images, and one for the original post. This can hurt SEO in more ways than one:

  • Google may start bringing more traffic to the attachment pages instead of the original post to which they belong. Like 404 errors for example
  • This standalone attachment appears out of context, and a visitors landing on the image or attachment page are likely to close the link and move away.
  • It’s possible that Google may index all the image files and consider it as duplicate content.

Finally, there’s also a niche situation where access to content in pages and posts is restricted by a password. It may happen that someone shares your images on social media. By clicking on the image URLs, an unauthorized visitor may be able to access the media content within these posts or pages despite not knowing the password.

You can help your readers skip these attachment pages by redirecting them to the original attachment files by enabling the options here.

Keep in mind that this redirect option will only work if the media item was uploaded to the original post in the first place.

Redirect Image Attachments
You can also choose to redirect only the image attachments to their original files and other attachments to their respective parent posts by enabling the Redirect Image Attachments Only checkbox.

Default Redirection TypeCopy heading anchor to clipboard

Here you can select the redirection type that you would like to be used as the default. There are various types of redirection, each serving a unique purpose.

301 Permanent Redirect

This is used when a URL has permanently moved to a new location. Search engines and browsers update their records, and users are automatically redirected to the unique URL. This is a permanent redirection, and it’s important for SEO as it passes on the SEO value from the old URL to the new one.

302 Temporary Redirect

This redirect is used when a URL has been temporarily moved to a different location. It’s often used for short-term changes. However, in terms of SEO, search engines may continue to index the original URL, and the SEO value might not pass on to the new URL.

307 Temporary Redirect

This redirect is used for temporary moves, preserving the original HTTP method. It’s similar to a 302 redirect but emphasizes preserving the technique differently. You can use it to ensure that the original form (e.g., POST) is retained when the client is redirected.

410 Content Deleted

This status code indicates that the URL is no longer available and there is no replacement. When search engines encounter a 410, they may de-index the page more quickly than other status codes, which signifies that the content is intentionally gone.

451 Content Unavailable for Legal Reasons

This status code indicates that a web resource is inaccessible due to legal obligations, such as government-mandated censorship or copyright issues. It’s used to inform users and clients about content restrictions based on legal grounds.

Location-based RulesCopy heading anchor to clipboard

Location-based redirection enables to set up rules to redirect URLs based on the site visitor’s geographical location.

Location-based redirection requires you to sign up for a MaxMind account to get the license key, which is free, although paid services are available.

To sign up, click the Create a free account link in the Location-based rules module.

Complete the MaxMind GeoLite Sign Up form, then click Continue.

MaxMind will send an email containing verification information. Follow the directions in the email to verify and activate the account.

After a successful verification, log in to your MaxMind account.

In the menu on the left, click Manage License Key. Then, click on Generate New License Key.

In the modal, enter a name for your license key and click Confirm.

The License Key will be generated and displayed. Copy and paste the generated license key in the MaxMind License Key field and click Download.

Add Maxmind license key in SmartCrawl

A notice will pop up to remind you that it takes up to 5 minutes for Maxmind to activate your new key.

Once you have clicked the Download button and the license key has been activated, you can start adding the redirect rules, as seen in Location Rules above.

DeactivateCopy heading anchor to clipboard

If you no longer need the URL Redirection feature, you can click the button here to deactivate it. Deactivating the feature will disable all existing redirects, but will not delete them.

3.9.3 WooCommerce SEO

Link to chapter 9

WooCommerce adds basic schema data by default when activated on a site. When SmartCrawl detects WooCommerce on your site, you can enable this feature to enhance that basic SEO configuration.

However, if you want full control over all specifics of your WooCommerce Product schema, you can create a product type in the Types Builder and fine-tune everything there instead.

Click Activate to begin.


Once activated, the following WooCommerce configuration options are available.

Improve Woo SchemaCopy heading anchor to clipboard

Configure WooCommerce schema in SmartCrawl

  • Brand – This option enables you to select a default taxonomy to use as the Brand schema for all products. If you specify a different brand taxonomy for any specific WooCommerce Product type in the Types Builder, that taxonomy will override the default set here.
  • Global Identifier – This option enables you to specify a global GTIN or MPN to use throughout all products. When enabled, the selected identifier will be added as an input field under the Inventory tab in the product editor. If you add the same identifier as a Property to any specific WooCommerce Product schema in the Types Builder and select a different value, that will override what you set here.
  • Enable Shop Schema – When enabled, this option will automatically add the CollectionPage markup to your WooCommerce shop and product archive pages to help search engines better understand that there are several items on the pages.

Improve Woo MetaCopy heading anchor to clipboard

Configure WooCommerce meta in SmartCrawl

  • Enable Product Open Graph – Enable this to add product price and currency meta to Open Graph if that is also enabled in Title & Meta > Post Types > Products.
  • Remove Generator Tag – Enable this option to remove the WooCommerce generator tag from your site: <meta name=”generator” content=”WooCommerce x.x.x” />

Restrict Search EnginesCopy heading anchor to clipboard

Configure WooCommerce search engine options in SmartCrawl

  • No-Index Hidden Products – If you have set Catalog Visibility to hidden in your WooCommerce settings, enable this option to prevent search engines from indexing your product pages.
  • Disallow Crawling of Cart, Checkout & My Account Pages – Enable this to automatically add the following entries to your Robots.txt file when using the Robots.txt Editor feature:
    • Disallow: /add-to-cart=
    • Disallow: /cart/
    • Disallow: /checkout/
    • Disallow: /my-account/

3.9.4 Moz

Link to chapter 9

Moz is the industry leader in SEO reports, and we make it easy to integrate with their API. Note that configuring this is entirely optional.

To take advantage of the Moz reporting tools you need Moz API Access ID and Secret Key.

To get your Moz API credentials, create a new Moz Pro account or sign in to your existing account.

After creating a Moz Pro account, navigate to https://moz.com/products/api/pricing to create a Moz API account.

Opt for a free or paid plan, enter the required information, and complete the Moz API account creation. After a successful Moz API account creation, the subscription summary will be displayed.

Now, to create or access your Moz API credentials, navigate to the API dashboard and click on the Add Token button.

Enter an identifier for the token and click Create to generate credentials.

To view the generated API credentials, click on the Show Legacy Credentials link.


Copy the Access ID and Secret Key.

Paste the generated API credentials in the Moz integration modal and click Connect.

After a successful connection, it’ll take a few minutes to see metrics specific to your site (in a multisite install, metrics specific to each site in your network appear in the dashboard of each site). You can also see individual stats per post in the post editor under the SEOmoz URL Metrics module.

Moz metrics in SmartCrawl

You’ll find a wealth of information about good SEO practices, and details about your site metrics, by visiting the Moz Learn Section.

Moz API Subscription & Usage QuotaCopy heading anchor to clipboard

Note that the Moz API account is separate from the standard Moz account. If you already have a Moz account, make sure to create a Moz API account to create/access the API credentials.

Additionally, ensure you have an active API subscription and that your account has not exceeded its usage quota. You can check your current usage in the API dashboard and your API subscription on the subscriptions page.

DeactivateCopy heading anchor to clipboard

Moz settings in SmartCrawl

If you no longer need the Moz feature, you can click the button here to deactivate it. Note that deactivating the feature will reset your Moz credentials.

3.9.5 Robots.txt Editor

Link to chapter 9

This tool enables you to directly edit the robots.txt file for your site without having to go into the file system via FTP or cPanel.

Robots.txt is used to tell web crawlers what they should or should not index. For example, you could include directives in your robots.txt file to prevent Google and other search engines from indexing certain files on your website (images, PDFs, etc.) so they don’t appear in search results.

Start by enabling the module:

IMPORTANT

This tool cannot be used to edit an existing physical robots.txt file. If such a file already exists, the Activate button will be replaced by a message alerting you that you will need to remove that file before proceeding.

Robots-txt-editor-alert

OutputCopy heading anchor to clipboard

The top section here will give you a link to the location of the virtual robots.txt file created by the plugin and show you the current contents of the file that search engines will see.

Robots-txt-output

Include SitemapCopy heading anchor to clipboard

This setting enables you to automatically include the URL to your site’s sitemap.xml file in your robots.txt file. It’s a good idea to ensure this option is enabled, and the sitemap.xml URL is entered here, so search engines know where to find that file as they use it to crawl your site.

Robots-txt-sitemap-no-sc

Note that if you have enabled the Sitemap module in SmartCrawl, this URL will be automatically filled in for you and cannot be changed.

Robots-txt-sitemap-sc

CustomizeCopy heading anchor to clipboard

This is where you can add any directives you need in your robots.txt file to instruct web crawlers on what they should do on your site.

Robots-txt-customize

By default, this section only contains the following, which allows all user agents (search engines) to access and index all content:

User-agent: *
Disallow:

  • User-agent: * means all ( * ) search engines.
  • Disallow: with nothing after, it means nothing is disallowed, so access to all content is allowed.

You can add any additional directives you need in this section but can leave it as-is if you don’t need or want to restrict web crawler access.

Customization ExamplesCopy heading anchor to clipboard

If you want to prevent search engines from indexing WooCommerce cart & checkout pages on your site, you could add rules like these to your robots.txt file:

Disallow: /*add-to-cart=*
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/

If you’re having trouble with bots scanning your WooCommerce pages, adding products to the cart and slowing down your site, you could add rules like these to your robots.txt file to block them:

User-agent: *
Disallow: /wp-admin/
Disallow: /?s=
Disallow: /search
Disallow: /wp-json
Disallow: /cart
Disallow: /wishlist
Disallow: /checkout
Disallow: /my-account
Disallow: /*?orderby*
Disallow: /*?filter
Disallow: /*add-to-cart=*
Disallow: /*?add_do_wishlist=*

Allow: /wp-admin/admin-ajax.php
Crawl-delay: 10

For more information on robots.txt directives, please visit this handy article at moz.com https://moz.com/learn/seo/robotstxt

Google and Wikipedia also have some good articles to help out if needed:
https://support.google.com/webmasters/answer/6062608?hl=en
https://support.google.com/webmasters/answer/6062596?hl=en
https://en.wikipedia.org/wiki/Robots_exclusion_standard#About_the_standard

This section contains various overall settings where you can enable/disable specific modules, and adjust plugin behavior.

3.10.1 General Settings

Link to chapter 10

Plugin ModulesCopy heading anchor to clipboard

Here, you can enable or disable any of the SmartCrawl modules. For example, if you’re using some other means to create sitemaps of your site, you’d want to disable the SmartCrawl sitemap module to avoid conflicts.

In case you have multiple SEO plugins installed on your website, here you can deactivate the modules that conflict with other SEO plugins.

Enable the Hide disabled modules from the dashboard and sub-menu checkbox to hide the disabled plugin modules from the plugin dashboard and sub-menu.

In-Post AnalysisCopy heading anchor to clipboard

These options enable modules that appear inside the post editor and provide per-page analysis and opportunities to fine-tune the SEO for each post or page. Our SEO analysis engine supports the languages below:

  • Czech
  • Dutch
  • English
  • French
  • German
  • Italian
  • Russian
  • Spanish

If your site language is set to any other language, you’ll see a notice informing you that the language is not supported by these features.

Site language not supported notice

See the In-Post SEO Analysis chapter below for more info.

Enable in-page analysis options in SmartCrawl

  • Page Analysis – This option enables you to fine tune numerous SEO tweaks for each page and post on your site.
  • Readability Analysis – Enable this to use the Flesch-Kincaid readability tests in each post or page to indicate how easy or difficult a passage is to understand. Note that your post must be in either Draft or Published status for this analysis option to work, as it scans the content output, not what is in the post editor.

If you enable either of the in-post analysis options above, you’ll want to select the scope of the analysis engine.

Select analysis engine in SmartCrawl

  • Content – Recommended for most websites as it only reviews the_content() output.
  • Wide – Includes everything, except for your header, nav, footer and sidebars. This can be helpful for page builders and themes with custom output.
  • All – Checks your entire page’s content including elements like nav and footer. Due to analysing everything, you might miss key analysis of your real content so we don’t recommend this approach.
  • None – Only analyzes content you tell it to programmatically. If you have a fully custom setup, this is the option for you. If you select this option, add the class .smartcrawl-checkup-included to container elements you want to include in the SEO and Readability Analysis.

Note that if you want the engine to analyze the SEO of featured images in your posts, you’ll want to select either the Wide or All option above.

Disable Page Analysis Check on Pages/Posts ScreenCopy heading anchor to clipboard

By default, SmartCrawl will analyze posts and pages one at a time in your Posts and Pages lists to avoid excessive server load. However, you can toggle on this option to disable these checks altogether if needed.

Admin BarCopy heading anchor to clipboard

Enable this option to add a SmartCrawl menu item to your admin bar.

Meta TagsCopy heading anchor to clipboard

Meta tag options in SmartCrawl

  • Hide Generator Meta Tag – With this option enabled you can hide your WordPress version visible to the public and improve the security of your site.
  • Hide Redundant Canonical Link Tags – WordPress automatically generates a default canonical tag for each of your website’s pages & posts. In many cases, this isn’t needed, so you can enable this option to remove them and avoid any potential SEO “duplicate content” backlash from search engines. See Working with Canonicals in SmartCrawl below for more info.
  • Enforce Meta Tag Character Limits – Each meta tag type has recommended maximum characters lengths to follow. Enabling this will prevent you from adding too many characters.

Search EnginesCopy heading anchor to clipboard

This tool will add the meta tags required by search engines to your website’s <head> tag to verify your site with their SEO management tools.

Verification is the process of proving that you own the site or app that you claim to own. Search engines need to confirm ownership because once you are verified for a site or app you have access to its private Google/Bing Search data, and can affect how Google/Bing search crawls it.

Add meta tags in SmartCrawl

Google VerificationCopy heading anchor to clipboard

To add Google verification follow the instructions given here https://support.google.com/webmasters/answer/35179?hl=en

Bing VerificationCopy heading anchor to clipboard

To add Bing verification follow the instructions given here using verification method #3 https://www.bing.com/webmasters/help/add-and-verify-site-12184f8b

Add Verification Code toCopy heading anchor to clipboard

Here you can choose to add the verification code to All Pages or just the Home Page.

Custom Meta TagsCopy heading anchor to clipboard

A short example of what you could enter here would be to help Bing by adding:<

<meta name=”geo.position” content=”latitude; longitude”>

<meta name=”geo.placename” content=”Place Name”>

<meta name=”geo.region” content=”Country Subdivision Code”>

Working with Canonicals in SmartCrawlCopy heading anchor to clipboard

By default, WordPress automatically generates and adds a rel=canonical link in the source code of every page & post of your site, which is the same as the post permalink.

In many cases, these links are not needed and this default behavior can be disabled by toggling on the option to Hide Redundant Canonical Link Tags above.

Whether the Hide Redundant Canonical Link Tags option is enabled or not, you can specify your own custom canonical links for each page or post as needed under SmartCrawl > Advanced in the in-post editor.

Here is the behavior to expect depending on your configuration:

With the Hide Redundant Canonical Link Tags option disabled:

  • WordPress default canonical links are added to each post & page as expected.
  • The SmartCrawl in-post option can be used to override the WordPress default canonical link for any post or page if needed.

With the Hide Redundant Canonical Link Tags option enabled:

  • All WordPress default canonical links are removed from your site.
  • The SmartCrawl in-post option can be used to add a custom canonical link to any page or post if needed.

Usage TrackingCopy heading anchor to clipboard

Usage tracking is incredibly useful for our designers, and enables us to learn more about what features you use and don’t use. It is a completely anonymous feature, and helps us deliver more relevant features in the future. See our Privacy documentation for more information about the data we collect.

Usage Tracking when White Label is enabled

When White Label is activated in the WPMU DEV Dashboard, only users with permission to access the WPMU DEV Dashboard will be able to see the usage tracking option.

3.10.2 Configs

Link to chapter 10

The Configs module allows you to save your SmartCrawl configurations so that they can be applied to another site in just a few clicks.

To help you get started, we provide a Default SEO Config denoted with a blue checkmark. It includes the recommended settings that are ideal for fresh installations.

SmartCrawl default SEO Config

Save a ConfigurationCopy heading anchor to clipboard

To save your current configuration, click Save Config.

Save button

Then, enter a name and optional description for the configuration and click Save.

Save config window

Saved configurations are listed alphabetically. To view details about a saved configuration, click the configuration in the list.

config details

To make changes to a saved configuration, click the gear icon. The available actions are:

  • Apply – Apply the saved configuration to this SmartCrawl installation.
  • Download – Download the saved configuration as a .json file.
  • Name & Description – Edit the name and description for the saved configuration.
  • Delete – Permanently delete the saved configuration.

Additional actions menu

Upload a ConfigurationCopy heading anchor to clipboard

To upload a configuration that you’ve downloaded from another site, click Upload, and select the relevant .json file from your computer.

Upload config

SmartCrawl will import the uploaded configuration and add it to the Preset Configs list.

Apply a ConfigurationCopy heading anchor to clipboard

To apply a saved configuration to your SmartCrawl installation, click Apply.

Apply config

You will be asked to confirm that you would like to replace your current settings with the saved configuration. Click Apply to proceed.

Apply config window

Sync with the HubCopy heading anchor to clipboard

SmartCrawl configurations will automatically be synced with the Hub. Synced configs can be accessed and applied directly from the Config or SEO modules in the Hub, or from the SmartCrawl installation of any of your sites.

If a new config created in the Hub doesn’t appear immediately in the SmartCrawl Settings module, click Check again to refresh your data.

Sync with the Hub

NOTE

This feature is only available to users with the WPMU DEV Dashboard plugin and an active WPMU DEV subscription.

3.10.3 User Roles

Link to chapter 10

User Roles allows you to configure access of user roles to certain parts of SmartCrawl. You can manage access to In Page SEO, 301 Redirects and Moz Data.

Set user role permissions in SmartCrawl

As always, don’t forget to click SAVE SETTINGS when you’re done.

3.10.4 Import

Link to chapter 10

ImportCopy heading anchor to clipboard

Third PartyCopy heading anchor to clipboard

You can import settings from another SEO plugin you may have installed before deactivating it. Currently, import is only supported from the Yoast SEO plugin.

Click Import to import your third-party settings.

SmartCrawl third-party settings import

3.10.5 Data & Settings

Link to chapter 10

The options here enable you to specify what should be done with your settings and data.

UninstallationCopy heading anchor to clipboard

SmartCrawl uninstall options

  • Settings – Select to either Preserve or Reset the plugin settings if you need to uninstall the plugin for any reason. By default, the Preserve option is selected. Settings include everything related to the plugin configuration.
  • Data – Select to either Keep or Reset all data stored by the plugin when you uninstall it. By default, the Keep option is selected. This includes data from logs, sitemap crawler, SEO checkup, etc.

Subsites (Multisite only)Copy heading anchor to clipboard

If you need to clear all settings and data from all subsites in your multisite network, click the Reset Subsites button in this option.

SmartCrawl option to delete subsite settings and data

Factory ResetCopy heading anchor to clipboard

If you ever need to completely wipe all data and reset settings back to their defaults, click the Reset button in this option.

SmartCrawl reset option

3.10.6 Accessibility

Link to chapter 10

Enable the High Contrast Mode option here to enhance visibility of the plugin interface and meet WCAG AAA requirements.

SmartCrawl accessibility option

3.11 Network Settings

Copy chapter anchor to clipboard

For WordPress multisite installations, SmartCrawl includes a Network Settings page where you can configure how much control your sub-site admins have over the SmartCrawl settings for their sites.

Choose whether Super Admins or Site Admins should control sub-site settings by selecting either Super Admins or Site Admins from the Admin Access dropdown menu.

SmartCrawl network settings

You can also choose which SmartCrawl modules should be available and configurable on sub-sites.

Settings modules

If you’d like new sub-sites to have the desired SmartCrawl settings pre-applied, select a preset config from the New Sub-sites dropdown menu.

Settings new sub-sites

3.12 In-Post SEO Analysis

Copy chapter anchor to clipboard

SmartCrawl includes several tools on the post editor screens to help you maximize SEO benefits for your individual posts and pages. Our SEO analysis engine supports the languages below:

  • Czech
  • Dutch
  • English
  • French
  • German
  • Italian
  • Russian
  • Spanish

If your site language is set to any other language, you’ll see a notice informing you that the language is not supported by these features.

Site language not supported notice

Do you have a multilingual website?

Check out our blog post on Multilingual Readability and Post SEO Analysis for more information on how to optimize your multilingual content for search engines.

3.12.1 In Post/Page/Taxonomy Lists

Link to chapter 12

The first thing you’ll notice on your All Posts or All Pages screens is a new SEO Meta [+] link just above the regular WordPress action links.

seo-editor

Clicking that will open up a details pane with some basic info about the post’s current SEO-optimized title and description, including colored indicators showing their character counts. Those indicators will appear green if the count is within the recommended range, or yellow if the count is less than or greater than the recommended range.

seo-editor-meta

You’ll also see a few new columns in the post/page list.

  • SEO – If Page Analysis is enabled in SmartCrawl > Settings. Note that this column will only show data if the SEO Analysis has already been run for focus keywords at least once.
  • Readability – If Readability Analysis is enabled in SmartCrawl > Settings. Note that this column will only show data if the Readability test has already been run at least once.
  • Robots Meta – Shows the index settings you have enabled for crawler instructions.

Hovering your mouse over the number indicator in the SEO column will reveal a quick overview of the recommendations SmartCrawl has for that page or post.

seo-editor-tooltip

If you find those columns take up too much space on your screen, or you don’t really need them there, they can be toggled on/off by opening up the Screen Options tab and checking/unchecking those columns.

screen-options

You’ll also find a handy SEO Status column on categories, tags, and custom taxonomy pages to indicate whether the SEO description is set and has the recommended length.

You’ll see a green checkmark if the SEO description is set and contains the recommended 120 to 160 characters. If it is missing, you’ll see a red (x) and if it is too long or too short, you’ll see a yellow (!). Hover over any icon to reveal a tooltip with additional information.

SEO status in taxonomies

3.12.2 In Post Editor

Link to chapter 12

In the post editor for any post or page, scroll down to where you’ll see the SmartCrawl metabox. Depending on the options and features you’ve enabled in SmartCrawl > Settings, you’ll see several tabs there:

  • SEO – This is where you can customize the SEO title and description that appears in search engine results pages, and run a full SEO analysis of your post.
  • Readability – This uses the Flesch-Kincaid Test to determine how easily readable your content is for the average reader.
  • Social – Customize the post’s OpenGraph elements for sharing to social networks.
  • Advanced – Includes indexing, sitemap and automatic linking options for the post.

SEO TabCopy heading anchor to clipboard

Under the SEO tab, you’ll see a preview of how Google sees your post meta, with the title at the top, followed by the post permalink, and your post description.

Edit MetaCopy heading anchor to clipboard

Click the Edit Meta button to pop open an editor where you can customize the title and description. You’ll see the same color-coded character counts as previously described to help you get the perfect length.

Click the [+] icon in either field to add dynamic data using the built-in macros. Click any to automatically add to the corresponding field; you can add as many as you need to create the ideal structure for your title and description. You can also add plain text in each as well if you like.

Edit SEO meta in WordPress posts and pages

The following macros are currently supported in the title and description fields:

  • Post/page ID – %%id%%
  • Title of the post/page – %%title%%
  • Post/page excerpt (or auto-generated if it does not exist) – %%excerpt%%
  • Post/page excerpt (without auto-generation) – %%excerpt_only%%
  • Post/page modified time – %%modified%%
  • Date of the post/page – %%date%%
  • Post/page author’s ‘nicename’ – %%name%%
  • Post/page author’s userid – %%userid%%
  • Post/page author’s description – %%user_description%%
  • Post categories (comma separated) – %%category%%
  • Current tag/tags – %%tag%%
  • Separator – %%sep%%
  • Site’s name – %%sitename%%
  • Site’s tagline / description – %%sitedesc%%
  • Current page number (i.e. page 2 of 4) – %%page%%
  • Current page total – %%pagetotal%%
  • Current page number – %%pagenumber%%
  • Current page number, spelled out as numeral in English – %%spell_pagenumber%%
  • Current page total, spelled out as numeral in English – %%spell_pagetotal%%
  • Current page number, spelled out as numeral in English – %%spell_page%%
  • Current time – %%currenttime%%
  • Current date – %%currentdate%%
  • Current month – %%currentmonth%%
  • Current year – %%currentyear%%

SEO AnalysisCopy heading anchor to clipboard

Scroll a little further down under the SEO tab and you’ll see the SEO Analysis section if you’ve enabled that option in SmartCrawl Settings.

You can analyze the post content for up to three focus keywords. Enter each keyword individually and click the Add Keyword button for each one, or enter all keywords separated by commas and click the Add Keyword button just once.

The first keyword you enter will be considered as your primary keyword, while the optional second and third ones will be analyzed as secondary keywords.

Analysis results for each keyword will appear under their own separate tabs so you can adjust your post content according to recommendations for each one as needed.

In-post SEO analysis results

After each content adjustment, click the Refresh button to run the analysis again for the same keywords.

For each focus keyword, SmartCrawl will give you a list of recommendations to improve the SEO of your post. Recommendations will be highlighted in yellow, while passed audits will be highlighted in green.

seo-editor-post-analysis-results

Click any recommendation to reveal details about what you can do to improve things. If you decide any recommendation is not needed, you can click the Ignore button so it doesn’t keep popping up every time you run the analysis. Every ignored recommendation is accompanied by a Restore button. Click Restore to revert the ignored recommendation.

seo-editor-post-analysis-results-open

Note that the Focus Keyword feature does not add anything to the source code of your post. It simply enables you to analyze the post content in real-time based on a specified keyword, and get recommendations to improve the post SEO for that keyword.

Readability TabCopy heading anchor to clipboard

Under the Readability tab, simply click the Refresh button to run the Flesch-Kincaid Test and see how well your post reads for the average English reader. Any score above 70 is considered good here.

Note that your post must be in either Draft or Published status for the Readability analysis to work, as it scans the content output, not what is in the post editor.

Social TabCopy heading anchor to clipboard

If you would like your post’s title, description and/or featured image(s) to be different when shared to social networks, enable OpenGraph support for the post under the Social tab, and configure the specifics you need.

Advanced TabCopy heading anchor to clipboard

Under the Advanced tab, you can adjust the indexing options for the post if you want them to be other than what you have set for the post type’s defaults in SmartCrawl > Title & Meta.

  • Index – Instruct search engines whether or not you want this post to appear in search results.
  • Follow – Tells search engines whether or not to follow the links on your page and crawl them too.
  • Archive – Instructs search engines whether to store a cached version of this page or not.
  • Snippet – Allows search engines to show a snippet of this page in the search results and prevents them from caching the page.

seo-editor-post-advanced-index

Under this same tab, you can also set optional canonical or redirect URLs, and enable/disable automatic linking in the post if that feature is enabled in SmartCrawl > Advanced Tools.

SEO editor - Post advanced links

3.12.3 In-Post SEO Recommendations

Link to chapter 12

SmartCrawl suggests In-Post SEO Recommendations for every focus keyword for which your post content has been analyzed.

Click on any recommendation to expand it and find additional information about how it can benefit your post SEO.

Below is a list of all the available SEO Recommendations included with SmartCrawl:

Your focus keyword(s) aren’t used in the SEO titleCopy heading anchor to clipboard

Try Including the focus keyword(s) in the SEO title of a post. Including focus keyword(s) in the SEO title increases the chances of your article ranking higher up in the search results for the targeted keyword(s).

Your SEO title is too shortCopy heading anchor to clipboard

Your SEO title should have a nice length, neither too short nor too long. Keeping the length of your title between 50 – 65 characters (including spaces) is recommended, 60 being the sweet spot.

Title is longer than 65 charactersCopy heading anchor to clipboard

SEO titles are cut off visually to a certain degree. Therefore, having long titles can affect the readability and SEO of your content.

The SEO description doesn’t contain your focus keywordsCopy heading anchor to clipboard

An SEO description without your focus keywords has less chance of matching what your visitors are searching for, compared to a description that includes them. It’s worth trying to get your focus keywords in there, just remember to keep it readable and natural.

Your meta description is less than 120 charactersCopy heading anchor to clipboard

Meta descriptions provide users a short summary of what a page is about. Keeping your meta descriptions between 120 – 160 characters (including spaces) is recommended. This helps in achieving a right balance between readability and keyword density.

You haven’t used your focus keywords in the page URLCopy heading anchor to clipboard

Google does index your page URL. Using your focus keywords in the page slug can help your page rank higher as it has a greater chance of matching search terms. Try getting your focus keywords in there.

You haven’t added any imagesCopy heading anchor to clipboard

Images are a great addition to any piece of content and it’s highly recommended to have images in your content. Consider adding a few images that relate to your content to enhance the readability. It’s also a great opportunity to include your focus keyword(s) in image ALT attributes to better associate your content with the target keyword(s).

You haven’t used any keywords yetCopy heading anchor to clipboard

Your content should be populated with an optimal amount of focus keywords to give it a better chance of ranking higher in the search results. Use the focus keyword(s) as naturally possible in your content. The recommended keyword density is 1-3%.

Internal links help search engines crawl your website, effectively pointing them to more pages to crawl and index on your website. You should consider adding at least one internal link to another related article within your site.

You haven’t included the focus keywords in the first paragraph of your articleCopy heading anchor to clipboard

It’s good practice to include your focus keywords in the first paragraph of your content so that search engines and visitors can quickly figure out the topic of your post.

You don’t have any subheadingsCopy heading anchor to clipboard

Using subheadings in your content (such as H2’s or H3’s) will help both users and search engines quickly figure out what your content is about. Creating a visual hierarchy improves readability and user experience. We recommend you have at least one subheading.

None of your image alt texts contain the focus keyword(s)Copy heading anchor to clipboard

Including your focus keyword(s) in image alt attributes helps search engines to better associate your content with the topic you’re writing about, and helps visually impaired visitors who use assistive technology. Add your focus keyword(s) to one or more of your images, but be careful not to overdo it.

Note that if you want SmartCrawl to analyze the SEO of featured images in your posts, you’ll want to select either the Wide or All option in the In-Post Analysis Engine.

The text contains ‘x’ words which is less than the recommended minimum of 300 wordsCopy heading anchor to clipboard

The recommended minimum content length for the web is 300 words. Aim at least for the recommended length of 300-1600 words.

Your keyword density is greater than 3%Copy heading anchor to clipboard

Including focus keywords in your content is recommended, but make sure that your content is not littered with too many of them. The recommended range of keyword density is 1-3%. Keyword stuffing can result in search engines penalizing your content and marking it as spam.

There are stop words in focus keywordsCopy heading anchor to clipboard

Stop words don’t convey much information, and are generally considered insignificant and are often filtered out from a search query. It is recommended not to include them as a part of your focus keyword(s).

Your URL contains underscoresCopy heading anchor to clipboard

Google recommends ‘-’(hyphens) over underscores in URL. It makes it easier for the search engines to crawl and interpret the URL better. However if you have already published your content we recommend not to remove the ‘_'(underscore) as it can cause short term ranking loss.

Your meta description is greater than 160 charactersCopy heading anchor to clipboard

Just like SEO titles, the SEO descriptions are also cut off to a certain degree. Therefore, keeping your meta description too long can affect the readability and makes it unclear what your content is about. Recommended length of meta description is 120 – 160 characters.

Your meta description is autogeneratedCopy heading anchor to clipboard

It is recommended to handcraft your meta description with the relevant information about the page content. Follow the meta description best practices to further increase the chances of your content ranking higher in search results.

Primary focus keyword is already used on another post/pageCopy heading anchor to clipboard

Focus keywords are intended to be unique. Optimizing more than one post for the same focus keyword confuses search engines on which content to rank for that specific focus keyword, affecting the SEO and page rankings.

Reusing primary focus keyword

Expand this recommendation to display a list of other posts/pages with the same primary focus keyword.

Note: The list will contain only a maximum of 10 pages/posts, there can still be other pages/posts with the same primary focus keyword.

Linking to external web pages in your site redirects your site traffic to other sites. However, adding relevant outbound links improves your site credibility, authority, adds more value to users, and also helps search engines determine the relevance and quality of your content. Hence, it is recommended to have at least one external dofollow link in your site.

3.13 SmartCrawl & WPML Integration

Copy chapter anchor to clipboard

SmartCrawl’s integrations with WPML allows you to optimize the pages you have translated for search engines.

3.13.1 Translating Global Meta Text

Link to chapter 13

You can translate the global title and meta values such as the home page SEO title, description, OpenGraph details, and Twitter details. Simply, go to WPML > String Translation and look up the global meta text you want to translate. Then, click the Plus icon, add the translation, and hit enter to save it.

Translating global SEO text via WPML's String Translation screen

If you are using macros, make sure you copy the default language macro text in the translation. Translating the macro text will result in breaking its functionality for the secondary language pages.

It is worth noting that you can find all the global meta text under the admin_texts_wds_onpage_options domain on the WPML String Translation screen.

SmartCrawl's text domain

3.13.2 Translating the Page Title and Meta

Link to chapter 13

Smartcrawl Titles & Meta section enables you to set the SEO title, meta description, focus keyword, OpenGraph, and Twitter values for pages, posts, and custom post types.

You can translate these meta elements along with the page content using WPML’s Advanced Translation Editor. Additionally, you can set non-textual meta elements such as the indexing options to be different from those on the default language page.

Translating textual meta elementsCopy heading anchor to clipboard

On the page or post editing screen, click on the Plus icon in the Language metabox corresponding to the language you want to translate your page into. This takes you to the Advanced Translation Editor page.

Clicking the Plus icon to translate the page content and textual meta elements

Translate the textual meta elements and the page content, then click the Complete button to save the translation.

WPML's Advanced Translation Editor page

Setting non-textual meta elements on the secondary language pagesCopy heading anchor to clipboard

SmartCrawl enables you to configure indexing metatags for your pages. On a multilingual website, you might want to have different indexing options on the secondary language page than the one you have for your default language page.

To do this, go to the secondary language editing page. You will see a message warning you to edit the translated page using the standard WordPress editor. Click the Edit anyway button.

Please make sure that you don’t make any changes to the secondary language content using the default WordPress editor. These changes will get overwritten if you use the Advanced Translation editor to update the translated content.

Click Edit Anyway

On SmartCrawl’s Advanced tab, adjust the indexing options. Once done, Update the page to save the changes.

Set the secondary language page indexing meta tags and update

3.14 White Label SmartCrawl

Copy chapter anchor to clipboard

Remove Code CommentsCopy heading anchor to clipboard

When activated, SmartCrawl will add meta tags to your site’s source code according to options you configure in the plugin. These tags are wrapped in a comment that identifies the plugin, like so:

<!-- SEO meta tags powered by SmartCrawl https://wpmudev.com/project/smartcrawl-wordpress-seo/ -->...tags here...<!-- /SEO -->

If you prefer to not have that comment in your site’s source code, you can remove it in one of the following two ways:

  • Enable the White Label feature in the WPMU DEV Dashboard plugin.
  • Or add this constant to your wp-config.php file, before the line that says “That’s all, stop editing”:
    define ('SMARTCRAWL_WHITELABEL_ON', true);