Merge pull request #19 from camya/feature/robots-tag-support

Robots Tag feature (+ pest tests + documentation)

README.md

@@ -6,13 +6,13 @@ Currently there aren't that many SEO-packages for Laravel and the available ones
 
 This package generates **valid and useful meta tags straight out-of-the-box**, with limited initial configuration, whilst still providing a simple, but powerful API to work with. It can generate:
 
-1. Robots tag
-2. Title tag (with sitewide suffix)
-3. Meta tags (author, description, image, etc.)
-4. OpenGraph Tags (Facebook, LinkedIn, etc.)
-5. Twitter Tags
-6. Structured data (Article and Breadcrumbs)
-7. Favicon
+1. Title tag (with sitewide suffix)
+2. Meta tags (author, description, image, robots, etc.)
+3. OpenGraph Tags (Facebook, LinkedIn, etc.)
+4. Twitter Tags
+5. Structured data (Article and Breadcrumbs)
+6. Favicon
+7. Robots tag
 
 If you're familiar with Spatie's media-library package, this package works in almost the same way, only then for SEO. I'm sure it will be very helpful for you, as it's usually best to SEO attention right from the beginning.
 
@@ -105,6 +105,26 @@ return [
      */
     'canonical_link' => true,
 
+    'robots' => [
+        /**
+         * Use this setting to specify the default value of the robots meta tag. `<meta name="robots" content="noindex">`
+         * Overwrite it with the robots attribute of the SEOData object. `SEOData->robots = 'noindex, nofollow'`
+         * "max-snippet:-1" Use n chars (-1: Search engine chooses) as a search result snippet.
+         * "max-image-preview:large" Max size of a preview in search results.
+         * "max-video-preview:-1" Use max seconds (-1: There is no limit) as a video snippet in search results.
+         * See https://developers.google.com/search/docs/advanced/robots/robots_meta_tag
+         * Default: 'max-snippet:-1, max-image-preview:large, max-video-preview:-1'
+         */
+        'default' => 'max-snippet:-1,max-image-preview:large,max-video-preview:-1',
+
+        /**
+         * Force set the robots `default` value and make it impossible to overwrite it. (e.g. via SEOData->robots)
+         * Use case: You need to set `noindex, nofollow` for the entire website without exception.
+         * Default: false
+         */
+        'force_default' => false,
+    ],
+
     /**
      * Use this setting to specify the path to the favicon for your website. The url to it will be generated using the `secure_url()` function,
      * so make sure to make the favicon accessibly from the `public` folder.
@@ -211,6 +231,9 @@ On the SEO model, you may **update the following properties**:
 2. `description`: this will be used for the `<meta>` description tag and all the related tags (OpenGraph, Twitter, etc.)
 3. `author`: this should be the name of the author and it will be used for the `<meta>` author tag and all the related tags (OpenGraph, Twitter, etc.)
 4. `image`: this should be the path to the image you want to use for the `<meta>` image tag and all the related tags (OpenGraph, Twitter, etc.). The url to the image is generated via the `secure_url()` function, so be sure to check that the image is publicly available and that you provide the right path.
+5. `robots`
+    - Overwrites the default robots value, which is set in the config. (See `'seo.robots.default'`).
+    - String like `noindex,nofollow` [(Specifications)](https://developers.google.com/search/docs/advanced/robots/robots_meta_tag), which is added to `<meta name="robots">`
 
 ```php
 $post = Post::find(1);
@@ -247,8 +270,9 @@ You are allowed to only override the properties you want and omit the other prop
 9. `modified_time` (should be a `Carbon` instance with the published time. By default this will be the `updated_at` property of your model)
 10. `section` (should be the name of the section of your content. It is used for OpenGraph article tags and it could be something like the category of the post)
 11. `tags` (should be an array with tags. It is used for the OpenGraph article tags)
-12. 'schema' (this should be a SchemaCollection instance, where you can configure the JSON-LD structured data schema tags)
+12. `schema` (this should be a SchemaCollection instance, where you can configure the JSON-LD structured data schema tags)
 13. `locale` (this should be the locale of the page. By default this is derived from `app()->getLocale()` and it looks like `en` or `nl`.)
+14. `robots` (should be a string with the content value of the robots meta tag, like `nofollow,noindex`). You can also use the `$SEOData->markAsNoIndex()` to prevent a page from being indexed.
 
 Finally, you should update your Blade file, so that it can receive your model when generating the tags:
 

config/seo.php

@@ -33,6 +33,26 @@
      */
     'canonical_link' => true,
 
+    'robots' => [
+        /**
+         * Use this setting to specify the default value of the robots meta tag. `<meta name="robots" content="noindex">`
+         * Overwrite it with the robots attribute of the SEOData object. `SEOData->robots = 'noindex, nofollow'`
+         * "max-snippet:-1" Use n chars (-1: Search engine chooses) as a search result snippet.
+         * "max-image-preview:large" Max size of a preview in search results.
+         * "max-video-preview:-1" Use max seconds (-1: There is no limit) as a video snippet in search results.
+         * See https://developers.google.com/search/docs/advanced/robots/robots_meta_tag
+         * Default: 'max-snippet:-1, max-image-preview:large, max-video-preview:-1'
+         */
+        'default' => 'max-snippet:-1,max-image-preview:large,max-video-preview:-1',
+
+        /**
+         * Force set the robots `default` value and make it impossible to overwrite it. (e.g. via SEOData->robots)
+         * Use case: You need to set `noindex, nofollow` for the entire website without exception.
+         * Default: false
+         */
+        'force_default' => false,
+    ],
+
     /**
      * Use this setting to specify the path to the favicon for your website. The url to it will be generated using the `secure_url()` function,
      * so make sure to make the favicon accessibly from the `public` folder.

database/migrations/create_seo_table.php.stub

@@ -17,6 +17,7 @@ return new class extends Migration
             $table->string('title')->nullable();
             $table->string('image')->nullable();
             $table->string('author')->nullable();
+            $table->string('robots')->nullable();
 
             $table->timestamps();
         });

src/Models/SEO.php

@@ -44,6 +44,7 @@ public function prepareForUsage(): SEOData
             schema: $overrides->schema ?? null,
             type: $overrides->type ?? null,
             locale: $overrides->locale ?? null,
+            robots: $overrides->robots ?? $this->robots,
         );
     }
 }

src/Support/RenderableCollection.php

@@ -12,7 +12,7 @@ public function render(): string
         return $this->reduce(function (string $carry, Renderable $item): string {
             return $carry .= Str::of(
                 $item->render()
-            )->trim()->trim(PHP_EOL);
+            )->trim().PHP_EOL;;
         }, '');
     }
 }

src/Support/SEOData.php

@@ -30,6 +30,7 @@ public function __construct(
         public ?string $site_name = null,
         public ?string $favicon = null,
         public ?string $locale = null,
+        public ?string $robots = null,
     ) {
         if ( $this->locale === null ) {
             $this->locale = Str::of(app()->getLocale())->lower()->kebab();
@@ -44,4 +45,11 @@ public function imageMeta(): ?ImageMeta
 
         return null;
     }
+
+    public function markAsNoindex(): static
+    {
+        $this->robots = 'noindex, nofollow';
+
+        return $this;
+    }
 }

src/TagCollection.php

@@ -7,11 +7,13 @@
 use RalphJSmit\Laravel\SEO\Support\SchemaTagCollection;
 use RalphJSmit\Laravel\SEO\Support\SEOData;
 use RalphJSmit\Laravel\SEO\Tags\AuthorTag;
+use RalphJSmit\Laravel\SEO\Tags\CanonicalTag;
 use RalphJSmit\Laravel\SEO\Tags\DescriptionTag;
 use RalphJSmit\Laravel\SEO\Tags\FaviconTag;
 use RalphJSmit\Laravel\SEO\Tags\ImageTag;
 use RalphJSmit\Laravel\SEO\Tags\OpenGraphTags;
-use RalphJSmit\Laravel\SEO\Tags\RobotsTags;
+use RalphJSmit\Laravel\SEO\Tags\RobotsTag;
+use RalphJSmit\Laravel\SEO\Tags\SitemapTag;
 use RalphJSmit\Laravel\SEO\Tags\TitleTag;
 use RalphJSmit\Laravel\SEO\Tags\TwitterCardTags;
 
@@ -22,7 +24,9 @@ public static function initialize(SEOData $SEOData = null): static
         $collection = new static();
 
         $tags = collect([
-            RobotsTags::initialize($SEOData),
+            RobotsTag::initialize($SEOData),
+            CanonicalTag::initialize($SEOData),
+            SitemapTag::initialize($SEOData),
             DescriptionTag::initialize($SEOData),
             AuthorTag::initialize($SEOData),
             TitleTag::initialize($SEOData),

src/TagManager.php

@@ -107,7 +107,7 @@ public function render(): string
         return $this->tags
             ->pipeThrough(SEOManager::getTagTransformers())
             ->reduce(function (string $carry, Renderable $item) {
-                return $carry .= Str::of($item->render())->trim()->trim(PHP_EOL);
+                return $carry .= Str::of($item->render())->trim().PHP_EOL;
             }, '');
     }
 

src/Tags/RobotsTags.php → src/Tags/CanonicalTag.php

@@ -5,29 +5,21 @@
 use Illuminate\Contracts\Support\Renderable;
 use Illuminate\Support\Collection;
 use RalphJSmit\Laravel\SEO\Support\LinkTag;
-use RalphJSmit\Laravel\SEO\Support\MetaTag;
 use RalphJSmit\Laravel\SEO\Support\RenderableCollection;
 use RalphJSmit\Laravel\SEO\Support\SEOData;
-use RalphJSmit\Laravel\SEO\Support\SitemapTag;
 
-class RobotsTags extends Collection implements Renderable
+class CanonicalTag extends Collection implements Renderable
 {
     use RenderableCollection;
 
     public static function initialize(SEOData $SEOData = null): static
     {
         $collection = new static();
 
-        $collection->push(new MetaTag('robots', 'max-snippet:-1,max-image-preview:large,max-video-preview:-1'));
-
         if ( config('seo.canonical_link') ) {
             $collection->push(new LinkTag('canonical', $SEOData->url));
         }
 
-        if ( $sitemap = config('seo.sitemap') ) {
-            $collection->push(new SitemapTag($sitemap));
-        }
-
         return $collection;
     }
 }

src/Tags/RobotsTag.php

@@ -0,0 +1,29 @@
+<?php
+
+namespace RalphJSmit\Laravel\SEO\Tags;
+
+use Illuminate\Contracts\Support\Renderable;
+use Illuminate\Support\Collection;
+use RalphJSmit\Laravel\SEO\Support\MetaTag;
+use RalphJSmit\Laravel\SEO\Support\RenderableCollection;
+use RalphJSmit\Laravel\SEO\Support\SEOData;
+
+class RobotsTag extends Collection implements Renderable
+{
+    use RenderableCollection;
+
+    public static function initialize(SEOData $SEOData = null): static
+    {
+        $collection = new static();
+
+        $robotsContent = config('seo.robots.default');
+
+        if ( ! config('seo.robots.force_default') ) {
+            $robotsContent = $SEOData?->robots ?? $robotsContent;
+        }
+
+        $collection->push(new MetaTag('robots', $robotsContent));
+
+        return $collection;
+    }
+}

src/Tags/SitemapTag.php

@@ -0,0 +1,25 @@
+<?php
+
+namespace RalphJSmit\Laravel\SEO\Tags;
+
+use Illuminate\Contracts\Support\Renderable;
+use Illuminate\Support\Collection;
+use RalphJSmit\Laravel\SEO\Support\RenderableCollection;
+use RalphJSmit\Laravel\SEO\Support\SEOData;
+use RalphJSmit\Laravel\SEO\Support\SitemapTag as SitemapTagSupport;
+
+class SitemapTag extends Collection implements Renderable
+{
+    use RenderableCollection;
+
+    public static function initialize(SEOData $SEOData = null): static
+    {
+        $collection = new static();
+
+        if ( $sitemap = config('seo.sitemap') ) {
+            $collection->push(new SitemapTagSupport($sitemap));
+        }
+
+        return $collection;
+    }
+}

tests/Feature/Tags/CanonicalTagTest.php

@@ -0,0 +1,20 @@
+<?php
+
+use Illuminate\Support\Str;
+
+use function Pest\Laravel\get;
+
+it('can display the canonical URL if allowed', function () {
+    config()->set('seo.canonical_link', true);
+
+    get($url = route('seo.test-plain', ['name' => 'robots']))
+        ->assertSee('<link rel="canonical" href="' . Str::before($url, '?name') . '">', false);
+});
+
+it('cannot display the canonical url if not allowed', function () {
+    config()->set('seo.canonical_link', false);
+
+    get($url = route('seo.test-plain', ['name' => 'robots']))
+        ->assertDontSee('rel="canonical"', false);
+});
+

tests/Feature/Tags/RobotsTagTest.php

@@ -0,0 +1,69 @@
+<?php
+
+use RalphJSmit\Laravel\SEO\Support\SEOData;
+use RalphJSmit\Laravel\SEO\Tests\Fixtures\Page;
+
+use function Pest\Laravel\get;
+use function Pest\Laravel\post;
+
+it('can output the robots tag "default" value', function () {
+    config()->set('seo.robots.default', 'max-snippet:-1');
+
+    get($url = route('seo.test-plain'))
+        ->assertSee('<meta name="robots" content="max-snippet:-1">', false);
+});
+
+it('can overwrite the robots tag "default" value with the robots attribute (SEOData)', function () {
+    config()->set('seo.robots.default', 'max-snippet:-1');
+    config()->set('seo.robots.force_default', false);
+
+    $SEOData = new SEOData(
+        robots: 'noindex,nofollow',
+    );
+    $SEODataOutput = (string) seo($SEOData);
+
+    $this->assertStringContainsString('<meta name="robots" content="noindex,nofollow">', $SEODataOutput);
+});
+
+it('cannot overwrite the robots tag "default" value with the robots attribute if "force_default" is set (SEOData)', function () {
+    config()->set('seo.robots.default', 'max-snippet:-1');
+    config()->set('seo.robots.force_default', true);
+
+    $SEOData = new SEOData(
+        robots: 'noindex,nofollow',
+    );
+    $SEODataOutput = (string) seo($SEOData);
+
+    $this->assertStringContainsString('<meta name="robots" content="max-snippet:-1">', $SEODataOutput);
+});
+
+it('can overwrite the robots tag "default" value with the robots attribute (DB Model)', function () {
+    config()->set('seo.robots.default', 'max-snippet:-1');
+
+    $page = Page::create();
+
+    $page->seo->update([
+        'robots' => 'noindex,nofollow',
+    ]);
+
+    $page->refresh();
+
+    get(route('seo.test-page', ['page' => $page]))
+        ->assertSee('<meta name="robots" content="noindex,nofollow">', false);
+});
+
+it('cannot overwrite the robots tag "default" value with the robots attribute if "force_default" is set (DB Model)', function () {
+    config()->set('seo.robots.default', 'max-snippet:-1');
+    config()->set('seo.robots.force_default', true);
+
+    $page = Page::create();
+
+    $page->seo->update([
+        'robots' => 'noindex,nofollow',
+    ]);
+
+    $page->refresh();
+
+    get(route('seo.test-page', ['page' => $page]))
+        ->assertSee('<meta name="robots" content="max-snippet:-1">', false);
+});