Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bulk enable/disable audits #302

Draft
wants to merge 30 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
00435c9
feat: bulk toggle audits
alinarublea May 17, 2024
6bbbffa
Merge branch 'main' into issue-115
alinarublea May 17, 2024
00fa62c
Merge branch 'main' into issue-115
alinarublea May 21, 2024
924650c
Update src/controllers/sites.js
alinarublea May 23, 2024
2a97058
Merge branch 'main' into issue-115
alinarublea May 23, 2024
6014416
feat: bulk toggle audits
alinarublea May 23, 2024
5971d85
Merge branch 'main' into issue-115
alinarublea May 23, 2024
ff6e142
feat: bulk toggle audits
alinarublea May 23, 2024
53b90ad
Merge branch 'main' into issue-115
alinarublea May 24, 2024
8685326
feat: code review
alinarublea May 24, 2024
5f4e530
Merge branch 'issue-115' of github.com:adobe/spacecat-api-service int…
alinarublea May 24, 2024
e0ca1b6
feat: temp disable tests for manual testing purposes
alinarublea May 27, 2024
fb45552
feat: temp disable tests for manual testing purposes
alinarublea May 27, 2024
57dc03a
feat: test slack command
alinarublea May 27, 2024
c30427a
feat: test slack command
alinarublea May 27, 2024
0c3e480
feat: test slack command
alinarublea May 27, 2024
1db4063
feat: test slack command
alinarublea May 27, 2024
c4ec341
feat: test slack command
alinarublea May 27, 2024
0a28d20
Merge branch 'main' into issue-115
alinarublea May 27, 2024
747e4c9
feat: test slack command
alinarublea May 27, 2024
6e3dd9e
feat: test slack command
alinarublea May 27, 2024
7f234e5
feat: slack command + partial unit tests
alinarublea May 28, 2024
cc5f505
Merge branch 'main' into issue-115
alinarublea May 29, 2024
859f899
feat: slack command + partial unit tests
alinarublea May 29, 2024
3e01356
Merge branch 'issue-115' of github.com:adobe/spacecat-api-service int…
alinarublea May 29, 2024
6e696ba
feat: slack command + partial unit tests
alinarublea May 29, 2024
c694b40
fix: allow default organization
alinarublea Jun 14, 2024
24bbd14
feat: apis for updating metadata and redirects
alinarublea Aug 29, 2024
5db3d4d
feat: apis for updating metadata and redirects
alinarublea Aug 29, 2024
1d2d0e8
Merge branch 'main' into issue-115
alinarublea Sep 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/openapi/api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ paths:
$ref: './site-api.yaml#/site-by-base-url'
/sites/with-latest-audit/{auditType}:
$ref: './sites-api.yaml#/sites-with-latest-audit'
/sites/bulk-audit-config:
$ref: './sites-api.yaml#/bulk-update-sites-audit-config'
/sites/{siteId}:
$ref: './site-api.yaml#/site'
/sites/{siteId}/audits:
Expand Down
22 changes: 22 additions & 0 deletions docs/openapi/schemas.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,28 @@ SiteCreate:
organizationId: 'o1p2q3r4-s5t6-u7v8-w9x0-yz12x34y56z'
baseURL: 'https://www.newsite.com'
deliveryType: 'aem_cs'
BulkUpdateSitesConfigResponse:
type: array
items:
type: object
properties:
baseURL:
type: string
description: The base URL of the site
response:
type: object
description: The response for the site update operation
properties:
status:
type: string
description: The status of the operation
message:
type: string
description: The message of the operation
site:
$ref: './schemas.yaml#/SiteDto'
required:
- status
SiteUpdate:
type: object
properties:
Expand Down
44 changes: 44 additions & 0 deletions docs/openapi/sites-api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -125,3 +125,47 @@ sites-for-organization:
$ref: './responses.yaml#/500'
security:
- api_key: [ ]
bulk-update-sites-audit-config:
patch:
tags:
- site
summary: Enable/Disable audits for multiple sites
description: |
This endpoint is useful for enabling audit types for multiple sites.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description and operationId don't seem to match to what the code does, which is also affect organization config...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is because as of now, we cannot enable a site without enabling the org. Please confirm @iuliag or @ekremney

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's fine - all i'm saying is the API doc should transparently describe the effects.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree to rename to make this more clear.

Specifically for enabling the org, I will add here the discussion I had with @alinarublea :
Users might not always want to enable/disable at the org level when they do this for the site.
E.g. case we had with a customer, where we had to enable the whole org and disable a part of the sites within that org. So we need a way to disable at site level without disabling at the same time at org level.

For enable = true, it's pointless to enable just the site, because anyways the audit doesn't run unless the org is enabled as well. In this case, we'd want to unify those 2 changes in 1 API call.

operationId: bulkUpdateAuditConfigForSites
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
baseURLs:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the primary key of sites is the siteId (as used in routes like /sites/<siteId> for example) - is it a conscious decision to use baseURL instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is coming from #115, because if we want to easily use the API, we know the baseURL of a site, whereas for the id we need to do another request

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iuliag can you elaborate? i don't see in #115 where the requirement/fact that we only have baseURLs comes from. the API usually works with ids.

Copy link
Contributor

@iuliag iuliag May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@solaris007 , the consumers generally know the site URL and it's a hassle for them to:

In all requests we've received about configuring/looking at some data for a site, only the URL is mentioned (or sometimes just the customer name), not the site id.

type: array
items:
$ref: './schemas.yaml#/URL'
auditTypes:
type: array
items:
type: string
examples:
- 404
- broken-backlinks
- organic-traffic
enableAudits:
type: boolean
responses:
'207':
description: A list of sites
content:
application/json:
schema:
$ref: './schemas.yaml#/BulkUpdateSitesConfigResponse'
'401':
$ref: './responses.yaml#/401'
'500':
$ref: './responses.yaml#/500'
security:
- admin_key: [ ]


3 changes: 1 addition & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,8 @@
"@adobe/spacecat-shared-data-access": "1.23.4",
"@adobe/spacecat-shared-http-utils": "1.3.0",
"@adobe/spacecat-shared-ims-client": "1.3.5",
"@adobe/spacecat-shared-slack-client": "1.3.5",
"@adobe/spacecat-shared-rum-api-client": "1.8.2",
"@adobe/spacecat-shared-slack-client": "1.3.5",
"@adobe/spacecat-shared-utils": "1.15.2",
"@aws-sdk/client-s3": "3.577.0",
"@aws-sdk/client-sqs": "3.577.0",
Expand All @@ -89,7 +89,7 @@
"chai-as-promised": "7.1.2",
"dotenv": "16.4.5",
"eslint": "8.57.0",
"esmock": "2.6.5",
"esmock": "^2.6.5",
alinarublea marked this conversation as resolved.
Show resolved Hide resolved
"husky": "9.0.11",
"junit-report-builder": "3.2.1",
"lint-staged": "15.2.2",
Expand Down
73 changes: 73 additions & 0 deletions src/controllers/sites.js
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,78 @@ function SitesController(dataAccess) {
return ok(sites);
};

/** Bulk update audit configuration for multiple sites.
* @param {object} context - Context of the request.
* @returns {Promise<Response>} Array of sites response.
*/
const bulkUpdateSitesConfig = async (context) => {
const { baseURLs, enableAudits, auditTypes } = context.data;

if (!Array.isArray(baseURLs) || baseURLs.length === 0) {
return badRequest('Base URLs are required');
}
if (!Array.isArray(auditTypes) || auditTypes.length === 0) {
return badRequest('Audit types are required');
}
if (!isBoolean(enableAudits)) {
return badRequest('enableAudits is required');
}

const organizationsMap = new Map();

const sites = await Promise.all(baseURLs.map(async (baseURL) => {
const site = await dataAccess.getSiteByBaseURL(baseURL);
if (!site) {
return { baseURL, site: null };
}
const organizationId = site.getOrganizationId();

if (organizationId !== 'default' && !organizationsMap.has(organizationId)) {
const organization = await dataAccess.getOrganizationByID(organizationId);
if (!organization) {
return { baseURL, response: { message: `Organization with ID: ${organizationId} not found`, status: 404 } };
}
organizationsMap.set(organizationId, organization);
}

return { baseURL, site };
}));

const responses = await Promise.all(sites.map(async ({ baseURL, site }) => {
if (!site) {
return { baseURL, response: { message: `Site with baseURL: ${baseURL} not found`, status: 404 } };
}
const organizationId = site.getOrganizationId();
const organization = organizationsMap.get(organizationId);

auditTypes.forEach((auditType) => {
if (organization) {
organization.getAuditConfig().updateAuditTypeConfig(
auditType,
);
}
site.getAuditConfig().updateAuditTypeConfig(auditType, { auditsDisabled: !enableAudits });
});

if (organization && enableAudits) {
try {
await dataAccess.updateOrganization(organization);
} catch (error) {
return { baseURL: site.getBaseURL(), message: `Error updating organization with id: ${organizationId}`, status: 500 };
}
}
try {
await dataAccess.updateSite(site);
} catch (error) {
return { baseURL: site.getBaseURL(), response: { message: `Error updating site with id: ${site.getId()}`, status: 500 } };
}

return { baseURL: site.getBaseURL(), response: { body: SiteDto.toJSON(site), status: 200 } };
}));

return createResponse(responses, 207);
};

/**
* Gets all sites as an XLS file.
* @returns {Promise<Response>} XLS file.
Expand Down Expand Up @@ -377,6 +449,7 @@ function SitesController(dataAccess) {
getAuditForSite,
getByBaseURL,
getAllByDeliveryType,
bulkUpdateSitesConfig,
getByID,
removeSite,
updateSite,
Expand Down
1 change: 1 addition & 0 deletions src/routes/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ export default function getRouteHandlers(
'GET /sites/by-base-url/:baseURL': sitesController.getByBaseURL,
'GET /sites/by-delivery-type/:deliveryType': sitesController.getAllByDeliveryType,
'GET /sites/with-latest-audit/:auditType': sitesController.getAllWithLatestAudit,
'PATCH /sites/bulk-audit-config': sitesController.bulkUpdateSitesConfig,
'GET /slack/events': slackController.handleEvent,
'POST /slack/events': slackController.handleEvent,
'POST /slack/channels/invite-by-user-id': slackController.inviteUserToChannel,
Expand Down
64 changes: 64 additions & 0 deletions src/support/slack/commands/bulk-enable-audits.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
/*
* Copyright 2024 Adobe. All rights reserved.
* This file is licensed to you under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License. You may obtain a copy
* of the License at http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under
* the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR REPRESENTATIONS
* OF ANY KIND, either express or implied. See the License for the specific language
* governing permissions and limitations under the License.
*/

import BaseCommand from './base.js';
import bulkUpdateSitesConfig from '../../../controllers/sites.js';

const PHRASES = ['bulk enable audits'];

function BulkEnableAuditsCommand(context) {
const baseCommand = BaseCommand({
id: 'bulk-enable-audits',
name: 'Bulk Enable Audits',
description: 'Enables audits for multiple sites.',
phrases: PHRASES,
usageText: `${PHRASES[0]} {site1,site2,...} {auditType1,auditType2,...}`,
});

const { log } = context;

const handleExecution = async (args, slackContext) => {
const { say } = slackContext;

try {
const [baseURLsInput, auditTypesInput] = args;

const baseURLs = baseURLsInput.split(',');
const auditTypes = auditTypesInput.split(',');

const enableAudits = true;

const responses = await bulkUpdateSitesConfig({
data: { baseURLs, enableAudits, auditTypes },
});

let message = 'Bulk update completed with the following responses:\n';
responses.forEach((response) => {
message += `- ${response.baseURL}: ${response.response.status}\n`;
});

await say(message);
} catch (error) {
log.error(error);
await say(`Error during bulk update: ${error.message}`);
}
};

baseCommand.init(context);

return {
...baseCommand,
handleExecution,
};
}

export default BulkEnableAuditsCommand;
66 changes: 66 additions & 0 deletions test/controllers/sites.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ describe('Sites Controller', () => {
'getAll',
'getAllByDeliveryType',
'getAllWithLatestAudit',
'bulkUpdateSitesConfig',
'getAllAsCSV',
'getAllAsXLS',
'getAuditForSite',
Expand Down Expand Up @@ -639,4 +640,69 @@ describe('Sites Controller', () => {
expect(result.status).to.equal(404);
expect(error).to.have.property('message', 'Site not found');
});
describe('bulkUpdateSitesConfig', () => {
it('updates multiple sites and returns their responses', async () => {
const baseURLs = ['https://site1.com', 'https://site2.com'];
const enableAudits = true;
const auditTypes = ['type1', 'type2'];

const site1 = SiteDto.fromJson({ id: 'site1', baseURL: 'https://site1.com', deliveryType: 'aem_edge' });
const site2 = SiteDto.fromJson({ id: 'site2', baseURL: 'https://site2.com', deliveryType: 'aem_edge' });

mockDataAccess.getSiteByBaseURL.withArgs('https://site1.com').resolves(site1);
mockDataAccess.getSiteByBaseURL.withArgs('https://site2.com').resolves(site2);

const response = await sitesController.bulkUpdateSitesConfig({
data: { baseURLs, enableAudits, auditTypes },
});

expect(mockDataAccess.getSiteByBaseURL.calledTwice).to.be.true;
expect(mockDataAccess.updateSite.calledTwice).to.be.true;
expect(response.status).to.equal(207);
const multiResponse = await response.json();
expect(multiResponse).to.be.an('array').with.lengthOf(2);
expect(multiResponse[0].baseURL).to.equal('https://site1.com');
expect(multiResponse[0].response.status).to.equal(200);
expect(multiResponse[1].baseURL).to.equal('https://site2.com');
expect(multiResponse[1].response.status).to.equal(200);
});

it('returns bad request when baseURLs is not provided', async () => {
const response = await sitesController.bulkUpdateSitesConfig({ data: {} });
const error = await response.json();

expect(response.status).to.equal(400);
expect(error).to.have.property('message', 'Base URLs are required');
});

it('returns bad request when auditTypes is not provided', async () => {
const response = await sitesController.bulkUpdateSitesConfig({ data: { baseURLs: ['https://site1.com'] } });
const error = await response.json();

expect(response.status).to.equal(400);
expect(error).to.have.property('message', 'Audit types are required');
});

it('returns bad request when enableAudits is not provided', async () => {
const response = await sitesController.bulkUpdateSitesConfig({ data: { baseURLs: ['https://site1.com'], auditTypes: ['type1'] } });
const error = await response.json();

expect(response.status).to.equal(400);
expect(error).to.have.property('message', 'enableAudits is required');
});

it('returns not found when site is not found', async () => {
mockDataAccess.getSiteByBaseURL.withArgs('https://site1.com').resolves(null);

const response = await sitesController.bulkUpdateSitesConfig({
data: { baseURLs: ['https://site1.com'], enableAudits: true, auditTypes: ['type1'] },
});
const responses = await response.json();

expect(responses).to.be.an('array').with.lengthOf(1);
expect(responses[0].baseURL).to.equal('https://site1.com');
expect(responses[0].response.status).to.equal(404);
expect(responses[0].response.message).to.equal('Site with baseURL: https://site1.com not found');
});
});
});
Loading
Loading