Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated testcases related to Origins API #2

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Updated testcases related to Origins API #2

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
37ddc9b
Updated testcases related to Origins API
yashkohli88 Dec 18, 2024
ac4ff44
Updated testcase for maven and gradle
yashkohli88 Dec 20, 2024
f14fdb5
Refactored code for better performance
yashkohli88 Jan 8, 2025
a99f7cd
Update copyright header
yashkohli88 Jan 8, 2025
df1c517
Updated originsTest to handle timeout
yashkohli88 Jan 13, 2025
b5defbf
Fix formatting
yashkohli88 Jan 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 47 additions & 42 deletions tools/integration/test/integration/e2e-test-service/originsTest.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,61 +2,67 @@ const assert = require('assert')
const { callFetch } = require('../../../lib/fetch')
const { devApiBaseUrl, prodApiBaseUrl, getComponents, definition } = require('../testConfig')

const ORIGIN_EXCLUSIONS = ['go/golang', 'debsrc/debian', 'maven/mavengoogle']
const ORIGIN_REVISIONS_EXCLUSIONS = ['debsrc/debian']
const EXTRA_COMPONENTS = ['maven/mavencentral/org.apache.httpcomponents', 'maven/mavencentral/org.apache.httpcompon']
const ORIGIN_EXCLUSION_LIST = ['go/golang', 'debsrc/debian', 'maven/mavengoogle']
const ORIGIN_REVISIONS_EXCLUSION_LIST = ['debsrc/debian']

const MAVEN_COMPONENT_GROUP_ID = 'maven/mavencentral/org.apache.httpcomponents'
const MAVEN_COMPONENT_PARTIAL_GROUP_ID = 'maven/mavencentral/org.apache.httpcompon'
const GRADLE_COMPONENT_ENDPOINT = 'gradleplugin/io.github.lognet.grpc-spring-boot'

;(async function validateOriginsApi() {
const components = await getComponents()

describe('Validate origins API between dev and prod', function () {
describe('Validate Origins API Between Dev and Prod', function () {
this.timeout(definition.timeout)

// Rest interval to avoid overloading the servers
afterEach(() => new Promise(resolve => setTimeout(resolve, definition.timeout / 2)))

components.forEach(component => {
if (shouldCheckOrigins(component)) {
it(`checks origins API response for ${component}`, () => fetchAndCompareOrigins(component))
}
if (shouldCheckOriginsWithRevisions(component)) {
it(`checks origins API with revisions response for ${component}`, () =>
fetchAndCompareOriginsWithRevisions(component))
}
components.filter(isOriginAllowed).forEach(component => {
it(`Validates Origins API response for ${component}`, () => compareOrigins(component))
})

EXTRA_COMPONENTS.forEach(component => {
if (shouldCheckOrigins(component)) {
it(`checks origins API response for ${component}`, () => fetchAndCompareOrigins(component))
}
components.filter(isOriginWithRevisionsAllowed).forEach(component => {
it(`Validates Origins API response with revisions for ${component}`, () => compareOriginsWithRevisions(component))
})

it('Validates Origins API response for a Maven component with only a group ID', () =>
compareOrigins(MAVEN_COMPONENT_GROUP_ID))

it('Validates Origins API response for a Maven component with a partial group ID for suggestion checks', () =>
compareOrigins(MAVEN_COMPONENT_PARTIAL_GROUP_ID))

it('Validates Origins API response for a Gradle plugin component', () =>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we have to separate gradle components?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have used separate test cases for gradle since the endpoint for it is origins/gradleplugin/:pluginID where pluginID is not consistent with the name of group id or artefact id like

Here is the case where pluginId is combination of group id and artefact id
https://plugins.gradle.org/plugin/io.outfoxx.sunday-generator
https://mvnrepository.com/artifact/io.outfoxx.sunday/generator

Here is the case where pluginId is only group id -
https://plugins.gradle.org/plugin/io.github.lognet.grpc-spring-boot
https://mvnrepository.com/artifact/io.github.lognet.grpc-spring-boot/io.github.lognet.grpc-spring-boot.gradle.plugin

Please let me know if I could handle this better. Most of the cases corresponds pluginId to group id. In that case we would also need to add another buildurl function for gradleplugin.

compareEndpoints(GRADLE_COMPONENT_ENDPOINT))

it('Validates Origins API with revisions response for a Gradle plugin component', () =>
compareEndpoints(`${GRADLE_COMPONENT_ENDPOINT}/revisions`))
})
})()

function extractIds(response) {
return response.map(item => item.id)
}

function assertOriginsMatch(actual, expected) {
function assertResponsesMatch(actual, expected) {
const sortedActualIds = extractIds(actual).sort()
const sortedExpectedIds = extractIds(expected).sort()

assert.deepStrictEqual(sortedActualIds, sortedExpectedIds)
}

function isNotInExclusionList(list, coordinate) {
return !list.some(excluded => coordinate.includes(excluded))
function isCoordinateAllowed(coordinate, exclusionList) {
return !exclusionList.some(excluded => coordinate.includes(excluded))
}

function shouldCheckOrigins(coordinate) {
return isNotInExclusionList(ORIGIN_EXCLUSIONS, coordinate)
function isOriginAllowed(coordinate) {
return isCoordinateAllowed(coordinate, ORIGIN_EXCLUSION_LIST)
}

function shouldCheckOriginsWithRevisions(coordinate) {
return isNotInExclusionList(ORIGIN_REVISIONS_EXCLUSIONS, coordinate)
function isOriginWithRevisionsAllowed(coordinate) {
return isCoordinateAllowed(coordinate, ORIGIN_REVISIONS_EXCLUSION_LIST)
}

function resolveProviderType(type, provider) {
function getProviderType(type, provider) {
switch (type) {
case 'git':
return 'github'
Expand All @@ -72,37 +78,36 @@ function resolveProviderType(type, provider) {
}

function parseCoordinates(coordinates) {
const [type, provider, namespaceToken, name, version] = coordinates.split('/')
const namespace = namespaceToken === '-' ? '' : namespaceToken
const [type, provider, namespaceOrEmpty, name, version] = coordinates.split('/')
const namespace = namespaceOrEmpty === '-' ? '' : namespaceOrEmpty
return { type, provider, namespace, name, version }
}
function buildCondaURL(coordinates) {

function buildCondaUrl(coordinates) {
const { type, provider, name } = parseCoordinates(coordinates)
return `${type}/${provider}/${name}`
}

function buildOriginUrl(coordinates) {
const { type, provider, namespace, name } = parseCoordinates(coordinates)
const resolvedType = resolveProviderType(type, provider)
const resolvedType = getProviderType(type, provider)
return `${resolvedType}${namespace ? `/${namespace}` : ''}${name ? `/${name}` : ''}`
}

async function fetchAndCompareOriginsWithRevisions(coordinates) {
const originUrl = buildOriginUrl(coordinates)
async function compareEndpoints(endpoint) {
const [devResponse, prodResponse] = await Promise.all([
callFetch(`${devApiBaseUrl}/origins/${originUrl}/revisions`).then(res => res.json()),
callFetch(`${prodApiBaseUrl}/origins/${originUrl}/revisions`).then(res => res.json())
callFetch(`${devApiBaseUrl}/origins/${endpoint}`).then(res => res.json()),
callFetch(`${prodApiBaseUrl}/origins/${endpoint}`).then(res => res.json())
])
assertOriginsMatch(devResponse, prodResponse)
assertResponsesMatch(devResponse, prodResponse)
}

async function fetchAndCompareOrigins(coordinates) {
const originUrl = coordinates.startsWith('conda/') ? buildCondaURL(coordinates) : buildOriginUrl(coordinates)

const [devResponse, prodResponse] = await Promise.all([
callFetch(`${devApiBaseUrl}/origins/${originUrl}`).then(res => res.json()),
callFetch(`${prodApiBaseUrl}/origins/${originUrl}`).then(res => res.json())
])
async function compareOriginsWithRevisions(coordinates) {
const originUrl = buildOriginUrl(coordinates)
compareEndpoints(`${originUrl}/revisions`)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

await needs to be added

}

assertOriginsMatch(devResponse, prodResponse)
async function compareOrigins(coordinates) {
const originUrl = coordinates.startsWith('conda/') ? buildCondaUrl(coordinates) : buildOriginUrl(coordinates)
compareEndpoints(`${originUrl}`)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

await needs to be added

}
Loading