Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EDSC-3773: Refactor buildAccessMethods.js #1767

Merged
merged 41 commits into from
Aug 29, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
0eb7d13
EDSC-3773:
bnp26 Jul 17, 2024
88be2fc
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Jul 17, 2024
bcf7ead
removed dangling cammas
bnp26 Jul 17, 2024
cd4600f
Merge branch 'main' into EDSC-3773
bnp26 Jul 17, 2024
cd93a33
EDSC-3773: a bit of cleanup
bnp26 Jul 17, 2024
81621ad
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Jul 17, 2024
43bde25
Merge branch 'EDSC-3773' of https://github.com/nasa/earthdata-search …
bnp26 Jul 17, 2024
c543c69
EDSC-3773: added some code coverage
bnp26 Jul 17, 2024
483ed3f
EDSC-3773: updated documentation and how we run buildAccessMethods.
bnp26 Jul 19, 2024
2338f90
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Jul 19, 2024
d7993e2
EDSC-3773: changed lodash to lodash-es
bnp26 Jul 19, 2024
e9df974
EDSC-3773: fixed the indexing issue and cleaned up some stuff
bnp26 Jul 22, 2024
5c49a41
EDSC-3773: fixed harmony tests
bnp26 Jul 23, 2024
d03175a
Merge branch 'main' into EDSC-3773
bnp26 Jul 23, 2024
8388e9c
Merge branch 'main' into EDSC-3773
bnp26 Jul 23, 2024
b111fab
EDSC-3773: addressed matthew's comments
bnp26 Jul 24, 2024
444015b
Merge branch 'EDSC-3773' of https://github.com/nasa/earthdata-search …
bnp26 Jul 24, 2024
84d7d88
EDSC-3773: added some comments and pushed a small lint fix
bnp26 Jul 25, 2024
62d8a48
EDSC-3773: fixed some eslint issues
bnp26 Jul 25, 2024
898652c
fixed some wording
bnp26 Jul 25, 2024
c37b9fe
Merge branch 'main' into EDSC-3773
bnp26 Jul 29, 2024
62d2014
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Jul 29, 2024
15cd00b
Merge branch 'EDSC-3773' of https://github.com/nasa/earthdata-search …
bnp26 Jul 29, 2024
1ab5025
EDSC-3773: added a test that needs to be finished in buildAccessMethods
bnp26 Jul 29, 2024
080a64e
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Aug 14, 2024
2a5c1b8
Merge branch 'main' into EDSC-3773
bnp26 Aug 16, 2024
737fe81
EDSC-3773: fixed up the format and tests of Harmony, Opendap, and Swo…
bnp26 Aug 19, 2024
3d22271
EDSC-3773: removed console.logs
bnp26 Aug 19, 2024
d26308f
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Aug 19, 2024
3d59e59
Merge branch 'EDSC-3773' of https://github.com/nasa/earthdata-search …
bnp26 Aug 19, 2024
56a5e74
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Aug 19, 2024
feab563
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Aug 22, 2024
7f99a33
Merge branch 'main' into EDSC-3773
bnp26 Aug 26, 2024
49755c1
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Aug 27, 2024
afe3ca8
EDSC-3773: fixed up syntax/linting so it matches our standards
bnp26 Aug 27, 2024
ef0ce8a
Merge branch 'EDSC-3773' of https://github.com/nasa/earthdata-search …
bnp26 Aug 27, 2024
9ee253d
EDSC-3773: forgot a couple of the lint changes
bnp26 Aug 27, 2024
11975a8
adding test and seeing weirdness
bnp26 Aug 29, 2024
552c806
Merge branch 'main' of https://github.com/nasa/earthdata-search into …
bnp26 Aug 29, 2024
5f8a29e
fixed up the test
bnp26 Aug 29, 2024
fa630bf
EDSC-3773: merged new test and removed console.logs and fixed the ord…
bnp26 Aug 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,598 changes: 935 additions & 663 deletions static/src/js/util/accessMethods/__tests__/buildAccessMethods.test.js
bnp26 marked this conversation as resolved.
Show resolved Hide resolved

Large diffs are not rendered by default.

230 changes: 47 additions & 183 deletions static/src/js/util/accessMethods/buildAccessMethods.js
Original file line number Diff line number Diff line change
@@ -1,16 +1,8 @@
import { camelCase, uniq } from 'lodash-es'

import { getApplicationConfig } from '../../../../../sharedUtils/config'

import { isDownloadable } from '../../../../../sharedUtils/isDownloadable'
import { generateFormDigest } from './generateFormDigest'
import { getVariables } from './getVariables'
import { supportsBoundingBoxSubsetting } from './supportsBoundingBoxSubsetting'
import { supportsConcatenation } from './supportsConcatenation'
import { defaultConcatenation } from './defaultConcatenation'
import { supportsShapefileSubsetting } from './supportsShapefileSubsetting'
import { supportsTemporalSubsetting } from './supportsTemporalSubsetting'
import { supportsVariableSubsetting } from './supportsVariableSubsetting'
import { buildEsiEcho } from './buildAccessMethods/buildEsiEcho'
import { buildOpendap } from './buildAccessMethods/buildOpendap'
import { buildHarmony } from './buildAccessMethods/buildHarmony'
import { buildSwodlr } from './buildAccessMethods/buildSwodlr'
import { buildDownload } from './buildAccessMethods/buildDownload'

/**
* Builds the different access methods available for the provided collection
Expand All @@ -25,25 +17,27 @@ export const buildAccessMethods = (collectionMetadata, isOpenSearch) => {
variables: collectionAssociatedVariables = {}
} = collectionMetadata

const accessMethods = {}
let harmonyIndex = 0
let echoIndex = 0
let esiIndex = 0

const { items: serviceItems = null } = services

const { disableOrdering, disableSwodlr } = getApplicationConfig()
const buildMethods = {
esi: (serviceItem, params) => buildEsiEcho(serviceItem, params),
'echo orders': (serviceItem, params) => buildEsiEcho(serviceItem, params),
opendap: (serviceItem, params) => buildOpendap(serviceItem, params),
harmony: (serviceItem, params) => buildHarmony(serviceItem, params),
swodlr: (serviceItem) => buildSwodlr(serviceItem),
downloads: () => buildDownload(granules, isOpenSearch)
}

if (serviceItems !== null) {
serviceItems.forEach((serviceItem) => {
const nonDownloadMethods = serviceItems === null
? {}
: serviceItems.reduce((methods, serviceItem) => {
let associatedVariables = collectionAssociatedVariables
const {
conceptId: serviceConceptId,
description,
orderOptions,
type: serviceType,
url,
longName,
maxItemsPerOrder,
name,
supportedReformattings,
variables: serviceAssociatedVariables = {}
} = serviceItem

Expand All @@ -52,175 +46,45 @@ export const buildAccessMethods = (collectionMetadata, isOpenSearch) => {
associatedVariables = serviceAssociatedVariables
}

const lowerServiceType = serviceType.toLowerCase()

// Only process service types that EDSC supports
const supportedServiceTypes = ['esi', 'echo orders', 'opendap', 'harmony', 'swodlr']
if (!supportedServiceTypes.includes(serviceType.toLowerCase())) return

const { urlValue } = url

const supportsOrderOptions = ['esi', 'echo orders']

// Only process orderOptions if the service type uses orderOptions
// Do not include access if orders are disabled
if (supportsOrderOptions.includes(serviceType.toLowerCase()) && (disableOrdering !== 'true')) {
const { items: orderOptionsItems } = orderOptions
if (orderOptionsItems === null) return

orderOptionsItems.forEach((orderOptionItem, orderOptionIndex) => {
const {
conceptId: orderOptionConceptId,
form,
name: orderOptionName
} = orderOptionItem

const method = {
type: serviceType,
maxItemsPerOrder,
url: urlValue,
optionDefinition: {
conceptId: orderOptionConceptId,
name: orderOptionName
},
form,
formDigest: generateFormDigest(form)
}

let methodKey = camelCase(serviceType)

// `echoOrders` needs to be singular to match existing savedAccessConfigurations
if (methodKey === 'echoOrders') {
methodKey = 'echoOrder'
}

accessMethods[`${methodKey}${orderOptionIndex}`] = method
})
}
if (!supportedServiceTypes.includes(lowerServiceType)) return {}

if (serviceType.toLowerCase() === 'opendap') {
const {
hierarchyMappings,
keywordMappings,
variables
} = getVariables(associatedVariables)

const outputFormats = []

if (supportedReformattings) {
supportedReformattings.forEach((reformatting) => {
const { supportedOutputFormats } = reformatting

// Collect all supported output formats from each mapping
outputFormats.push(...supportedOutputFormats)
})
}

accessMethods.opendap = {
hierarchyMappings,
id: serviceConceptId,
isValid: true,
keywordMappings,
longName,
name,
supportedOutputFormats: uniq(outputFormats),
supportsVariableSubsetting: supportsVariableSubsetting(serviceItem),
type: serviceType,
variables
}
const params = {
harmonyIndex,
echoIndex,
esiIndex,
associatedVariables
}

if (serviceType.toLowerCase() === 'harmony') {
const {
hierarchyMappings,
keywordMappings,
variables
} = getVariables(associatedVariables)
const {
supportedOutputProjections
} = serviceItem

const outputFormats = []

if (supportedReformattings) {
supportedReformattings.forEach((reformatting) => {
const { supportedOutputFormats } = reformatting

// Collect all supported output formats from each mapping
outputFormats.push(...supportedOutputFormats)
})
}

let outputProjections = []
if (supportedOutputProjections) {
outputProjections = supportedOutputProjections.filter((projection) => {
const { projectionAuthority } = projection

return projectionAuthority != null
}).map((projection) => {
const { projectionAuthority } = projection

return projectionAuthority
})
}

accessMethods[`harmony${harmonyIndex}`] = {
description,
enableTemporalSubsetting: true,
enableSpatialSubsetting: true,
hierarchyMappings,
id: serviceConceptId,
isValid: true,
keywordMappings,
longName,
name,
supportedOutputFormats: uniq(outputFormats),
supportedOutputProjections: outputProjections,
supportsBoundingBoxSubsetting: supportsBoundingBoxSubsetting(serviceItem),
supportsShapefileSubsetting: supportsShapefileSubsetting(serviceItem),
supportsTemporalSubsetting: supportsTemporalSubsetting(serviceItem),
supportsVariableSubsetting: supportsVariableSubsetting(serviceItem),
supportsConcatenation: supportsConcatenation(serviceItem),
defaultConcatenation: defaultConcatenation(serviceItem),
enableConcatenateDownload: defaultConcatenation(serviceItem),
type: serviceType,
url: urlValue,
variables
}
const builtMethod = buildMethods[lowerServiceType](serviceItem, params)

harmonyIndex += 1
}
const {
accessMethods: newAccessMethods,
esiIndex: newEsiIndex,
echoIndex: newEchoIndex
} = builtMethod

if (serviceType.toLowerCase() === 'swodlr' && (disableSwodlr !== 'true')) {
accessMethods.swodlr = {
id: serviceConceptId,
isValid: true,
longName,
name,
type: serviceType,
supportsSwodlr: true,
url: urlValue
}
}
})
}
esiIndex = newEsiIndex
echoIndex = newEchoIndex

// Determine if the collection should have the downloadable accessMethod
let onlineAccessFlag = false
const updatedMethods = {
...methods,
...newAccessMethods
}

if (granules) {
// If the collection has granules, check their online access flags to
// determine if this collection is downloadable
const { items: granuleItems } = granules
if (lowerServiceType === 'harmony') {
harmonyIndex += 1
}

if (granuleItems) {
onlineAccessFlag = isDownloadable(granuleItems)
}
return updatedMethods
}, {})

if (onlineAccessFlag || isOpenSearch) {
accessMethods.download = {
isValid: true,
type: 'download'
}
}
const accessMethods = {
...nonDownloadMethods,
...buildMethods.downloads()
}

return accessMethods
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import { buildDownload } from '../buildDownload'

describe('buildDownload', () => {
test('returns a download access method', () => {
const collectionMetadata = {
granules: {
items: [{
online_access_flag: true
}]
}
}
const isOpenSearch = false

const {
granules
} = collectionMetadata

const method = buildDownload(granules, isOpenSearch)

expect(method).toEqual({
download: {
isValid: true,
type: 'download'
}
})
})

test('returns a download access method for open search', () => {
const collectionMetadata = {}
const isOpenSearch = true

const {
granules = {}
} = collectionMetadata

const method = buildDownload(granules, isOpenSearch)

expect(method).toEqual({
download: {
isValid: true,
type: 'download'
}
})
})
})
Loading