Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test kit e2e #337

Merged
merged 34 commits into from
Jul 31, 2022
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
6dc2ba6
change e2e, app structure
Idokah Jul 20, 2022
3ff85ed
close server instead of reload
Idokah Jul 20, 2022
e1e30d8
testkit lib
Idokah Jul 20, 2022
b672151
testkit lib and commons
Idokah Jul 20, 2022
8ebe7c2
e2e resources
Idokah Jul 20, 2022
b2b6edb
mysql test resources
Idokah Jul 21, 2022
49a28bd
testkit fixes
Idokah Jul 21, 2022
52d12bd
spanner test resources
Idokah Jul 21, 2022
92fa7db
postgres test resources
Idokah Jul 21, 2022
841d24a
firestore test resources
Idokah Jul 21, 2022
739bf47
mssql test resources
Idokah Jul 21, 2022
dcdba46
mongo e2e- not finished
Idokah Jul 26, 2022
9f233f0
fix mongo
Idokah Jul 26, 2022
fd1fbca
google-sheets
Idokah Jul 26, 2022
db5bfa0
fix export google-sheets
Idokah Jul 26, 2022
09e5065
airtable
Idokah Jul 26, 2022
158e67d
dynamodb
Idokah Jul 26, 2022
77bc122
e2e_resources
Idokah Jul 26, 2022
427de5d
bigquery
Idokah Jul 26, 2022
756429f
remove only
Idokah Jul 26, 2022
1c325f9
fix imports
Idokah Jul 26, 2022
f4d023d
provider resources, delete engines
Idokah Jul 26, 2022
cb73fea
add globalSetUp and globalTearDown
Idokah Jul 27, 2022
8e87a26
rebase firestore after ts
Idokah Jul 27, 2022
1debce9
bigquery rebase after ts
Idokah Jul 27, 2022
9b9f3fa
change imports
Idokah Jul 27, 2022
eab655a
fix imports
Idokah Jul 28, 2022
5abc4c4
add jest to tsconfig and remove ts-ignore
Idokah Jul 28, 2022
4c05786
export supportedOperations only from test resources
Idokah Jul 28, 2022
eb981fb
fix postgres import
Idokah Jul 28, 2022
59da64d
add networks to docker-compose for all images
Idokah Jul 28, 2022
48f1e18
upgrade versions
Idokah Jul 31, 2022
4eaeba7
nop
Idokah Jul 31, 2022
2e111bd
upgrade types package
Idokah Jul 31, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/velo-external-db/project.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"outputs": [ "{options.outputPath}" ],
"options": {
"outputPath": "dist/apps/velo-external-db",
"main": "apps/velo-external-db/src/app.ts",
"main": "apps/velo-external-db/src/main.ts",
"tsConfig": "apps/velo-external-db/tsconfig.app.json",
"generatePackageJson": true,
"progress": true,
Expand Down
31 changes: 8 additions & 23 deletions apps/velo-external-db/src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,16 @@ import express = require('express')
import { create, readCommonConfig } from '@wix-velo/external-db-config'
import { ExternalDbRouter } from '@wix-velo/velo-external-db-core'
import { engineConnectorFor } from './storage/factory'
import { Server } from 'http'
import { ConnectionCleanUp, ISchemaProvider } from '@wix-velo/velo-external-db-types'
import { Hooks } from 'libs/velo-external-db-core/src/types'

let started = false
let server: Server, _schemaProvider: ISchemaProvider, _cleanup: ConnectionCleanUp
let externalDbRouter: ExternalDbRouter



const initConnector = async (hooks?: Hooks) => {
const { vendor, type: adapterType } = readCommonConfig()
const configReader = create()
const { authorization, secretKey, ...dbConfig } = await configReader.readConfig()

const { connector: engineConnector, providers, cleanup } = await engineConnectorFor(adapterType, dbConfig)

externalDbRouter = new ExternalDbRouter({
const externalDbRouter = new ExternalDbRouter({
connector: engineConnector,
config: {
authorization: {
Expand All @@ -33,21 +25,14 @@ const initConnector = async (hooks?: Hooks) => {
hooks
})

_cleanup = async () => {
await cleanup()
}
_schemaProvider = providers.schemaProvider

return { externalDbRouter }
return { externalDbRouter, cleanup: async () => await cleanup(), schemaProvider: providers.schemaProvider }
}

initConnector().then(({ externalDbRouter }) => {
export const createApp = async () => {
const app = express()
app.use(externalDbRouter.router)

server = app.listen(8080, () => console.log('Connector listening on port 8080'))

started = true
})
const initConnectorResponse = await initConnector()
app.use(initConnectorResponse.externalDbRouter.router)
const server = app.listen(8080, () => console.log('Connector listening on port 8080'))

export const internals = () => ({ server, schemaProvider: _schemaProvider, cleanup: _cleanup, started, reload: initConnector, externalDbRouter })
return { server, ...initConnectorResponse, reload: () => initConnector() }
}
3 changes: 3 additions & 0 deletions apps/velo-external-db/src/main.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import { createApp } from "velo-external-db";
Idokah marked this conversation as resolved.
Show resolved Hide resolved

createApp()
2 changes: 1 addition & 1 deletion apps/velo-external-db/test/e2e/app.e2e.spec.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { authOwner } from '../drivers/auth_test_support'
import { authOwner } from '@wix-velo/external-db-testkit'
import { initApp, teardownApp, dbTeardown, setupDb, currentDbImplementationName } from '../resources/e2e_resources'

const axios = require('axios').create({
Expand Down
2 changes: 1 addition & 1 deletion apps/velo-external-db/test/e2e/app_auth.e2e.spec.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { Uninitialized, gen } from '@wix-velo/test-commons'
import { authVisitor, authOwnerWithoutSecretKey, errorResponseWith } from '../drivers/auth_test_support'
import { authVisitor, authOwnerWithoutSecretKey, errorResponseWith } from '@wix-velo/external-db-testkit'
import each from 'jest-each'
import { initApp, teardownApp, dbTeardown, setupDb, currentDbImplementationName } from '../resources/e2e_resources'

Expand Down
3 changes: 1 addition & 2 deletions apps/velo-external-db/test/e2e/app_data.e2e.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import * as gen from '../gen'
import * as schema from '../drivers/schema_api_rest_test_support'
import * as data from '../drivers/data_api_rest_test_support'
import * as matchers from '../drivers/schema_api_rest_matchers'
import { authAdmin, authOwner, authVisitor } from '../drivers/auth_test_support'
import { authAdmin, authOwner, authVisitor } from '@wix-velo/external-db-testkit'
import * as authorization from '../drivers/authorization_test_support'
import Chance = require('chance')
import { initApp, teardownApp, dbTeardown, setupDb, currentDbImplementationName, supportedOperations } from '../resources/e2e_resources'
Expand All @@ -20,7 +20,6 @@ const axios = require('axios').create({
describe(`Velo External DB Data REST API: ${currentDbImplementationName()}`, () => {
beforeAll(async() => {
await setupDb()

await initApp()
}, 20000)

Expand Down
2 changes: 1 addition & 1 deletion apps/velo-external-db/test/e2e/app_data_hooks.e2e.spec.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { authOwner, errorResponseWith } from '../drivers/auth_test_support'
import { authOwner, errorResponseWith } from '@wix-velo/external-db-testkit'
import { testSupportedOperations } from '@wix-velo/test-commons'
const { UpdateImmediately, DeleteImmediately, Aggregate } = require('@wix-velo/velo-external-db-commons').SchemaOperations
const each = require('jest-each').default
Expand Down
2 changes: 1 addition & 1 deletion apps/velo-external-db/test/e2e/app_schema.e2e.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { SchemaOperations } from '@wix-velo/velo-external-db-types'
const { RemoveColumn } = SchemaOperations
import * as schema from '../drivers/schema_api_rest_test_support'
import * as matchers from '../drivers/schema_api_rest_matchers'
import { authOwner } from '../drivers/auth_test_support'
import { authOwner } from '@wix-velo/external-db-testkit'
import * as gen from '../gen'
import Chance = require('chance')
import { initApp, teardownApp, dbTeardown, setupDb, currentDbImplementationName, supportedOperations } from '../resources/e2e_resources'
Expand Down
3 changes: 2 additions & 1 deletion apps/velo-external-db/test/e2e/app_schema_hooks.e2e.spec.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { authOwner, errorResponseWith } from '../drivers/auth_test_support'
import { authOwner, errorResponseWith } from '@wix-velo/external-db-testkit'
const each = require('jest-each').default
import { initApp, teardownApp, dbTeardown, setupDb, currentDbImplementationName, env } from '../resources/e2e_resources'
import gen = require('../gen')
Expand All @@ -7,6 +7,7 @@ import schema = require('../drivers/schema_api_rest_test_support')
import hooks = require('../drivers/hooks_test_support')

import { Uninitialized } from '@wix-velo/test-commons'

import { ServiceContext } from 'libs/velo-external-db-core/src/types'

const axios = require('axios').create({
Expand Down
20 changes: 10 additions & 10 deletions apps/velo-external-db/test/env/env.db.setup.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@ import { registerTsProject } from 'nx/src/utils/register'
registerTsProject('.', 'tsconfig.base.json')


const postgres = require('../resources/engines/postgres_resources')
const mysql = require('../resources/engines/mysql_resources')
const spanner = require('../resources/engines/spanner_resources')
const firestore = require('../resources/engines/firestore_resources')
const mssql = require('../resources/engines/mssql_resources')
const mongo = require ('../resources/engines/mongo_resources')
const googleSheet = require('../resources/engines/google_sheets_resources')
const airtable = require ('../resources/engines/airtable_resources')
const dynamoDb = require ('../resources/engines/dynamodb_resources')
const bigquery = require ('../resources/engines/bigquery_resources')
const { testResources: postgres } = require ('@wix-velo/external-db-postgres')
const { testResources: mysql } = require ('@wix-velo/external-db-mysql')
const { testResources: spanner } = require ('@wix-velo/external-db-spanner')
const { testResources: firestore } = require ('@wix-velo/external-db-firestore')
const { testResources: mssql } = require ('@wix-velo/external-db-mssql')
const { testResources: mongo } = require ('@wix-velo/external-db-mongo')
const { testResources: googleSheet } = require('@wix-velo/external-db-google-sheets')
const { testResources: airtable } = require('@wix-velo/external-db-airtable')
const { testResources: dynamoDb } = require('@wix-velo/external-db-dynamodb')
const { testResources: bigquery } = require('@wix-velo/external-db-bigquery')

const { sleep } = require('@wix-velo/test-commons')
const ci = require('./ci_utils')
Expand Down
24 changes: 15 additions & 9 deletions apps/velo-external-db/test/env/env.db.teardown.js
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
const postgres = require('../resources/engines/postgres_resources')
const mysql = require('../resources/engines/mysql_resources')
const spanner = require('../resources/engines/spanner_resources')
const firestore = require('../resources/engines/firestore_resources')
const mssql = require('../resources/engines/mssql_resources')
const googleSheet = require('../resources/engines/google_sheets_resources')
const airtable = require('../resources/engines/airtable_resources')
const dynamo = require('../resources/engines/dynamodb_resources')
const mongo = require ('../resources/engines/mongo_resources')
const { testResources: postgres } = require ('@wix-velo/external-db-postgres')
const { testResources: mysql } = require ('@wix-velo/external-db-mysql')
const { testResources: spanner } = require ('@wix-velo/external-db-spanner')
const { testResources: firestore } = require ('@wix-velo/external-db-firestore')
const { testResources: mssql } = require ('@wix-velo/external-db-mssql')
const { testResources: mongo } = require ('@wix-velo/external-db-mongo')
const { testResources: googleSheet } = require('@wix-velo/external-db-google-sheets')
const { testResources: airtable } = require('@wix-velo/external-db-airtable')
const { testResources: dynamo } = require('@wix-velo/external-db-dynamodb')
const { testResources: bigquery } = require('@wix-velo/external-db-bigquery')

const ci = require('./ci_utils')

const shutdownEnv = async(testEngine) => {
Expand Down Expand Up @@ -46,6 +48,10 @@ const shutdownEnv = async(testEngine) => {
case 'mongo':
await mongo.shutdownEnv()
break

case 'bigquery':
await bigquery.shutdownEnv()
break
}
}

Expand Down
102 changes: 34 additions & 68 deletions apps/velo-external-db/test/resources/e2e_resources.ts
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
import { Uninitialized, sleep } from '@wix-velo/test-commons'
import { suiteDef } from './test_suite_definition'
import { authInit } from '../drivers/auth_test_support'
import { waitUntil } from 'async-wait-until'
import { createApp } from '../../src/app'

import * as postgres from'./engines/postgres_resources'
import * as mysql from'./engines/mysql_resources'
import * as spanner from'./engines/spanner_resources'
import * as firestore from'./engines/firestore_resources'
import * as mssql from'./engines/mssql_resources'
import * as mongo from'./engines/mongo_resources'
import * as airtable from'./engines/airtable_resources'
import * as dynamo from'./engines/dynamodb_resources'
import * as bigquery from'./engines/bigquery_resources'
import { testResources as postgres } from '@wix-velo/external-db-postgres'
import { testResources as mysql } from '@wix-velo/external-db-mysql'
import { testResources as spanner } from '@wix-velo/external-db-spanner'
import { testResources as firestore } from '@wix-velo/external-db-firestore'
import { testResources as mssql } from '@wix-velo/external-db-mssql'
import { testResources as mongo } from '@wix-velo/external-db-mongo'
import { testResources as airtable } from '@wix-velo/external-db-airtable'
import { testResources as googleSheet } from '@wix-velo/external-db-google-sheets'
import { testResources as dynamo } from '@wix-velo/external-db-dynamodb'
import { testResources as bigquery } from '@wix-velo/external-db-bigquery'

import { E2EResources } from '@wix-velo/external-db-testkit'
import { Uninitialized } from '@wix-velo/test-commons'
import { ExternalDbRouter } from '@wix-velo/velo-external-db-core'
import { Server } from 'http'
import { ConnectionCleanUp, ISchemaProvider } from '@wix-velo/velo-external-db-types'
import { ExternalDbRouter } from '@wix-velo/velo-external-db-core'
// const googleSheet = require('./engines/google_sheets_resources')

interface App {
server: Server;
Expand All @@ -30,7 +30,7 @@ interface App {

type Internals = () => App

export const env:{
export let env:{
app: App,
externalDbRouter: ExternalDbRouter,
internals: Internals
Expand All @@ -40,61 +40,27 @@ export const env:{
externalDbRouter: Uninitialized
}

export const initApp = async() => {
process.env.CLOUD_VENDOR = 'azure'

if (env.app) {
await env.app.reload()
} else {
authInit()
env.internals = require('../../src/app').internals

await waitUntil(() => env.internals().started)
}
env.app = env.internals()
env.externalDbRouter = env.app.externalDbRouter
}

export const teardownApp = async() => {
await sleep(500)
await env.app.server.close()
}

const dbInit = async impl => {
await impl.cleanup()
impl.setActive()
}

export const dbTeardown = async() => {
await env.app.cleanup()
}

const postgresTestEnvInit = async() => await dbInit(postgres)
const mysqlTestEnvInit = async() => await dbInit(mysql)
const spannerTestEnvInit = async() => await dbInit(spanner)
const firestoreTestEnvInit = async() => await dbInit(firestore)
const mssqlTestEnvInit = async() => await dbInit(mssql)
const mongoTestEnvInit = async() => await dbInit(mongo)
const airTableTestEnvInit = async() => await dbInit(airtable)
const dynamoTestEnvInit = async() => await dbInit(dynamo)
const bigqueryTestEnvInit = async() => await dbInit(bigquery)
// const googleSheetTestEnvInit = async() => await dbInit(googleSheet)

const testSuits = {
mysql: suiteDef('MySql', mysqlTestEnvInit, mysql.supportedOperations),
postgres: suiteDef('Postgres', postgresTestEnvInit, postgres.supportedOperations),
spanner: suiteDef('Spanner', spannerTestEnvInit, spanner.supportedOperations),
firestore: suiteDef('Firestore', firestoreTestEnvInit, firestore.supportedOperations),
mssql: suiteDef('Sql Server', mssqlTestEnvInit, mssql.supportedOperations),
mongo: suiteDef('Mongo', mongoTestEnvInit, mongo.supportedOperations),
airtable: suiteDef('Airtable', airTableTestEnvInit, airtable.supportedOperations),
dynamodb: suiteDef('DynamoDb', dynamoTestEnvInit, dynamo.supportedOperations),
bigquery: suiteDef('BigQuery', bigqueryTestEnvInit, bigquery.supportedOperations),
// 'google-sheet': suiteDef('Google-sheet', googleSheetTestEnvInit, googleSheet.supportedOperations),
mysql: new E2EResources(mysql, createApp),
postgres: new E2EResources(postgres, createApp),
spanner: new E2EResources(spanner, createApp),
firestore: new E2EResources(firestore, createApp),
mssql: new E2EResources(mssql, createApp),
mongo: new E2EResources(mongo, createApp),
'google-sheet': new E2EResources(googleSheet, createApp),
airtable: new E2EResources(airtable, createApp),
dynamodb: new E2EResources(dynamo, createApp),
bigquery: new E2EResources(bigquery, createApp),
}

const testedSuit = () => testSuits[process.env.TEST_ENGINE]

export const testedSuit = () => testSuits[process.env.TEST_ENGINE]
export const supportedOperations = testedSuit().supportedOperations

export const setupDb = () => testedSuit().setup()
export const setupDb = () => testedSuit().setUpDb()
export const currentDbImplementationName = () => testedSuit().name
export const initApp = async () => {
env = await testedSuit().initApp()
}
export const teardownApp = async () => testedSuit().teardownApp()
export const dbTeardown = async () => testedSuit().dbTeardown()

This file was deleted.

Loading