Separated database for store loader

README.md

@@ -8,11 +8,12 @@
 # TdRules - Test Data Coverage Evaluation
 
 This repository contains a set of components to generate *Full Predicate Coverage Rules* and *SQL Mutants*
-to assess the coverage of the test data in relation to a query:
+to assess the coverage of the test data in relation to a query and load test data:
 
 - Generate the FPC Rules and SQL Mutants ([TdRules Service](https://in2test.lsi.uniovi.es/tdrules/)).
-- Discover the data store schema from an open JDBC connection to a relational database.
+- Discover the data store schema from an OpenApi specification or from a JDBC connection to a relational database.
 - Models to manipulate the rules (both FPC Rules or SQL Mutants) and the schema.
+- Load test data through a REST API or a JDBC connection.
 - Available for Java 8 and higher, and .NET (netstandard 2.0).
 
 NOTE: The name *TdRules* (Test Data Rules) is replacing the former *SQLRules* 
@@ -23,6 +24,8 @@ to enable generation of rules for data stores other than relational databases.
 - On Java, include the client api dependencies
 [tdrules-client](https://central.sonatype.com/artifact/io.github.giis-uniovi/tdrules-client)
 and 
+[tdrules-client-oa](https://central.sonatype.com/artifact/io.github.giis-uniovi/tdrules-client-oa)
+or
 [tdrules-client-rdb](https://central.sonatype.com/artifact/io.github.giis-uniovi/tdrules-client-rdb)
 available in Maven Central.
 A bom is also available:
@@ -32,23 +35,34 @@ A bom is also available:
 [TdRules](https://www.nuget.org/packages/TdRules/)
 available in NuGet.
 
-**Example:** To generate the FPC Rules for a query `query`
-that executes against a relational database that can be reached by an open JDBC Connection `conn`,
+**Example:** To generate the FPC Rules for a query (`query`)
+that executes against 
+an OpenApi schema specification in a file or url (`spec`)
+or a relational database that can be reached by an open JDBC Connection (`conn`),
 you first get the data store schema model and then the rules model as follows:
 
-<details open><summary><strong>Java</strong></summary>
+<details open><summary><strong>Java (OpenApi)</strong></summary>
 
 ```Java
-TdSchema schemaModel = new DbSchemaApi(conn).getDbSchema();
+TdSchema schemaModel = new OaSchemaApi(spec).getSchema();
 TdRules rulesModel = new TdRulesApi().getRules(schemaModel, query, "");
 ```
 
 </details>
 
-<details><summary><strong>.NET</strong></summary>
+<details open><summary><strong>Java (RDB)</strong></summary>
+
+```Java
+TdSchema schemaModel = new DbSchemaApi(conn).getSchema();
+TdRules rulesModel = new TdRulesApi().getRules(schemaModel, query, "");
+```
+
+</details>
+
+<details><summary><strong>.NET (RDB only)</strong></summary>
 
 ```C#
-TdSchema schemaModel = new DbSchemaApi(conn).GetDbSchema();
+TdSchema schemaModel = new DbSchemaApi(conn).GetSchema();
 TdRules rulesModel = new TdRulesApi().GetRules(schemaModel, query, "");
 ```
 
@@ -73,18 +87,23 @@ Modules currently available in this repo are:
 - `tdrules-bom`: The bill of materials of all TdRules components.
 - `tdrules-client`: Client api to generate FPC Rules and SQL Mutants.
 - `tdrules-model`: Models of the FPC Rules, SQL Mutants and the data store schema.
+- `tdrules-client-oa`: Client api to generate the data store schema from an OpenApi specification.
 - `tdrules-client-rdb`: Client api to generate the data store schema from a live JDBC connection.
 - `tdrules-store-rdb`: Core component used to discover the schema of relational databases.
 - `tdrules-store-shared`: Shared components for all data stores.
+- `tdrules-store-loader`: Load test data through a REST API or a JDBC connection.
 - `setup`: A folder with the configuration of test database containers to use in your development environment.
 - `net`: A folder with the source of the .NET implementation.
 
 ```mermaid
 flowchart TD
-    client(client) --> model(model)
+    client(client) ---> model(model)
+    clientoa(client-oa) ---> model
     clientrdb(client-rdb) --> model
     clientrdb --> storerdb(store-rdb)
     storerdb --> storeshared(store-shared)
+    storeloader(store-loader) ---> model
+    storeloader ---> storeshared
 ```
 
 To set-up the test database containers in a local development environment, see the `setup` folder.

pom.xml

@@ -172,6 +172,8 @@
 			<dependency>
 				<groupId>com.oracle.database.jdbc</groupId>
 				<artifactId>ojdbc8</artifactId>
+				<!-- latest: 23.4.0.24.05 does not work on windows with gvenzl/oracle-xe:21.3.0-slim-faststart
+				     error: ORA-01005 null password given, use 21.14.0.0 -->
 				<version>23.4.0.24.05</version>
 				<scope>test</scope>
 			</dependency>

setup/database.properties

@@ -7,6 +7,9 @@ tdrules.java.tdclirdb.postgres.user=tdclirdb
 tdrules.java.tdstorerdb2.postgres.url=jdbc:postgresql://localhost:5432/tdstorerdb2
 tdrules.java.tdstorerdb2.postgres.user=tdstorerdb2
 
+tdrules.java.tdloadrdb.postgres.url=jdbc:postgresql://localhost:5432/tdloadrdb
+tdrules.java.tdloadrdb.postgres.user=tdloadrdb
+
 tdrules.java.tdstorerdb2.postgres.meta.dbms=Database: PostgreSQL - Version: 14.
 tdrules.java.tdstorerdb2.postgres.meta.driver=Driver: PostgreSQL JDBC Driver - Version: 42.
 
@@ -17,14 +20,20 @@ tdrules.java.tdclirdb.sqlserver.user=tdclirdb
 tdrules.java.tdstorerdb2.sqlserver.url=jdbc:sqlserver://localhost:1433;databaseName=tdstorerdb2;encrypt=true;trustServerCertificate=true
 tdrules.java.tdstorerdb2.sqlserver.user=tdstorerdb2
 
+tdrules.java.tdloadrdb.sqlserver.url=jdbc:sqlserver://localhost:1433;databaseName=tdloadrdb;encrypt=true;trustServerCertificate=true
+tdrules.java.tdloadrdb.sqlserver.user=tdloadrdb
+
 tdrules.java.tdstorerdb2.sqlserver.meta.dbms=Database: Microsoft SQL Server - Version: 14.
 tdrules.java.tdstorerdb2.sqlserver.meta.driver=Driver: Microsoft JDBC Driver 12.
 
-# Oracle is not iested in the client (tdclirdb) 
+# Oracle is not tested in the client-rdb (tdclirdb) 
 # and needs two additional databases for TestOracleSchemaMulti (see internal doc)
 tdrules.java.tdstorerdb2.oracle.url=jdbc:oracle:thin:@localhost:1521:XE
 tdrules.java.tdstorerdb2.oracle.user=tdstorerdb2
 
+tdrules.java.tdloadrdb.oracle.url=jdbc:oracle:thin:@localhost:1521:XE
+tdrules.java.tdloadrdb.oracle.user=tdloadrdb
+
 tdrules.java.tdstorerdb0.oracle.url=jdbc:oracle:thin:@localhost:1521:XE
 tdrules.java.tdstorerdb0.oracle.user=tdstorerdb0
 tdrules.java.tdstorerdb1.oracle.url=jdbc:oracle:thin:@localhost:1521:XE

setup/oracle/oracle-setup.sh

@@ -32,5 +32,15 @@ sqlplus system/${TEST_ORACLE_PWD}@XE  <<-EOSQL
   grant create view  to tdstorerdb1; 
   grant dba to tdstorerdb1;
   grant unlimited tablespace to tdstorerdb1;
+----
+  create user tdloadrdb identified by "$TEST_ORACLE_PWD" ACCOUNT UNLOCK;
+  grant connect to tdloadrdb;
+  grant create session to tdloadrdb;
+  grant resource to tdloadrdb;
+  grant create table to tdloadrdb; 
+  grant create procedure to tdloadrdb; 
+  grant create view to tdloadrdb;
+  grant unlimited tablespace to tdloadrdb;
+ 
 EOSQL
 echo "-- End setup"

setup/postgres/postgres-setup.sh

@@ -7,5 +7,8 @@ psql -v ON_ERROR_STOP=1   <<-EOSQL
 
   CREATE USER tdstorerdb2 with encrypted password '$TEST_POSTGRES_PWD';
   CREATE DATABASE tdstorerdb2;
+
+  CREATE USER tdloadrdb with encrypted password '$TEST_POSTGRES_PWD';
+  CREATE DATABASE tdloadrdb;
 EOSQL
 echo "-- END SETUP!"

setup/sqlserver/sqlserver-setup.sh

@@ -26,5 +26,14 @@ echo "-- Begin setup"
   CREATE USER [tdstorerdb2] FOR LOGIN [tdstorerdb2] --WITH DEFAULT_SCHEMA=[dbo]
   EXEC sp_addrolemember 'db_owner', 'tdstorerdb2'
   GO
+  ----
+  CREATE LOGIN tdloadrdb WITH PASSWORD = '$TEST_SQLSERVER_PWD', CHECK_POLICY=OFF, CHECK_EXPIRATION=OFF, DEFAULT_LANGUAGE=spanish;
+  GO
+  CREATE DATABASE tdloadrdb
+  GO
+  USE [tdloadrdb]
+  CREATE USER [tdloadrdb] FOR LOGIN [tdloadrdb] --WITH DEFAULT_SCHEMA=[dbo]
+  EXEC sp_addrolemember 'db_owner', 'tdloadrdb'
+  GO
 EOSQL
 echo "-- END SETUP!"

tdrules-store-loader/pom.xml

@@ -9,7 +9,7 @@
 	<packaging>jar</packaging>
 
 	<name>tdrules-store-loader</name>
-	<description>Load data through the API or a JDBC connection</description>
+	<description>Load test data through a REST API or a JDBC connection</description>
 	<url>http://github.com/giis-uniovi/tdrules</url>
 	<organization>
 		<name>Software Engineering Research Group (GIIS) - Universidad de Oviedo, ES</name>

tdrules-store-loader/src/test/java/test4giis/tdrules/store/loader/sql/Base.java

@@ -26,7 +26,7 @@ public class Base {
 	private static final String ENVIRONMENT_PROPERTIES = FileUtil.getPath(SETUP_PATH, "environment.properties");
 	private static final String DATABASE_PROPERTIES = FileUtil.getPath(SETUP_PATH, "database.properties");
 
-	public static final String TEST_DBNAME = "tdstorerdb2";
+	public static final String TEST_DBNAME = "tdloadrdb";
 
 	protected String dbmsname = "sqlserver";