From 743986f6583c63578b4a8a3eff25eea4a316b15b Mon Sep 17 00:00:00 2001 From: Josip Mrden Date: Tue, 25 Jun 2024 11:57:24 +0200 Subject: [PATCH 1/4] Add postgresql migration docs --- .../available-algorithms/migrate.mdx | 113 ++++++++++++++++-- 1 file changed, 101 insertions(+), 12 deletions(-) diff --git a/pages/advanced-algorithms/available-algorithms/migrate.mdx b/pages/advanced-algorithms/available-algorithms/migrate.mdx index ccc03672a..a469a674f 100644 --- a/pages/advanced-algorithms/available-algorithms/migrate.mdx +++ b/pages/advanced-algorithms/available-algorithms/migrate.mdx @@ -29,15 +29,17 @@ A module that contains procedures describing graphs on a meta-level. ### `mysql()` -With the `migrate.mysql()` procedure you can access MySQL and execute queries. +With the `migrate.mysql()` procedure you can access MySQL and migrate your data to Memgraph. The result table is converted into a stream, and the returned rows can be used to create graph structures. The value of the `config` parameter must be at least -an empty map. If `config_path` is passed, every key,value pair from JSON file +an empty map. If `config_path` is passed, every key-value pair from JSON file will overwrite any values in `config` file. {

Input:

} -* `table_or_sql: str` ➡ Table name or an SQL query. +* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module + will migrate all the rows from the table. In the case that a SQL query is provided, the module + will migrate the rows returned from the queries. * `config: mgp.Map` ➡ Connection configuration parameters (as in `mysql.connector.connect`). * `config_path` ➡ Path to a JSON file containing configuration parameters (as in `mysql.connector.connect`). * `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values. @@ -59,17 +61,31 @@ YIELD row RETURN count(row); ``` +In the case you want to migrate specific results from a SQL query, it is enough to modify the +first argument of the query module call: + +```cypher +CALL migrate.mysql('SELECT * FROM example_table', {user:'memgraph', + password:'password', + host:'localhost', + database:'demo_db'} ) +YIELD row +RETURN count(row); +``` + ### `sql_server()` -With the `migrate.sql_server()` procedure you can access SQL Server and execute -queries. The result table is converted into a stream, and the returned rows can +With the `migrate.sql_server()` procedure you can access SQL Server and migrate your data +to Memgraph. The result table is converted into a stream, and the returned rows can be used to create graph structures. The value of the `config` parameter must be -at least an empty map. If `config_path` is passed, every key,value pair from +at least an empty map. If `config_path` is passed, every key-value pair from JSON file will overwrite any values in `config` file. {

Input:

} -* `table_or_sql: str` ➡ Table name or an SQL query. +* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module + will migrate all the rows from the table. In the case that a SQL query is provided, the module + will migrate the rows returned from the queries. * `config: mgp.Map` ➡ Connection configuration parameters (as in `pyodbc.connect`). * `config_path` ➡ Path to the JSON file containing configuration parameters (as in `pyodbc.connect`). * `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values. @@ -91,17 +107,31 @@ YIELD row RETURN row; ``` +In the case you want to migrate specific results from a SQL query, it is enough to modify the +first argument of the query module call: + +```cypher +CALL migrate.sql_server('SELECT * FROM example_table', {user:'memgraph', + password:'password', + host:'localhost', + database:'demo_db'} ) +YIELD row +RETURN count(row); +``` + ### `oracle_db()` -With the `migrate.oracle_db` you can access Oracle DB and execute queries. The -result table is converted into a stream, and the returned rows can be used to +With the `migrate.oracle_db` you can access Oracle DB and migrate your data to Memgraph. +The result table is converted into a stream, and the returned rows can be used to create graph structures. The value of the `config` parameter must be at least an -empty map. If `config_path` is passed, every key,value pair from JSON file will +empty map. If `config_path` is passed, every key-value pair from JSON file will overwrite any values in `config` file. {

Input:

} -* `table_or_sql: str` ➡ Table name or an SQL query. +* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module + will migrate all the rows from the table. In the case that a SQL query is provided, the module + will migrate the rows returned from the queries. * `config: mgp.Map` ➡ Connection configuration parameters (as in `oracledb.connect`). * `config_path` ➡ Path to the JSON file containing configuration parameters (as in `oracledb.connect`). * `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries may be parameterized. In that case, `params` provides parameter values. @@ -122,4 +152,63 @@ CALL migrate.oracle_db('example_table', {user:'memgraph', YIELD row RETURN row LIMIT 5000; -``` \ No newline at end of file +``` + +In the case you want to migrate specific results from a SQL query, it is enough to modify the +first argument of the query module call: + +```cypher +CALL migrate.oracle_db('SELECT * FROM example_table', {user:'memgraph', + password:'password', + host:'localhost', + database:'demo_db'} ) +YIELD row +RETURN count(row); +``` + +### `postgresql()` + +With the `migrate.postgresql` you can access PostgreSQL and migrate your data to Memgraph. +The result table is converted into a stream, and the returned rows can be used to +create graph structures. The value of the `config` parameter must be at least an +empty map. If `config_path` is passed, every key-value pair from JSON file will +overwrite any values in `config` file. + +{

Input:

} + +* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module + will migrate all the rows from the table. In the case that a SQL query is provided, the module + will migrate the rows returned from the queries. +* `config: mgp.Map` ➡ Connection configuration parameters (as in `psycopg2.connect`). +* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `psycopg2.connect`). +* `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries may be parameterized. In that case, `params` provides parameter values. + +{

Output:

} + +* `row: mgp.Map`: The result table as a stream of rows. + +{

Usage:

} + +To get the first 5000 rows from a database, use the following query: + +```cypher +CALL migrate.postgresql('example_table', {user:'memgraph', + password:'password', + host:'localhost', + database:'demo_db'} ) +YIELD row +RETURN row +LIMIT 5000; +``` + +In the case you want to migrate specific results from a SQL query, it is enough to modify the +first argument of the query module call: + +```cypher +CALL migrate.postgresql('SELECT * FROM example_table', {user:'memgraph', + password:'password', + host:'localhost', + database:'demo_db'} ) +YIELD row +RETURN count(row); +``` From c74e88868d5b03951b25d86b2e2ffa7ce0a997af Mon Sep 17 00:00:00 2001 From: Josip Mrden Date: Tue, 25 Jun 2024 12:03:00 +0200 Subject: [PATCH 2/4] Add IBM DB2 migration --- .../available-algorithms/migrate.mdx | 94 ++++++++++++++----- 1 file changed, 70 insertions(+), 24 deletions(-) diff --git a/pages/advanced-algorithms/available-algorithms/migrate.mdx b/pages/advanced-algorithms/available-algorithms/migrate.mdx index a469a674f..9ad73494b 100644 --- a/pages/advanced-algorithms/available-algorithms/migrate.mdx +++ b/pages/advanced-algorithms/available-algorithms/migrate.mdx @@ -27,21 +27,21 @@ A module that contains procedures describing graphs on a meta-level. ## Procedures -### `mysql()` +### `db2()` -With the `migrate.mysql()` procedure you can access MySQL and migrate your data to Memgraph. -The result table is converted into a stream, and the returned rows can be used -to create graph structures. The value of the `config` parameter must be at least -an empty map. If `config_path` is passed, every key-value pair from JSON file -will overwrite any values in `config` file. +With the `migrate.db2()` procedure you can access IBM DB2 and migrate your data +to Memgraph. The result table is converted into a stream, and the returned rows can +be used to create graph structures. The value of the `config` parameter must be +at least an empty map. If `config_path` is passed, every key-value pair from +JSON file will overwrite any values in `config` file. {

Input:

} * `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module will migrate all the rows from the table. In the case that a SQL query is provided, the module will migrate the rows returned from the queries. -* `config: mgp.Map` ➡ Connection configuration parameters (as in `mysql.connector.connect`). -* `config_path` ➡ Path to a JSON file containing configuration parameters (as in `mysql.connector.connect`). +* `config: mgp.Map` ➡ Connection configuration parameters (as in `ibm_db.connect`). +* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `ibm_db.connect`). * `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values. {

Output:

} @@ -50,22 +50,22 @@ will overwrite any values in `config` file. {

Usage:

} -To get count of rows, use the following query: +To get all data from database in form of map, use the following query: ```cypher -CALL migrate.mysql('example_table', {user:'memgraph', +CALL migrate.db2('example_table', {user:'memgraph', password:'password', host:'localhost', database:'demo_db'} ) YIELD row -RETURN count(row); +RETURN row; ``` In the case you want to migrate specific results from a SQL query, it is enough to modify the first argument of the query module call: ```cypher -CALL migrate.mysql('SELECT * FROM example_table', {user:'memgraph', +CALL migrate.db2('SELECT * FROM example_table', {user:'memgraph', password:'password', host:'localhost', database:'demo_db'} ) @@ -73,21 +73,21 @@ YIELD row RETURN count(row); ``` -### `sql_server()` +### `mysql()` -With the `migrate.sql_server()` procedure you can access SQL Server and migrate your data -to Memgraph. The result table is converted into a stream, and the returned rows can -be used to create graph structures. The value of the `config` parameter must be -at least an empty map. If `config_path` is passed, every key-value pair from -JSON file will overwrite any values in `config` file. +With the `migrate.mysql()` procedure you can access MySQL and migrate your data to Memgraph. +The result table is converted into a stream, and the returned rows can be used +to create graph structures. The value of the `config` parameter must be at least +an empty map. If `config_path` is passed, every key-value pair from JSON file +will overwrite any values in `config` file. {

Input:

} * `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module will migrate all the rows from the table. In the case that a SQL query is provided, the module will migrate the rows returned from the queries. -* `config: mgp.Map` ➡ Connection configuration parameters (as in `pyodbc.connect`). -* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `pyodbc.connect`). +* `config: mgp.Map` ➡ Connection configuration parameters (as in `mysql.connector.connect`). +* `config_path` ➡ Path to a JSON file containing configuration parameters (as in `mysql.connector.connect`). * `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values. {

Output:

} @@ -96,22 +96,22 @@ JSON file will overwrite any values in `config` file. {

Usage:

} -To get all data from database in form of map, use the following query: +To get count of rows, use the following query: ```cypher -CALL migrate.sql_server('example_table', {user:'memgraph', +CALL migrate.mysql('example_table', {user:'memgraph', password:'password', host:'localhost', database:'demo_db'} ) YIELD row -RETURN row; +RETURN count(row); ``` In the case you want to migrate specific results from a SQL query, it is enough to modify the first argument of the query module call: ```cypher -CALL migrate.sql_server('SELECT * FROM example_table', {user:'memgraph', +CALL migrate.mysql('SELECT * FROM example_table', {user:'memgraph', password:'password', host:'localhost', database:'demo_db'} ) @@ -212,3 +212,49 @@ CALL migrate.postgresql('SELECT * FROM example_table', {user:'memgraph', YIELD row RETURN count(row); ``` + +### `sql_server()` + +With the `migrate.sql_server()` procedure you can access SQL Server and migrate your data +to Memgraph. The result table is converted into a stream, and the returned rows can +be used to create graph structures. The value of the `config` parameter must be +at least an empty map. If `config_path` is passed, every key-value pair from +JSON file will overwrite any values in `config` file. + +{

Input:

} + +* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module + will migrate all the rows from the table. In the case that a SQL query is provided, the module + will migrate the rows returned from the queries. +* `config: mgp.Map` ➡ Connection configuration parameters (as in `pyodbc.connect`). +* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `pyodbc.connect`). +* `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values. + +{

Output:

} + +* `row: mgp.Map`: The result table as a stream of rows. + +{

Usage:

} + +To get all data from database in form of map, use the following query: + +```cypher +CALL migrate.sql_server('example_table', {user:'memgraph', + password:'password', + host:'localhost', + database:'demo_db'} ) +YIELD row +RETURN row; +``` + +In the case you want to migrate specific results from a SQL query, it is enough to modify the +first argument of the query module call: + +```cypher +CALL migrate.sql_server('SELECT * FROM example_table', {user:'memgraph', + password:'password', + host:'localhost', + database:'demo_db'} ) +YIELD row +RETURN count(row); +``` From 24c5419ff061d4884f7cdb2e27d6dc12dc7a375d Mon Sep 17 00:00:00 2001 From: Josip Mrden Date: Tue, 2 Jul 2024 12:09:23 +0200 Subject: [PATCH 3/4] Fix from PR review --- .../available-algorithms/migrate.mdx | 44 +++++++++++++------ 1 file changed, 30 insertions(+), 14 deletions(-) diff --git a/pages/advanced-algorithms/available-algorithms/migrate.mdx b/pages/advanced-algorithms/available-algorithms/migrate.mdx index 9ad73494b..97424c737 100644 --- a/pages/advanced-algorithms/available-algorithms/migrate.mdx +++ b/pages/advanced-algorithms/available-algorithms/migrate.mdx @@ -50,7 +50,7 @@ JSON file will overwrite any values in `config` file. {

Usage:

} -To get all data from database in form of map, use the following query: +To inspect all data from database in form of map, use the following query: ```cypher CALL migrate.db2('example_table', {user:'memgraph', @@ -62,7 +62,8 @@ RETURN row; ``` In the case you want to migrate specific results from a SQL query, it is enough to modify the -first argument of the query module call: +first argument of the query module call, and continue to use the Cypher query language to +shape your results: ```cypher CALL migrate.db2('SELECT * FROM example_table', {user:'memgraph', @@ -70,15 +71,18 @@ CALL migrate.db2('SELECT * FROM example_table', {user:'memgraph', host:'localhost', database:'demo_db'} ) YIELD row -RETURN count(row); +WITH row +WHERE row.age >= 30 +RETURN row; ``` ### `mysql()` With the `migrate.mysql()` procedure you can access MySQL and migrate your data to Memgraph. The result table is converted into a stream, and the returned rows can be used -to create graph structures. The value of the `config` parameter must be at least -an empty map. If `config_path` is passed, every key-value pair from JSON file +to create graph structures. The value of the `config` parameter must be +a map, either empty or populated with configuration key/value pairs. +If `config_path` is passed, every key-value pair from JSON file will overwrite any values in `config` file. {

Input:

} @@ -104,11 +108,12 @@ CALL migrate.mysql('example_table', {user:'memgraph', host:'localhost', database:'demo_db'} ) YIELD row -RETURN count(row); +RETURN row; ``` In the case you want to migrate specific results from a SQL query, it is enough to modify the -first argument of the query module call: +first argument of the query module call, and continue to use the Cypher query language to +shape your results: ```cypher CALL migrate.mysql('SELECT * FROM example_table', {user:'memgraph', @@ -116,7 +121,9 @@ CALL migrate.mysql('SELECT * FROM example_table', {user:'memgraph', host:'localhost', database:'demo_db'} ) YIELD row -RETURN count(row); +WITH row +WHERE row.age >= 30 +RETURN row; ``` ### `oracle_db()` @@ -155,7 +162,8 @@ LIMIT 5000; ``` In the case you want to migrate specific results from a SQL query, it is enough to modify the -first argument of the query module call: +first argument of the query module call, and continue to use the Cypher query language to +shape your results: ```cypher CALL migrate.oracle_db('SELECT * FROM example_table', {user:'memgraph', @@ -163,7 +171,9 @@ CALL migrate.oracle_db('SELECT * FROM example_table', {user:'memgraph', host:'localhost', database:'demo_db'} ) YIELD row -RETURN count(row); +WITH row +WHERE row.age >= 30 +RETURN row; ``` ### `postgresql()` @@ -202,7 +212,8 @@ LIMIT 5000; ``` In the case you want to migrate specific results from a SQL query, it is enough to modify the -first argument of the query module call: +first argument of the query module call, and continue to use the Cypher query language to +shape your results: ```cypher CALL migrate.postgresql('SELECT * FROM example_table', {user:'memgraph', @@ -210,7 +221,9 @@ CALL migrate.postgresql('SELECT * FROM example_table', {user:'memgraph', host:'localhost', database:'demo_db'} ) YIELD row -RETURN count(row); +WITH row +WHERE row.age >= 30 +RETURN row; ``` ### `sql_server()` @@ -248,7 +261,8 @@ RETURN row; ``` In the case you want to migrate specific results from a SQL query, it is enough to modify the -first argument of the query module call: +first argument of the query module call, and continue to use the Cypher query language to +shape your results: ```cypher CALL migrate.sql_server('SELECT * FROM example_table', {user:'memgraph', @@ -256,5 +270,7 @@ CALL migrate.sql_server('SELECT * FROM example_table', {user:'memgraph', host:'localhost', database:'demo_db'} ) YIELD row -RETURN count(row); +WITH row +WHERE row.age >= 30 +RETURN row; ``` From 014b908c6bd3a4dc18d5ccd416e0d398693620e7 Mon Sep 17 00:00:00 2001 From: Josip Mrden Date: Tue, 2 Jul 2024 12:16:35 +0200 Subject: [PATCH 4/4] Another fix --- .../available-algorithms/migrate.mdx | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/pages/advanced-algorithms/available-algorithms/migrate.mdx b/pages/advanced-algorithms/available-algorithms/migrate.mdx index 97424c737..f7db4272f 100644 --- a/pages/advanced-algorithms/available-algorithms/migrate.mdx +++ b/pages/advanced-algorithms/available-algorithms/migrate.mdx @@ -50,7 +50,7 @@ JSON file will overwrite any values in `config` file. {

Usage:

} -To inspect all data from database in form of map, use the following query: +To inspect the first 5000 rows from a database, use the following query: ```cypher CALL migrate.db2('example_table', {user:'memgraph', @@ -58,7 +58,8 @@ CALL migrate.db2('example_table', {user:'memgraph', host:'localhost', database:'demo_db'} ) YIELD row -RETURN row; +RETURN row +LIMIT 5000; ``` In the case you want to migrate specific results from a SQL query, it is enough to modify the @@ -149,7 +150,7 @@ overwrite any values in `config` file. {

Usage:

} -To get the first 5000 rows from a database, use the following query: +To inspect the first 5000 rows from a database, use the following query: ```cypher CALL migrate.oracle_db('example_table', {user:'memgraph', @@ -199,7 +200,7 @@ overwrite any values in `config` file. {

Usage:

} -To get the first 5000 rows from a database, use the following query: +To inspect the first 5000 rows from a database, use the following query: ```cypher CALL migrate.postgresql('example_table', {user:'memgraph', @@ -249,7 +250,7 @@ JSON file will overwrite any values in `config` file. {

Usage:

} -To get all data from database in form of map, use the following query: +To inspect the first 5000 rows from a database, use the following query: ```cypher CALL migrate.sql_server('example_table', {user:'memgraph',