Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IBM DB2 migration #868

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

IBM DB2 migration #868

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
743986f
Add postgresql migration docs
Josipmrden Jun 25, 2024
c74e888
Add IBM DB2 migration
Josipmrden Jun 25, 2024
24c5419
Fix from PR review
Josipmrden Jul 2, 2024
014b908
Another fix
Josipmrden Jul 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
206 changes: 179 additions & 27 deletions pages/advanced-algorithms/available-algorithms/migrate.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,70 @@ A module that contains procedures describing graphs on a meta-level.

## Procedures

### `db2()`

With the `migrate.db2()` procedure you can access IBM DB2 and migrate your data
to Memgraph. The result table is converted into a stream, and the returned rows can
be used to create graph structures. The value of the `config` parameter must be
at least an empty map. If `config_path` is passed, every key-value pair from
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/at least an empty map/a map, either empty or populated with configuration key/value pairs

JSON file will overwrite any values in `config` file.

{<h4> Input: </h4>}

* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module
will migrate all the rows from the table. In the case that a SQL query is provided, the module
will migrate the rows returned from the queries.
* `config: mgp.Map` ➡ Connection configuration parameters (as in `ibm_db.connect`).
* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `ibm_db.connect`).
* `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values.

{<h4> Output: </h4>}

* `row: mgp.Map`: The result table as a stream of rows.

{<h4> Usage: </h4>}

To inspect the first 5000 rows from a database, use the following query:

```cypher
CALL migrate.db2('example_table', {user:'memgraph',
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
RETURN row
LIMIT 5000;
```

In the case you want to migrate specific results from a SQL query, it is enough to modify the
first argument of the query module call, and continue to use the Cypher query language to
shape your results:

```cypher
CALL migrate.db2('SELECT * FROM example_table', {user:'memgraph',
hal-eisen-MG marked this conversation as resolved.
Show resolved Hide resolved
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
WITH row
WHERE row.age >= 30
RETURN row;
```

### `mysql()`

With the `migrate.mysql()` procedure you can access MySQL and execute queries.
With the `migrate.mysql()` procedure you can access MySQL and migrate your data to Memgraph.
The result table is converted into a stream, and the returned rows can be used
to create graph structures. The value of the `config` parameter must be at least
an empty map. If `config_path` is passed, every key,value pair from JSON file
to create graph structures. The value of the `config` parameter must be
a map, either empty or populated with configuration key/value pairs.
If `config_path` is passed, every key-value pair from JSON file
will overwrite any values in `config` file.

{<h4> Input: </h4>}

* `table_or_sql: str` ➡ Table name or an SQL query.
* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module
will migrate all the rows from the table. In the case that a SQL query is provided, the module
will migrate the rows returned from the queries.
* `config: mgp.Map` ➡ Connection configuration parameters (as in `mysql.connector.connect`).
* `config_path` ➡ Path to a JSON file containing configuration parameters (as in `mysql.connector.connect`).
* `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values.
Expand All @@ -56,54 +109,89 @@ CALL migrate.mysql('example_table', {user:'memgraph',
host:'localhost',
database:'demo_db'} )
YIELD row
RETURN count(row);
RETURN row;
```

### `sql_server()`
In the case you want to migrate specific results from a SQL query, it is enough to modify the
first argument of the query module call, and continue to use the Cypher query language to
shape your results:

With the `migrate.sql_server()` procedure you can access SQL Server and execute
queries. The result table is converted into a stream, and the returned rows can
be used to create graph structures. The value of the `config` parameter must be
at least an empty map. If `config_path` is passed, every key,value pair from
JSON file will overwrite any values in `config` file.
```cypher
CALL migrate.mysql('SELECT * FROM example_table', {user:'memgraph',
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
WITH row
WHERE row.age >= 30
RETURN row;
```

### `oracle_db()`

With the `migrate.oracle_db` you can access Oracle DB and migrate your data to Memgraph.
The result table is converted into a stream, and the returned rows can be used to
create graph structures. The value of the `config` parameter must be at least an
empty map. If `config_path` is passed, every key-value pair from JSON file will
overwrite any values in `config` file.

{<h4> Input: </h4>}

* `table_or_sql: str` ➡ Table name or an SQL query.
* `config: mgp.Map` ➡ Connection configuration parameters (as in `pyodbc.connect`).
* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `pyodbc.connect`).
* `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values.
* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module
will migrate all the rows from the table. In the case that a SQL query is provided, the module
will migrate the rows returned from the queries.
* `config: mgp.Map` ➡ Connection configuration parameters (as in `oracledb.connect`).
* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `oracledb.connect`).
* `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries may be parameterized. In that case, `params` provides parameter values.

{<h4> Output: </h4>}

* `row: mgp.Map`: The result table as a stream of rows.

{<h4> Usage: </h4>}

To get all data from database in form of map, use the following query:
To inspect the first 5000 rows from a database, use the following query:

```cypher
CALL migrate.sql_server('example_table', {user:'memgraph',
CALL migrate.oracle_db('example_table', {user:'memgraph',
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
RETURN row
LIMIT 5000;
```

In the case you want to migrate specific results from a SQL query, it is enough to modify the
first argument of the query module call, and continue to use the Cypher query language to
shape your results:

```cypher
CALL migrate.oracle_db('SELECT * FROM example_table', {user:'memgraph',
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
WITH row
WHERE row.age >= 30
RETURN row;
```

### `oracle_db()`
### `postgresql()`

With the `migrate.oracle_db` you can access Oracle DB and execute queries. The
result table is converted into a stream, and the returned rows can be used to
With the `migrate.postgresql` you can access PostgreSQL and migrate your data to Memgraph.
The result table is converted into a stream, and the returned rows can be used to
create graph structures. The value of the `config` parameter must be at least an
empty map. If `config_path` is passed, every key,value pair from JSON file will
empty map. If `config_path` is passed, every key-value pair from JSON file will
overwrite any values in `config` file.

{<h4> Input: </h4>}

* `table_or_sql: str` ➡ Table name or an SQL query.
* `config: mgp.Map` ➡ Connection configuration parameters (as in `oracledb.connect`).
* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `oracledb.connect`).
* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module
will migrate all the rows from the table. In the case that a SQL query is provided, the module
will migrate the rows returned from the queries.
* `config: mgp.Map` ➡ Connection configuration parameters (as in `psycopg2.connect`).
* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `psycopg2.connect`).
* `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries may be parameterized. In that case, `params` provides parameter values.

{<h4> Output: </h4>}
Expand All @@ -112,14 +200,78 @@ overwrite any values in `config` file.

{<h4> Usage: </h4>}

To get the first 5000 rows from a database, use the following query:
To inspect the first 5000 rows from a database, use the following query:

```cypher
CALL migrate.oracle_db('example_table', {user:'memgraph',
CALL migrate.postgresql('example_table', {user:'memgraph',
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
RETURN row
LIMIT 5000;
```
```

In the case you want to migrate specific results from a SQL query, it is enough to modify the
first argument of the query module call, and continue to use the Cypher query language to
shape your results:

```cypher
CALL migrate.postgresql('SELECT * FROM example_table', {user:'memgraph',
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
WITH row
WHERE row.age >= 30
RETURN row;
```

### `sql_server()`

With the `migrate.sql_server()` procedure you can access SQL Server and migrate your data
to Memgraph. The result table is converted into a stream, and the returned rows can
be used to create graph structures. The value of the `config` parameter must be
at least an empty map. If `config_path` is passed, every key-value pair from
JSON file will overwrite any values in `config` file.

{<h4> Input: </h4>}

* `table_or_sql: str` ➡ Table name or an SQL query. When the table name is specified, the module
will migrate all the rows from the table. In the case that a SQL query is provided, the module
will migrate the rows returned from the queries.
* `config: mgp.Map` ➡ Connection configuration parameters (as in `pyodbc.connect`).
* `config_path` ➡ Path to the JSON file containing configuration parameters (as in `pyodbc.connect`).
* `params: mgp.Nullable[mgp.Any] (default=None)` ➡ Optionally, queries can be parameterized. In that case, `params` provides parameter values.

{<h4> Output: </h4>}

* `row: mgp.Map`: The result table as a stream of rows.

{<h4> Usage: </h4>}

To inspect the first 5000 rows from a database, use the following query:

```cypher
CALL migrate.sql_server('example_table', {user:'memgraph',
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
RETURN row;
```

In the case you want to migrate specific results from a SQL query, it is enough to modify the
first argument of the query module call, and continue to use the Cypher query language to
shape your results:

```cypher
CALL migrate.sql_server('SELECT * FROM example_table', {user:'memgraph',
password:'password',
host:'localhost',
database:'demo_db'} )
YIELD row
WITH row
WHERE row.age >= 30
RETURN row;
```