diff --git a/Recipes/ClearScape_Functions/BinarySeriesOp.ipynb b/Recipes/ClearScape_Functions/BinarySeriesOp.ipynb new file mode 100644 index 00000000..7de68efd --- /dev/null +++ b/Recipes/ClearScape_Functions/BinarySeriesOp.ipynb @@ -0,0 +1,354 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "bc549e6c-0cc4-4188-94a3-a9bdd3ae3dfa", + "metadata": {}, + "source": [ + "
\n", + "

\n", + " BinarySeriesOp function in Vantage\n", + "
\n", + " \"Teradata\"\n", + "

\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "7ae7611a-0795-4168-b716-01fee6880cbd", + "metadata": {}, + "source": [ + "

Introduction

\n", + "\n", + "

BinarySeriesOp

\n", + "

The BinarySeriesOp() function performs a point wise mathematical operation on two time series of equal size. The principal mathematical operation can be subtraction, addition, multiplication and division. It is called a point wise operation because it performs the mathematical operation one sample point at a time.

\n", + "

Common uses of BinarySeriesOp() are:

\n", + "
  • Subtracting trends from a time series to create a model from it.
    • \n", + "
  • Restoring trends to a time series before using the model for forecasting.
    • \n", + "
  • As a building block to formulate more complex functions.

    • \n", + " \n", + "

    The following procedure is an example of how to use BinarySeriesOp() to convolve two series with digital signal processing:

    \n", + "
  • Use DFFT() function on series 1 and series 2 to get dataframes named 'dfftRes1' and 'dfftRes2', respectively.
    • \n", + "
  • Use BinarySeriesOp() to do point-wise multiplication using 'dfftRes1' and 'dfftRes2'.
    • \n", + "
  • Use IDFFT() on the output of BinarySeriesOp() to get the convolved result of the two series.
    • \n", + "

    " + ] + }, + { + "cell_type": "markdown", + "id": "6b3a00b4-6661-4c91-9b2d-cb7b0b403140", + "metadata": {}, + "source": [ + "
    \n", + "1. Initiate a connection to Vantage" + ] + }, + { + "cell_type": "markdown", + "id": "2346857f-e0d3-488a-8a3f-ac6dff752c2b", + "metadata": {}, + "source": [ + "

    In the section, we import the required libraries and set environment variables and environment paths (if required)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c5af5af3-29d5-4f6a-8334-9df6924e7787", + "metadata": {}, + "outputs": [], + "source": [ + "from teradataml import (\n", + " create_context,\n", + " execute_sql,\n", + " load_example_data,\n", + " DataFrame,\n", + " in_schema,\n", + " TDSeries,\n", + " BinarySeriesOp,\n", + " db_drop_view,\n", + " remove_context,\n", + " copy_to_sql,\n", + " db_drop_table\n", + " )\n", + "\n", + "from teradatasqlalchemy.types import *\n", + "\n", + "from teradataml import to_numeric\n", + "# Modify the following to match the specific client environment settings\n", + "display.max_rows = 5" + ] + }, + { + "cell_type": "markdown", + "id": "ad3dd7b4-831c-4fb3-ab71-719c8c99a71c", + "metadata": {}, + "source": [ + "

    \n", + "

    1.1 Connect to Vantage

    \n", + "

    You will be prompted to provide the password. Enter your password, press the Enter key, and then use the down arrow to go to the next cell.

    " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2742444c-4349-4b0f-b4e5-b068a8785cd9", + "metadata": {}, + "outputs": [], + "source": [ + "%run -i ../../UseCases/startup.ipynb\n", + "eng = create_context(host = 'host.docker.internal', username='demo_user', password = password)\n", + "print(eng)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "01c4a128-d106-46ea-8dee-34acc5abd29f", + "metadata": {}, + "outputs": [], + "source": [ + "%%capture\n", + "execute_sql('''SET query_band='DEMO=PP_BinarySeriesOp_Python.ipynb;' UPDATE FOR SESSION; ''')" + ] + }, + { + "cell_type": "markdown", + "id": "efe2fd2d-63ff-4278-9157-8b9110d682e8", + "metadata": {}, + "source": [ + "

    Begin running steps with Shift + Enter keys.

    " + ] + }, + { + "cell_type": "markdown", + "id": "4a6e3852-3b15-4cbd-844d-e7194b3bd204", + "metadata": {}, + "source": [ + "
    \n", + "\n", + "

    1.2 Getting Data for This Demo

    \n", + "\n", + "

    Here, we will get the time series data which is available in the teradataml library and use the same to show the usage of the function.

    " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "398fa0aa-c0df-4130-9dea-53b776fbaf80", + "metadata": {}, + "outputs": [], + "source": [ + "load_example_data(\"uaf\", [\"binary_complex_left\", \"binary_complex_right\"])" + ] + }, + { + "cell_type": "markdown", + "id": "2a3762ac-ba27-4fa3-adba-d577262a4290", + "metadata": {}, + "source": [ + "
    \n", + "2. Data Exploration\n", + "

    Create a \"Virtual DataFrame\" that points to the data set in Vantage. Check the shape of the dataframe as check the datatype of all the columns of the dataframe.

    " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9672d80d-cdcc-4f4c-9399-7c367e7970b1", + "metadata": {}, + "outputs": [], + "source": [ + "data1 = DataFrame.from_table(\"binary_complex_left\")\n", + "data2 = DataFrame.from_table(\"binary_complex_right\")\n", + "data1 " + ] + }, + { + "cell_type": "markdown", + "id": "4a2f79a0-eb16-4c3b-bec2-c313e97ac940", + "metadata": {}, + "source": [ + "
    \n", + "3. BinarySeriesOp\n", + "

    The BinarySeriesOp() function can be configured to operate in one of three input modes - ONE2ONE, MANY2ONE, and MATCH. These modes determine the number of primary series and number of secondary series involved in the operation, as well as determining how the primary and secondary series will be matched together.

    \n", + "\n", + "

    The first step is to convert the data into TDSeries, which is required for the input time series which are passed to the BinarySeriesOp function.

    \n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a711f58c-fbd3-4457-9f6a-9cd5c1b192dc", + "metadata": {}, + "outputs": [], + "source": [ + "data1_series_df = TDSeries(data=data1,\n", + " id=\"id\",\n", + " row_index=\"seq\",\n", + " row_index_style=\"SEQUENCE\",\n", + " payload_field=[\"real_val\", \"imaginary_val\"],\n", + " payload_content=\"COMPLEX\") " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "73e8def9-f8a7-40b0-bbf0-4cb3e4295794", + "metadata": {}, + "outputs": [], + "source": [ + "data2_series_df = TDSeries(data=data2,\n", + " id=\"id\",\n", + " row_index=\"seq\",\n", + " row_index_style= \"SEQUENCE\",\n", + " payload_field=[\"real_val\", \"imaginary_val\"],\n", + " payload_content=\"COMPLEX\")" + ] + }, + { + "cell_type": "markdown", + "id": "d33d2bcc-c87c-4fd4-913a-79e330bd2da2", + "metadata": {}, + "source": [ + "

    Than, we form the filter expressions to filter the series with id=1.

    " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d9cc417c-b6f2-40a1-810b-bbfa893b10ba", + "metadata": {}, + "outputs": [], + "source": [ + "data1_filter_expr=data1_series_df.id==1\n", + "data2_filter_expr=data2_series_df.id==1" + ] + }, + { + "cell_type": "markdown", + "id": "0bd6859c-b85a-4599-90f5-1f6621980fab", + "metadata": {}, + "source": [ + "

    We use the BinarySeriesOp to perform addition of two time series of equal size.

    " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8965392c-404e-4e25-b5b8-c7ef06069154", + "metadata": {}, + "outputs": [], + "source": [ + "uaf_out = BinarySeriesOp(data1=data1_series_df,\n", + " data1_filter_expr=data1_filter_expr,\n", + " data2=data2_series_df,\n", + " data2_filter_expr=data2_filter_expr,\n", + " math_op=\"ADD\",\n", + " input_fmt_input_mode=\"MANY2ONE\")\n", + "\n", + "binaryseries_df = uaf_out.result\n", + "binaryseries_df" + ] + }, + { + "cell_type": "markdown", + "id": "2f66bf78-4ea4-4387-b078-a91f491931e6", + "metadata": {}, + "source": [ + "

    The result series always inherits the identifiers, series_id from the primary series.

    " + ] + }, + { + "cell_type": "markdown", + "id": "c25fdaf1-098b-4dea-984b-579c6412557e", + "metadata": {}, + "source": [ + "
    \n", + "4. Cleanup" + ] + }, + { + "cell_type": "markdown", + "id": "a5d460af-1a32-4681-8094-360c3f25dc1f", + "metadata": {}, + "source": [ + "

    Work Tables

    \n", + "

    The following code will clean up tables created above.

    " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9dff9677-dae1-4cc5-b78e-3d197cc41e0c", + "metadata": {}, + "outputs": [], + "source": [ + "db_drop_table(\"binary_complex_left\")\n", + "db_drop_table(\"binary_complex_right\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9b96168c-7973-4259-b3e0-668b92fd846e", + "metadata": {}, + "outputs": [], + "source": [ + "remove_context()" + ] + }, + { + "cell_type": "markdown", + "id": "4317a6cf-1479-4aa8-b30a-ee0a3b5231a8", + "metadata": {}, + "source": [ + "
    \n", + "\n", + "

    Links:

    \n", + "" + ] + }, + { + "cell_type": "markdown", + "id": "b2dcca28-5de5-44d7-88cb-45a12153b3f8", + "metadata": {}, + "source": [ + "" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.10" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}