Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recipe notebook for BinarySeriesOp #834

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Recipe notebook for BinarySeriesOp #834

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bb3fa24
Recipe notebook for BinarySeriesOp
shilpa-nalkande Jan 30, 2025
dc8b80c
corrected a minor error in Markdown
shilpa-nalkande Jan 31, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
354 changes: 354 additions & 0 deletions Recipes/ClearScape_Functions/BinarySeriesOp.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,354 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "bc549e6c-0cc4-4188-94a3-a9bdd3ae3dfa",
"metadata": {},
"source": [
"<header>\n",
" <p style='font-size:36px;font-family:Arial; color:#F0F0F0; background-color: #00233c; padding-left: 20pt; padding-top: 20pt;padding-bottom: 10pt; padding-right: 20pt;'>\n",
" BinarySeriesOp function in Vantage\n",
" <br>\n",
" <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
" </p>\n",
"</header>"
]
},
{
"cell_type": "markdown",
"id": "7ae7611a-0795-4168-b716-01fee6880cbd",
"metadata": {},
"source": [
"<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>Introduction</b></p>\n",
"\n",
"<p style = 'font-size:20px;font-family:Arial;color:#00233C'><b>BinarySeriesOp</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The BinarySeriesOp() function performs a point wise mathematical operation on two time series of equal size. The principal mathematical operation can be subtraction, addition, multiplication and division. It is called a point wise operation because it performs the mathematical operation one sample point at a time.</p>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Common uses of BinarySeriesOp() are:</p>\n",
"<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Subtracting trends from a time series to create a model from it.</li>\n",
"<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Restoring trends to a time series before using the model for forecasting.</li>\n",
"<li style = 'font-size:16px;font-family:Arial;color:#00233C'>As a building block to formulate more complex functions.</li></p>\n",
" \n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The following procedure is an example of how to use BinarySeriesOp() to convolve two series with digital signal processing:</p>\n",
"<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Use DFFT() function on series 1 and series 2 to get dataframes named 'dfftRes1' and 'dfftRes2', respectively.</li>\n",
"<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Use BinarySeriesOp() to do point-wise multiplication using 'dfftRes1' and 'dfftRes2'.</li>\n",
"<li style = 'font-size:16px;font-family:Arial;color:#00233C'>Use IDFFT() on the output of BinarySeriesOp() to get the convolved result of the two series.</li>\n",
"</p>"
]
},
{
"cell_type": "markdown",
"id": "6b3a00b4-6661-4c91-9b2d-cb7b0b403140",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
"<b style = 'font-size:20px;font-family:Arial;color:#00233C'>1. Initiate a connection to Vantage</b>"
]
},
{
"cell_type": "markdown",
"id": "2346857f-e0d3-488a-8a3f-ac6dff752c2b",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>In the section, we import the required libraries and set environment variables and environment paths (if required)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5af5af3-29d5-4f6a-8334-9df6924e7787",
"metadata": {},
"outputs": [],
"source": [
"from teradataml import (\n",
" create_context,\n",
" execute_sql,\n",
" load_example_data,\n",
" DataFrame,\n",
" in_schema,\n",
" TDSeries,\n",
" BinarySeriesOp,\n",
" db_drop_view,\n",
" remove_context,\n",
" copy_to_sql,\n",
" db_drop_table\n",
" )\n",
"\n",
"from teradatasqlalchemy.types import *\n",
"\n",
"from teradataml import to_numeric\n",
"# Modify the following to match the specific client environment settings\n",
"display.max_rows = 5"
]
},
{
"cell_type": "markdown",
"id": "ad3dd7b4-831c-4fb3-ab71-719c8c99a71c",
"metadata": {},
"source": [
"<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
"<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>1.1 Connect to Vantage</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>You will be prompted to provide the password. Enter your password, press the Enter key, and then use the down arrow to go to the next cell.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2742444c-4349-4b0f-b4e5-b068a8785cd9",
"metadata": {},
"outputs": [],
"source": [
"%run -i ../../UseCases/startup.ipynb\n",
"eng = create_context(host = 'host.docker.internal', username='demo_user', password = password)\n",
"print(eng)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "01c4a128-d106-46ea-8dee-34acc5abd29f",
"metadata": {},
"outputs": [],
"source": [
"%%capture\n",
"execute_sql('''SET query_band='DEMO=PP_BinarySeriesOp_Python.ipynb;' UPDATE FOR SESSION; ''')"
]
},
{
"cell_type": "markdown",
"id": "efe2fd2d-63ff-4278-9157-8b9110d682e8",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Begin running steps with Shift + Enter keys. </p>"
]
},
{
"cell_type": "markdown",
"id": "4a6e3852-3b15-4cbd-844d-e7194b3bd204",
"metadata": {},
"source": [
"<hr style='height:1px;border:none;background-color:#00233C;'>\n",
"\n",
"<p style = 'font-size:18px;font-family:Arial;color:#00233c'><b>1.2 Getting Data for This Demo</b></p>\n",
"\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Here, we will get the time series data which is available in the teradataml library and use the same to show the usage of the function.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "398fa0aa-c0df-4130-9dea-53b776fbaf80",
"metadata": {},
"outputs": [],
"source": [
"load_example_data(\"uaf\", [\"binary_complex_left\", \"binary_complex_right\"])"
]
},
{
"cell_type": "markdown",
"id": "2a3762ac-ba27-4fa3-adba-d577262a4290",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
"<b style = 'font-size:20px;font-family:Arial;color:#00233C'>2. Data Exploration</b>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Create a \"Virtual DataFrame\" that points to the data set in Vantage. Check the shape of the dataframe as check the datatype of all the columns of the dataframe.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9672d80d-cdcc-4f4c-9399-7c367e7970b1",
"metadata": {},
"outputs": [],
"source": [
"data1 = DataFrame.from_table(\"binary_complex_left\")\n",
"data2 = DataFrame.from_table(\"binary_complex_right\")\n",
"data1 "
]
},
{
"cell_type": "markdown",
"id": "4a2f79a0-eb16-4c3b-bec2-c313e97ac940",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
"<b style = 'font-size:20px;font-family:Arial;color:#00233C'>3. BinarySeriesOp</b>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The BinarySeriesOp() function can be configured to operate in one of three input modes - ONE2ONE, MANY2ONE, and MATCH. These modes determine the number of primary series and number of secondary series involved in the operation, as well as determining how the primary and secondary series will be matched together.</p>\n",
"\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The first step is to convert the data into TDSeries, which is required for the input time series which are passed to the BinarySeriesOp function.</p>\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a711f58c-fbd3-4457-9f6a-9cd5c1b192dc",
"metadata": {},
"outputs": [],
"source": [
"data1_series_df = TDSeries(data=data1,\n",
" id=\"id\",\n",
" row_index=\"seq\",\n",
" row_index_style=\"SEQUENCE\",\n",
" payload_field=[\"real_val\", \"imaginary_val\"],\n",
" payload_content=\"COMPLEX\") "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "73e8def9-f8a7-40b0-bbf0-4cb3e4295794",
"metadata": {},
"outputs": [],
"source": [
"data2_series_df = TDSeries(data=data2,\n",
" id=\"id\",\n",
" row_index=\"seq\",\n",
" row_index_style= \"SEQUENCE\",\n",
" payload_field=[\"real_val\", \"imaginary_val\"],\n",
" payload_content=\"COMPLEX\")"
]
},
{
"cell_type": "markdown",
"id": "d33d2bcc-c87c-4fd4-913a-79e330bd2da2",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>Than, we form the filter expressions to filter the series with id=1.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d9cc417c-b6f2-40a1-810b-bbfa893b10ba",
"metadata": {},
"outputs": [],
"source": [
"data1_filter_expr=data1_series_df.id==1\n",
"data2_filter_expr=data2_series_df.id==1"
]
},
{
"cell_type": "markdown",
"id": "0bd6859c-b85a-4599-90f5-1f6621980fab",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>We use the BinarySeriesOp to perform addition of two time series of equal size.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8965392c-404e-4e25-b5b8-c7ef06069154",
"metadata": {},
"outputs": [],
"source": [
"uaf_out = BinarySeriesOp(data1=data1_series_df,\n",
" data1_filter_expr=data1_filter_expr,\n",
" data2=data2_series_df,\n",
" data2_filter_expr=data2_filter_expr,\n",
" math_op=\"ADD\",\n",
" input_fmt_input_mode=\"MANY2ONE\")\n",
"\n",
"binaryseries_df = uaf_out.result\n",
"binaryseries_df"
]
},
{
"cell_type": "markdown",
"id": "2f66bf78-4ea4-4387-b078-a91f491931e6",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The result series always inherits the identifiers, series_id from the primary series.</p>"
]
},
{
"cell_type": "markdown",
"id": "c25fdaf1-098b-4dea-984b-579c6412557e",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;background-color:#00233C;\">\n",
"<b style = 'font-size:20px;font-family:Arial;color:#00233C'>4. Cleanup</b>"
]
},
{
"cell_type": "markdown",
"id": "a5d460af-1a32-4681-8094-360c3f25dc1f",
"metadata": {},
"source": [
"<p style = 'font-size:18px;font-family:Arial;color:#00233C'><b>Work Tables</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'>The following code will clean up tables created above.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9dff9677-dae1-4cc5-b78e-3d197cc41e0c",
"metadata": {},
"outputs": [],
"source": [
"db_drop_table(\"binary_complex_left\")\n",
"db_drop_table(\"binary_complex_right\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9b96168c-7973-4259-b3e0-668b92fd846e",
"metadata": {},
"outputs": [],
"source": [
"remove_context()"
]
},
{
"cell_type": "markdown",
"id": "4317a6cf-1479-4aa8-b30a-ee0a3b5231a8",
"metadata": {},
"source": [
"<hr style=\"height:1px;border:none;background-color:#00233C;\">\n",
"\n",
"<p style = 'font-size:16px;font-family:Arial;color:#00233C'><b>Links:</b></p>\n",
"<ul style = 'font-size:16px;font-family:Arial'>\n",
" <li>Teradataml Python reference: <a href = 'https://docs.teradata.com/search/all?query=Python+Package+User+Guide&content-lang=en-US'>here</a></li>\n",
" <li>BinarySeriesOp function reference: <a href = 'https://docs.teradata.com/search/all?query=BinarySeriesOp&content-lang=en-US'>here</a></li>\n",
" \n",
"</ul>"
]
},
{
"cell_type": "markdown",
"id": "b2dcca28-5de5-44d7-88cb-45a12153b3f8",
"metadata": {},
"source": [
"<footer style=\"padding-bottom:35px; background:#f9f9f9; border-bottom:3px solid #00233C\">\n",
" <div style=\"float:left;margin-top:14px\">ClearScape Analytics™</div>\n",
" <div style=\"float:right;\">\n",
" <div style=\"float:left; margin-top:14px\">\n",
" Copyright © Teradata Corporation - 2025. All Rights Reserved\n",
" </div>\n",
" </div>\n",
"</footer>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}