update

serverless-tutorial/README.md

@@ -8,15 +8,247 @@ Video: TODO: add video link
 ---
 ## Prerequisites
 
-TODO: add prerequisites
+Before you can create serverless functions in **DigitalOcean**, you need to sign up at TODO: add referral code here.
+
+*TODO: add referral description here*
+
+*Some examples that get information about the latest video of a YouTube channel, need to use the YouTube Data API. The YouTube Data API allows you to retrieve information about YouTube videos, channels, playlists, and more. To use the API, you'll need to have a project on the Google Cloud Platform, and you'll need to enable the YouTube Data API for that project.*
 
 ---
+## Deploy a web function
+
+### Create a namespace
+
+Log in to your **DigitalOcean** Account and in the web UI, create a new **namespace** in the **manage -> functions** section.
+
+### Create a function
+
+In the **namespace** section, you can create a new **function** under **actions -> create function**.
+
+Select your desired **runtime**, e.g. `Python`, `NodeJS`, etc. In this tutorial I'm using `Python` You can also add your function to a **package**, which is optional. Make sure the **web function** checkbox is enabled.
+
+In the code editor, you can add your desired code or modify the example.
+
+### Run the function
+
+To run the function from the web ui, you can click on **run**, this will simulate a web request to the function. You can also change the **parameters**, which is optional.
 
-TODO: add tutorial
+You can also run the function from your PC by opening a web request.
+
+```sh
+curl $(doctl sls fn get your-package/your-function --url)
+```
+
+### Authentication
+
+To require a secret to run your function, you can edit the function settings it in the **settings -> access & security -> web function** menu. 
+
+Enable the checkbox **secure web function**, and add a custom secret, or use the generated one.
+
+Test the authentication with the following command.
+
+```sh
+curl -X GET $(doctl sls fn get your-package/your-function --url) \
+     -H "Content-Type: application/json" \
+     -H "X-Require-Whisk-Auth: your-secret"
+```
 
 ---
-## References
+## Develop functions in vscode
+
+### Install doctl
+
+Install `doctl` following the directions for your package manager or operating system, e.g. macOS:
+
+```sh
+brew install doctl
+```
+
+### Install serverless support
+
+Install the support for serverless functions in **doctl**.
+
+```sh
+doctl serverless install
+```
+
+### Connect to your namespace
+
+Connect to your **namespace**, you have created before.
+
+```sh
+doctl serverless connect your-namespace
+```
+
+### Initialize a new project
+
+You can initialize a new project by executing the following command.
+
+```sh
+doctl serverless init your-namespace --language python
+```
+
+You can now open the **namespace** project directory in a tool like **VSCode** and start developing your **packages** and **functions**.
+
+### Edit the project file
+
+All project settings of a **namespace** , **packages**, and **functions** are described in a `project.yml` file.
+
+```yaml
+packages:
+    - name: your-package
+      functions:
+        - name: your-function
+          runtime: python:default
+          web: false
+```
+
+### Edit packages and functions
+
+All **packages** need to be created in a separate folder e.g. `packages/`.  All **functions** are created by adding files in the package folders.
+
+A project structure should look like this.
+
+```text
+namespace/project
+ ↳ packages
+    ↳ your-package-1
+        ↳ your-function-1.py
+        ↳ your-function-2.py
+    ↳ your-package-1
+        ↳ your-function-3.py
+ ↳ .gitignore
+ ↳ project.yml
+```
 
-TODO: add references
+### Deploy the package
+
+Upload the **package** code to DigitalOcean.
+
+```sh
+doctl serverless deploy your-package
+```
+
+### Invoke a function
+
+You can test and run a non-web function with the following command.
+
+```sh
+doctl serverless functions invoke your-package/your-function
+```
+
+### Invoke a function with parameters
+
+You can invoke **functions** with optional or required parameters.
+
+```sh
+doctl serverless functions invoke httpreq-demo-1/hello -p name:Christian
+```
+
+In the code, you can read parameters like the following.
+
+```py
+def main(args):
+    name = args.get("name", "stranger")
+    greeting = "Hello " + name + "!"
+    print(greeting)
+    return {"body": greeting}
+```
+
+### Environment Variables
+
+You can add environment variables by updating your `project.yml` like the following.
+
+```yml
+...
+packages:
+    - name: httpreq-demo-1
+      functions:
+        - name: hello
+          binary: false
+          main: ""
+          runtime: python:default
+          web: false
+          websecure: false
+          parameters: {}
+          environment: {
+            PERSON: "Christian"
+          }
+
+```
+
+In the code, you can refer to the environment variables like the following.
+
+```py
+import os
+
+
+def read_env_variable(variable_name):
+    try:
+        return os.environ.get(variable_name)
+    except Exception:
+        return False
+
+
+def main(args):
+    greeting = f"Hello {read_env_variable('PERSON')}"
+    print(greeting)
+    return {"body": greeting}
+```
+
+---
+## Troubleshooting
+
+### Build requirements
+
+You can add other Python Libraries by adding a `requirements.txt` and a `build.sh` script into your project.
+
+Change your **project** structure according to the following.
+
+```text
+namespace/project
+ ↳ packages
+    ↳ your-package-1
+        ↳ your-function-1
+	        ↳ __main__.py
+	        ↳ build.sh
+	        ↳ requirements.txt
+```
+
+**Example `requirements.txt`:**
+```sh
+google-api-python-client==2.72.0
+```
+
+**Example `build.sh`:**
+```sh
+#!/bin/bash
+
+set -e
+
+virtualenv virtualenv
+source virtualenv/bin/activate
+pip install -r requirements.txt
+deactivate
+```
+
+### Increase Limits
+
+When you get timeout errors, try increasing the function **limits**.
+
+```yaml
+...
+packages:
+    - name: your-package
+      functions:
+        - name: your-function
+          ...
+		  limits:
+            timeout: 3000  # timeout in seconds
+            memory: 512  # memory in mb
+```
+
+---
+## References
 
-- []()
+- [DigitalOcean Functions Docs](https://docs.digitalocean.com/products/functions/) 

serverless-tutorial/files/youtubify-demo-2/.gitignore

@@ -0,0 +1,7 @@
+.nimbella
+.deployed
+__deployer__.zip
+__pycache__/
+node_modules
+package-lock.json
+.DS_Store

serverless-tutorial/files/youtubify-demo-2/packages/youtubify/getlatestvideo/__main__.py

@@ -0,0 +1,117 @@
+import os
+import requests
+from googleapiclient.discovery import build
+import boto3
+import botocore
+
+# read youtube api key and channel id from environment variables
+youtube_api_key = os.environ.get("YOUTUBE_API_KEY")
+youtube_channel_id = os.environ.get("YOUTUBE_CHANNEL_ID")
+
+# read spaces access key and secret from environment variables
+spaces_access_key = os.environ.get("SPACES_ACCESS_KEY")
+spaces_secret_key = os.environ.get("SPACES_SECRET_KEY")
+
+# read discord webhook url
+discord_webhook_url = os.environ.get("DISCORD_WEBHOOK_URL")
+
+
+# get latest video from a channel
+def get_latest_video(channel_id):
+    # get latest video from a channel
+    youtube = build("youtube", "v3", developerKey=youtube_api_key)
+
+    request = youtube.search().list(
+        part="snippet",
+        channelId=channel_id,
+        order="date",
+        maxResults=1
+    )
+
+    # execute the request
+    response = request.execute()
+
+    # get the video id from the response
+    video_id = response["items"][0]["id"]["videoId"]
+
+    # get the video meta data
+    request = youtube.videos().list(
+        part="snippet,contentDetails,statistics",
+        id=video_id
+    )
+
+    # execute the request
+    response = request.execute()
+
+    # return the video meta data
+    return response["items"][0]
+
+
+# get latest video from s3, if it exists, if not return None
+def get_latest_video_from_s3():
+    # get latest video from s3, if it exists
+
+    client = boto3.client(
+        "s3",
+        region_name="fra1",
+        endpoint_url="https://fra1.digitaloceanspaces.com",
+        aws_access_key_id=spaces_access_key,
+        aws_secret_access_key=spaces_secret_key
+    )
+
+    try:
+        response = client.get_object(
+            Bucket="youtubify-demo-2-space",
+            Key="latest_video_id.txt"
+        )
+
+        # return the video id
+        return response["Body"].read().decode("utf-8")
+    except botocore.exceptions.ClientError as e:
+        if e.response["Error"]["Code"] == "NoSuchKey":
+            # the object does not exist
+            return None
+
+
+# push the latest video id to s3
+def push_latest_video_to_s3(video_id):
+    # push the latest video id to s3
+
+    client = boto3.client(
+        "s3",
+        region_name="fra1",
+        endpoint_url="https://fra1.digitaloceanspaces.com",
+        aws_access_key_id=spaces_access_key,
+        aws_secret_access_key=spaces_secret_key
+    )
+
+    client.put_object(
+        Bucket="youtubify-demo-2-space",
+        Key="latest_video_id.txt",
+        Body=video_id
+    )
+
+
+def main(args):
+    # get latest youtube video from a channel
+    video = get_latest_video(youtube_channel_id)
+
+    # get latest video id from s3
+    latest_video_id = get_latest_video_from_s3()
+
+    # if the latest video id is not the same as the latest video id from s3
+    if latest_video_id != video["id"]:
+        # create a message with video url that will be pushed to discord
+        message = "New video: https://www.youtube.com/watch?v=" + video["id"]
+
+        # push the video url to discord webhook
+        requests.post(discord_webhook_url, json={
+            "content": message
+        })
+
+        # push the latest video id to s3
+        push_latest_video_to_s3(video["id"])
+
+        return {"body": "New video"}
+    else:
+        return {"body": "No new video"}

serverless-tutorial/files/youtubify-demo-2/packages/youtubify/getlatestvideo/build.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+
+set -e
+
+virtualenv virtualenv
+source virtualenv/bin/activate
+
+pip install -r requirements.txt
+deactivate

serverless-tutorial/files/youtubify-demo-2/packages/youtubify/getlatestvideo/requirements.txt

@@ -0,0 +1 @@
+google-api-python-client==2.72.0