Used cosmos as db #1

lilyjma · 2019-11-05T01:23:34Z

App is working but with following problems:

A logged in user can mark other user's tasks as complete or not and can delete other user's tasks
(it can't add tasks on behalf of other users though)

(UPDATE 11/7: this might've been a problem that existed in the original forked repo)
2. Tasks have correct owners when a new user creates an account and log in, but wrong owners if an existing user logs in; however, refreshing the page corrects the problem. (Still trying to figure out what went wrong as the front end is getting the right data.)

(UPDATE 11/7: new bug introduced)
3. A user making a new account with an existing email should trigger an error message saying there's an existing user with that email--the error message is sent to the browser but not displayed for some reason.

After fixing these two issues, the next steps are:

Use Key Vault to manage the key to Cosmos and other creds. (UPDATE 11/7: DONE)
Use Identity as way to authorize log in
Host the app using Azure App Service

Please feel free to suggest other services to use, other ways to doing things, and best practices in general for app building.

jongio · 2019-11-06T23:18:26Z

app/models/models.py

+
+    @staticmethod
+    def get_user_by_email(user_email):
+        query = "SELECT * FROM users u where u.email='" + user_email + "'"


Cosmos SQL is subject to SQL injection.

Read this: https://en.wikipedia.org/wiki/SQL_injection and this https://michaelhowardsecure.blog/2019/03/05/cosmosdb-and-sql-injection/
And then re-write these queries as parameterized queries: https://azuresdkdocs.blob.core.windows.net/$web/python/azure-cosmos/4.0.0b5/index.html?highlight=parameterized%20queries#query-the-database

jongio · 2019-11-06T23:21:00Z

README.md

@@ -1,4 +1,4 @@
-# Flask + React + Postgres Starter 
+# Flask + React + Cosmos 


Seems like this would be a different sample. So let's chat before you attempt to merge back into this repo. We'll likely create a new one.

Okay. I'll keep it as is for now and change after we chat.

jongio · 2019-11-06T23:32:52Z

static/src/index.js

@@ -1,5 +1,7 @@
 import React from "react";


I suggest using https://create-react-app.dev/ to help setup the app, wire up dependencies, and so on.

Hi Jon,

I'm not quite sure what you mean by this. My understanding is that since this is not purely a React app so I can't really just follow the code structure given on that site. From what I've read there, I haven't found any material on integrating React with a framework like Flask; I only came across something on integration with an API backend. Upon Googling 'flask react boilerplate', one of the Github repos that came up was the the one that joshgav used to base his Flask-React-Postgres app on.

Could you elaborate on what you mean here? I'm probably misunderstanding you or missing some pieces of info here.

Thanks!

-Lily

Create react app is kind of a standard these days for creating react apps, it handles a lot of the stuff that I'm seeing issues with above, like css/js dependency managements, etc. It sets you up with a good base to work from. You can put this on the backlog to do later if you'd like.

jongio · 2019-11-07T00:09:53Z

app/__init__.py

+key = app.config["SECRET_KEY"]
+uri = app.config["COSMOS_DB_URI"]
+
+client = CosmosClient(uri, credential=key)


I'm not crazy about having the cosmos client as a global variable for the app. I would like all the "cosmos" stuff to be encapsulated so the app is only coupled to that encapsulation versus directly to cosmos.

Done, I think. Added a db.py in /utils and kept cosmos things there. Imported the db in models.py to use. I thought this way, if in the future, we want to use other db, we can keep all db things there and just import the appropriate dbs to use in models.py or other scripts.

lilyjma · 2019-11-07T20:08:54Z

More changes. Invited Maggie to be collaborator.

…Flask-React-Postgres into team_standup_app_cosmos

maggiepint · 2019-11-08T19:14:48Z

app/models/models.py

    def __init__(self, task, user_id, status):
-        self.date = datetime.utcnow().date()
+        self.id = str(randint(_MIN, _MAX))


I would think Cosmos has a way to generate random identifiers for documents - what happens if you just don't assign an id? I would think it would auto generate one.

Also, randint doesn't guarantee unique across all items, so please switch over to using Guid for ids. If you don't provide an "id" field, then Cosmos should auto create an id field and insert a guid.

You have to add an id to each item you insert into the container according to the docs here.
https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/cosmos/azure-cosmos#insert-data

I'll try to use Guid.

jongio · 2019-11-08T20:07:03Z

README.md

+## 1. Setting up Azure Services
+First, sign up for a free [Azure](https://azure.microsoft.com/en-us/free/) account if you don't already have one. Sign into https://portal.azure.com.
+
+[Create a resource group](https://github.com/lilyjma/azurethings/blob/master/createResourceGroup.md) to store the resources that you'll be using here--Azure Cosmos DB and Key vault. Then follow the instructions in the links below to create each resource:


This references your fork directly. Make a relative path please.

I thought this should be in a repository of its own, because I created a repository called 'azurethings' to store small tutorials/things that I learned about Azure.

jongio · 2019-11-08T20:07:57Z

README.md

+   1. When creating the database, name it 'team_standup'. For this app, you also need two containers named 'tasks' and 'users'. The [partition key](https://docs.microsoft.com/en-us/azure/cosmos-db/partitioning-overview#choose-partitionkey) for both is '/id'. 
+2. [Key Vault](https://docs.microsoft.com/en-us/azure/key-vault/quick-create-portal#create-a-vault)
+   1. This will store the credentials for the resources this app uses. For example, it'll store the key to the cosmos database. This way, you don't reveal any key in your code. 
+   2. You'll add two secrets called 'cosmosKey' and 'cosmosURI' to Key Vault to hold the cosmos key and URI respectively. To find these, click into the Cosmos DB account created, go to 'Keys' tab and get the Primary Key and the URI. 


Anytime you use 'cosmos' it should be "Cosmos DB" to adhere to branding

jongio · 2019-11-08T20:08:53Z

README.md

+
+(Remember to store them in the resource group you created; this will make it easier to clean up the resources in the future.)
+
+1. [Azure Cosmos DB](https://docs.microsoft.com/en-us/azure/cosmos-db/create-cosmosdb-resources-portal#create-an-azure-cosmos-db-account)


I like to include the CLI commands along side the portal instructures or a section which is

Azure Portal

Portal steps here

Azure CLI

CLI steps here

The CLI commands are also executed on Portal though--in the Cloud Shell. You could also execute CLI commands on your local machine but that requires some extra configuration so I thought it'd be the easiest to just do it on Cloud Shell.

jongio · 2019-11-08T20:09:55Z

README.md

+
+    ```az ad sp create-for-rbac --name http://my-applications-url --skip-assignment```
+
+    Use your web app's url. To find that, click on the app in Portal, go to the 'Overview' tab and look for 'URL' on the top right portion of the page. It looks something like https://myUniqueAppName.azurewebsites.net. 


What do you mean use your web apps url? In the above or below? Please be explicit in everything you ask the user to do.

jongio · 2019-11-08T20:10:26Z

README.md

-cd flask-react-postgres
-```
+   ```bash
+   git clone https://github.com/lilyjma/Flask-React-Postgres/tree/team_standup_app_cosmos


I don't think this path is right, but doesn't matter for now

jongio · 2019-11-08T20:14:52Z

app/models/models.py

-        if user and bcrypt.check_password_hash(user.password, password):
-            return user
+        users = user_container.query_items(
+            query="SELECT * FROM users u WHERE u.email = @userEmail AND u.password = @userPassword",


Just @ email and @ password should be ok.

jongio · 2019-11-08T20:17:17Z

app/models/models.py

    def __init__(self, task, user_id, status):
-        self.date = datetime.utcnow().date()
+        self.id = str(randint(_MIN, _MAX))


Also, randint doesn't guarantee unique across all items, so please switch over to using Guid for ids. If you don't provide an "id" field, then Cosmos should auto create an id field and insert a guid.

jongio · 2019-11-08T20:19:30Z

app/models/models.py

            return True
-        except IntegrityError:
+        except exceptions.CosmosHttpResponseError:  # item couldn't be upserted
            return False


Let's do some research on how we want to handle error messages from the model. Typically the UI doesn't want to show the error details, but something to consider, versus just returning false.

Oh the UI wouldn't just show 'False.' It'll have an error message that's appropriate for the specific situation. Please take a look at the routes.py script.

jongio · 2019-11-08T20:21:24Z

static/src/index.js

@@ -1,5 +1,7 @@
 import React from "react";


Create react app is kind of a standard these days for creating react apps, it handles a lot of the stuff that I'm seeing issues with above, like css/js dependency managements, etc. It sets you up with a good base to work from. You can put this on the backlog to do later if you'd like.

jongio · 2019-11-08T20:30:26Z

app/models/models.py

+            enable_cross_partition_query=True,
+        )
+        user = list(users)[0]
+
        return user

    @staticmethod
    def get_user_with_email_and_password(email, password):


We should be using OAuth to manage users and permissions. Having a method with email and password basically means that your app controls user access, permissions, etc and storing of credentials in the database. Typically you avoid with this access tokens and OAuth. It's a bigger more complicated implementation that you should learn, but we can discuss later.

Yes, I'd love to have a discussion on this. Right now, one problem I'm seeing is that when a user is logged in, he/she can edit other people's tasks, instead of just his/her own. So basically, a logged in user has access to the whole database, instead of just part of it that stores his/her own info. I don't know if this can be solved by OAuth, but the way this works now is not ideal.

Also, can we discuss the create-react-app too, like how to make it integrate with Flask? I didn't change the structure of the code at all. I think the original app is based on this boilerplate that combines Flask and React, which I guess is not ideally structured.

…eyvault

Changed what needs to be done to create a service principal. Previously misunderstood the --name parameter in the command--it just has to be a name that starts with 'http://' and doesn't have to be the URI to a real, existing site.

Lily Ma added 3 commits October 29, 2019 09:17

working prototype with postgres

6fa55e4

this app uses cosmos instead of postgres

d8a8cb2

working prototype with cosmos as db

c35e265

lilyjma requested a review from jongio November 5, 2019 01:25

used key vault to store cosmos creds

3796753

jongio requested changes Nov 7, 2019

View reviewed changes

Lily Ma added 5 commits November 6, 2019 16:20

use dotenv to read env vars

0863ec1

add dotenv to requirement.txt

1158785

parameterized sql queries

d5e183b

encapsulated all db things in utils/db.py

8023c90

pass in user/task as dict to create/upsert item

75fc439

lilyjma requested a review from jongio November 7, 2019 20:09

Lily Ma and others added 5 commits November 7, 2019 17:00

updating readme

476098e

Update README.md

60604a0

update readme

d2a11d5

Merge branch 'team_standup_app_cosmos' of https://github.com/lilyjma/…

4fe77ea

…Flask-React-Postgres into team_standup_app_cosmos

fixed path to get env vars correctly

a8b1407

maggiepint reviewed Nov 8, 2019

View reviewed changes

Lily Ma and others added 4 commits November 8, 2019 11:21

update readme

6ccc15f

Update README.md

f27e7dc

Update README.md

fb646a2

Update README.md

9834e34

jongio requested changes Nov 8, 2019

View reviewed changes

Lily Ma and others added 6 commits November 8, 2019 14:27

used uuid for task id

166b2d6

update readme

144a8cc

added .env.tmp

5291854

update readme

fbf02c6

update readme: authorize service principal to perform operations to k…

7d1f785

…eyvault

Update README.md

bd86c40

lilyjma added 2 commits November 11, 2019 10:37

Update README.md

efe01c5

Service principal change

b422d27

Changed what needs to be done to create a service principal. Previously misunderstood the --name parameter in the command--it just has to be a name that starts with 'http://' and doesn't have to be the URI to a real, existing site.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Used cosmos as db #1

Used cosmos as db #1

lilyjma commented Nov 5, 2019 •

edited

Loading

jongio Nov 6, 2019

lilyjma Nov 7, 2019

jongio Nov 6, 2019

lilyjma Nov 7, 2019 •

edited

Loading

jongio Nov 6, 2019

lilyjma Nov 7, 2019 •

edited

Loading

jongio Nov 8, 2019

jongio Nov 7, 2019

lilyjma Nov 7, 2019 •

edited

Loading

lilyjma commented Nov 7, 2019 •

edited

Loading

maggiepint Nov 8, 2019

jongio Nov 8, 2019

lilyjma Nov 8, 2019

jongio Nov 8, 2019

lilyjma Nov 8, 2019

jongio Nov 8, 2019

jongio Nov 8, 2019

lilyjma Nov 8, 2019 •

edited

Loading

jongio Nov 8, 2019

jongio Nov 8, 2019

jongio Nov 8, 2019

jongio Nov 8, 2019

jongio Nov 8, 2019

lilyjma Nov 8, 2019

jongio Nov 8, 2019

jongio Nov 8, 2019

lilyjma Nov 8, 2019 •

edited

Loading

		@@ -1,4 +1,4 @@
		# Flask + React + Postgres Starter
		# Flask + React + Cosmos


		(Remember to store them in the resource group you created; this will make it easier to clean up the resources in the future.)

		1. [Azure Cosmos DB](https://docs.microsoft.com/en-us/azure/cosmos-db/create-cosmosdb-resources-portal#create-an-azure-cosmos-db-account)


		```az ad sp create-for-rbac --name http://my-applications-url --skip-assignment```

		Use your web app's url. To find that, click on the app in Portal, go to the 'Overview' tab and look for 'URL' on the top right portion of the page. It looks something like https://myUniqueAppName.azurewebsites.net.

Used cosmos as db #1

Are you sure you want to change the base?

Used cosmos as db #1

Conversation

lilyjma commented Nov 5, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lilyjma Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lilyjma Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lilyjma Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

lilyjma commented Nov 7, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Azure Portal

Azure CLI

lilyjma Nov 8, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lilyjma Nov 8, 2019 • edited Loading

Choose a reason for hiding this comment

lilyjma commented Nov 5, 2019 •

edited

Loading

lilyjma Nov 7, 2019 •

edited

Loading

lilyjma Nov 7, 2019 •

edited

Loading

lilyjma Nov 7, 2019 •

edited

Loading

lilyjma commented Nov 7, 2019 •

edited

Loading

lilyjma Nov 8, 2019 •

edited

Loading

lilyjma Nov 8, 2019 •

edited

Loading