Skip to content

Commit df0ea3b

Browse files
mcburtonmcburton
committed
adding pandas extra and requirements
1 parent c38deba commit df0ea3b

12 files changed

+7794
-0
lines changed

‎day-two/5-apis-and-json.ipynb

Lines changed: 391 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,391 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"## Application Programming Interfaces (APIs)\n",
8+
"\n",
9+
"![Diagram of an API, Image from Software AG](https://www.softwareag.com/pl/images/SAG_API_Portal_KD_1_Large_tcm403-160297.png)\n",
10+
"\n",
11+
"* API or application programming interface is not a new term, but it has taken on a new significance with the Web\n",
12+
"* Now we talk about \"Web APIs\" or \"Web Services\" which are APIs that use web technologies and standards ;)\n",
13+
"* If UIs are interaces for the user to access a system, APIs are software interfaces for systems to access other systems (with the help of a Developer).\n"
14+
]
15+
},
16+
{
17+
"cell_type": "markdown",
18+
"metadata": {},
19+
"source": [
20+
"## What might an API look like?\n",
21+
"\n",
22+
"* Imagine having an API that takes a time zone string and returns the current time for that time zone\n",
23+
"* This API would take a string like \"America/Los_Angeles\" and return \"2019-02-28T20:09:45-07:00\"\n",
24+
"* One design of the API might look like this:\n",
25+
" * http://api.example.com/timezone?tz=America+Los_Angeles\n",
26+
"\n",
27+
"```json\n",
28+
"{\n",
29+
" \"time\": \"2019-02-28T20:09:45-07:00\",\n",
30+
" \"zone\": \"America/Los_Angeles\"\n",
31+
"}\n",
32+
"```\n",
33+
"\n",
34+
"* This API has an *endpoint*, `/timezone` that expects a *query paramater, `tz={Timezone location}`"
35+
]
36+
},
37+
{
38+
"cell_type": "markdown",
39+
"metadata": {},
40+
"source": [
41+
"## APIs Support Multiple Interfaces\n",
42+
"\n",
43+
"* With an API you don't need to build multiple interfaces to applications and platforms\n",
44+
"* You build a generic interface and the platforms conform to the API specification and data formats\n",
45+
"\n",
46+
"![A web page that accesses an API. Image from Express In Action](https://cdn.glitch.com/47219279-662d-49cf-9388-c11e70fac7be%2FScreenshot%202019-02-28%2013.06.03.png?1551377204363)\n",
47+
"\n",
48+
"* By separating the content from the representation we can support multiple modes of access\n",
49+
"\n",
50+
"![A command line application accessing an API. Image from Express In Action](https://cdn.glitch.com/47219279-662d-49cf-9388-c11e70fac7be%2FScreenshot%202019-02-28%2013.06.31.png?1551377204295)\n",
51+
"\n",
52+
"* APIs can use different data formats like XML or proprietary formats\n",
53+
"* These days many web-based APIs, including the LC API, use a data format called JSON"
54+
]
55+
},
56+
{
57+
"cell_type": "markdown",
58+
"metadata": {},
59+
"source": [
60+
"## What is JSON\n",
61+
"\n",
62+
"* JSON stands for [*JavaScript Object Notation*](https://www.json.org/)\n",
63+
"* Is a text-based format that is valid JavaScript code\n",
64+
"* It is \"self-describing\" and easy to read and understand\n",
65+
"* A lightweight format for storing and transferring data\n",
66+
"* While it uses JavaScript syntax, it is supported by nearly every programming language."
67+
]
68+
},
69+
{
70+
"cell_type": "markdown",
71+
"metadata": {},
72+
"source": [
73+
"## JSON Data Types\n",
74+
"\n",
75+
"* Strings - Strings must be double quotes. `{\"name\":\"Bob\"}`\n",
76+
"* Numbers - Numbers must be integer or floating point numbers. `5, 5.6`\n",
77+
"* Objects - Values can be JSON objects. `{\"employee\":{ \"name\":\"John\", \"age\":30, \"city\":\"New York\" }}` (look like anything you know?)\n",
78+
"* Arrays - Arrays must be an order list of any values. `{\"employees\":[ \"John\", \"Anna\", \"Peter\" ]}`\n",
79+
"* Boolean - Must be true or false value. `{\"sale\":true}`\n",
80+
"* Null - Values can also be null. `{\"middlename\":null}`\n"
81+
]
82+
},
83+
{
84+
"cell_type": "markdown",
85+
"metadata": {},
86+
"source": [
87+
"## Working with JSON in Python\n",
88+
"\n"
89+
]
90+
},
91+
{
92+
"cell_type": "code",
93+
"execution_count": 2,
94+
"metadata": {},
95+
"outputs": [],
96+
"source": [
97+
"# import the JSON module so we can load and save JSON data\n",
98+
"import json\n"
99+
]
100+
},
101+
{
102+
"cell_type": "markdown",
103+
"metadata": {},
104+
"source": [
105+
"* "
106+
]
107+
},
108+
{
109+
"cell_type": "markdown",
110+
"metadata": {},
111+
"source": [
112+
"## Reading JSON\n",
113+
"\n",
114+
"* In Python you parse JSON text into Python data structures using the `json.loads()` and `json.load()` functions.\n",
115+
"* "
116+
]
117+
},
118+
{
119+
"cell_type": "code",
120+
"execution_count": 5,
121+
"metadata": {},
122+
"outputs": [
123+
{
124+
"data": {
125+
"text/plain": [
126+
"'\\n[\\n{\"first\":\"Matt\",\"last\":\"Burton\",\"city\":\"Pittsburgh\"},\\n{\"name\":\"John\", \"age\":30, \"city\":\"New York\"},\\n{\"first\": \"Jessica\", \"city\":\"Chicago\"}\\n]\\n'"
127+
]
128+
},
129+
"execution_count": 5,
130+
"metadata": {},
131+
"output_type": "execute_result"
132+
}
133+
],
134+
"source": [
135+
"# create a string that represents some JSON data\n",
136+
"json_string = \"\"\"\n",
137+
"[\n",
138+
"{\"first\":\"Matt\",\"last\":\"Burton\",\"city\":\"Pittsburgh\"},\n",
139+
"{\"name\":\"John\", \"age\":30, \"city\":\"New York\"},\n",
140+
"{\"first\": \"Jessica\", \"city\":\"Chicago\"}\n",
141+
"]\n",
142+
"\"\"\"\n",
143+
"json_string"
144+
]
145+
},
146+
{
147+
"cell_type": "markdown",
148+
"metadata": {},
149+
"source": [
150+
"* This is a Python string, but it can be parsed into a list of dictionaries using `json.loads()`"
151+
]
152+
},
153+
{
154+
"cell_type": "code",
155+
"execution_count": 6,
156+
"metadata": {},
157+
"outputs": [
158+
{
159+
"data": {
160+
"text/plain": [
161+
"[{'city': 'Pittsburgh', 'first': 'Matt', 'last': 'Burton'},\n",
162+
" {'age': 30, 'city': 'New York', 'name': 'John'},\n",
163+
" {'city': 'Chicago', 'first': 'Jessica'}]"
164+
]
165+
},
166+
"execution_count": 6,
167+
"metadata": {},
168+
"output_type": "execute_result"
169+
}
170+
],
171+
"source": [
172+
"json_data = json.loads(json_string)\n",
173+
"json_data"
174+
]
175+
},
176+
{
177+
"cell_type": "markdown",
178+
"metadata": {},
179+
"source": [
180+
"* Now the data had been loaded into Python and we can access it "
181+
]
182+
},
183+
{
184+
"cell_type": "code",
185+
"execution_count": 10,
186+
"metadata": {},
187+
"outputs": [
188+
{
189+
"data": {
190+
"text/plain": [
191+
"{'city': 'Pittsburgh', 'first': 'Matt', 'last': 'Burton'}"
192+
]
193+
},
194+
"execution_count": 10,
195+
"metadata": {},
196+
"output_type": "execute_result"
197+
}
198+
],
199+
"source": [
200+
"# get the first item \n",
201+
"json_data[0]"
202+
]
203+
},
204+
{
205+
"cell_type": "code",
206+
"execution_count": 11,
207+
"metadata": {},
208+
"outputs": [
209+
{
210+
"data": {
211+
"text/plain": [
212+
"'New York'"
213+
]
214+
},
215+
"execution_count": 11,
216+
"metadata": {},
217+
"output_type": "execute_result"
218+
}
219+
],
220+
"source": [
221+
"# get the city field of the second item\n",
222+
"json_data[1]['city']"
223+
]
224+
},
225+
{
226+
"cell_type": "markdown",
227+
"metadata": {},
228+
"source": [
229+
"* Parsing JSON can be very picky and cryptic\n",
230+
"* For example, why doesn't this work?"
231+
]
232+
},
233+
{
234+
"cell_type": "code",
235+
"execution_count": 7,
236+
"metadata": {},
237+
"outputs": [
238+
{
239+
"ename": "JSONDecodeError",
240+
"evalue": "Expecting property name enclosed in double quotes: line 5 column 22 (char 124)",
241+
"output_type": "error",
242+
"traceback": [
243+
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
244+
"\u001b[0;31mJSONDecodeError\u001b[0m Traceback (most recent call last)",
245+
"\u001b[0;32m<ipython-input-7-9c7bb4442ce1>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 7\u001b[0m ]\n\u001b[1;32m 8\u001b[0m \"\"\"\n\u001b[0;32m----> 9\u001b[0;31m \u001b[0mjson\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mloads\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mbad_json_string\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
246+
"\u001b[0;32m~/miniconda3/lib/python3.7/json/__init__.py\u001b[0m in \u001b[0;36mloads\u001b[0;34m(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)\u001b[0m\n\u001b[1;32m 346\u001b[0m \u001b[0mparse_int\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mparse_float\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 347\u001b[0m parse_constant is None and object_pairs_hook is None and not kw):\n\u001b[0;32m--> 348\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_default_decoder\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 349\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mcls\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 350\u001b[0m \u001b[0mcls\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mJSONDecoder\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
247+
"\u001b[0;32m~/miniconda3/lib/python3.7/json/decoder.py\u001b[0m in \u001b[0;36mdecode\u001b[0;34m(self, s, _w)\u001b[0m\n\u001b[1;32m 335\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 336\u001b[0m \"\"\"\n\u001b[0;32m--> 337\u001b[0;31m \u001b[0mobj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraw_decode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0midx\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0m_w\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 338\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_w\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 339\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m!=\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
248+
"\u001b[0;32m~/miniconda3/lib/python3.7/json/decoder.py\u001b[0m in \u001b[0;36mraw_decode\u001b[0;34m(self, s, idx)\u001b[0m\n\u001b[1;32m 351\u001b[0m \"\"\"\n\u001b[1;32m 352\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 353\u001b[0;31m \u001b[0mobj\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscan_once\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0midx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 354\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mStopIteration\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 355\u001b[0m \u001b[0;32mraise\u001b[0m \u001b[0mJSONDecodeError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Expecting value\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ms\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
249+
"\u001b[0;31mJSONDecodeError\u001b[0m: Expecting property name enclosed in double quotes: line 5 column 22 (char 124)"
250+
]
251+
}
252+
],
253+
"source": [
254+
"# create a string that represents some JSON data\n",
255+
"bad_json_string = \"\"\"\n",
256+
"[\n",
257+
"{\"first\":\"Matt\",\"last\":\"Burton\",\"city\":\"Pittsburgh\"},\n",
258+
"{\"name\":\"John\", \"age\":30, \"city\":\"New York\"},\n",
259+
"{\"first\": \"Jessica\", city:\"Chicago\"}\n",
260+
"]\n",
261+
"\"\"\"\n",
262+
"json.loads(bad_json_string)"
263+
]
264+
},
265+
{
266+
"cell_type": "markdown",
267+
"metadata": {},
268+
"source": [
269+
"### Reading JSON from files\n",
270+
"\n",
271+
"* If you have a JSON file you can read it from disk using `json.load()`\n",
272+
" * Example file comes from [wikipedia](https://en.wikipedia.org/wiki/JSON)\n",
273+
"* "
274+
]
275+
},
276+
{
277+
"cell_type": "code",
278+
"execution_count": 9,
279+
"metadata": {},
280+
"outputs": [
281+
{
282+
"data": {
283+
"text/plain": [
284+
"{'address': {'city': 'New York',\n",
285+
" 'postalCode': '10021-3100',\n",
286+
" 'state': 'NY',\n",
287+
" 'streetAddress': '21 2nd Street'},\n",
288+
" 'age': 27,\n",
289+
" 'children': [],\n",
290+
" 'firstName': 'John',\n",
291+
" 'isAlive': True,\n",
292+
" 'lastName': 'Smith',\n",
293+
" 'phoneNumbers': [{'number': '212 555-1234', 'type': 'home'},\n",
294+
" {'number': '646 555-4567', 'type': 'office'},\n",
295+
" {'number': '123 456-7890', 'type': 'mobile'}],\n",
296+
" 'spouse': None}"
297+
]
298+
},
299+
"execution_count": 9,
300+
"metadata": {},
301+
"output_type": "execute_result"
302+
}
303+
],
304+
"source": [
305+
"# open a file handler in read-only mode\n",
306+
"with open(\"test.json\", \"r\") as f:\n",
307+
" # pass the file handler to the json parser\n",
308+
" json_from_file = json.load(f)\n",
309+
" \n",
310+
"json_from_file"
311+
]
312+
},
313+
{
314+
"cell_type": "markdown",
315+
"metadata": {},
316+
"source": [
317+
"* This has been parsed into a JSON dictionary\n",
318+
"* The JSON keys are now Python Dictionary keys"
319+
]
320+
},
321+
{
322+
"cell_type": "code",
323+
"execution_count": 12,
324+
"metadata": {},
325+
"outputs": [
326+
{
327+
"data": {
328+
"text/plain": [
329+
"'John'"
330+
]
331+
},
332+
"execution_count": 12,
333+
"metadata": {},
334+
"output_type": "execute_result"
335+
}
336+
],
337+
"source": [
338+
"json_from_file['firstName']"
339+
]
340+
},
341+
{
342+
"cell_type": "code",
343+
"execution_count": 13,
344+
"metadata": {},
345+
"outputs": [
346+
{
347+
"data": {
348+
"text/plain": [
349+
"'John Smith'"
350+
]
351+
},
352+
"execution_count": 13,
353+
"metadata": {},
354+
"output_type": "execute_result"
355+
}
356+
],
357+
"source": [
358+
"full_name = json_from_file[\"firstName\"] + \" \" + json_from_file[\"lastName\"]\n",
359+
"full_name"
360+
]
361+
},
362+
{
363+
"cell_type": "code",
364+
"execution_count": null,
365+
"metadata": {},
366+
"outputs": [],
367+
"source": []
368+
}
369+
],
370+
"metadata": {
371+
"kernelspec": {
372+
"display_name": "Python 3",
373+
"language": "python",
374+
"name": "python3"
375+
},
376+
"language_info": {
377+
"codemirror_mode": {
378+
"name": "ipython",
379+
"version": 3
380+
},
381+
"file_extension": ".py",
382+
"mimetype": "text/x-python",
383+
"name": "python",
384+
"nbconvert_exporter": "python",
385+
"pygments_lexer": "ipython3",
386+
"version": "3.7.3"
387+
}
388+
},
389+
"nbformat": 4,
390+
"nbformat_minor": 2
391+
}

0 commit comments

Comments
 (0)