-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathmodule-three.html
511 lines (447 loc) · 42.5 KB
/
module-three.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
<!DOCTYPE html>
<html lang="en">
<head>
<!-- Global site tag (gtag.js) - Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', '{{ site.google_analytics }}', 'auto');
ga('send', 'pageview');
ga('set', 'anonymizeIp', true);
</script>
<title>Data for Decision-Making | Collecting Data</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" type="image/png" sizes="196x196" href="../template-assets/images/favicon.png">
<link rel="stylesheet" href="https://tascha.github.io/D4D/template-assets/css/style.css">
<script src="https://code.jquery.com/jquery-2.1.4.min.js" type="text/javascript" charset="utf-8"></script>
<script src="https://tascha.github.io/D4D/template-assets/js/template.js" type="text/javascript" charset="utf-8"></script>
</head>
<body>
<div class="wrapper" mode="overview">
<aside>
<div class="image">
<img src="images/TASCHA_logo_stacked_color.png" alt="TASCHA logo">
</div>
<ul class="agenda-navigation">
<li><a href="#overview">Overview</a></li>
<li><a href="#step-1">Review</a></li>
<li><a href="#step-2">Introduction to Key Concepts: Data Lifecycle</a></li>
<li><a href="#step-3">Providing Context</a></li>
<li><a href="#step-4">Student Activity 3.1</a></li>
<li><a href="#step-5">Understanding Data Collection Methods</a></li>
<li><a href="#step-6">Introduction to Key Concepts</a></li>
<li><a href="#step-7">Key Considerations in Designing a Data Collection Plan</a></li>
<li><a href="#step-8">Providing Future Resources</a></li>
<li><a href="#step-9">Student Activity 3.2</a></li>
<li><a href="#step-10">Lying With Data</a></li>
<li><a href="#step-11">Debrief</a></li>
</ul>
</aside>
<article class="main">
<div class="activity-menu">
<a class="toggle"><strong>Data for Decison-Making</strong> - Module 3 of 6</a>
<div class= "header-links"><a href="../D4D/index.html" target="_blank">HOME</a>
<a href="../D4D/about/" target="_blank">ABOUT</a>
<a href="../D4D/curriculum/" target="_blank">CURRICULUM</a>
<a href="../D4D/downloads/" target="_blank">DOWNLOADS</a></div>
<ol class="activity-list">
<li><a href="module-one.html">Introduction to Data for Decision-Making</a></li>
<li><a href="module-two.html">Assessing your Organization’s Data Culture</a></li>
<li class="current"><a href="module-three.html">Introduction to the Data Lifecycle - Collecting Data</a></li>
<li><a href="module-four.html">Introduction to the Data Lifecycle - Analyzing and Sharing Data</a></li>
<li><a href="module-five.html">Applying Data for Decision-Making
to Your Organization, Part 1</a></li>
<li><a href="module-six.html">Applying Data for Decision-Making
to Your Organization, Part 2</a></li>
<br>
<li><a href="downloads/">Visit our downloads page for more options</a></li>
</ol>
</div>
<h1 class="activity-title">Introduction to Data Lifecycle - Collecting Data</a></h1>
<section class="overview">
<p class="made-by">
<a href="https://creativecommons.org/licenses/by-sa/3.0/">CC-BY-SA</a> by <a href="https://tascha.uw.edu">TASCHA</a>
</p>
<p class="time total-time">
3 hours
</p>
<br><p class="download-box">
<a href="../D4D/documents/Module3_Contents.zip">Download this module</a> or visit our <a href="../D4D/downloads/" target="_blank">downloads page</a> for more options
</p>
<!--<p class="summary">-->
<p><h4>Student Objectives</h4>
<ul>
<li>Understand basic concepts of data lifecycle - including the collection, analysis, and sharing of data for decision-making.</li>
<li>Apply the data life cycle to a real life scenario at their own organization.</li>
<li>Be able to identify different potential future sources of data.</li>
<li>Understand data collection protocol.</li>
<li>Raise awareness of free online data collection and data management sources.</li>
<li>Recognize misleading data sources and visualizations.</li>
<li>Understand the pitfalls of lying with data and how to avoid it.</li>
</ul>
</p>
<h4>Materials</h4>
<ul>
<li>Projector</li>
<li>Computer</li>
<li>Blackboard/whiteboard (ideally)</li>
<li>Paper</li>
<li>Pencils</li>
<li><a href="../D4D/documents/Mod3_Images.doc">Printout of images</a></li>
<li><a href="http://tascha.github.io/D4D/documents/Activity_3.1.doc">Activity packet 3.1</a></li>
<li><a href="../D4D/documents/Acivity_3.2.doc">Activity packet 3.2</a></li>
<li><a href="../D4D/documents/StudentHandbook_Module3.doc">Student handbook</a></li>
<li><a href="../D4D/documents/Module3_DataforDecisions.ppt">Instructor Powerpoint slides</a></li>
</ul>
</section>
<ul class="agenda">
<li>
<h1>Review</h1>
<div class="time step-time">15 minutes</div>
<p>Welcome the participants back. Remind the participants of the structure for today: two, three-hour workshops with a break in between for lunch. Also use this time to go over any administrative tasks as necessary.</p>
<p>Then, review the following key concepts with the participants. Again, as before, try to have the participants provide their own definitions before you provide them.</p>
<ul>
<li>Data</li>
<li>Data for decision-making</li>
<li>Stakeholders</li>
<li>Assessment</li>
<li>Mission</li>
<li>Vision</li>
<li>Data producer</li>
<li>Data consumer</li>
</ul>
<p>Ask, what are the key steps to using data for decision-making? Remind the participants:</p>
<ul>
<li>Identify a problem or research question</li>
<li>Assess data available to you and your data needs</li>
<li>Identify stakeholders</li>
<li>Plan for how data will be used, analyze, and shared</li>
</ul>
<p>Pause to ask if anyone has questions so far.</p>
</li> <!--End of step-->
<!-- Copy and paste this <li> tag to add additional steps -->
<li>
<h1>Introduction to Key Concepts: Data Lifecycle</h1>
<div class="time step-time">15 minutes</div>
<p>Write the word “lifecycle” on the board or project it on the screen. Ask the class what they think the word means. Where have they seen this word before? What kind of examples of lifecycles can they provide? If the class is quiet or slow to respond, feel free to provide some guiding questions, such as what is an example of a lifecycle within their own life? An animal’s life? A plant’s life?</p>
<p>Tell the class that much like living things, data has a lifecycle of its own.</p>
<p>Project this image on the screen, or pass it around to participants.</p>
<p><img src="http://tascha.github.io/D4D/images/data-lifecycle.jpg" alt="Data Lifecycle Chart: Collect, Analyze, Share" width="100%"></p>
<p>Then, tell the class that this image represents a data life cycle. There are three stages:</p>
<ul>
<li>Collecting data</li>
<li>Analyzing data</li>
<li>Sharing data</li>
</ul>
<p>Remind the class that when using data for decision-making, it is often best to begin the data lifecycle with a question or problem that they are trying to solve. This helps to make their work more efficient and effective. There can be cases, however, when they have a dataset first and want to see what kinds of decisions they can make by using these data.</p>
<p>Then, move on to the following definitions. As before, solicit the class for their own definitions before providing them with the ones below.</p>
<ul>
<li>Data collection: data collection is the process of gathering information in a systematic way. Collected data are generally intended to answer questions and/or evaluate outcomes.</li>
</ul>
<p>Remind the class that data can be collected from surveys, questionnaires, interviews, or observations.</p>
<ul>
<li>Analyzing data: data analysis is the process of inspecting, cleaning, transforming, and visualizing data with the goal of discovering its useful information, suggesting conclusions, and supporting decision-making. (Wikipedia)</li>
</ul>
<p>Take time to explain to the participants that data analysis is made up of many stages:</p>
<ul>
<li>Inspecting the data</li>
<li>Cleaning the data</li>
<li>Transforming the data</li>
<li>Visualizing the data</li>
</ul>
<p>Introduce the above concepts, but make note that they will be explained in more detail in the next module. Then, provide the definition for the last part of the data lifecycle:</p>
<ul>
<li>Data sharing: data sharing is the process of making data that are used in problem solving, research, or evaluation available to others. </li>
</ul>
<p>Underscore to the participants the importance of sharing data. Many funding agencies, institutions, and publication venues around the world have policies regarding data sharing because transparency and openness are considered to be key parts of research. In addition to the data themselves, metadata and documentation should also be shared if possible.</p>
</li> <!-- end of step-->
<li>
<h1>Providing Context</h1>
<div class="time step-time">30 minutes</div>
<p>Provide context by going through these steps in a country specific example and a generic example.</p>
<p>Below, two examples have been provided of applications of the data lifecycle. One is a generic example, and one is a country specific example for Myanmar. If implanting this curriculum outside of Myanmar, it would be helpful to provided participants with data lifecycle examples in their own countries.</p>
<p><b>Generic Example: Digital Green, India</b></p>
<p>The use of information and communications technology (ICT) in agricultural services is becoming increasingly common. These technologies—which include radio, SMS, television, video, and Internet services—have the potential to help smallholder farmers increase their incomes by making it easier for them to learn about and adopt new farming methods, grow higher-value crops, or connect with new markets.</p>
<p>Digital Green, an international non-profit organization based in India, uses locally-produced videos and in-person facilitation to share knowledge about improved agricultural and nutrition practices. The program aims to help rural communities across South Asia and Sub-Saharan Africa understand and adopt better agricultural and nutrition practices, and the ultimate goal of the program is to have a positive impact on individual well-being. Digital Green is currently working in nine states in India, and also in Afghanistan, Ethiopia, Ghana, Niger, Tanzania, Malawi, and Papua New Guinea. Since its start in 2008, Digital Green’s program has produced over 4,000 videos reaching more than 800,000 viewers across more than 9,000 villages.</p>
<p>Digital Green is in the process of measuring the program’s impact on farmer livelihoods, and health status using evaluations in both India and Ethiopia. In India, Digital Green is also measuring the program’s effect on improving nutrition-related behaviors. The organization has also invested in an activity monitoring system that reports data on program implementation and tracks the adoption of Digital Green-promoted practices from remote locations. One challenge in the activity monitoring system is its reliance on data from partner organizations, which varies in quality. Recognizing the issue, Digital Green has instituted a series of data quality checks and procedures to improve quality. The Goldilocks Initiative’s recommendations for Digital Green focus on its agricultural activities, and include refining and consolidating the program’s theory of change and conducting a systematic review of data quality. (<i>Innovations for Poverty Action Lab, 2016</i>)</p>
<p><b>Myanmar Specific Example: MIMU 3W</b></p>
<p>In 2008, after Cyclone Nargis, the Myanmar Information Management Unit (MIMU) launched their 3W project to track and share humanitarian and development activities undertaken by all agencies across Myanmar.</p>
<p>3W stands for “Who does What, Where.” By keeping an up-to-date database on which organizations (who) are conducting certain activities (what) in Myanmar (where), other organizations and donors can better target beneficiaries to ensure humanitarian and development needs are met.</p>
<p>The partnered agencies share their information on planned, ongoing, and recently completed activities every six months using standard sector definitions and MIMU compiles these data. They clean and digitize the data, creating quality baseline data sets on humanitarian and development efforts in Myanmar and on responding agencies in the country. They then make the data publically available for other agencies to use. The intended impact is that agencies and donors can use these data to determine which areas, sectors, and populations to target in their future projects. (<a href="http://themimu.info/3w-maps-and-reports"><i>http://themimu.info/3w-maps-and-reports</i></a>)</p>
<p>After each example, take time for discussion with the participants regarding data life cycles. Some guiding discussion questions include: What was the problem the organization was trying to solve? How did the organization collect data? How did they plan to analyze data? How will they use data to make a decision? How can the organization share their data? How does the organization question their data quality? Can all the data be shared openly? What kinds of issues need to be considered if data are or are not shared?</p>
</li> <!-- end of step-->
<li>
<h1>Activity 3.1: Data Collection Telephone</h1>
<div class="time step-time">25 minutes</div>
<p class="download-box"><a href="http://tascha.github.io/D4D/documents/Activity_3.1.doc">Download Activity 3.1</a></p>
<p><b>Objectives:</b></p>
<ul>
<li>Apply the data lifecycle to a classroom activity</li>
<li>Understand each phase of the cycle and questions to consider throughout the cycle</li>
<li>Think creatively and critically about how to apply the data lifecycle to different situations</li>
</ul>
<p><b>Materials Needed:</b></p>
<ul>
<li>Paper</li>
<li>Pencils</li>
<li>Data management plan template (located in the Appendix of the Activity <a href="http://tascha.github.io/D4D/documents/Activity_3.1.doc">Download</a>)</li>
<li>Timer or stopwatch</li>
</ul>
<p><b>Introduction:</b> (Use the following information to introduce and explain the activity to the class)</p>
<p>This activity will help participants apply the data lifecycle by having them think through various stages of the lifecycle from unique, real world examples from other participants. Explain to the participants that this game builds upon a popular children’s game in the United States called “telephone”. In this game, a phrase is whispered from one individual to the next, and the individual is not allowed to ask for the phrase to be repeated. Then, the individual whispers this word to another individual, and so on and so forth until the phrase is passed on to all the individuals participating in the game.</p>
<p>If need be, feel free to play the game as an example to provide context for the students. Example phrases to use in the game could be anything, such as “dogs dig holes for big bones” or “beggars can’t be choosers”.</p>
<p>Keep in mind the phrase that you use should be short and simple in order to not make the game unnecessarily complicated. Start the chain yourself, and then have the participants pass the phrase along by whispering into each other’s ear one by one, until the phrase gets to the last participant. Have the last participant stand up and say the phrase. If you are lucky, the phrase has been kept intact throughout the chain. However, often the phrase will have been muddled and may turn into something completely different.</p>
<p>Introduce to the participants that the class is going to do a similar activity, but revolving around the data lifecycle. Break the class up into groups of 2-3. Pass out the data management plan template to each group. Once each group has one, explain that each team should begin by writing a question they want to answer with data in the provided space on the sheet. Then once everyone is finished, groups will pass their data management plan to a different group. Then, that group will have four minutes to think through one phase of the data lifecycle for the problem or question from the other team. Then, once time is up, the paper is passed to another team, until the problem has gone through the phases of collection, analysis, and sharing.</p>
<p>Once the phases are complete, return each group’s original data collection plan to the appropriate owners. Have the class look at the responses before leadings a discussion about what happened in the activity. Some guiding questions include: What was planned for data collection, analysis, and sharing? Is it different from what you would have done yourself? How has the telephone game enhanced your understanding of the importance of collaboration? What issues around data privacy and security should be considered if sharing data?</p>
<p class="summary">After activity, dismiss the class for a ten-minute break.</p>
</li> <!-- end of step-->
<li>
<h1>Understanding Data Collection Methods</h1>
<div class="time step-time">10 minutes</div>
<p>Transition with the participants by saying that now that they have a deeper understanding of data collection as a whole, the next part of the module will focus on data collection processes, protocols, and concepts. </p>
<p>Ask the class if they know what “primary” and “secondary” data are. Solicit answers from the class. Then, provide them with the following simplified definitions:</p>
<ul>
<li>Primary data: information collected by you or your team.</li>
<li>Secondary data: information that is collected by a third party.</li>
</ul>
<p>Take time to discuss with the class the benefits and drawbacks of each kind of data. Primary data are valuable in that you have control over the collection of the data and direct knowledge of how the data have been managed. But they are more costly, time-consuming, and certain expertise is needed for data collection design.</p>
<p>Take the time to walk the class through the different forms of primary research.</p>
<p>Then, state that secondary data are also valuable because they can improve your insights, help plan new data collection, or answer existing questions without having to spend time and money collecting data. Walk the class through different sources of secondary data, such as:</p>
<ul>
<li>Journals</li>
<li>Books</li>
<li>Newspapers</li>
<li>Records</li>
<li>Previous reports and analyses</li>
</ul>
<p>Ask the class for examples of primary and secondary data at their own organizations. If they collected primary data, what were the benefits of that? What were the disadvantages? What about the benefits and disadvantages of using secondary data for a project?</p>
<p>Underscore to the class that in many cases it is valuable to use a combination of both secondary and primary resources in making a decision with data.</p>
</li> <!-- end of step-->
<li>
<h1>Introduction to Key Concepts</h1>
<div class="time step-time">15 minutes</div>
<p>A key part of any data collection plan are data collection protocols. Ask the class what a protocol is. Once they have provided their own definitions, ask the class what they think a data collection protocol might be. Then, provide the following definitions: </p>
<ul>
<li>Protocols are systematic plans for how a set of operations are to be carried out
</li>
<li>Data protocols are systematic plan for how data are to be collected, stored, and described</li>
</ul>
<p>Data collection protocols can help save time and resources by specifying the format the data should be collected in, the types of data you want to generate, the instruments you will use to generate and collect data, how the data should be stored, and how it should be shared.</p>
<p>Introduce the concept of “metadata” to the participants. Solicit the participants for their own definition of metadata, then provide the following</p>
<ul>
<li>Metadata: information that describes, explains, or gives context for other data. They are provided to make it easier to interpret, use, and manage data.</li>
</ul>
<p>Metadata are important because they are used to add context to data. Metadata are the key for primary data to be used as secondary data. Two examples of metadata types are:</p>
<ul>
<li>Descriptive metadata (Such as who created the data, what was the data created for, where was the data collected, and when the data was collected)</li>
<li>Administrative metadata (Why these data were collected)</li>
</ul>
<p>Ask the group for examples of metadata.</p>
</li> <!-- end of step-->
<li>
<h1>Key Considerations in Designing a Data Collection Plan</h1>
<div class="time step-time">15 minutes</div>
<p>There are key questions that should be considered when designing any data management plan. Walk through the below questions with the class:</p>
<ul>
<li><p>What questions are trying to be answered?</p>
<p>The first step before collecting data is to clearly understand the problem. Write down a set of questions and potential ways they can be answered with new or existing data.</p></li>
<li><p>What do you need to know?</p>
<p>Often when we list the kinds of data we want it can become a wishlist of things that are all “interesting” but may not all be necessary. Make sure everything in your list of “need-to-know” data are necessary for answering the questions or problem you stated. Go back and forth between the data and problems to make sure both match. You may need to remove some data from your wish list if they do not support the problem statements. You may also need to revise your problem statements if the data you list do not support what is there but you feel those data are vital.</p></li>
<li><p>When to collect new data (primary), and when to use existing data (secondary)?</p>
<p>This can be answered by exploring existing data sources, writing down explicit questions to be asked and trying to answer those with existing data sources, etc.</p></li>
<li><p>What instruments will you need to create?</p>
<p>Collecting new primary data requires significant time developing new instruments, making sure the questions are phrased accurately to get the data you need, and testing them before administering them.</p></li>
<li><p>Who will be involved in data collection, and for how long?</p>
<p>Plan to have people involved that can contribute data collection and analysis stages.</p></li>
<li><p>What documentation will be needed to use the data again?</p>
<p>This should tell the history of the data. Who created it? When was it created? Why was it created? What information does it include?</p></li>
</ul>
<p>Project the steps of the data collection plan process on the screen or pass it around to the participants. State that this can help provide a helpful guide in the future when designing data collection protocols. Walk through the steps with the participants, taking time for discussion. If possible, have a student volunteer a problem they would like to see solved with data from the “telephone” activity. Then, walk through the steps with the class. Pause and answer any questions.</p>
</li> <!-- end of step-->
<li>
<h1>Providing Future Resources</h1>
<div class="time step-time">20 minutes</div>
<p>The following section of this module provides participants with sample resources surrounding data management and collection that they can return to in the future. If using a projector, take time to go to each website and click around, providing commentary with participants about the website’s purpose and how they can use the website in the future. If not, provide screenshots of each source that can be passed around to the participants as the instructor describes each data resource.</p>
<p>Sample resources:</p>
<ul>
<li>Data Management Plan tool: <a href="https://dmponline.dcc.ac.uk/">https://dmponline.dcc.ac.uk/</a></li>
<li>Following best practices in choosing a sample (size, diversity, relevant population, etc.) <a href="https://resolutionresearch.com/page/results-calculate/">https://resolutionresearch.com/page/results-calculate/</a></li>
<li>What are databases? How to design one? <a href="www.dartmouth.edu/~bknauff/dwebd/2004-02/DB-intro.pdf">www.dartmouth.edu/~bknauff/dwebd/2004-02/DB-intro.pdf</a></li>
<li>Creating a Google Form (see activity 3.2)</li>
</ul>
<br>
<br>
<br>
<br>
<br>
<br>
</li> <!-- end of step-->
<li>
<h1>Activity 3.2: Data Collection with Google Forms</h1>
<div class="time step-time">30 minutes</div>
<p class="download-box"><a href="http://tascha.github.io/D4D/documents/Activity_3.2.doc">Download Activity 3.2</a></p>
<p><b>Objectives:</b></p>
<ul>
<li>Create a survey using Google Forms</li>
<li>Understand how to design a survey, the types of questions that can be asked, and how to view responses</li>
<li>View response in Google Sheets, and use these responses for your own analysis </li>
</ul>
<p><b>Materials Needed:</b></p>
<ul>
<li>Shared Computer</li>
<li>Paper</li>
<li>Pencils</li>
<li>Google Account</li>
</ul>
<p><b>Introduction:</b> (Use the following information to introduce and explain the activity to the class)</p>
<p>Google Forms are free and openly accessible digital tools that can be used to design a survey. For our benefit, many of these tools work together – so we can use Forms to collect data, and we can use Sheets to view aggregated data.</p>
<p>In this example, we are going to design a simple survey for the sake of collecting data about people’s experience attending a movie. We want to understand what is typical of their experience.</p>
<p>Surveys are a valuable instrument for collecting data when doing evaluation, gathering information about an unknown subject, or generally understanding users behaviors. Some additional features that make for a good survey are as follows:</p>
<ul>
<li>The population is known, and can be separated (differentiated) in some meaningful way, such as by</li>
<ul>
<li>Age</li>
<li>Gender</li>
<li>Income level</li>
<li>Education level</li>
</ul>
<li>You have a small set of questions that you want each participant to supply</li>
<li>The questions you want answered are simple, and have straightforward answers</li>
<li>The responses might vary by the type of participant</li>
</ul>
<p>Survey questions can take a variety of forms – the most common types of questions that can be asked are:</p>
<ul>
<li>Multiple choice - participants choose from a set of examples</li>
<ul>
<li>When asking many multiple choice, or check-box questions, you can also use a “grid” format to ask many questions in a row. We will look at an example of this in our first exercise.</li>
</ul>
<li>Rank order - participants are asked to rank a set of options</li>
<li>Likert scale – a participant is able to offer a judgement based on a numerical value (e.g. 1 being the best, and 5 being the worst)</li>
<li>Open ended or Short answers – a participant is given the opportunity to, in their own words, respond with an explanation.</li>
</ul>
<p>To create a new google form</p>
<ul>
<li>Log into your Google Drive account</li>
<li>Go to <a href="https://docs.google.com/forms">https://docs.google.com/forms</a></li>
<li>We can either create a new form, or select an existing template</li>
</ul>
<p>For the first exercise, let’s select the “Exit Ticket” template</p>
<ul>
<li>Here we see an example of a form that allows participants to leave Short Answer responses about their experience in a class.</li>
<li>Notice that in designing the form you can click on an individual question</li>
<ul>
<li>This gives you the option to change the type of question that is being asked, edit the text of the question.</li>
<li>You can also select whether the question is “Required” meaning it has to be answered by each participant, delete, or duplicate the question. </li>
<li>To move questions around in order, simply drag the 6 dots at the top of the question</li>
</ul>
<p><img src="http://tascha.github.io/D4D/images/GoogleForm1.png" alt="Google Forms Screenshot" width="100%"></p>
<p><img src="http://tascha.github.io/D4D/images/GoogleForm2.png" alt="Google Forms Screenshot" width="100%"></p>
<li>After we have finished designing our form, we can also see what the responses will look like by clicking the ‘RESPONSES’ tab.</li>
<ul>
<li>The responses can be viewed by a Summary of all responses, or by each individual response</li>
</ul>
<p><img src="http://tascha.github.io/D4D/images/GoogleForm3.png" alt="Google Forms Screenshot" width="100%"></p>
<li>It will be hard to use this information within the Form itself. To create a spreadsheet of all answers we can click the green button with a cross (the spreadsheet button). This will give us the option to create a google Sheets form to view our answers. Click the Sheets button and create a new sheet with the responses.</li>
<p><img src="http://tascha.github.io/D4D/images/GoogleForm4.png" alt="Google Forms Screenshot" width="100%"></p>
<li>The Sheet that opens has each question as a Column, and each row as a response to our questions. So, we can now view (in aggregate) all responses to our questions.</li>
<p><img src="http://tascha.github.io/D4D/images/GoogleForm5.png" alt="Google Forms Screenshot" width="100%"></p>
</ul>
</li> <!-- end of step-->
<li>
<h1>Lying with Data</h1>
<div class="time step-time">25 minutes</div>
<p>We have noted before and want to again make the strong point that the same data can easily be manipulated and used to tell different, opposing, or inaccurate stories.</p>
<p>This can be intentional or unintentional. Not all bad or misleading storytelling is done with ill-intent.</p>
<p>Using data to mislead can be particularly challenging when there are powers in our work that may want to see certain things from data and directly or indirectly pressure us to find ways to say them.</p>
<p>It is important to always be thoughtful about how you and your peers are using data and be careful of political and other pressures that may lead to inappropriate use of data.</p>
<p>Find ways to respectfully talk with your colleagues and even supervisors if you feel data are being used incorrectly or inappropriately.</p>
<p>This training will not go into the many ways data can be used poorly. That is an entire course to itself. In fact, many universities now offer courses just on how you can spot data that are used to spread misleading information. However, here are a few of the many things to be careful of. Keep your data radar always active!</p>
<ul>
<li>Correlation vs causation</li>
<ul>
<li>A correlation describes a relationship between two or more variables. For example, both variables may increase or decrease at a similar rate, or one may decrease while the other increases. It does not, however, mean that one variable impacts the other or “causes” the other to change.</li>
<ul>
<li>If there is time, use the following <a href="http://www.tylervigen.com/spurious-correlations">website</a> to show how correlations cannot be used to say one thing causes another. The examples show how real data trends could be spuriously used to show causation. More cheese consumption does not lead to more PhDs in Engineering or more people dying from being tangled in their bedsheets. These are very obviously not true. But it is harder to detect with other your data that are of similar subject matter.</li>
</ul>
<figure>
<p><img src="http://tascha.github.io/D4D/images/mozzarellachart.png" alt="Chart Showing Mozzarella Cheese Consumption Correlation with Civil Engineering Doctorates Awarded" width="100%"></p>
<figcaption>Source: <a href="http://www.tylervigen.com/spurious-correlations">http://www.tylervigen.com/spurious-correlations</a></figcaption>
</figure>
<figure>
<p><img src="http://tascha.github.io/D4D/images/cheesechart.png" alt="Chart Showing Cheese Consumption Correlation with Death by Tangled Bedsheets" width="100%"></p>
<figcaption>Source: <a href="http://www.tylervigen.com/spurious-correlations">http://www.tylervigen.com/spurious-correlations</a></figcaption>
</figure>
<li>Causation shows that the change in one variable is the result of a change in the other. In other words, a change in one <b>causes</b> a change in the other.</li>
<li>Examples both show how real data trends could be used to show causation. More cheese consumption does not lead to more PhDs in Engineering or more people dying from being tangled in their bedsheets. These are very obviously not true. But it is harder to detect with other data that are of similar subject matter.</li>
</ul>
<li>Misleading visualizations</li>
<ul>
<li>There are many ways visualizations can be misleading. Always be skeptical of the visualizations you and others create, even if published in reputable journals, papers, magazines, or other sources. The slides display a few examples.</li>
<li>In the examples, the first graph doesn’t include 0. It gives the impression that German workers spend much more time working than workers in other countries. Once 0 is added, this doesn’t seem the case, does it? The differences are much less dramatic.</li>
</ul>
<figure>
<p><img src="http://tascha.github.io/D4D/images/GTAI_average_hours.gif" alt="Average Number of Hours Worked Barchart by Country" width="100%"></p>
<figcaption>Source: <a href="https://callingbullshit.org/tools/tools_misleading_axes.html">Calling Bullshit</a></figcaption>
</figure>
<figure>
<p><img src="http://tascha.github.io/D4D/images/GTAI_redrawn.png" alt="Average Number of Hours Worked Barchart by Country" width="100%"></p>
<figcaption>Source: <a href="https://callingbullshit.org/tools/tools_misleading_axes.html">Calling Bullshit</a></figcaption>
</figure>
<li>Using bad data</li>
<ul>
<li>Sometimes bad data are used to tell stories without clearly indicating the data should not be fully trusted. Know the limitations of your data.</li>
</ul>
<li>Selective storytelling</li>
<ul>
<li>One can leave out pieces of a dataset that do not fit the “story” they want to tell with the data. You may not know if others do it, but you certainly can know if you do. Always stay honest in your analyses and be open to other stories that arise in the data, even if they don’t support what you want to tell.</li>
<li>In this picture, FOX News did an analysis of unemployment rates under President Obama from 2009-2012, with the conclusion that Obama’s policies have led to a near doubling of unemployment. However, the methods they used to calculate the 2009 figure were different than the ones for the 2012 figure. The actual comparable figures should have shown a change from 14.2% to 14.7%. We should not assume the intent of using incorrect data was to lie, but the incorrect comparison did support a narrative the show was trying to tell and were clearly misleading.</li>
</ul>
<figure>
<p><img src="http://tascha.github.io/D4D/images/foxnewsgraphic.jpg" alt="Fox News Screen Grab" width="100%"></p>
<figcaption>Source: <a href="http://www.msnbc.com/rachel-maddow-show/lies-damned-lies-and-statistics-fox-0">MSNBC The Rachel Maddow Show</a></figcaption>
</figure>
<li>Key takeaways</li>
<ul>
<li>Summarize as seems most relevant to the group. Add expansion or other takeaways as needed</li>
<li>With regards to the FOX News slide, note that FOX did apologize by saying "We mixed up the numbers on Wednesday, so we wanted to clear things up.” However, they left it at that and didn’t explain how they mixed up the numbers or what the correct numbers were. Humility is very important and we should be prepared to admit our biases and mistakes, and correct any story we may get wrong.</li>
</ul>
</ul>
</li> <!-- end of step-->
<li>
<h1>Debrief</h1>
<div class="time step-time">5 minutes</div>
<p>Take time to answer any questions the participants may have. Then, announce to the participants that the next module will focus on analysis and data sharing. Then, dismiss the participants for lunch.</p>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
</li> <!-- end of step-->
</ul>
</section>
</article>
</div>
</body>
</html>