diff --git a/Entry 0 -- March 27.md b/Entry 0 -- March 27.md new file mode 100644 index 0000000..adf14b8 --- /dev/null +++ b/Entry 0 -- March 27.md @@ -0,0 +1 @@ +This is the 0th notebook entry, stay tuned for more exciting entries in upcoming weeks. \ No newline at end of file diff --git a/Entry 1 -- April 3.md b/Entry 1 -- April 3.md index 67f8f0a..78afc8a 100755 --- a/Entry 1 -- April 3.md +++ b/Entry 1 -- April 3.md @@ -2,22 +2,29 @@ ## Last week's critique -**TODO:** Fill in this part with a summary and reflection on the critque you -received for last week's work. Answer questions such as: How, specifically, did -the feedback help improve the project? Did the feedback point out or offer -something you hadn't considered? Did it help you make a design decision? Was it -helpful in addressing the most pressing issues in your project? How will you -incorporate the feedback into your work? Will you change something about the -design, implementation, or evaluation as a result? +I receieved a lot of useful notes from the critique last week. For starters, I got some feedback on the scope of the project. Rebekah cautioned me about trying to do a lot of different things in the project and mentioned that I should wait to implement a database interface or a UI. This didn't change much for my overall plans because I already knew that I needed to prioritize the tasting notes language format in the DSL about tasting notes. However, the fact that she mentioned that made me realize that I had started thinking a lot about the UI and database interface in my head. The critique was the type of reminder that I needed to re-focus on the important parts of the project and direct all of my attention towards the language. + +A lot of the critique was focused on the specific language I will use to describe wines and the amount of freedom I plan to grant users of the DSL. I hadn't made any concrete decisions on linguistic user freedom at the time of the critique, so this gave me a lot of food for thought. She discussed limiting the language of the users in order to make the "wine cellar" database searchable. One suggestion that she had was to couple a limited set of searchable keywords with an optional expanded description. One example that she used was ("raspberries", "like in my grandfather's garden"). I think that this suggestion will really improve the project, since it's a good way to make the database more searchable while still giving the user the freedom of expression that they should have in tasting notes. + +I had to think a lot more about the syntax of the language this week, and the critique gave me a lot of places to start thinking. I wasn't thinking about how to make the database searchable. One thing I'm still wrestling with is how much I want to focus on making different queries to the database (i.e. searching by specific flavor tasting notes) versus just having the database as a basic storage mechanism (queries might have to be very specific for a particular wine instead of more general searches). This is something I would love to get feedback on and would impact how much time I spend trying to create a finite set of flavor/aroma keywords. + ## Description -**TODO:** Fill in this part with information about your work this week: -important design decisions, changes to previous decisions, open questions, -exciting milestones, preliminary results, etc. Feel free to include images -(e.g., a sketch of the design or a screenshot of a running program), links to -code, and any other resources that you think will help clearly convey your -design process. +I have made a substantial amount of design decisions for my project this week. All of these decisions were so substantial that I have not actually done any coding for the project yet. I plan to start this coding very early in the coming week, but there were a lot of things I needed to think about before I could actually start writing. For the record, I kept notes while working over the course of the week and am summarizing those notes and that progress in this notebook at the end of the week. That explains why this section of the notebook will read more as a weekly reflection, since I know everything I did this past week. + +My critique last week suggested thinking of a list of what I want users of my language to be able to express about a given wine. My goals for the week were to find common attributes among styles of wine tasting notes and to come up with a basic language for wine that took the best features from these formats. I focused on how to make these attributes as flexible as possible to appeal to a wide user base. I began the week by searching through many different tasting notes and extracting the common themes among them. [This Google Doc](https://docs.google.com/a/g.hmc.edu/document/d/1XCLj4SaLaCd-cl_7Xwdeo2VM-2kYo6yXD4Uw0Fq0sPw/edit?usp=sharing) is a summary of that research. It contains links to various articles on tasting notes as well as the relevant tasting attributes and descriptions mentioned in each of those articles. + +I wasn't especially surprised by the wine attributes that I found in the articles, since I have some amount of experience in what people look for when drinking a wine. I started to get an idea of the general attributes I wanted to represent with my language, but I still had the more thorny problem of how to encode individual flavors. I was debating whether to let users pick from a specific set of tastes and then add description. I could also let users write paragraphs about their tastes or aromas, but this would be impossible to compare to other note formats unless I wanted to get into NLP (I don't, and this will come up later). I currently have a primary solution that I plan to describe later, but I would love feedback on how feasible that representation will be. + +I looked into a lot of tasting note formats and compiled the examples into a different [Google Sheet](https://docs.google.com/a/g.hmc.edu/spreadsheets/d/1YixnUrSxKshJENVIG6wQ9mP44irSelkOzrnIm6kUH3s/edit?usp=sharing). I was curious how premade tasting notes sheets were set up, since I've rarely written wine notes for myself. I tried reviewing wines using some of the worksheets for tasting notes and found things that I liked and didn't like about each of the formats. Sometimes I felt too constrained and sometimes I felt like I did not have enough guidance or inspiration for the notes. This motivated me to settle for a structure somewhere in the middle of that spectrum, something that would allow users to write general impressions for categories like Color/Appearance and Overall Impression. However, I decided to break Aroma/Flavor down into subcategories so that users can identify individual flavors and write notes on them but not output a frustrating (programmatically) block of text. + +At this point in the week, I knew I had to start thinking more about the technical end of the problem. I had a list of wine APIs and was very aware that I needed to find one where the tasting notes would somewhat match up to my preliminary format. Cue instant problems. No matter how much I searched, the most helpful tasting notes I could find for wine searching APIs consisted of a block of text in a review as well as a numerical score. Thus, if I wanted to compare a personal tasting note in my language to something more professional, I would only be able to match the two overall scores and then access a paragraph of a wine review. + +As I mentioned before, I think that it would be possible to do some NLP on the block of text in a review to extract the desired features. However, most of the reviews are not long enough to extract a lot of information ([vintank's API](http://developer.cruvee.com/search-api/sample-wine-response-in-json/) provided the most helpful reviews in the description of a response). It would be hard to get a good comparison between personal tasting notes and professional opinions. Also, I would prefer to focus on different technical goals for the project than learning about NLP. I am more technically interested in the language design as well as the prospect of building a visual tool with this DSL. + +At this point, I had a decision to make. It seemed like my initial idea of comparing personal and professional notes was not very feasible. Note that this revalation occurred fairly late in the week, so I'm still wrestling with a few of these decisions. I think I'll continue a lot of this in the Questions section, so I'm going to give a brief overview of my decisions here. At this point, I want to turn my DSL into more of a personal visualization tool for wine tasting notes. I could still keep the "wine cellar" aspect of the project and let users write their own wine tasting notes. A program in the language would be a set of tasting notes, and the user would be able to create visualizations from these tasting notes. I have a sketch of a sample program [here](https://goo.gl/photos/Kqqp9qn346wKMWQR8). For example, a user could list flavors that they detect in the wine and the strength of each of those flavors. This information could get compiled into a pie chart describing the taste of the wine. Users could even compare different wines in their own personal cellar by this method, something that would be helped by the visualization. I have a lot of questions for the project at this point, so I'll take it into the next section. Overall, I came up with a tasting notes format this week that I think can be used in a lot of different ways. Since I have this format and I'm happy with the user flexibility, I feel as though I have accomplished some good goals in the past week. + ## Questions @@ -25,10 +32,19 @@ design process. you need to make, what implementation issue are you trying to solve, or how are you evaluating your design and implementation?** +I think it's pretty obvious, but the most pressing issue for my project is what I want to do with the tasting notes. Would the visualization of a wine based on the notes be a good application? I think it would help the user see their opinions on a wine in a different format. Would it be all right if the users can't compare their opinions with something more professional? Again, I think that I want to keep the primary structure of users writing tasting notes that get put in their own personal "wine cellar." If the notes in the cellar could be compared, would this also be appealing to users? I need to make sure that this design decision would be enough of a DSL and still feasible given the time constraints of the project. Again, I plan to start coding for the project at the very beginning of the upcoming week. + **What questions do you have for your critique partners? How can they best help you?** +I have two primary areas in which I want feedback. First is the format of my tasting notes. I linked to an image above which is a sketch of my note format, but [here](https://goo.gl/photos/Kqqp9qn346wKMWQR8) is the image again. A sample program is written on the left side of the paper. I think that, to start, a program would just be a file where the different tasting notes are separated by key headers. My questions are: is this a good format for tasting notes? I don't expect specific feedback on the intricacies of wine tasting, but is it a good idea to structure the notes this way? Should I keep the fields that have numbers? Should I add more numbers (the ability to score each attribute) and determine the overall score of the wine at the end? Currently, I'm letting users arbitrarily assign a score at the end but this could be changed. I also structured aroma/flavor attributes in the format of (attribute, score, modifier), where the attribute is a single arbitrary word to name the flavor/aroma, the score is the strength of the attribute, and the modifier is an optional word such as "aroma" "finish" or "taste" to specify where that attribute was detected. I might also expand this to include Rebekah's suggestion of more specific attributes, so an optional "expanded definition" where the user could get more specific on the nature of the flavor. Would this be feasible for the program? + +The second thing I want feeback for is all of the questions in the previous section. I'm worried that I don't have a completely concrete project direction at this point. I'm happy to do the visualization project, but would this be enough of a DSL? Do you think it would be useful for users? Would it be beneficial to let users compare multiple different wine visualizations? If I do this, when should I generate the graphs and how should I store wines into the database? By that, I mean should I make a tasting note and focus on immediately generating a visualization, or should I store it into a database and generate a visualization after I access it in the database? Actually, I probably should just focus on things that don't deal with databases and think about how to make a visualization from a single program. Would this be a good direction for the project? If so I think my next steps are: write the tasting notes format, generate a visualization, then add the ability to generate multiple visualizations. Are there alternative next steps or directions I could do with that project? + +One final question I have: if I end up doing the visualizations, any suggestions for a language I could use? I know Scala has limited tools for it, but if I wanted to make the visualizations more complex, I would think about using another language in conjunction with Scala. Suggestions? I've done d3/SVG for web visualizations before, but I think I'd be looking for something else. I haven't looked into this at all yet, so I'm also going to do research on this in the very near future. + + **How much time did you spend on the project this week? If you're working in a team, how did you share the labor?** - +Not including the time I spent at a wine bar doing research, I spent around 10 hours on the project this week. Most of it was spent compiling tasting notes formats and refining my own tasting notes design. The rest of the week was spent frantically searching through APIs, getting disappointed that none had well-formatted tasting notes, and trying to think of new directions for my project. diff --git a/Entry 2 -- April 10.md b/Entry 2 -- April 10.md index c3ed055..bd7509e 100755 --- a/Entry 2 -- April 10.md +++ b/Entry 2 -- April 10.md @@ -2,33 +2,46 @@ ## Last week's critique -**TODO:** Fill in this part with a summary and reflection on the critque you -received for last week's work. Answer questions such as: How, specifically, did -the feedback help improve the project? Did the feedback point out or offer -something you hadn't considered? Did it help you make a design decision? Was it -helpful in addressing the most pressing issues in your project? How will you -incorporate the feedback into your work? Will you change something about the -design, implementation, or evaluation as a result? +The critique that I received last week gave some good recommendations for storing tasting notes into a database. Daniel recommended that I create a database with future features in mind so as not to make it too closed. I think that's a good idea, since I could easily see my keywords changing as small things about the tasting notes format are altered in implementation. Unfortunately, I've moved away from a lot of the database aspects of my project and more towards visualizations. However, if I have time at the end it might be worth implementing more of a data storage mechanism than "store all tasting notes in a directory." I'm glad that he brought this up in the critique because I had mainly forgotten about that aspect of my project. + +Daniel also talked about the ability to compare tasting notes with professionals. He said that it might be a good idea to link to professional opinions on a wine, even if they're not in the correct format. I agree with this and would like to keep that as a potential direction for the proejct. Even though I'm mostly focused on the visualization of tasting notes, there's no reason why I shouldn't also be able to include links to more information about the wine or pull a basic description of the wine from a database. It would be interesting to try and integrate this functionality later in the project. + +Several more things that Daniel mentioned in his critique were comparing reviews with friends and altering the visualization format. I really like the idea of comparing reviews with friends, especially if a group goes out wine tasting. It's an interesting way to see what other people thought of the wine beyond just discussion of flavors. I'll probably talk about this below, but it seems like a great reach goal for the project, though I would be more likely to focus on it if I were building an actual app. Daniel talked about creating a customizable GUI where users could set the colors of various wine attributes, which got me thinking about users interacting with the visualizations. I could let users adjust colors and things like that, but I might try to let users select different types of charts for wine attributes. It's interesting to think about users comparing wines with their friends and then being able to edit their tasting notes or save their friends' tasting notes. Again, some of these would be more of a consideration for an app, but it's interesting to think about. + +Finally, Daniel talked about assigning an overall score to a wine and how it might make more sense for the score to be a product of different wine attributes. I can see arguments for and against this, the latter being that it might be more difficult for users to score individual attributes of wines in my tasting notes format. Again, this is something to keep thinking about as I implement the tasting notes in the coming weeks. + ## Description -**TODO:** Fill in this part with information about your work this week: -important design decisions, changes to previous decisions, open questions, -exciting milestones, preliminary results, etc. Feel free to include images -(e.g., a sketch of the design or a screenshot of a running program), links to -code, and any other resources that you think will help clearly convey your -design process. +This week, I determined the exact nature of my project and made some significant technical progress on it. I came out of last week uncertain about the best way to implement my project. I brought up the idea of representing user tasting notes with visualzations. I had a good idea for how the tasting notes would be formatted but did not have many concrete plans for how I would implement the graphic side of the project. + +At the beginning of the week, I talked to Prof Ben and my critique partners about where to go with the project. I came into class knowing that I wanted to try and visualize tasting notes. First, I tried to find visualization languages that would allow me to represent the information I wanted. I had used the javascript library [d3](https://d3js.org/) in the past and was comfortable trying to write a DSL that would interact with it. I also found other DSLs such as [Raw](http://raw.densitydesign.org/) and [Timeline](http://timeline.knightlab.com/) that did similar things to my proposed project. They both allowed user-friendly input formats for visualization of data. Raw is built on d3, which gave me hope that my project would be similar to it once completed, though with the more specific domain of wine. + +In general, my goal for a DSL is something that takes wine tasting notes in a text file and transforms them into graphical visualizations. I was worried that it would be closer to a web app than a DSL, but I talked about some of the technical background with Prof Ben. He suggested parsing the text files into JSON format and then visualizing that in d3. I decided to go with that suggestion. Another place I could add more linguistic functionality to the app is through different types of visualizations. I might be able to expand the language to support other tables or graphs, or possibly expand it to process different parts of the wine tasting notes. One thing that my critique partners suggested was side-by-side wine comparison with others. This is definitely outside the scope of my project, but I think it would eventually be fun to write an app that lets people take their own tasting notes and then compare their resulting visualizations with the people around them. It leaves room for wine as both an individual and a social experience. + +Once I knew how my project would be structured, I started getting into the technical details. As Prof Ben suggested in class, I began to look at it from both the user and final output ends. I knew that I wanted the users to write their initial tasting notes in text files (for ease in this project) and use d3 to output the files as graphics. In between, the text files would get parsed to JSON and passed into the d3 javascript. I had to figure out how to structure the project in order to convert between all of these different data formats. + +I quickly set up a basic d3 program using html and javascript. I used a [radial sunburst basic example](http://bl.ocks.org/mbostock/4348373) from mbostock, the creator of d3. This example is used to graph a [coffee flavor wheel](https://www.jasondavies.com/coffee-wheel/) and I thought it might be a good starting point for my wine visualizations. At this point, I was just trying to test if I could display anything. I got this visualization working and saw that all of the radial sunburst data being used to create the specific wheel pieces and layers was loaded from a JSON file. I then spent a while messing with the data and changing various parts of the visualization. It occurred to me that I could change the JSON file being loaded and get visualizations of different tasting notes. I spend far too long looking into application building platforms like [Meteor](https://www.meteor.com/) and [Node.js](https://nodejs.org/en/) because I knew I would be dealing with a lot of different files. I was also using Python to run a simple server from the command line in order to display the visualizations in Chrome. After wasting a lot of time, I realized that it would be simpler to get a small part of the project working and not spend so long on file structure. + +My primary concern at this point was to build a sample program demonstrating the data flow from user to visualization. I decided to write most of the project in Python because it provides very easy tools for reading text files. I was already serving content on a Python server, so it made sense to build this functionality into my project file. On Prof Ben's recommendation, I picked that as my big language and started hooking the pieces of my project together. It took a lot of Googling, but eventually I wrote a Python program that could be run from the command line by typing `python run-server.py syrah.txt`. The text file argument in that command represents the tasting notes file to be visualized. This is then read by Python, converted into JSON, and placed in a file named `wine.json`. The `wine.json` file can then be accessed by the local server, since it lives in the same directory as the Python program. + +My current code starts the Python server and then opens Chrome to the url `http://localhost:8080/?wine=wineFileName` where `wineFileName` is the name of the JSON file. In the future, I will save the JSON file by the same name as the text file, which should make the URL less confusing. Since Chrome is being opened to that URL on the local server, it runs the `index.html` file in the same directory. This is where all of my Javascript code is stored to take the name of the wine JSON file from the URL, add the `.json` extension, and visualize it in d3. Finally, the visualization corresponding to the initial text file is output in Chrome. + +At this point, my main focus for next week is the txt to JSON parser. The current text file I'm passing as an argument is JSON in text format purely for the proof of concept. This means that my Python program reads the JSON string from the text file, converts it from a string to JSON, and dumps it into the JSON file. I have the software stack built in addition to the tasting notes format from the previous week so it's time to combine the two in this parser. Ideally, I'll be able to parse the tasting notes and visualize in this sunburst shape by the end of next week. In the fourth week, I'll be able to experiment with other visualization formats and expanded functionality for the parts of the tasting notes that can be represented graphically. As I mentioned in my critique, I would also like to be able to link to professional opinions about a given wine, so I might go back to some of the APIs I researched in the first week and look into how they could be integrated. + ## Questions -**What is the most pressing issue for your project? What design decision do -you need to make, what implementation issue are you trying to solve, or how -are you evaluating your design and implementation?** +As described above, my next big goals for the project are to work on the text file to JSON parser. I think that this is a big linguistic decision for me, especially since the grammar of the tasting notes will determine how flexible the format can be. I'm trying to decide which parts of the tasting notes can be optional, how to deal with files that don't contain some of the mandatory tasting notes attributes (error messages), and how I want to visualize different aspects of the tasting notes. + +At this point, I'll be happy with the basic structure of the project once I get the parser working. I think I'll still have some important implementation decisions to make for the visualization side of the process. If I choose to represent all attributes of the tasting notes in a single visualization, I need to determine a standard format. I also might make it so that users can customize their visualization (what parts of the tasting notes are displayed, the format in which they are shown, etc) which could get into the territory of another mini DSL. That's something I have to keep in the back of my mind as the project progresses, but it's a fun direction to consider. + +At this point my biggest question is whether my software stack seems efficient. To display the JSON file on my local Python server, I pass the file name by URL (in Python) and then (in Javascript) load the corresponding JSON file in the directory to display. This works when I test it and I'm happy to keep using it for the remainder of the project, but it seems a bit convoluted. Are there any tools that you know about which would make this process easier? If possible, I would love something that doesn't necessitate passing parameters via URL. -**What questions do you have for your critique partners? How can they best help -you?** +I intend to keep asking this question every week, but are there any other helpful visualizations for my tasting notes format? So far, I like the sunburst and a potential bar graph for charting the various flavors that people taste in a glass of wine. The rough sketch of my tasting notes format from last week is [here](https://goo.gl/photos/Kqqp9qn346wKMWQR8). Can you think of any way I could display information about those specific categories that would be appealing to users? Anything that users might want to customize? **How much time did you spend on the project this week? If you're working in a team, how did you share the labor?** +Including the time spent in class this week and the time to do this notebook entry, I spend about 10 hours on the project. I intend to spend a bit more before class on Monday so that I'll have an interesting example program to demonstrate. diff --git a/Entry 3 -- April 17.md b/Entry 3 -- April 17.md index bafd527..153f312 100755 --- a/Entry 3 -- April 17.md +++ b/Entry 3 -- April 17.md @@ -2,22 +2,30 @@ ## Last week's critique -**TODO:** Fill in this part with a summary and reflection on the critque you -received for last week's work. Answer questions such as: How, specifically, did -the feedback help improve the project? Did the feedback point out or offer -something you hadn't considered? Did it help you make a design decision? Was it -helpful in addressing the most pressing issues in your project? How will you -incorporate the feedback into your work? Will you change something about the -design, implementation, or evaluation as a result? +The advice in last week's critique was very encouraging to me. I keep losing sight of my project direction and considering too many things. Rebekah helped me commit to a few concrete decisions that were well within the scope of my project. She recommended that I continue with visualizations and said that my current software stack was a positive contribution to the project. This was the encouragement that I needed to do more in-depth thinking about visualizations as opposed to broader thinking about all the different directions I could take the project. + +Some of the specific suggestions Rebekah gave for visualizations were particularly interesting to me. She liked the idea of integrating the opinions of a professional taster, especially since that was more in line with my original project goals. I still don't have a firm idea of how I will do that, but I plan to consider the problem in future weeks. I want to incorporate the ability to compare notes in some way, be it with a professional or with other sets of notes. As per Rebekah's suggestion, I plan to (at minimum) add a side-by-side tasting notes comparison feature. + +The critique mentioned that I should not invest a great deal of time in error-checking at the expense of other features. I agree that it might be time-intensive to try and develop specific messages for things like obscure parsing errors. Instead, I'll probably structure the language so that many of the fields are optional and the user has the flexibility to customize their tasting notes. I also got a recommendation for the Python json library to aid in the conversion between text and JSON. I used this for the demo on Monday (reading in a JSON-formatted text file) and plan to continue using it as I do more work on my project. + +Finally, the critique made me think more about the types of visualizations I want to produce. I got some good suggestions for depicting a wine glass filled with flavors. Rebekah suggested that I separate the flavors by the point at which they are detected, which is a fun idea I was already considering. It was good to have someone else mention the idea and shows that this style of visualization may appeal to different users. She briefly mentioned that I could include various types of visualizations. I want to discuss this more in my entry for this week, but I think this is a good idea that can be scaled up or down depending on how much time I have to spare. ## Description -**TODO:** Fill in this part with information about your work this week: -important design decisions, changes to previous decisions, open questions, -exciting milestones, preliminary results, etc. Feel free to include images -(e.g., a sketch of the design or a screenshot of a running program), links to -code, and any other resources that you think will help clearly convey your -design process. +As is becoming very common from week to week, I had a mini-identity crisis about the exact nature of my project. Usually, these crises are themed around the concern that I am creating more of an application and not a DSL. This week was no different. I started out with an demonstration that could take a text file organized into a pseudo-JSON format and display it in a sunburst d3 graph in a browser. I didn't receive any particularly negative reviews of my URL-based file name parameter passing method (as mentioned in last week's notebook), so I decided not to spend time thinking of other solutions. My intentions for the week were to create a parser that could take tasting notes in a particular text format and convert them to this JSON format for display. As I considered this approach to the project, I became more and more convinced that the parser would have little substance. I worried about the language design aspects of the project would only involve filling holes in a JSON template. + +On Monday, I shared my concerns with my critique group and Prof Ben. After a bit of discussion, I was advised to consider the visualization end of the project as a place to do more language design work. I started looking into differences between chart formats in d3, since I thought that I could let users adjust various parameters in charts. That way they could customize the chart types to their specifications, with my wine visualizations being used for a proof of concept. I found [Datamaps](http://datamaps.github.io/), which has this specific functionality for map visualizations in d3. If I decided to go that way with my project, I intended to implement this for several different chart styles. However, I was not sold. I decided it would be a lot of work at the cost of everything I had already done on the project. I wanted to stay true to my original project and didn't want to start something fairly different with a vague scope. + +Eventually, I came back to my original intentions for the week. I wanted to write a parser to convert the text tasting notes format into JSON for visualizations. After thinking about the problem for a while, I decided that my ideal tasting notes format would have enough complexity that I could develop a grammar and write my own parser. My earlier hesitations revolved around the fear that my tasting notes format would not be syntactically complex enough to warrant a parser. I also knew that I was not the first person to want to convert between text and JSON in Python, so there would probably be more efficient tools for this conversion than writing my own parser. However, I looked into the problem a bit more and found that the text to JSON tools would still require a parser for my tasting notes in order to build the correct form of dictionary for JSON conversion. + +With this in mind, I set out to write a parser for my tasting notes format. The code for this lives in `jsonParser.py` and takes in the text from a specifically formatted file to output the appropriate JSON for a d3 visualization. I began by sketching out an abstract syntax tree in `AST.py`. I don't use this file anywhere in the actual code stack, but it helped me determine the order in which I wanted to parse the tasting notes to create the JSON. I then wrote methods for each node in the AST, though my initial approach to the problem was incorrect. I thought that I could output a string with the appearance of a JSON object that could then be converted to actual JSON with the Python `json` library. However, the library takes in text data in the form of a Python dictionary. I had forgotten this about the `json` library and didn't think to check before implementing the parser. After a lot of print statements, I tweaked a lot of the code by building up the end result in a dictionary instead of a string. Thankfully, these changes resulted in a functional parser. I re-learned the annoying lesson to double check documentation in order to save time in implementation, which will be helpful in future coding projects. + +At this point, I was mainly finished with my work for the week but decided to do a little bit of work on the other end of the project. I knew that I could parse a very basic tasting notes program and get a visualization, but the visualization needed a lot of work. I hadn't made any modifications to the sunburst and wanted to create a visual output that would include all parts of the tasting notes as well as separate charts for the flavor and profile of a wine. After some experimentation with d3, I was able to visualize all parts of a user's tasting notes, something I intend to demonstrate to my critique partners on Monday. I intend to discuss my project goals for the coming week in the next section of the journal. + + +- Mention user-defined types of visualizations +- Comparing with friends/pro wine people +- Custom GUI (users adjust colors and things like that, but I might try to let users select different types of charts for wine attributes) ## Questions @@ -25,10 +33,22 @@ design process. you need to make, what implementation issue are you trying to solve, or how are you evaluating your design and implementation?** +In the coming week, I have a lot of things I can improve on the project as well as many additional features I could implement. The two main areas I want to explore are the comparison of multiple wines and more visualization options. I have consistently received feedback that the project would be more interesting and novel with a review comparison feature. I do not know whether this would best be implemented in a comparison against other wines reviewed in my format. It would be a very reasonable goal to let users input two text files as arguments for side-by-side comparison. The idea of bringing in professional tasting notes or reviews is a bit more daunting but could potentially add a great deal to the project. I would have to review some of the APIs I researched earlier in the semester but could potentially find some wine database housing professional tasting notes. Even if these weren't formatted in a particular manner, it could be interesting to include them in a visualization. I could make that a user-defined feature accessible via an optional command line argument. + +As I mentioned, I also want to explore additional visualization options. This is a more open-ended problem, but I like the idea that users could specify the format in which they could view the notes. It would be convenient for someone who prefers pie charts over wine glass visualizations to see their tasting notes in a friendly format. + +In general, I'm trying to decide what features are within the scope of my project. I want to focus on parts of the project that I can reasonably implement before the end of the year. This could be a fun project to extend (and there's certainly a lot of directions I could take it) but I want it to have working features at the end of the project term. The most important issue for the project is: given what I have at this moment, what can I resonably accomplish by the end of the semester? I want to refine the parser and some of the primary visualization code, but I think I can add several more features to the current project core. I would appreciate advice on which features would be the most desired and the most feasible to implement. + **What questions do you have for your critique partners? How can they best help you?** +As I mentioned above, I would like to know how to structure the rest of my work time on the project. I am satisfied that my project is currently working from end-to-end. At this point, I want to know how to make it better. I also have questions about whether my current system seems efficient. Could I organize the project structure in a different way? Should the files be stored somewhere else? Is it all right to call the parser in this way, and is it fine to have all of that parsing logic in the same file? Should I create a better AST? + +Like every other week, what visualizations should I make? I'm going to make the wine glass and some pie/direction charts, but hopefully I can add the option for users to declare their preferred viewing format. If this feature gets implemented, I want to have several appealing chart options for users. Any suggestions? + **How much time did you spend on the project this week? If you're working in a team, how did you share the labor?** +Including time in class and time writing this entry, I spent approximately 9 hours on the project this week. + diff --git a/Entry 4 -- April 24.md b/Entry 4 -- April 24.md index 72425b0..354badf 100755 --- a/Entry 4 -- April 24.md +++ b/Entry 4 -- April 24.md @@ -2,22 +2,30 @@ ## Last week's critique -**TODO:** Fill in this part with a summary and reflection on the critque you -received for last week's work. Answer questions such as: How, specifically, did -the feedback help improve the project? Did the feedback point out or offer -something you hadn't considered? Did it help you make a design decision? Was it -helpful in addressing the most pressing issues in your project? How will you -incorporate the feedback into your work? Will you change something about the -design, implementation, or evaluation as a result? +My critique from last week was generally positive. I got a lot of good feedback on writing the tasting notes parser, especially in the context of language design. Daniel recommended I include more error messages in my parser, since the tasting notes need to be in a very specific format. Specific error checking would help the user of my DSL to understand how they could alter their input to be accepted by the parser. I'm not sure whether I want to focus on this with the time left in the project, though it seems feasible. It certainly makes sense to me. One alternate solution to this could be to write a very quick GUI with predefined fields which would automatically control the format for user input. + +I also got the feedback that I should try to find a way to pass the JSON object directly to my Javascript visualization code. I think that there are ways to do this, primarily by passing data through my server connection. I know that I could use something like [Flask](http://stackoverflow.com/questions/33788341/passing-data-to-d3-after-preprocessing-in-a-flask-app) but that wasn't really my primary focus in the project. It's definitely a more efficient solution for the future, but I'm happy that I have something working for this project. + +The last bit of feedback I received was a suggestion for more visualizations. Daniel said that he thought it would be interesting to view flavors in the order that they were tasted, which got me thinking about how some flavors could be represented in a type of graph. Each line could correspond to a different flavor and the axes would be time versus intensity. I'm not sure how feasible this is in the current time scope, but it could be an interesting visualization to extend the project (see Description section for more comments on that). ## Description -**TODO:** Fill in this part with information about your work this week: -important design decisions, changes to previous decisions, open questions, -exciting milestones, preliminary results, etc. Feel free to include images -(e.g., a sketch of the design or a screenshot of a running program), links to -code, and any other resources that you think will help clearly convey your -design process. +This week I learned a lot more about d3 visualizations and put a lot of thought into how my project could be extensible for future developers. Since I had the entire stack of the project working, my goal was to produce the visualization of one wine, including a display of all text elements and graphical elements. I experimented with pie charts for flavor display. + +While trying to get this working, I was also talking with my critique partners and Prof Ben about how I could present a somewhat completed project. Prof Ben mentioned that I could think about ways to design both the language and the visualization code to make it accessible for developers. I struggled for a while about how this could best be done. There are a lot of ways to tackle this problem. + +One approach would be to alter the parser to process different formats of data objects in the tasting notes. Instead of including checks for a particular attribute name ("Flavor", "Aroma", etc), I would include AttrList data types or TextDefault data types. The parser already has most of this functionality but the user would be able to define the elements in the tasting notes corresponding to that format. The default format of each tasting note attribute would be a text block and attributes could be classified differently based on user arguments or more specification in the tasting notes text. + +I also considered expanding the parser based on specifically named attributes and limiting the attributes that could be visualized beyond showing a block of text. However, I wanted to make something more extensible and thought it could be done most efficiently by declaring general types (lists, text boxes, etc) for the parsing language and letting users incorporate their own visualizations. For the purposes of the project and the scope of the demonstration, I decided to focus specifically on lists of attributes and notes in clumps of text. + +If I have time before my project presentation, it seems like the most efficient way to determine the behavior of each tasting notes attribute is to include a config file or dictionary. It would store the names of standard attributes parsed differently than the default. For example, I could write tasting notes with a "Flavor" attribute and the dictionary would say that "Flavor" is parsed as a list by default. That would affect the final JSON object and the visualizations which could work with that attribute. + +This dictionary feature would also give users the option to override the default options. For example, if the default type for "Flavor" was a list of attributes but I wanted to parse and visualize "Flavor" as a block of text, I could override this from the command line by entering somethign like `python run-server.py -v "Flavor" Text` where the arguments after the `-v` flag are the attribute to override and the type to which it is converted in the parser. + +On the visualization side of the project, I tried to think about ways that developers could easily add new visualization behavior. I think that by providing some default types in the language, users could predict the JSON format of each attribute and write visualizations based on that expected data. Currently, it is easy to add new visualizations to the system. Each tasting notes attribute is extracted from the JSON object and given as input to a d3 visualization method. The user has to specify the visualization function corresponding to each attribute, though the default is to [append text to the SVG element](https://www.dashingd3js.com/svg-text-element). Ideally, the attributes would all have default visualizations depending on their type, but this works for the time being. If a developer wanted to write their own visualizations, they would need to add a Javascript file using d3 to the visualizations directory of the project and then specify the attribute they want to visualize in the `index.html` file. + +Most of my time this week has gone into learning about d3 and thinking of ways to make my system more extensible and accessible to developers. I will present my DSL to the class on Wednesday. At the very least, I will be able to show them a text file containing wine notes and then produce a d3 visualization of those notes. Depending on what I can get working by then, I could have a more extensible and user-friendly system. + ## Questions @@ -25,10 +33,9 @@ design process. you need to make, what implementation issue are you trying to solve, or how are you evaluating your design and implementation?** -**What questions do you have for your critique partners? How can they best help -you?** +I don't think I have a lot of issues with the project, I just have way too many things I would like to implement and not enough time. I initially chose a project with a reasonable scope but it got twisted into a much larger project once I met a few roadblocks along the development process. My biggest decisions for the project still revolve around how to make it as user- and developer-friendly as possible. I have been struggling to resolve these two audiences while also considering the language design components of the project (and the class). I want to focus on a nice user experience that doesn't require precise tasting note formatting. I also want users to be able to customize their visualizations, compare different wines, and see tasting notes for a given wine. A lot of these features are not related to language design and would take a while to implement. In the context of this class, I have prioritized the developer experience because it involves the most significant language design questions. Though this is not my preference and the project has not turned out like I initially envisioned, I think that it involves a lot of interesting issues and unique challenges. My goal for the end of the project is to present a coherent demo to the class and share the lessons that I have learned about making a DSL for both developers and users. **How much time did you spend on the project this week? If you're working in a team, how did you share the labor?** - +I spent upwards of 10 hours on the project this week, including time in class and writing this notebook entry.