Week 5: ML with Python

[BekahHW]

Getting Started

The goal this week is to get to "Training the Model" lesson from freeCodeCamp's ML with python and check-in here.

Learning Resources

• https://towardsdatascience.com/recurrent-neural-networks-by-example-in-python-ffd204f99470
• https://www.analyticsvidhya.com/blog/2019/01/fundamentals-deep-learning-recurrent-neural-networks-scratch-python/
• RNN with Keras Colab: https://colab.research.google.com/github/tensorflow/docs/blob/snapshot-keras/site/en/guide/keras/rnn.ipynb
• https://machinelearningmastery.com/understanding-simple-recurrent-neural-networks-in-keras/
• https://www.simplilearn.com/tutorials/deep-learning-tutorial/rnn

Check-in

1. What have you learned about Recurrent neural networks since last week? What have you learning about creating and training a model?
2. What are some more use cases for RNNs?
3. What do you know today that you didn't when you started?
4. What are some key terms or ideas from these lessons?
5. What do you need to be clarified or what questions do you have?
6. What are you excited to learn more about?

[BekahHW]

1. Idk if I have a good answer for this, but I have some ideas based on all the things we're learning and that's exciting. As a community builder, I listen to lots of community podcasts. And I'd love to figure out how to process the transcripts of the episodes, and be able to identify commonalities and main takeaways. I know with DG I can create summarization and topics, but I'd like to see overlapping takeaways. It might be similar to the Romeo and Juliet example but at a larger scale.
2. Financial transactions, predictions
4. stateful, recurrent_initializer
5. I really enjoyed this resource: https://www.simplilearn.com/tutorials/deep-learning-tutorial/rnn
6. Honestly, a good overview of the models we can use for NLP would be super. I need to do more exploration of Tensorflow hub.

[paceaux]

1. what have I learned since last week? The size of your data and epochs matter somehow.
2. Use-cases for RNN. Text predictions. Language detection. Language generation
3. What do I know now that I didn't before? Size of data + epochs + temperature really affects predictions
4. Key terms? Checkpoints? load weights? Temperature?
5. What do I need clarified? I need to go over the batch predictions again. absolutely none of that sunk in.
6. What am I excited to learn more about? HOW THE FRIGGIN' MODELS LOOK/ WORK. I want to see the model, in all its glory.

Also
I decided to train it on a few pages of Les Misérables, with a low number of epochs.

with a two-word prompt (dès les) I got a glorious bit of absolute nonsense, but it looked very French.

When I trained on the entire book of Les Mis, and made it 20 epochs, it was able to make a sensible prediction that was almost completely readable.

But what I'm REALLY fascinated by is how, with a small set of data, it was able to still have a model that knew what French looked like. I'm more interested in determining what the base-level "look/sound" of a language is, and so a poorly-trained model is actually really interesting in this case.