Summify

Unlocking knowledge from audio and visual content: Revolutionizing information extraction and synthesis.

Inspiration 📜

Through the first-hand pain of a student sitting through long audio and video content for things like school assignments, podcast analysis, making meeting minutes for recordings of long online meetings, and other such time-consuming tasks, we got our inspiration to innovate a solution through this project. Automating the process is essential as students have limited time, so every minute is crucial to be efficient. At the same time, we realized that this went far beyond us. It would also be beneficial to 432 million adults and 34 million children around the globe with hearing loss by having a tool to get visual information extracted from audio resources, in a quick manner. By unlocking knowledge from audio and visual content, our goal is to revolutionize information extraction and synthesis.

What it does 💻

• In today's fast-paced world, whilst information is abundant, time is limited. Students and professionals often find themselves sifting through lengthy audio and video content like podcasts, interviews, and online meetings to extract key ideas and topics. This is a very time-consuming process which is also inefficient.

• Summify is also beneficial for differently abled individuals with impaired hearing as our solution caters to them by concisely extracting key information from audio/video samples and presenting it to them in an easy-to-understand manner visually.

• Our tech solution addresses this issue head-on. Summify aims to help users effectively work with audio and video content through automated summarization, key points identification, topic extraction, and sentimental analysis. Summify’s audio summarization feature supports various formats such as mp3, wav, amr, aac, ac3, aiff, flac, ogg, opus, wma, and m4a. Summify’s video summarization feature supports mp4.

How we built it ⚙️

• Since our team consisted mostly of beginners in web development, we decided to use pure JavaScript/HTML/CSS for our project so that we could all work on the solution.

• Summify was built using various software found through our own research that include:

  • For Front-End: HTML and CSS
  • For Back-End: JavaScript, Symbl.ai API, and Conversation Intelligence
  • For Website Design Mockup: Figma

• We also made use of online documentation available on our chosen API’s website that guided us toward building the various components of Summify.

Challenges we ran into 🤺

• In the beginning stages of building Summify, we were having trouble fetching information from API due to our inexperience in API development and found everything quite complicated
• We needed to work on resolving authentication issues related to API configuration
• Due to this, while we were still integrating it into our website, we created a backup plan with whisperAI click here to check it out! in python.
• We had issues with the API slowing down as we developed due to which, we had to add a loader. Getting the job status and integrating the loader took a while.

Accomplishments that we're proud of 🏆

• We were able to resolve all the challenges we faced by working for long hours and constantly looking for ways to improve our project to present Summify: a completely finished, fully functional and user-friendly tech innovation
• We successfully responded to a real user need and automated our solution to this problem through Summify
• We were able to configure the API, fetch information from it, and successfully set authentication even when we faced some serious issues -Our innovation caters to differently-abled individuals who have impaired hearing problems as well as students and professionals which makes Summify an inclusive solution
• We were able to set up the backend as well as frontend and create a fully functional website within the hackathon time frame of only 36 hours
• We worked our way through all the different types of error codes and resolved every issue
• We were able to enjoy the entire process even when we faced multiple challenges
• We made friends and had a blast working together

What we learnt 📝

• Through the workshop hosted by Postman which was on “Introduction to APIs”, we were able to deepen our understanding of and knowledge on APIs and how Postman facilitates the information retrieval process
• Through the “Introduction to Web Development, React, and DevOps” workshop, we further reinforced our knowledge of the fundamentals of making a website and app as well as how they intersect with businesses
• Through our own research and trial-and-error process, we were able to learn how to set up the authentication end of the API we used and connect it to our website. We learned about how to integrate bearer tokens, API keys and conversation IDs to be able to manipulate the API to get and push requests according to Summify’s various use cases
• We learned how to collaborate with completely new individuals and how to manage our time to finish our project a few hours before the deadline

What's next for Summify 💡

• Our next steps for Summify include implementing language detection from any audio and video content which can help the user learn about which language their content is in to better help them find resources to translate it if needed.
• Plus, we want to include the ability to have the output in various languages so it suits the user better with the languages they are fluent in
• We also want to give our users the ability to do any file uploads directly on Summify
• In the future, we want to enable direct audio or video input right from redirected sources like Spotify, YouTube, and more.

Built With 🦾

HTML, CSS, JavaScript, Conversation Intelligence, Artificial Intelligence, Figma, Symbl.ai and API