RAG News

Overview

RAG News is a Python application that leverages the Groq LLM API and retrieval augmented generation (RAG) to run interactive QA sessions with news articles. The application allows users to add new articles to a database and query the database for information using natural language.

Features

• Interactive QA Sessions: Ask questions about the news articles in the database and get responses generated by the Groq LLM.
• Article Management: Add new articles to the database using URLs.
• Text Summarization: Summarize the content of news articles.
• Translation: Translate non-English articles into English.
• Keyword Extraction: Extract keywords from text to facilitate article retrieval.

Installation

1. Clone the repository:

   git clone https://github.com/finnless/ragnews.git
   cd ragnews

2. Install the required dependencies:

   pip install -r requirements.txt

3. Install metahtml:

   cd metahtml
   
   git init
   git remote add origin https://github.com/mikeizbicki/metahtml
   git config core.sparseCheckout true
   echo '/*' > .git/info/sparse-checkout
   echo '!/tests' >> .git/info/sparse-checkout
   git pull origin master
   
   pip install .
   
   cd ..
   

Usage

Running Interactive QA Sessions

To run an interactive QA session, simply run the script without additional parameters:

python ragnews.py

Adding New Articles

To add a new article to the database, use the --add parameter:

python ragnews.py --add "https://www.nytimes.com/2024/09/16/us/politics/trump-violence-assassination-attempt.html"

Querying the Database

To query the database for specific information, use the --query parameter:

python ragnews.py --query "Who is the current democratic presidential nominee?"

Environment Variables

The application uses environment variables for configuration. Ensure you have a .env file in the root directory with the following variables:

GROQ_API_KEY=your_groq_api_key