🤖 Crypto News Assistant

An intelligent crypto news analysis system powered by RAG (Retrieval Augmented Generation) that automatically scrapes, processes, and analyzes cryptocurrency news to provide real-time insights and answers to your questions! 🚀

✨ Features

• 🔄 Real-time News Scraping: Automatically scrapes crypto news from TradingView
• 🧠 Intelligent Analysis: Uses advanced RAG architecture to provide context-aware responses
• 💡 Flexible Model Support: Works with both OpenAI and local models
• 🔍 Hybrid Search: Combines vector and keyword search for better results
• 🎯 Smart Reranking: Uses Cohere for intelligent result reranking

🛠️ Installation

1. Clone the repository:

git clone https://github.com/marsidmali/Crypto-News-Assistant.git
cd Crypto-News-Assistant

2. Create and activate a virtual environment:

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

pip install -r requirements.txt

4. Set up environment variables:
   • Creat config.json (see Configuration example)
   • Add your API keys and configuration

🤖 Model Setup

Using OpenAI

1. Get your API key from OpenAI
2. Add it to your config.json:

{
    "model_type": "openai",
    "openai_api_key": "your-key-here",
    "openai_model": "gpt-4o-mini"
}

Using Ollama (Local Models)

1. Install Ollama from ollama.ai
2. Pull your preferred model:

ollama pull deepseek-r1:32b

3. Update your config.json:

{
    "model_type": "ollama",
    "ollama_model": "deepseek-r1:32b"
}

Cohere API Setup

1. Create an account at Cohere
2. Get your API key and add it to your config.json:

{
    "cohere_api_key": "your-cohere-key-here"
}

🚀 Usage

1. Start the application:

streamlit run src/app.py

2. Wait for the initial news scraping and indexing
3. Start asking questions about crypto news and trends!

🏗️ RAG Architecture

The app uses the following RAG (Retrieval Augmented Generation) architecture:

1. Data Ingestion:

   • Scrapes crypto news from TradingView
   • Processes and chunks articles for optimal retrieval

2. Hybrid Retrieval:

   • Vector Search (FAISS) with MiniLM embeddings
   • Keyword Search (BM25)
   • Weighted ensemble combination

3. Smart Reranking:

   • Cohere Rerank for context-aware result selection
   • Top-N filtering for most relevant context

4. Response Generation:

   • Context-aware prompt engineering
   • Flexible model support (OpenAI/Ollama)
   • Streaming response generation

📁 Project Structure

crypto-news-assistant/
│
├── src/                 # Source code
│   ├── app.py          # Streamlit application
│   ├── scraper.py      # TradingView scraping logic
│   └── rag.py          # RAG system implementation
│   
├── config.json         # Configuration file
├── requirements.txt    # Dependencies
└── README.md          # Documentation

⚙️ Configuration

Example config.json:

{
    "model_type": "openai or ollama"
    "openai_api_key": "",
    "openai_model": "gpt-4o-mini",
    "ollama_model": "deepseek-r1:32b",
    "cohere_api_key": ""
}

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.