This project aims to develop an automated sentiment analysis system for bank branches using data mining techniques. Our goal is to collect customer feedback to gain insights into their sentiments regarding banking services. The results of this sentiment analysis will provide valuable information to bank branches, allowing them to identify potential issues, enhance their customer service, and strengthen their competitive advantage. To achieve this, we rely on tools and technologies such as Apify, Airflow, BERT, PostgreSQL, and Power BI to collect, transform, store, and visualize data.
The main objectives of this project are as follows:
- Collect data from various sources.
- Apply advanced sentiment analysis using BERT.
- Store the results in a PostgreSQL database.
- Visualize the data using Power BI.
The project's architecture includes the following components:
- Apify: Used for data extraction from online sources.
- Airflow: Manages the scheduling and execution of data processing tasks.
- BERT: A natural language processing (NLP) model for sentiment analysis.
- PostgreSQL: The relational database where data is stored.
- Power BI: Used for data visualization.
The project structure is as follows:
extraction.py
: Script to extract data from online sources.transformation.py
: Script to clean and transform data.loading.py
: Script to load data into PostgreSQL.airflow_dag.py
: The Python file defining the Airflow DAG (Directed Acyclic Graph) for task scheduling.
- Ensure you have the required Python dependencies installed (see
requirements.txt
). - Configure the PostgreSQL connection parameters in
loading.py
. - Execute the Airflow DAG using
airflow_dag.py
to automate the process.
The results of the sentiment analysis are stored in PostgreSQL. You can visualize this data using Power BI to gain valuable insights into customer sentiments towards bank branches.
Here are some examples of visualizations that we have created:
Feel free to reach out to us if you have any questions or suggestions:
Chaimae BOUYARMANE
✨ Leverage sentiment analysis to enhance customer experience, address issues, and stand out from the competition! ✨