El-Emperador_Model Recipe

Welcome to the LeEmpereur_70-Base project! This repository contains the implementation recipe and fine-tuning scripts for the LeEmpereur_70-Base model.

•The Jupyter notebook contains the necessary libraries and steps to create a model using the ORPO Trainer.

Training Charts

Evaluation Charts:

Reference Articles:

1. https://www.databricks.com/blog/efficient-fine-tuning-lora-guide-llms

2. https://mlabonne.github.io/blog/posts/2024-04-19_Fine_tune_Llama_3_with_ORPO.html