This repository contains a Jupyter Notebook designed to run seamlessly on Google Colab, along with a docker-compose.yml file to set up and deploy Open WebUI for easy interaction with the model.
Please note that the maximum parameter size that can be run on Google Colab's free tier is 14B. Of course, you could run a larger model, but it will not operate seamlessly.