This code was the code of the winning team for the most impactfull award in the Merantix GenAI hackathon October 12th - 13th 2023.
Our teams application (TBD) - aims at making technical health records more accessible to everybody. The app lets you take photos of your documents and get a simplified description back. If you are not familiar with the output language you can translate the letter to your preferred language. Not only that, but you can interact with the documents by asking questions about the content of the letter.
here a small demo of what the app can do: link
The application was powered by firebase, with a flutter front end and google cloud functions in the backend.
The google cloud functions used in the backend are in the /gcp_functions folder.There are three main backend endpoints.
ocr- image to text. We used the google cloud's vision ocr library for this.gen_ai- Generating the simplified prognosis. We used google's Palm 2 for text with some additional prompt engineering.translation- translating text. We used google cloud's text translation for this.
The frontend code for the flutter app can be found in this repo