Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
iamarunbrahma authored Jan 24, 2025
1 parent fac4a75 commit ddc0daa
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -211,9 +211,9 @@ Note: For more details on custom model configuration i.e. `openai_config`, `gemi

## 📊 Benchmarks

I conducted benchmarking to evaluate Vision Parse's performance against MarkItDown and Nougat. The benchmarking was conducted using a curated dataset of 100 diverse machine learning papers from arXiv and used Marker library to generate the ground truth markdown formatted data.
I conducted benchmarking to evaluate Vision Parse's performance against MarkItDown and Nougat. The benchmarking was conducted using a curated dataset of 100 diverse machine learning papers from arXiv and used the Marker library to generate the ground truth markdown formatted data.

Since, there are no other ground truth data available for this task, I relied on Marker library to generate the ground truth markdown formatted data.
Since there are no other ground truth data available for this task, I relied on Marker library to generate the ground truth markdown formatted data.

### Results

Expand All @@ -223,7 +223,7 @@ Since, there are no other ground truth data available for this task, I relied on
| MarkItDown | 67% |
| Nougat | 79% |

Note: I used gpt-4o model for Vision Parse to extract markdown content from the pdf documents. I have used model parameter settings as in `scoring.py` script. The above results may vary depending upon the model you choose for Vision Parse and the model parameter settings.
Note: I used gpt-4o model for Vision Parse to extract markdown content from the pdf documents. I have used model parameter settings as in `scoring.py` script. The above results may vary depending on the model you choose for Vision Parse and the model parameter settings.

### Run Your Own Benchmarks

Expand Down

0 comments on commit ddc0daa

Please sign in to comment.