Release PeerQA dataset on Hugging Face

[NielsRogge]

Hi @timbmg 🤗

I'm Niels and work as part of the open-source team at Hugging Face. I discovered your work on Arxiv and was wondering whether you would like to submit it to hf.co/papers to improve its discoverability.If you are one of the authors, you can submit it at https://huggingface.co/papers/submit.

The paper page lets people discuss about your paper and lets them find artifacts about it (your dataset for instance),
you can also claim the paper as yours which will show up on your public profile at HF.

Would you like to host the datasets you've released on https://huggingface.co/datasets?
I see you're using tudatalib.ulb.tu-darmstadt.de for it. Hosting on Hugging Face will give you more visibility/enable better discoverability, and will also allow people to do:

from datasets import load_dataset

dataset = load_dataset("your-hf-org-or-username/your-dataset")

If you're down, leaving a guide here: https://huggingface.co/docs/datasets/loading.
We also support Webdataset, useful for image/video datasets: https://huggingface.co/docs/datasets/en/loading#webdataset.

Besides that, there's the dataset viewer which allows people to quickly explore the first few rows of the data in the browser.

After uploaded, we can also link the datasets to the paper page (read here) so people can discover your work.

Let me know if you're interested/need any guidance.

Kind regards,

Niels

[timbmg]

Thank you for checking out the code and data @NielsRogge! 🤗

I agree, making PeerQA available on Huggingface would be amazing! The dataset uses papers from OpenReview that have to be downloaded and processed with GROBID 8.0. Since they do not have a clear permissible license, we can not provide them directly. Do you know a convenient way to make this available in the huggingface datasets library? I saw that there are some spaces that host GROBID, so I am wondering do you think it would be possible to create a space like this and process the PDFs?

[NielsRogge]

Pinging @davanstrien here as he might have some ideas