Feat/select testset

[AJDERS]

Select a test according to the following restrictions:

1. Quantity: The total amount of test data in hours will be 15 hours, divided as follows: 7.5 hours of read aloud and 7.5 hours of conversational data.

2. Dialects: The speakers must be distributed so that each region is represented at least 15%.

3. Gender: Each gender must represent at least 45% of the total hours for both types of data.

4. Age: 20% of each age group, i.e. <25, >25,<50 and <50.

5. Accent: At least 10% of the total hours for both types of data must be represented by individuals with accents.

[AJDERS]

I ended up handpicking a testset. This selection of speakers (almost) satisfies the above mentioned criteria:

The shortcomings of this selection is the lack of fynske-dialects and bornholmsk, but besides this the dialect distribution is quite good. Once we've made recordings at Fyn and Bornholm the testset should be finished. The dialect and dialect group distribution can be seen here:

Once the database structure on the NAS has been implemented and the anonymization scheme has been decided we can create a script for uploading to huggingface, but we'll do that in a seperate PR.

[sorenmulli]

Looks like reasonable criteria and a very interesting graphs with nice distributions!

The accent category simply means all speakers for whom Danish is not first language with noticable accents?

And as I understand it, the test set division is made in blocks of speakers, that is, a speaker cannot both have some audio in test and train, right? We test generalization to new speakers? That makes sense to me.

[sorenmulli]

Additional thoughts after talking with Martin:

• The shown graphs use audio time proportion as X axis as I understand it. We should of course also remember to consider distribution of unique speakers, e.g. the current ~18 minutes of audio from Fyn is from one speaker, right? :)
• Similar criteria as you had for region would be nice to have for dialect in future versions.
• 15 hours (and more with Fyn+Bornholm additions) is a rather large test set for ASR - is it chosen b/c it is 10% of data?

[saattrupdan]

15 hours (and more with Fyn+Bornholm additions) is a rather large test set for ASR - is it chosen b/c it is 10% of data?

The large size is due to us wanting to both have a large diversity in the test set (different dialects, genders, ages and so on) and at the same time not have any speaker overlap between the train and test splits (each speaker has around 20-30 min speech).

[saattrupdan]

@AJDERS Can we get this merged in and get the test set up asap?

[AJDERS]

I have found a few inconsistencies in how the student-workers has input languages, which i am going to open an issue with and fix upstream.

[saattrupdan]

LGTM!