Team R&R (Role and Responsibility)
Project Kickup Meeting Minutes
This is an python supported web application using the Flask backend. Running following command to install all the dependency packages
pip install -r requirements.txt
Then simply run the app.py to render the server. Replacing {AUTH_TOKEN} with the authorization token from NeuralSpace. Currently for the functional demo usage, you can igonore the {AUTH_TOKEN}.
python app.py --auth_token {AUTH_TOKEN}
It can take some time when you fistly launch the system.
(This is a feature that is still under discussion, ideas and Pipeline codes are from Edulang )
.
├── app.py
├── data
│ ├── testing_set
│ │ └── demo.txt
│ └── validation_set
│ ├── arabic_text
│ ├── english_text
│ ├── WikiQAArticlesCorpusAr-En.tsv
│ └── WikiQAQuestionsCorpusAr-En.tsv
├── logic_graphviz.gv
├── logic.svg
├── README.md
├── requirements.txt
├── static
│ ├── img
│ │ ├── get_score_page.png
│ │ ├── LJL Official Logo Image.png
│ │ ├── LJL Official Logo Text.png
│ │ ├── neuralspace.png
│ │ ├── project_preview.png
│ │ ├── puzzLingLogo.png
│ │ └── puzzling_UI.png
│ ├── js
│ │ ├── easytimer.min.js
│ │ ├── package.json
│ │ ├── puzzling.js
│ │ └── timer.js
│ └── styles
│ └── style.css
├── team_roster.md
├── templates
│ ├── index.html
│ └── score.html
├── tools
│ └── get_data.py
9 directories, 27 files
PuzzLing combines the full system for the CALL(Computer Assisted Language Learning) platform especially focusing on low-resource languages, which includes language scoring and feedback functions. With the support from the latest language processing toolkit of the Neural Space, we aim to give a general evaluation and retrieve the error places that the testers can improve.
Our related using area can be automatic language scoring, evaluation, derivation— L2 teaching, Education of English as Foreign Language, etc.
For our target testers, their mother tongue (L1s) varies from many kinds of low resource languages such as Pashto, Arabic and Hindi, etc. And we intend to expand the scale of our current testing languages continuously in the future. To the best of our knowledge, our platform can be the first proposed non-native English scoring system for people who are speaking low-resource languages.
As we know, although the English language scoring system has made a promising breakthrough these years. Also, the Duolingo or Tofel already had their automatic language scoring system released, still, there is no total system that can efficiently work on especially focusing on low-resource languages (as far as the best we know). And this will be a big burden for those new-learners to learn English.
In the future, we have a strong belief that our Platform for those non-native English speakers will grow significantly. Meanwhile, we can help those low-source language speakers for our humanity.
Our system working flows from defining our problem to the conclusion and future works to compare the tester's textual input with pre-trained neural network model pipeline output (with the Support from the Neural Space API).
After the tester's testing data input into our scoring platform we build, and get the original text data passed through the Neural Space translation API pipeline, we can get the transcripted English result. We calculate the similarities score between those two results.
By comparing the difference between those two data and applied those weighting tactics, we can directly know how the tester's English language level.
------ test ------
+--------------------------------+--------+-------+--------+-----+-----------+
| | Somali | Xhosa | Pashto | Twi | Ukrainian |
+--------------------------------+--------+-------+--------+-----+-----------+
| unsup-simcse-bert-base-uncased | | | 95.74 | | |
+--------------------------------+--------+-------+--------+-----+-----------+
| unsup-simcse-bert-large-uncased| | | 96.17 | | |
+--------------------------------+--------+-------+--------+-----+-----------+
| unsup-simcse-roberta-base | | | 95.53 | | |
+--------------------------------+--------+-------+--------+-----+-----------+
| unsup-simcse-roberta-large | | | 95.61 | | |
+--------------------------------+--------+-------+--------+-----+-----------+
| sup-simcse-bert-base-uncased | | | 95.61 | | |
+--------------------------------+--------+-------+--------+-----+-----------+
| sup-simcse-bert-large-uncased | | | 96.92 | | |
+--------------------------------+--------+-------+--------+-----+-----------+
| sup-simcse-roberta-base | | | 96.23 | | |
+--------------------------------+--------+-------+--------+-----+-----------+
| sup-simcse-roberta-large | | | 96.33 | | |
+--------------------------------+--------+-------+--------+-----+-----------+
Results will be added soon!
We want to thank Princeton NLP group and NeuralSpace for their amazing works that makes this project possible.