Skip to content

Latest commit

 

History

History
100 lines (74 loc) · 2.79 KB

README.md

File metadata and controls

100 lines (74 loc) · 2.79 KB

Transformer Text Summarization

Final project of Statistical Learning Course.

Status

CodeFactor

Students

Here is our's information:

ID Name Contact
20120057 Trần Ngọc Đô Linkedin
20120024 Huỳnh Minh Tuấn Linkedin

Overview

This project is a small application of auto-text summarization using Transformer architecture. The codebase includes 2 parts

Techstack

1. Backend

Languages:

  • Python

Frameworks/Libraries:

  • Django - Build RESTful API
  • SpaceOCR - Process OCR Task
  • Pytorch - Build model and inference
  • HuggingFace - Using for Transformer architecture

2. Frontend

Languages:

  • Javascript
  • Typescript
  • CSS and HTML (markup language)

Frameworks/Libraries:

  • VueJS - Web structure
  • ElementPlus - UI supports
  • Axios - API communicating
  • Vite - Optimization

Getting started

# setup backend
$user cd server
$user python manage.py runserver <host>:<port>

# setup frontend
$user cd web
$user yarn dev --port <web_port>

Training

Hyper-parameters that were used to train the model:

Parameter Value
No. Epoch 3
Learning rate 1e-5 (First two epochs), 5e-6 (Last epoch)
Optimizer AdamW
Layers Full

The pre-trained model and weight are now available: here. Use this code snippet for inference:

from transformers import T5ForConditionalGeneration, AutoTokenizer

model =  T5ForConditionalGeneration.from_pretrained('ndtran/t5-small_cnn-daily-mail')
model.eval()
tokenizer = AutoTokenizer.from_pretrained('t5-small')
        
generated_ids = model.generate(
      tokenizer('summarize: ' + input('Enter your input'), return_tensors = 'pt').input_ids, 
      do_sample = True, 
      max_length = 256,
      top_k = 1, 
      temperature = 0.8
)
  
output = tokenizer.decode(generated_ids.squeeze(), skip_special_tokens = True)

Web capture

Or use this instance app: Huggingface Space

Issues

Feel free to open any issues.