Releases: neuralmagic/transformers
Releases · neuralmagic/transformers
NM Transformers 1.7
What's Changed
- GitHub Actions fixed to enable manual runs.
- Mistral model architecture was updated to enable quantization of the activations.
NM Transformers 1.6
fix reporting and manual run issues (#97) (#98) Co-authored-by: dhuang <dhuang@MacBook-Pro-2.local>
NM Transformers v1.5.1
What's Changed
Contributors
natuan, bfineran, and dhuangnm
Assets 3
NM Transformers Latest
nightly Refactor onto transformers v4.23.1
NM Transformers v1.4
Upgrade to transformers release V4.23.1 (#62) * Update trainer and model flows to accommodate sparseml Disable FP16 on QAT start (#12) * Override LRScheduler when using LRModifiers * Disable FP16 on QAT start * keep wrapped scaler object for training after disabling Using QATMatMul in DistilBERT model class (#41) Removed double quantization of output of context layer. (#45) Fix DataParallel validation forward signatures (#47) * Fix: DataParallel validation forward signatures * Update: generalize forward_fn selection Best model after epoch (#46) fix sclaer check for non fp16 mode in trainer (#38) Mobilebert QAT (#55) * Remove duplicate quantization of vocabulary. enable a QATWrapper for non-parameterized matmuls in BERT self attention (#9) * Utils and auxillary changes update Zoo stub loading for SparseZoo 1.1 refactor (#54) add flag to signal NM integration is active (#32) Add recipe_name to file names * Fix errors introduced in manual cherry-pick upgrade Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
NM Transformers v1.3
Upgrade to transformers release V4.23.1 (#62) * Update trainer and model flows to accommodate sparseml Disable FP16 on QAT start (#12) * Override LRScheduler when using LRModifiers * Disable FP16 on QAT start * keep wrapped scaler object for training after disabling Using QATMatMul in DistilBERT model class (#41) Removed double quantization of output of context layer. (#45) Fix DataParallel validation forward signatures (#47) * Fix: DataParallel validation forward signatures * Update: generalize forward_fn selection Best model after epoch (#46) fix sclaer check for non fp16 mode in trainer (#38) Mobilebert QAT (#55) * Remove duplicate quantization of vocabulary. enable a QATWrapper for non-parameterized matmuls in BERT self attention (#9) * Utils and auxillary changes update Zoo stub loading for SparseZoo 1.1 refactor (#54) add flag to signal NM integration is active (#32) Add recipe_name to file names * Fix errors introduced in manual cherry-pick upgrade Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
v1.5
What's Changed
- Question answering trainer with SparseML integration by @natuan in #1
- Use existing config for teacher model by @natuan in #3
- Export ONNX models with named inputs/outputs, samples by @natuan in #4
- Load and save SparseML QAT recipes by @natuan in #2
- Fix input/output shapes for exported samples by @natuan in #5
- Refactor SparseML trainer for multiple tasks by @natuan in #7
- enable a QATWrapper for non-parameterized matmuls in BERT self attention by @bfineran in #9
- Override LRScheduler when using LRModifiers by @bfineran in #11
- Disable FP16 on QAT start by @bfineran in #12
- Default behavior for sparseml w/o recipes by @natuan in #13
- Simplify saved recipes by @natuan in #17
- Integration of BERT QQP + MNLI(Aka GLUE Tasks) by @spacemanidol in #6
- Integrate SparseML with Masked LM training by @natuan in #16
- Use distillation modifier from SparseML for QA task by @natuan in #15
- Update setup.py by @rahul-tuli in #20
- updated fix: initialize is_qat_recipe in preprocess_state_dict by @bfineran in #18
- rm datasets/huggingface-hub versions by @rahul-tuli in #23
- SparseML integ for NER by @natuan in #21
- support to download and unwrap framework files from SparseZoo stubs by @bfineran in #19
- Add preprocessing num workers for GLUE by @natuan in #26
- changing qa to ner by @spacemanidol in #28
- Recipe args for NER, GLUE, QA and Masked LM by @natuan in #30
- Fix incorrect steps calculation when gradient acc. by @eldarkurtic in #31
- add flag to signal NM integration is active by @bfineran in #32
- update modifiers import for latest sparseml refactors by @bfineran in #34
- Update 4.7.0 -> 4.18.0 by @KSGulin in #37
- fix sclaer check for non fp16 mode in trainer by @bfineran in #38
- Add log_frequency arg by @KSGulin in #33
- Using QATMatMul in DistilBERT model class by @natuan in #41
- Fix distilbert scaling by @anmarques in #43
- Revert "Fix distilbert scaling (#43)" by @bfineran in #44
- Removed double quantization of output of context layer. by @anmarques in #45
- Fix DataParallel validation forward signatures by @KSGulin in #47
- Best model after epoch by @natuan in #46
- update Zoo stub loading for SparseZoo 1.1 refactor by @bfineran in #54
- Mobilebert QAT by @anmarques in #55
New Contributors
- @spacemanidol made their first contribution in #6
- @eldarkurtic made their first contribution in #31
- @anmarques made their first contribution in #43
Full Changelog: v1.4...v1.5
Contributors
natuan, spacemanidol, and 5 other contributors
Assets 4
NM Transformers v1.2
Mobilebert QAT (#55) * Remove duplicate quantization of vocabulary. * Remove duplicate quantization of vocabulary.
NM Transformers v1.1
Best model after epoch (#46)
NM Transformers v1.0
Best model after epoch (#46)