v0.10.1

🚀 Composer v0.10.1

Composer v0.10.1 is released! Install via pip:

pip install --upgrade mosaicml==0.10.1

New Features

1. 𐄷 Weight Standardization

   Weight Standardization reparametrizes convolutional weights such that the fan-in dimensions have zero mean and unit standard deviation. This could slightly improve performance at the expensive of 5% lower throughput. This has been used in several papers to train with smaller batch sizes, with normalization layers besides batch norm, and for transfer learning.

   Using Weight Standardization with the Composer Trainer:

   import composer
    
   # Apply Weight Standardization (when training is initialized)
   weight_std = composer.algorithms.WeightStandardization()
   
   # Train with Weight Standardization
   trainer = composer.trainer.Trainer(
       ...
       algorithms=[weight_std]
   )
   trainer.fit()

   Using Weight Standardization with the Composer functional interface:

   import composer
   from torchvision.models import resnet50
    
   my_model = resnet50()
    
   # Apply weight standardization to model
   my_model = composer.functional.weight_standardization(my_model)

   Please see the Weight Standardization Method Card for more details.

Bug Fixes

• Fix for checkpoints not being saved automatically at the end of a run (#1552)
• Fix Onnx export for Composer HuggingFaceModels (#1557)
• Fix for MIoU metric producing NaN's (#1558)
• CometML logger documentation updates and fixes (#1567, #1570, #1571)
• WandB image visualizer fix (#1591)

What's Changed

• Update evaluate_periodically() when eval interval is of type Duration by @karan6181 in #1523
• Quality of life updates to EMA by @coryMosaicML in #1524
• Add ADE20K and COCO v2 dataset behind a version flag by @karan6181 in #1528
• Pinned setuptools version to fix distutils version error by @karan6181 in #1536
• Less strict name formatting by @hanlint in #1535
• Defaulting streaming dataset version to 1 and add a deprecation warning by @karan6181 in #1532
• Changing 'stable' to 'latest' in notebooks in examples by @bcui19 in #1534
• Bump furo from 2022.6.21 to 2022.9.15 by @dependabot in #1540
• Bump fasteners from 0.17.3 to 0.18 by @dependabot in #1538
• Add Pandoc to Docker images, bump version to 2.19.2 by @bandish-shah in #1550
• Removed streaming version 2 from yaml since version 1 is default by @karan6181 in #1551
• Bump ipykernel from 6.15.2 to 6.15.3 by @dependabot in #1548
• Bump yamllint from 1.27.1 to 1.28.0 by @dependabot in #1546
• Bump traitlets from 5.3.0 to 5.4.0 by @dependabot in #1539
• Object Store Logger Race Condition + EMA Fix by @mvpatel2000 in #1552
• Adding in erroring for when using GradMonitor and DeepSpeed by @bcui19 in #1555
• Bump pypandoc from 1.8.1 to 1.9 by @dependabot in #1559
• Update context to raise errror by @mvpatel2000 in #1561
• Fix MIoU metric when self.total_union==0 by @abhi-mosaic in #1558
• Move dataloader initialize_object to factory methods by @hanlint in #1510
• Weight Standardization method by @Landanjs in #1562
• Update comet links to include query params and point to main site by @dakinggg in #1567
• remove dead line in alibi by @mvpatel2000 in #1568
• GLU Fixes by @mvpatel2000 in #1564
• Add FSDP strategy by @abhi-mosaic in #1553
• Comet example by @dakinggg in #1570
• Add missing _enabled flag, post_close, and clean up comet ml tests by @dakinggg in #1571
• Consistent Method Card Style by @growlix in #1407
• add missing return in context by @mvpatel2000 in #1574
• Remove eval batch split by @mvpatel2000 in #1576
• Fix Onnx Export for Composer HuggingFaceModels by @nik-mosaic in #1557
• Revert checkpoint rename by @hanlint in #1579

New Contributors

• @bcui19 made their first contribution in #1534

Full Changelog: v0.10.0...v0.10.1