Usability improvements and initial remote project support

This is the fifth feature release of the Skyline CLI. This release adds initial support for remote projects and introduces several usability improvements. It needs to be used with v0.5.0+ of the Skyline plugin.

As of this release, Skyline has been promoted from alpha to beta. The Skyline project now lives under the @skylineprof organization. Skyline also now has a website, which you can visit at https://skylineprof.github.io.

New Features

• Remote projects are now supported for advanced users. Read more about remote projects on the Skyline website.

Enhancements

• The providers have been simplified. Models returned by the skyline_model_provider() no longer need to produce a scalar tensor (i.e. Skyline no longer needs to call .backward() directly on the model's output).
• When an error occurs during profiling, Skyline will now display the file context (file name and line number) associated with the error when possible.

Bug Fixes

• Skyline has been updated to work with PyTorch 1.4.0 and 1.5.1

Interactive visualizations linked to batch size predictions

This is the fourth feature release of the Skyline CLI. This release adds support for interactive visualizations and needs to be used with v0.4.0+ of the Skyline Atom Plugin.

When profiling models interactively, Skyline will now generate predictive models for the model's training throughput and memory usage as a function of the batch size. Dragging the throughput and/or memory visualizations will result in mutations to the code, which indicate the predicted batch size that should be used to achieve the manipulated visualizations.

New Features

• Interactive key performance metric visualizations
• Batch size predictions for memory usage and training throughput

Interactive breakdown exploration and inline annotations

This is the third feature release of the Skyline CLI. This release changes the way the memory and iteration run time breakdowns are displayed and adds inline performance annotations to the code. This version of the CLI must be used with v0.3.0+ of the Skyline plugin.

The breakdowns are now displayed in a hierarchical way, based on the modules in the model. Double clicking on an entry will "zoom in" on the entry and show a breakdown of the operations within that module.

Skyline now also shows performance annotations inline. A marker will appear in the gutter for lines of code for which we have performance information. Hovering over these markers will show the contribution of the line of code to the overall memory footprint and iteration run time.

Memory and iteration run time profiling fixes

This release contains and bug fixes associated with both memory and iteration run time profiling.

Bug fixes

• Fix leftover function name changes from refactoring (9538245)
• Fixed an issue that caused extra memory to be tracked (0bb9403)
• Avoid unnecessarily cloning tensors during backward pass profiling (1a40832)
• Avoid overwriting report output files if they exist (e6ed70e)
• Exclude unrelated double underscore functions from being tracked (15cd27c)

Iteration run time profiling

This is the second feature release of the Skyline CLI. Skyline now performs iteration run time profiling in addition to memory and throughput profiling.

This release also adds a command line interface to Skyline's iteration run time and memory profilers:

# Generate a memory usage and breakdown report
# (Run skyline memory --help for more information)
skyline memory path/to/entry/point/file --output report.sqlite

# Generate an iteration run time breakdown report
# (Run skyline time --help for more information)
skyline time path/to/entry/point/file --output report.sqlite

The generated reports are SQLite databases. You can explore these reports using the sqlite3 command line tool (sudo apt-get install sqlite3 on Ubuntu 18.04).

New Features

• Iteration run time profiling
• A command line interface to the iteration run time and memory profilers

Multi-file project support fixes

This release fixes issues associated with multi-file project support.

Bug fixes

• Fix issues with importing user-defined modules
• Fix issues related to module caching

Memory profiling and multi-file project support

This is the first production release of the Skyline CLI. Skyline now supports multi-file projects and is more robust in handling profiling sessions.

Note: The Skyline CLI and plugin are released independently (i.e. they have independent version numbers). This repository acts as a "monorepository"; it stores the code for both components. However, releases here refer to CLI releases. The plugin's releases can be found at geoffxy/skyline-atom.

The Skyline CLI can now be installed using pip:

pip install skyline-cli

Features

• Memory profiling
• Multi-file project support

Changes since v0.0.0

• New name: INNPV is now Skyline
• Temporary removal of interactive features (throughput/memory vs. batch size predictions)
• Temporary removal of run time profiling and breakdowns
• Stability: handle connection loss and failures

Course project completion

This release marks the final state of Skyline when it was submitted as a project for the Winter 2019 offering of CSC2524 at the University of Toronto.

Supported Features

• Module-based iteration execution time measurement and analysis
• Single file analysis
• Remote machine analysis as long as the server is launched manually
• Throughput measurement and batch size predictions using a linear iteration execution time model
• Peak memory usage measurement and batch size predictions (using a linear model)
• Annotation mutations for batch size predictions only
• Source code highlighting within forward() and __init__()
• Performance annotations when iteration time measurements are dragged