Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support loss over all tokens to be back-propagated during training #28

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Support loss over all tokens to be back-propagated during training #28

wants to merge 5 commits into from
+70 −5

Commits on Aug 31, 2024

  1. Support for cross entropy loss over all tokens predicted

    @maydixit
    maydixit committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    1858946 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. Support for cross entropy loss over all tokens predicted

    @maydixit
    maydixit committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    a06293e View commit details
    Browse the repository at this point in the history

  2. Add support for all position loss

    @maydixit
    maydixit committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    307f2b4 View commit details
    Browse the repository at this point in the history

  3. Divide loss by position count

    @maydixit
    maydixit committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    7030034 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'maydixit' of https://github.com/maydixit/tiny-transformers 

     into maydixit
    @maydixit
    maydixit committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    0c37dd3 View commit details
    Browse the repository at this point in the history