v.0.1.1

Release Notes

Version 0.1.1 - February 6, 2025

Enhancements:

• Mathematical Constants: Replaced custom definitions of π with math.pi for consistency and clarity.
• Code Maintenance: Performed minor renaming and added logging to improve code readability and traceability.

Testing and Continuous Integration:

• Tests Added: Implemented tests for activation functions to ensure correctness and reliability.
• CI/CD Workflow: Established a CI/CD pipeline using GitHub Actions to automate testing and streamline development.

Dependencies:

• Requirements File: Introduced a requirements.txt file to specify project dependencies, facilitating easier setup and installation.

Initial release of torchortho

Release Notes - v1.0.0 (Initial Release)

• torchortho library for learnable activation functions based on:
  • Hermite Polynomials
  • Fourier Series
  • Tropical Polynomials & Rational Functions
• Adaptive activations that improve expressivity, gradient flow, and generalization.
• Features:
  • Hermite Activation: Smooth, variance-preserving polynomial activation.
  • Fourier Activation: Captures periodic structures in data.
  • Tropical Polynomial and Rational Activation: Convex activation for structured learning.
• Benchmarked on:
  • ConvNeXt-T on ImageNet-1K (improves accuracy).
  • GPT-2 on OpenWebText (reduces perplexity).
• Installation:
  • pip install torchortho
  • pip install git+https://github.com/K-H-Ismail/torchortho.git