Code

Update model builders#2282

Code

Merged

ebsmothers merged 11 commits intopytorch:mainpytorch/torchtune:mainfrom Ankur-singh:update-model-buildersAnkur-singh/torchtune:update-model-buildersCopy head branch name to clipboardJan 27, 2025

+570-512

Commits

Commits on Jan 26, 2025

• Refactor llama2 component_builder to pass multiple layers to TransformerDecoder

  Ankur-singh

  committed
  122cd32

  Copy full SHA for 122cd32

• Refactor TransformerDecoder to support multiple layers in gemma and lora_gemma functions

  Ankur-singh

  committed
  defc90d

  Copy full SHA for defc90d

• Refactor TransformerDecoder to support multiple layers in gemma2 and lora_gemma2 functions

  Ankur-singh

  committed
  1200ad5

  Copy full SHA for 1200ad5

• Refactor TransformerDecoder to support multiple layers in llama2, lora_llama2, llama2_classifier and lora_llama2_classifier functions

  Ankur-singh

  committed
  8b5d789

  Copy full SHA for 8b5d789

• Refactor T5 encoder and component builder to support multiple layers

  Ankur-singh

  committed
  1d37dda

  Copy full SHA for 1d37dda

• Refactor llama3 family component builders to pass ModuleList in TransformerDecoder

  Ankur-singh

  committed
  3999c4a

  Copy full SHA for 3999c4a

• Refactor mistral and lora_mistral functions to pass ModuleList to TransformerDecoder

  Ankur-singh

  committed
  faa38e5

  Copy full SHA for faa38e5

• Refactor phi3 and lora_phi3 functions to pass ModuleList in TransformerDecoder

  Ankur-singh

  committed
  525573a

  Copy full SHA for 525573a

• Refactor qwen2 and lora_qwen2 functions to pass ModuleList in TransformerDecoder

  Ankur-singh

  committed
  01903f5

  Copy full SHA for 01903f5

• add back support for num_layers in T5Encoder and other minor changes

  Ankur-singh

  committed
  084ac50

  Copy full SHA for 084ac50

• update expected output, thumbs up from pytest

  Ankur-singh

  committed
  b9a8284

  Copy full SHA for b9a8284