[Oneshot Refactor] dataclass Arguments

[horheynm]

ORDER OF REVIEWS:

1. [Oneshot Refactor] Rename get_shared_processor_src to get_processor_name_from_model #1108
2. [Oneshot Refactor] dataclass Arguments #1103 <- current PR
3. [Oneshot refactor] Refactor initialize_model_from_path #1109
4. [Oneshot Refactor] Main refactor #1110

SUMMARY:
Refactor dataclass used for llm-compressor entrypoints (oneshot, train, apply) to decouple non-relevant attributes from the existing dataclass. Ex. recipe in training_args. Recipe is contained in a session, not in the trainer that training_args govern.

Dataclass refactor details are in
https://docs.google.com/document/d/1YbR1dTQmCzqhGk74m5msBzqoPHQgB6dVxDtf6cTmetc/edit?usp=sharing

Note:
#1110 takes care of using a new entrypoint that will prohibit the post_train / oneshot call to use training_args. Current entrypoint will need training_args for oneshot to function - this PR is just for refactoring the dataclass.

Before:
ModelArguments:

llm-compressor/src/llmcompressor/transformers/finetune/model_args.py

Line 6 in 6fa5a5e

class ModelArguments:

DataTrainingArguments:

llm-compressor/src/llmcompressor/transformers/finetune/data/data_args.py

Line 70 in 6fa5a5e

class DataTrainingArguments(CustomDataTrainingArguments):

TrainingArguments:

llm-compressor/src/llmcompressor/transformers/finetune/training_args.py

Line 10 in 6fa5a5e

class TrainingArguments(HFTrainingArgs):

After:
ModelArguments: https://github.com/vllm-project/llm-compressor/pull/1103/files#diff-58fd0f7ae4564317960ae0d4d4b2cdb97b9588c1915f062915e74ecf51b5502cR6
DatasetArguments: https://github.com/vllm-project/llm-compressor/pull/1103/files#diff-5e43f74ba5d8327b937adada3c7f30a7efb13f9a44cb3fdb5e1a2a12b8b8ea27R70
RecipeArguments: https://github.com/vllm-project/llm-compressor/pull/1103/files#diff-0ff9c048a4deb55e5459054bdc61a5d8c81da9c94588ec2355e6b2c2ec8675d1R6
TrainingArguments: https://github.com/vllm-project/llm-compressor/pull/1103/files#diff-249ee96763dd50956a7309f898eda4bcaa91c6af653474568fbda10b5a39c817R12

TEST PLAN:

• Pass all existing tests
• Search dataclass arguments using pygrep

  3 function pygrep() {
  2     local search_term="$1"
  1     shift
126     local search_dirs="${*:-src examples tests}"                           
  1     grep -rn --include="*.py" -E "$search_term" $search_dirs
  2 }

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[dsikka]

@horheynm Can you touchbase with Mark on the data arguments he shared during the meeting this morning?

[kylesayrs]

Is this ready for review? I'm a little confused what the comment "make changes in a different pr" means

[horheynm]

@horheynm Can you touchbase with Mark on the data arguments he shared during the meeting this morning?

Yes

[horheynm]

Is this ready for review? I'm a little confused what the comment "make changes in a different pr" means

It means it still needs to be broken down, it should only contain dataargs only.

[kylesayrs]

@horheynm Is this ready for review now?

[horheynm]

@kylesayrs yes

[kylesayrs]

Missing changes

grep -r 'DataTrainingArguments' src/ tests/ examples/ --exclude="*.pyc"                
src/llmcompressor/transformers/tracing/debug.py:from llmcompressor.transformers import DataTrainingArguments, TextGenerationDataset
src/llmcompressor/transformers/tracing/debug.py:    data_args = DataTrainingArguments(**get_dataset_kwargs(modality))
src/llmcompressor/transformers/finetune/README.md:* DataTrainingArguments: `src/llmcompressor/transformers/finetune/data/data_training_args.py`

[horheynm]

grep -r 'DataTrainingArguments' src/ tests/ examples/ --exclude="*.pyc"

I see thats from new merge from main. Thanks

[kylesayrs]

examples/trl_mixin/sft_trainer.py
examples/trl_mixin/ex_trl_constant.py

ImportError: cannot import name 'TrainingArguments' from 'llmcompressor.transformers'

This is caused by the change in import path in src/llmcompressor/transformers/finetune/__init__.py. Since these are public primitives, I'd rather not change these paths unless it's necessary.

I did not find any issues by grepping for ModelArguments or DatasetArguments imports.

[kylesayrs]

I think the problem with defaulting output_dir to a string is the two different behaviors we want from the two different pathways.

We want oneshot to not save the model if no output_dir is specified.
We want train to default to "./outdir" if no output_dir is specified.

I recommend setting the default to None, then during validation, populate it with "./outdir" for train and None for oneshot, perhaps with a warning.

All of this should be explained in the help text as well.

[horheynm]

Changing saving logic and blocking this pr is out of scope. Any logic change outside of the dataclass argument population will be handled in the subsequent pr