Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feeling confused about num_interventions and intervention_locations #162

Open
NaOH678 opened this issue Mar 25, 2025 · 0 comments
Open

Feeling confused about num_interventions and intervention_locations #162

NaOH678 opened this issue Mar 25, 2025 · 0 comments

Comments

@NaOH678
Copy link

NaOH678 commented Mar 25, 2025

Hello! I'm interested in the ReFT but have some questions.

I want to intervene the 15th and 16th layer of the model, I use the following code:

TARGET_LAYER = 15

# get reft model
reft_config = ReftConfig(representations=[{
    "layer": TARGET_LAYER, "component": "block_output",
    "intervention": SubloreftIntervention(
    embed_dim=model.config.hidden_size, low_rank_dimension=8)},
    {
    "layer": TARGET_LAYER + 1, "component": "block_output",
    "intervention": SubloreftIntervention(
    embed_dim=model.config.hidden_size, low_rank_dimension=8)
    }])
reft_model = get_reft_model(model, reft_config)
reft_model.print_trainable_parameters()

and use the code to get intervention_locations:

dataset = make_multiple_position_supervised_data_module(tokenizer, model, num_interventions=len(reft_model.interventions),
                                                    inputs=subspace_dataset['instruction'], outputs=subspace_dataset['output'], 
                                                    positions='f5+l5') 

What I want to is to intervene the f5 and l5 tokens at both layer 15 and layer 16,but confused with the following results.
Here are first five results of all data:

dataset['train_dataset']['intervention_locations'][:5]

[
[[0, 1, 2, 3, 4], [22, 23, 24, 25, 26]],
 [[0, 1, 2, 3, 4], [8, 9, 10, 11, 12]],
 [[0, 1, 2, 3, 4], [13, 14, 15, 16, 17]],
 [[0, 1, 2, 3, 4], [17, 18, 19, 20, 21]],
 [[0, 1, 2, 3, 4], [13, 14, 15, 16, 17]]

]

I am confused about the three-dimention list. for each line in the list is a two-dimention list represents the first five tokens and last five tokens, does it mean intervene the first five token at layer 15 and last five token at layer 16 or intervene both first five and last five tokes at both layer 15 and layer 16? what is the meaning of the thee-dimention list?

Thanks for answering me question!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant