Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: CH04 - Is the test set part of the train set? #16

Open
dominicdill opened this issue Nov 23, 2024 · 0 comments
Open

Question: CH04 - Is the test set part of the train set? #16

dominicdill opened this issue Nov 23, 2024 · 0 comments

Comments

@dominicdill
Copy link

dominicdill commented Nov 23, 2024

I am confused by the use of train and test sets for these time series models.

Here you setup your entire dataset and split it into test and train sets.
image
image

It seems like you would want to evaluate your model by seeing how well it performs on the test set using only the train set. But in the notebook the test set is used when evaluating the model.

image

image

To actually see how well the model predicts into the future, wouldn't we instead want to set the df parameter of the rolling_forcast function to train and then add lines to that function that extend the supplied df with the predictions?

Currently, because df_dff (which holds both train and test data) is used, you're not truly seeing how well your models predict into the future.

Am I misunderstanding? Is this what we want to be doing?

I changed the code to do as I think I'm expecting:

image

And I see this:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant