Using system generated seed in RandomSampler #1441
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/data/1441
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit a5ec001 with merge base f15fd3a ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
divyanshk
reviewed
Feb 11, 2025
| seed = 1 | ||
| torch.manual_seed(seed) | ||
| dl3 = StatefulDataLoader(self.dataset, batch_size=1, shuffle=True) | ||
| data_dl3 = [] |
There was a problem hiding this comment.
Can we call this results3 ? and similarly results1 and results2 above.
divyanshk
reviewed
Feb 11, 2025
|
|
||
| seed = 1 | ||
| torch.manual_seed(seed) | ||
| dl3 = StatefulDataLoader(self.dataset, batch_size=1, shuffle=True) |
There was a problem hiding this comment.
We can rename dl3 to dataloader3. ditto for other dataloader variables.
divyanshk
reviewed
Feb 11, 2025
| ) | ||
|
|
||
| def test_seed_replicability(self): | ||
|
|
||
| seed = 0 |
There was a problem hiding this comment.
Instead of checking for specific seeds 0 and 1, we can generalize it to two randomly generated seeds.
And also add a assert to ensure both seeds are not equal.
andrewkho
reviewed
Feb 11, 2025
| data_source: Sized, | ||
| replacement: bool = False, | ||
| num_samples: Optional[int] = None, | ||
| generator=None, | ||
| ): | ||
| if generator is None: | ||
| # Ensure that underlying sampler has something repeatable |
There was a problem hiding this comment.
let's remove or update this comment
andrewkho
approved these changes
Feb 11, 2025
ramanishsingh
added a commit
that referenced
this pull request
Feb 19, 2025
* add new sampler tests * update seed generation in sampler * run precommit * update seed generation * change variable name * update comment * add seed to tests * run precommit
ramanishsingh
added a commit
that referenced
this pull request
Feb 19, 2025
* Fix end of epoch StatefulDataLoader restart (#1439) * add test for end of epoch state dict check * run precommit update stateful_dataloader run precommit local changes update test to test the order of batches update test update tests revert changes in SDL revert changes in SDL update tests run precommit * update sampler * run precommit * remove unnecessary comment * add test for statedict before and after endofepoch * run precommit * check if _sampler_iter is exhausted * run precommit * remove commented lines * remove default values * only exhaust sampler_iter if present in sd * update _StatefulRandomSamplerIterator update state dict if the iterator has finished add comment about why were updating state dict run precommit * update randomsampleriter state_dict fully * run precommit * fork torch.utils.data RandomSampler reverse changes to sdl.py generator to iterator run precommit update generator usage * update class name * run precommit * add a method to generate permutations * update return type * update next logic * add comment * update tests to include non stateful samplers * add comments * Using system generated seed in RandomSampler (#1441) * add new sampler tests * update seed generation in sampler * run precommit * update seed generation * change variable name * update comment * add seed to tests * run precommit
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently we are fixing the seed for
generatorinRandomSampleras 1.This leads to the generator not changing even when torch.manual_seed() seed is changed.
For the RandomSampler in
torch.utils.data.sampler, they useseed = int(torch.empty((), dtype=torch.int64).random_().item()). Using the same here.Fixes #1440