Skip to content

Commit

Permalink
removesuffix is not available 3.8, thus reverting the changes to rstr…
Browse files Browse the repository at this point in the history
…ip (#1239)

Summary:
Without these changes, it will result in CI failing. When 3.8 support is dropped, this change can be landed.

Fixes #{issue number}

### Changes
- change removesuffix back to rstrip

Pull Request resolved: #1239

Reviewed By: r-barnes

Differential Revision: D55433484

Pulled By: gokulavasan

fbshipit-source-id: 75ca1341e88378b6b3134c401a82d66b0129e8f4
  • Loading branch information
gokulavasan authored and facebook-github-bot committed Mar 27, 2024
1 parent 66f0b98 commit de72b34
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion torchdata/datapipes/iter/util/bz2fileloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def __iter__(self) -> Iterator[Tuple[str, BufferedIOBase]]:
pathname, data_stream = data
try:
extracted_fobj = bz2.open(data_stream, mode="rb") # type: ignore[call-overload]
new_pathname = pathname.removesuffix(".bz2")
new_pathname = pathname.rstrip(".bz2") # https://github.com/pytorch/data/issues/1240
yield new_pathname, StreamWrapper(extracted_fobj, data_stream, name=new_pathname) # type: ignore[misc]
except Exception as e:
warnings.warn(f"Unable to extract files from corrupted bzip2 stream {pathname} due to: {e}, abort!")
Expand Down
2 changes: 1 addition & 1 deletion torchdata/datapipes/iter/util/xzfileloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def __iter__(self) -> Iterator[Tuple[str, BufferedIOBase]]:
pathname, data_stream = data
try:
extracted_fobj = lzma.open(data_stream, mode="rb") # type: ignore[call-overload]
new_pathname = pathname.removesuffix(".xz")
new_pathname = pathname.rstrip(".xz") # https://github.com/pytorch/data/issues/1240
yield new_pathname, StreamWrapper(extracted_fobj, data_stream, name=pathname) # type: ignore[misc]
except Exception as e:
warnings.warn(f"Unable to extract files from corrupted xz/lzma stream {pathname} due to: {e}, abort!")
Expand Down

0 comments on commit de72b34

Please sign in to comment.