Skip to content

Commit

Permalink
remove{suffix,prefix} replaces {l,r}strip in pytorch/data/torchdata/d…
Browse files Browse the repository at this point in the history
…atapipes/iter/util/bz2fileloader.py +1

Summary:
`x.lstrip("string")` is equivalent to `re.sub(r"[string]", "", x)`. This means that `"fbcode/fbcode_file".lstrip("fbcode/")` returns `_file`. This can easily be an unintended behaviour!

Instead, as of Python 3.10, we should use `string.removeprefix` or `string.removesuffix`.

This diff makes one or more such fixes that we believe are likely to be bugs. Please double-check before approving.

Reviewed By: zsol

Differential Revision: D55386281

fbshipit-source-id: 16a082f06f2820e87bbe9df69877a8d07ee97218
  • Loading branch information
r-barnes authored and facebook-github-bot committed Mar 27, 2024
1 parent 88a1d34 commit 8f0b10a
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion torchdata/datapipes/iter/util/bz2fileloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ def __iter__(self) -> Iterator[Tuple[str, BufferedIOBase]]:
pathname, data_stream = data
try:
extracted_fobj = bz2.open(data_stream, mode="rb") # type: ignore[call-overload]
new_pathname = pathname.rstrip(".bz2")
new_pathname = pathname.removesuffix(".bz2")
yield new_pathname, StreamWrapper(extracted_fobj, data_stream, name=new_pathname) # type: ignore[misc]
except Exception as e:
warnings.warn(f"Unable to extract files from corrupted bzip2 stream {pathname} due to: {e}, abort!")
Expand Down

0 comments on commit 8f0b10a

Please sign in to comment.