Saving bandwidth on Amazon #3227
toxyy
started this conversation in
Ideas and feature requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I did web scraping a few years back, and a few tricks that I used to save bandwidth were:
This saved huge amounts of bandwidth. For the sites I had to scrape, the optimal file bytes to retrieve were:
Amazon - 700KB
Ebay - 100KB
Aliexpress: 2.5MB
Alibaba - 70KB
I haven't looked in the source to see if all of this was being taken place. The code I wrote was 6 years ago, though, I don't know if there are any better techniques that could be done.
Beta Was this translation helpful? Give feedback.
All reactions