Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add gmaps scraping blog #2772

Merged
merged 7 commits into from
Dec 24, 2024
Merged

docs: add gmaps scraping blog #2772

merged 7 commits into from
Dec 24, 2024

Conversation

souravjain540
Copy link
Collaborator

approved by adam and marketing.

@souravjain540 souravjain540 requested a review from vdusek December 13, 2024 06:09
Copy link
Contributor

@janbuchar janbuchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the article, it is very good and reads well, but it doesn't use the full potential of Crawlee in some places - let's improve that 🙂

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is huge, isn't there a more adequate format than gif?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't know. any suggestions? gif kinda fits here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WebP supports animation as well - I think that's the ideal candidate.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be webp as well?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@triposat the file was renamed, but it looks like it's still a png.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janbuchar I didn't get you, sorry :)

@vdusek vdusek changed the title docs: add gmaps scraping blog. docs: add gmaps scraping blog Dec 16, 2024
@triposat
Copy link

@souravjain540 @janbuchar

I've made all the required changes. I've updated the Notion file, including changing all the code snippets, descriptions, and adding more steps. I've also updated the code in the gist files where the complete code is stored.

One more thing, regarding the infinite_scrolling() method - it's not working. I tried but couldn't get proper results, so I opted for the general method of scrolling through pages. Apart from this, I have made all the requested changes.

Pls review again and let me know if anything else to be changed :)

@janbuchar janbuchar self-requested a review December 23, 2024 16:00
@janbuchar
Copy link
Contributor

One more thing, regarding the infinite_scrolling() method - it's not working. I tried but couldn't get proper results, so I opted for the general method of scrolling through pages. Apart from this, I have made all the requested changes.

I see. Could you please open an issue where you describe what you tried and how exactly it failed so that we can fix it?

@triposat
Copy link

One more thing, regarding the infinite_scrolling() method - it's not working. I tried but couldn't get proper results, so I opted for the general method of scrolling through pages. Apart from this, I have made all the requested changes.

I see. Could you please open an issue where you describe what you tried and how exactly it failed so that we can fix it?

Sure, @janbuchar. Will do it :)

Copy link
Contributor

@janbuchar janbuchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple minor comments/suggestions, great job otherwise!

Comment on lines 127 to 130
# Tell the crawler how to handle each page it visits
@crawler.router.default_handler
async def default_handler(context):
await scrape_google_maps(context)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should also probably just register scrape_google_maps as the default handler, like the example that follows this one.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@janbuchar Done. Pls check!

@triposat
Copy link

print(f"Error during scroll: {str(e)}")

@janbuchar Made all the changes in the Notion file and also changed the codes in the gist files

souravjain540 and others added 3 commits December 23, 2024 23:17
Co-authored-by: Jan Buchar <jan.buchar@apify.com>
@souravjain540 souravjain540 merged commit 3aeb014 into master Dec 24, 2024
9 checks passed
@souravjain540 souravjain540 deleted the gmaps branch December 24, 2024 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants