"comment_reactors": True still results in "comment_reactors": "<generator object PostExtractor.extract_reactors at 0x7f1fca7f4a40>",

[fraba]

Hello,

I am trying to write to json a post's comments and all reactions - including comments' reactors. I have set "comment_reactors": True. Still in the post dump I get a few fields such as "comment_reactors": "<generator object PostExtractor.extract_reactors at 0x7f1fca7f4a40>", although for others I actually get as expected:

            "comment_reactors": [
                {
                    "name": "Sara James",
...

Did I hit a limit in the number of nested levels the scrapers is able to get?

[neon-ninja]

As per https://github.com/kevinzg/facebook-scraper/blob/master/facebook_scraper/extractors.py#L222, you would get a comment_reactors generator if comments is set to "generator". 259352a should fix this.

[fraba]

No in my case comments is set to True also

[neon-ninja]

Either way, try my latest commit and see if that fixes your issue

[fraba]

Could allow_extra_requests help with this? Does it allow the scraper to go deeper in the replies?

[neon-ninja]

allow_extra_requests is True by default, if you're getting comments, that's a sufficient amount of extra requests. Did my latest commit help?

[fraba]

I can try this in a few hours! I’ll let you know

[fraba]

Yes, I can confirm I still get "comment_reactors": "<generator object PostExtractor.extract_reactors at 0x7f24b6a42d00>" in the replies to other comments with pip3 install git+https://github.com/kevinzg/facebook-scraper.git after your last commit.

[neon-ninja]

Can you post the exact code you're using so I can try reproduce this problem? As per the pinned issue template #273

[fraba]

This is my code:

post = next(get_posts(post_urls=['https://www.facebook.com/1576289275993875/posts/279258117330016'], cookies='cookies.json', options={"comments": True, "sharers": True, "reactors": True, "reactions": True, "comment_reactors": True}))

with open("1576289275993875-posts-279258117330016.json", mode='wt') as f:
     json.dump(post, f, indent=4, default=str)

In 1576289275993875-posts-279258117330016.json you will find the first "comment_reactors": "<generator object PostExtractor.extract_reactors at ...>" on line 399.

[l0rtk]

<generator object PostExtractor.extract_reactors at 0x7f44366c7350> is "comment_reactors" only in replies

[neon-ninja]

@l0rtk thanks, that useful. in that case, @fraba try 244c743

[fraba]

Thanks, @neon-ninja. Unfortunately after reinstalling the git version with pip3 install git+https://github.com/kevinzg/facebook-scraper.git I still get

                    "comment_reactors": "<generator object PostExtractor.extract_reactors at 0x7f10218f2518>",

[neon-ninja]

@fraba I get the feeling you're not actually updating your local copy of the library. Try run pip uninstall facebook-scraper twice first.

[fraba]

@neon-ninja Yep, that worked. I don't get a generator object anymore. Thanks!