Plugin using deprecated API

[sadatay]

Users of this plugin may have noticed that image retrieval stopped working yesterday. This appears to be because the plugin uses a Google API that has been deprecated since 2011, and has finally stopped working yesterday.

Read more here.

Is anybody currently working on this? If not I can take it up, I believe there is a new API that can be used according to the google documentation but I don't want to duplicate work if somebody is already on the case. 😄

[jimmycuadra]

I knew this day would come eventually. ;_;

No one is working on this so you are welcome to. Thanks for the report!

[sadatay]

Hmm, looks like the google "custom search" API that is suggested as an alternative has limitations.

For CSE users, the API provides 100 search queries per day for free. If you need more, you may sign up for billing in the Developers Console. Additional requests cost $5 per 1000 queries, up to 10k queries per day.

There is a paid option, but it's similarly limited, and considering the amount of people using the plugin it's pretty clear that we won't be able have the plugin use one key. A potential option might be to have users of lita-google-images register with google and provide their own key, but especially for larger teams 100 images per day doesn't seem realistic.

There's a similar conversation going on in the hubot plugin:
https://github.com/hubot-scripts/hubot-google-images/issues/29

One option might be scraping? I'm gonna start looking into that now, but long story short this won't be as simple as changing the endpoint and making some minor tweaks.

[jimmycuadra]

Yep, this is why I used the old API when building this plugin even though it was already deprecated at the time. Having it cost money for people is a huge deterrent, but it's not totally unreasonable since it's a company that provides the service, after all.

[sadatay]

https://github.com/nikhgupta/scrapix seems like a viable option. i'll play around with it tonight and see how it behaves in the lita dev environment.

[jimmycuadra]

I'm not sure scraping is really an option. It likely violates their TOS, as otherwise they wouldn't be able to sell API access. It's also highly error prone and probably not something we'd want to maintain long term.

[sadatay]

Hmm, we may be in a bind then. For what it's worth, yahoo and bing also require payment for their image APIs. Pixabay seems like the best alternative. I'll get cracking on a plugin later tonight.

By default, you can make up to 2,000 requests in 30 minutes (4,000 requests per hour). Requests are associated with the API key, and not with your IP address.

Sad that it went this way with google 😞

[tgraham]

I have fix in my fork, just needs the readme updated with how to setup CSE etc.

[vijaykramesh]

😭 I don't think I can go back to searching google images myself and copying-and-pasting links.

[taylorlapeyre]

Any new news on this? 😕

[tgraham]

If someone wants to look at the changes in my fork and let me know if that's something you'd want upstream (Google CSE), I'll update the README and make any other changes that might be recommended and submit a PR.

[taylorlapeyre]

@tgraham I tried using your fork in my lita instance as so:

gem "lita-google-images", github: 'tgraham/lita-google-images'

However, it looks like lita image me test did not work.

[tgraham]

There's a whole barrel of fun to do setting up the Google CSE to make it work, which is part of what needs to be addressed in the README if there's a consensus for going that route.

Happy to help debug once you've setup the CSE stuff or if you need me to add some quick notes to the README to get it into a good place for actual use, let me know and I can knock that out semi-quickly.

I've been using my fork both locally for dev and in our prod env since Saturday without issue other than no results being found. I've defaulted to not posting a message into our chatroom when that happens, but welcome to any and all feedback on additional changes the community at large would like to see.

[taylorlapeyre]

@tgraham ahhh, didn't know I had to set up all the Google CSE stuff. Man, what a pain.

I think that is the legitimate route, but I'll try to make a fork that scrapes google's results this afternoon. It will cause less pain for everyone, even if it's not totally legit.

[jimmycuadra]

Scraping is against their TOS, so I will not merge any code that does that into this repository, FYI.

[tgraham]

Yeah, and we (corporate "Royal We") cannot have any code in use that does naughty things like that either.

The actual process to get a CSE setup and running isn't really that bad, and it's only a one time thing. I think what most people don't like is the 100/day limit on the free tier. That's where the hubot crowd has ran with the Google/Bing/Something else route to address all their cat picture needs.

"We" don't have that high of a demand on cat pictures in chat, so the 100/day limit is more than sufficient for "Us".

[sadatay]

That's where the hubot crowd has ran with the Google/Bing/Something else route to address all their cat picture needs.

I'm currently working on a lita version of that: https://github.com/sadatay/lita-image-fetcher

Though I'm currently running it without issue, I wouldn't recommend doing so just yet, as it's still very young and there are quite a few places where it could break without grace.

[taylorlapeyre]

@sadatay 😲 👍

[zx1986]

WARN: Couldn't get image from Google: This API is no longer available. :-(

[jimmycuadra]

This was resolved by #5. Update to gem version 3.0.0 to get support for the new API.