Ensure fetching a set of items via a set of IDs never errors unless in "strict" mode...

[mikesname]

Quite often we want to fetch stuff from the search engine that's no longer in the DB. In most cases it's better to return a list missing those items and let the client handle it rather than 404ing the whole request.

[bencomp]

Are you sure it's better? In which cases wouldn't it be better?

[mikesname]

The new behaviour is more like an SQL IN statement where you get an empty set back if none of the in-clauses argument's exist, or rather, you get back just the arguments that do exist.

The advantage of the old behaviour (barfing if any items were not found) was that it gives you a heads-up in the form of lots of error emails when the search index is out of sync (since the search engine is the source of IDs we're fetching from the backend.)

With this new behaviour we're more likely to miss it when an item's still in the search index but not in the DB. The worst case here is that the user might do a query and it'll say it found, say, 5 results but only display 4. However I'm more-or-less past caring about that now.

[bencomp]

How often are items missing from the search index on the production platform? Does stuff get deleted or are there other causes too?

[mikesname]

Unfortunately stuff does get deleted a lot (someone got confused and added hundreds of erroneous authority files that they've now been deleting over the past few weeks 😦 ).

I'm now mentally switching over to "PHP" development mode, where I make all the things that go wrong into silent stealthy failures instead of loud obnoxious ones.

It be honest, I've only seen this a couple of times on the prod server, and then it could be down to a glitch that prevented the index delete handler getting called.