AWS Lambda - Apollo errors are taking multiple seconds. Successful requests are fast.

[slavaboiko]

Hi all,

I'm using Apollo-Lambda on AWS and noticing a very strange issue - throwing an exception will result in multi-second response time. from 2seconds to 6seconds (I think the upper limit might be Lambda's timeout).

I'm using serverless framework with esbuild to deploy. Locally with serverless-offline everything works fast.

Successful responses on deployed instances are also very fast. Only exceptions are taking super long time.

I have tried modying formatError and I see that there is even a big delay between me throwing an exception in the code and seeing the same exception in the formatError callback.

Honestly, no idea where to look anymore.

Does anyone have any ideas? Would appreciate any help!

[glasser]

That seems very strange. Are you talking about errors that are thrown inside your resolvers or elsewhere? Are you able to make a reproducible demo (eg, a small repo that we can even deploy to Lambda ourselves)? What version of Apollo Server? Do you have usage reporting to Apollo Studio set up?

formatError generally gets called as soon as the full execution is done (not just after that particular resolver ends). The didEncounterErrors plugin hook is invoked before formatError.

[slavaboiko]

@glasser thanks for your reply. I don't have reproducible demo right now, unfortunately. Trying right now to look into apollo code and yes, I have found the place with plugins being called, but having problems to get inside of the server and fetch a list of full plugins that are activated. the only thing that I have in my config is ApolloServerPluginLandingPageDisabled() which shouldn't cause any problems. Speaking about it - just tried removing plugin entry entirely from my config - didn't help either...

Wondering if there is any plugin that is activated which is trying to do something funky there... But I didn't have any additional things set up. Not using Apollo Studio either.

[slavaboiko]

There are some default plugins, but mostly if you're providing an Apollo API key or are implementing a subgraph with @apollo/subgraph.

yeah, found all default plugins and just tried disabling them all together as a bunch - didn't help either

[slavaboiko]

here you can see that the first message is happening 4 seconds earlier. and that's where I am throwing new Error() from my resolver

second line - that is formatError callback...

[slavaboiko]

SOURCE MAPS! It was source maps! 🤦

[glasser]

Interesting, is there something our project should be doing to avoid this issue?

[slavaboiko]

@glasser I think it was just very strange for me that it takes so much time while it is my first real encounter with such behavior. Never had such problems before and really surprised it takes so many seconds.
Also, the fact that it's just happening somewhere in internals of Node and slows down while stack trace is generated during exception creation (it seems) - that is insane. But totally makes sense. Not really sure how it can be of help here, but thanks for being around in this difficult minute ;)