HeartBeat too slow problems

[WeihanLi]

Recently, I got some the HeartBeat too slow warning and I can not see any thread starvation, the thread starvation data is from the thread pool etw event data collected by prometheus-net.DotNetRuntime

The metrics data:

The warning log:

There's a web API project running on asp.net core 3.1 and it's running in the docker container, the docker image I used is mcr.microsoft.com/dotnet/core/aspnet:3.1-alpine

[Tratcher]

When you see this happening you'll want to capture a dump and check for blocked threads.

Threadpool exhaustion isn't the only cause of this issue, but it is by far the most common.

[WeihanLi]

I mean there's some thread pool worker thread adjustment event caused by Thread timed out around the time of HeartBeat too slow warning

[Tratcher]

Ok, what were the details given with that event? Especially the "Reason".

[WeihanLi]

I can see there's two reasons for the thread adjustment, Climbing move and Thread timed out only, No more details.

I'm not sure if the metrics dashboard would help, maybe not

[Tratcher]

As I understand it, climbing_move is the precursor to full starvation. It's actively injecting new threads into the pool because the system is under pressure. Your system seems to be doing that continuously which suggests you are experiencing starvation inducing conditions. Figuring out what's consuming all of those threads is best done with a dump. I'm not aware of good alternatives.

[WeihanLi]

Great thanks for your information.