[release-1.14] Improve scheduler memory usage and remove scheduler waits to speed up recovery time

[release-1.14] Improve scheduler memory usage and remove scheduler waits to speed up recovery time #6803