Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-1.14] Improve scheduler memory usage and remove scheduler waits to speed up recovery time #8203

Merged

Commits on Sep 23, 2024

  1. Improve scheduler memory usage (knative#8144)

    * Improve scheduler memory usage
    
    - Create a namespaced-scoped statefulset lister instead of being
      cluster-wide
    - Accept a PodLister rather than creating a cluster-wide one
    
    Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
    
    * Update codegen
    
    Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
    
    ---------
    
    Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
    pierDipi committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    17ab884 View commit details
    Browse the repository at this point in the history
  2. Remove scheduler waits to speed up recovery time (knative#8200)

    Currently, the scheduler and autoscaler are single threads and use
    a lock to prevent multiple scheduling and autoscaling decision
    from happening in parallel; this is not a problem for our use
    cases, however, the multiple `wait` currently present are slowing
    down recovery time.
    
    From my testing, if I delete and recreate the Kafka control plane
    and data plane, without this patch it takes 1h to recover when there
    are 400 triggers or 20 minutes when there are 100 triggers; with the
    patch it is immediate (only a 2/3 minutes with 400 triggers).
    
    - Remove `wait`s from state builder and autoscaler
    - Add additional debug logs
    - Use logger provided through the context as opposed to gloabal loggers
      in each individual component to preserve `knative/pkg` resource aware
      log keys.
    
    Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>
    pierDipi committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    3060ca8 View commit details
    Browse the repository at this point in the history