Improve server listing performance #602
Conversation
databus23
requested review from
BugRoger,
jknipper and
onuryilmaz
as code owners
| var filteredNodes []Node | ||
|
|
||
| err := servers.List(c.ComputeClient, servers.ListOpts{}).EachPage(func(page pagination.Page) (bool, error) { | ||
| err := servers.List(c.ComputeClient, servers.ListOpts{Name: "^(kks-)?" + k.Spec.Name + "-" + pool.Name + "-"}).EachPage(func(page pagination.Page) (bool, error) { |
There was a problem hiding this comment.
@joker-at-work Is using a regex for listing servers by name safe and won't break in the future in your opinion? Seems like its depended on the database used for nova which unsettles me a bit.
There was a problem hiding this comment.
All I can say is that we don't plan to change the DB under Nova once again.
FYI this is the function translating the string into the query: https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L1943-L1973 Looks pretty DB-dependent. It basically just uses a sanitized version of whatever you provide.
databus23
force-pushed
the
list-servers
branch
from
6c88d3c to
099d894
Compare
Instead of listing all servers of a project we use a name filter the uses the regex `^(kks?)-$KLUSTERNAME-$POOLNAME` for the `name` attribute. In addition we also process the pages individually when filtering through the returned servers.
databus23
force-pushed
the
list-servers
branch
from
263521b to
a8b07b5
Compare
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
Instead of listing all servers of a project we use a name filter with the regex
^(kks-?)$KLUSTERNAME-$POOLNAMEfor thenameattribute.In addition we also process the pages individually when filtering through the returned servers.
This should avoid listing thousands of servers from s4 projects on a constant basis.
CAREFUL: This needs extra special testing and careful review. If the listing is somehow filtering out valid kubernikus nodes launchctl will spawn new servers until the quota is exceeded.