-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proxmox module refactoring #9225
Proxmox module refactoring #9225
Conversation
@Lithimlin this PR contains the following merge commits: Please rebase your branch to remove these commits. |
6fcef97
to
c2ffd98
Compare
This comment was marked as resolved.
This comment was marked as resolved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @Lithimlin thanks for your contribution!
I would recommend breaking this PR into smaller pieces, as it seems to contain both bugfixes as well as new features. Additionally, we must not use type hints, as Ansible still provides support for Python 2.7 - it's quite a nuisance, but that should be the case until that support is dropped, I want to say end of next year but I am not sure right now.
The last 2 versions of core already dropped support for 2.7 and 2.16 will be EOL in May 2025 |
Yes, but community.general still supports ansible-core 2.15. Felix mentioned the deadline more than once but it will be late next year - far enough into the future to be forgotten. Until then, in the collection, we must support Python 2.7. |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, I had approved it before, but will mark it "Request changes" to prevent other maintainers from accidentally merging it as-is now. Also, on second thought, this is quite a big revamp, so it would benefit from a more thorough examination before we merge it. I approved it before (and I should not, because the changelog needed fixing) and, days later, I cannot recall what I have seen here or not. I'm writing this to manage the expectations - this PR might take a tad longer to be merged - but please bear with us.
It looks like there's anohter conflict now (I guess due to #9524). Sorry for the hassle :( I hope this was the last reformat... |
This is a squash of the following commits for easier rebasing: proxmox module_utils: make use of choose_first_if_multiple in get_vm proxmox: refactor module proxmox: add changelog proxmox: fix deprecation message proxmox: remove type hints proxmox: remove spaces for keywords proxmox: run formatter proxmox: make compabtible with old python versions proxmox: remove f-strings proxmox: fix string formatting in build_volume proxmox: revert disk size parameter to simple integer proxmox: update changelog fragment proxmox: fix argument spec proxmox: fix size handling in build_volume proxmox: fix formatting proxmox: update changelog fragment
edab74c
to
0c87d71
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @Lithimlin
I have spent more time looking into it and made it through state=present
. Got some comments on it. Cheers
if not vmid and not hostname: | ||
self.module.fail_json(msg="Either VMID or hostname must be provided.") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this not something that should be guaranteed by the module required_*
arguments? It seems that all the module states call get_lxc_resource()
, so I reckon it would make more sense to add a required_one_of
with ("vmid", "hostname")
and remove this check from here, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can add the required_one_of
, but this is more a check in place for the function itself so it is not misused.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The functions within this file are not meant to be imported elsewhere, so in theory if not in practice, the only misuse that can happen is in this file as well. If you could guarantee that is not the case, then this check could be removed.
It already has a required_one_of vmid hostname
, so possibly that part is already done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suppose that means I should properly document all functions in this module as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, actually, I remember why I did it this way: This is preparation for being able to safely use this function in the module_utils
.
def get_lxc_resource_by_id(self, vmid): | ||
vms = self.get_vm_resources() | ||
|
||
if (proxmox.stop_instance(vm, vmid, timeout, force=module.params['force']) and | ||
proxmox.start_instance(vm, vmid, timeout)): | ||
module.exit_json(changed=True, vmid=vmid, msg="VM %s is restarted" % vmid) | ||
vms = [vm for vm in vms if vm["vmid"] == vmid] | ||
if len(vms) == 0: | ||
raise LookupError("VM with VMID %d does not exist in cluster." % vmid) | ||
|
||
return vms[0] | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method plus the get_vm_resources()
one below seem to do almost the exact same thing as ProxmoxAnsible.get_vm()
, defined in plugin/module_utils/proxmox.py:152-164.
The module_utils' code seems to have more checks and safety around types (the vmid parameter is an int in the module options, the PR code does not seem to deal with that, comparing int to str, for example), but the PR code does use a more specific exception (LookupError), which is usually a good thing.
Is there any compelling reason to have two new methods that do almost the same thing as existing code? If yes, please expand, else, please remove the new ones and make any improvements you feel necessary on the existing ones so that other modules may benefit from them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Somewhere down the line I'd love to put get_vm_resources()
and get_lxc_resource_by_id()
into the utils module, but I didn't want to replace it there as that would require me to check all other modules for functionality as well. I have neither the setup nor the time to do that unfortunately.
What I could do is add both of these new functions to the utils module so they can be used down the line and possibly deprecate the old one, but I'm not sure how good of an idea that is.
def get_vm_resources(self): | ||
try: | ||
return self.proxmox_api.cluster.resources.get(type="vm") | ||
except Exception as e: | ||
module.fail_json(vmid=vmid, msg="restarting of VM %s failed with exception: %s" % (vmid, e)) | ||
self.module.fail_json( | ||
msg="Unable to retrieve list of %s VMs from cluster resources: %s" | ||
% (self.VZ_TYPE, e) | ||
) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This indeed would be better placed in the module_utils, and methods get_vmid()
and get_vm()
(the ones on my screen at the moment, may others as well but not searching right now) could be refactored and benefit from it.
vm = proxmox.get_vm(vmid) | ||
if getattr(proxmox.proxmox_api.nodes(vm['node']), VZ_TYPE)(vmid).status.current.get()['status'] == 'running': | ||
module.exit_json(changed=False, vmid=vmid, msg="VM %s is already running" % vmid) | ||
lxc = self.get_lxc_resource(vmid, hostname) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that the module claims to work with both lxc and openvz VMs, it might be more neutral to call this variable vm
instead of lxc
.
lxc = self.get_lxc_resource(vmid, hostname) | |
vm = self.get_lxc_resource(vmid, hostname) |
Actually, the method also could have a more neutral name.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While that is true, this ties into the discussion of dropping openvz
support since it's from a Proxmox version that is way past its EOL.
self.module.fail_json( | ||
vmid=vmid, | ||
msg="VM %s does not exist but neither clone nor ostemplate were specified!" | ||
% identifier, | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this case ever going to happen? Line 742 (required_if state present) guarantee at least one of update
(in which case this method new_lxc_instance()
does not get called), clone
or ostemplate
is passed. Please double check, if sure it won't happen, please remove.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is possible for the LXC to not exist, but update
being set to true
, in which case neither clone
nor ostemplate
need to be set according to the module definition.
@russoz @Lithimlin what's the current state of the PR? Is everyone happy so we can merge this for 10.3.0? I guess we won't get more feedback from users of the module here, unless it ends up in a release... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, I'm approving on the basis that it seems to have been carefully tailored, but I have not really examined all the logic paths. Again, this PR is way too big - it would have been better to create a number of smaller ones.
@Lithimlin I understand/assume you have tested this thoroughly and I am approving based on that.
Backport to stable-10: 💚 backport PR created✅ Backport PR branch: Backported as #9634 🤖 @patchback |
* proxmox: Refactor This is a squash of the following commits for easier rebasing: proxmox module_utils: make use of choose_first_if_multiple in get_vm proxmox: refactor module proxmox: add changelog proxmox: fix deprecation message proxmox: remove type hints proxmox: remove spaces for keywords proxmox: run formatter proxmox: make compabtible with old python versions proxmox: remove f-strings proxmox: fix string formatting in build_volume proxmox: revert disk size parameter to simple integer proxmox: update changelog fragment proxmox: fix argument spec proxmox: fix size handling in build_volume proxmox: fix formatting proxmox: update changelog fragment * proxmox: Fix changelog fragment, doc, and deprecation string formatting. * proxmox: Fix formatting in imports * proxmox: require one of `vmid` or `hostname`, simplify checks * proxmox: apply check for supported features to entire module * proxmox: move parameter conversions inside create and update functions (cherry picked from commit d71ba0f)
@Lithimlin thanks a lot for your contribution! |
…9634) Proxmox module refactoring (#9225) * proxmox: Refactor This is a squash of the following commits for easier rebasing: proxmox module_utils: make use of choose_first_if_multiple in get_vm proxmox: refactor module proxmox: add changelog proxmox: fix deprecation message proxmox: remove type hints proxmox: remove spaces for keywords proxmox: run formatter proxmox: make compabtible with old python versions proxmox: remove f-strings proxmox: fix string formatting in build_volume proxmox: revert disk size parameter to simple integer proxmox: update changelog fragment proxmox: fix argument spec proxmox: fix size handling in build_volume proxmox: fix formatting proxmox: update changelog fragment * proxmox: Fix changelog fragment, doc, and deprecation string formatting. * proxmox: Fix formatting in imports * proxmox: require one of `vmid` or `hostname`, simplify checks * proxmox: apply check for supported features to entire module * proxmox: move parameter conversions inside create and update functions (cherry picked from commit d71ba0f) Co-authored-by: JL Euler <Lithimlin@users.noreply.github.com>
SUMMARY
This pull request refactors the historically grown
proxmox
module.The control flow was very opaque with some redundant checks and - most recently - features that were only introduced in parts of the module when they intended to target the whole module (see #9065).
This refactor aims to make the module more readable and more easily extendable.
Additionally, this request contains various bug fixes:
disk_volume
key is now properly parsed and passed on to the APIupdate=False
is now deprecated and will be changed toupdate=True
in version 11.0.0proxmox
utils module: The parameterchoose_first_if_multiple
ofget_vmid
is now used correctly, causing the function to return the first of multiple matches instead of failing.ISSUE TYPE
COMPONENT NAME
proxmox
ADDITIONAL INFORMATION
The following playbook should run correctly on both the old and refactored versions of this module, but the template creation will not be idempotent on the old version: