Azure gather metrics for vmss

[NissesSenap]

We have seen that even if a node is deleted from kubernetes they some time linger inside vmss.
Add metrics gathering from azure vmss metrics endpoint and expose them in prometheus.

Add the needed config to: https://github.com/XenitAB/terraform-modules/blob/main/modules/kubernetes/azure-metrics/charts/azure-metrics-exporter/templates/podmonitor.yaml

[NissesSenap]

@CalleB3 might this be something for you?

[CalleB3]

Unfortunately at the current time there isnt any supported metrics in Azure for this.

Under Microsoft.Compute/virtualmachineScaleSets at the link below, there isnt anything related to vmss and its instances.

https://docs.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-supported#microsoftcomputevirtualmachinescalesets

[NissesSenap]

It looks like there is: VmAvailabilityMetric but it's in preview, we could probably open a support ticket and ask to be added in this preview.

Another possible solution is that we actually don't need a specific metric we just need to know the count of nodes responding with metrics. Then we can compare this data with active k8s nodes in the cluster.
So it we gather any metric from vmss that is related to k8s we should be okay.

Do you agree @phillebaba?