Switch to GHCR due to docker.io pull rate limits

[juliusvonkohout]

Validation Checklist

• I confirm that this is a Kubeflow-related issue.
• I am reporting this in the appropriate repository.
• I have followed the Kubeflow installation guidelines.
• The issue report is detailed and includes version numbers where applicable.
• This issue pertains to Kubeflow development.
• I am available to work on this issue.
• You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

Version

master

Detailed Description

DckerHub seems to beending unauthenticated pulls from March 1, 2025.

We probably need to migrate all docker images to GHCR as soon as possible, possibly before 1.10 final is cut.

https://docs.docker.com/docker-hub/usage/pulls/

Steps to Reproduce

Pull too many images

Screenshots or Videos (Optional)

No response

[juliusvonkohout]

• KFP
• Katib
• Manifests/platform
• trainer
• kserve
• model registry

Some of them are already on GHCR according to the maintainers.

https://github.com/kubeflow/manifests/blob/master/hack/trivy_scan.py can give us all images. I'm each commit to master

@rimolive @tarekabouzeid

Here is the list

busybox:1.28
docker.io/istio/pilot:1.24.2
docker.io/istio/proxyv2:1.24.2
docker.io/kubeflow/model-registry-ui:v0.2.14
docker.io/kubeflowkatib/earlystopping-medianstop:v0.18.0-rc.0
docker.io/kubeflowkatib/enas-cnn-cifar10-cpu:v0.18.0-rc.0
docker.io/kubeflowkatib/file-metrics-collector:v0.18.0-rc.0
docker.io/kubeflowkatib/katib-controller:v0.18.0-rc.0
docker.io/kubeflowkatib/katib-db-manager:v0.18.0-rc.0
docker.io/kubeflowkatib/katib-ui:v0.18.0-rc.0
docker.io/kubeflowkatib/pytorch-mnist-cpu:v0.18.0-rc.0
docker.io/kubeflowkatib/suggestion-darts:v0.18.0-rc.0
docker.io/kubeflowkatib/suggestion-enas:v0.18.0-rc.0
docker.io/kubeflowkatib/suggestion-goptuna:v0.18.0-rc.0
docker.io/kubeflowkatib/suggestion-hyperband:v0.18.0-rc.0
docker.io/kubeflowkatib/suggestion-hyperopt:v0.18.0-rc.0
docker.io/kubeflowkatib/suggestion-optuna:v0.18.0-rc.0
docker.io/kubeflowkatib/suggestion-pbt:v0.18.0-rc.0
docker.io/kubeflowkatib/suggestion-skopt:v0.18.0-rc.0
docker.io/kubeflowkatib/tfevent-metrics-collector:v0.18.0-rc.0
docker.io/kubeflownotebookswg/centraldashboard:v1.10.0-rc.1
docker.io/kubeflownotebookswg/jupyter-web-app:v1.10.0-rc.1
docker.io/kubeflownotebookswg/kfam:v1.10.0-rc.1
docker.io/kubeflownotebookswg/notebook-controller:v1.10.0-rc.1
docker.io/kubeflownotebookswg/poddefaults-webhook:v1.10.0-rc.1
docker.io/kubeflownotebookswg/profile-controller:v1.10.0-rc.1
docker.io/kubeflownotebookswg/pvcviewer-controller:v1.10.0-rc.1
docker.io/kubeflownotebookswg/tensorboard-controller:v1.10.0-rc.1
docker.io/kubeflownotebookswg/tensorboards-web-app:v1.10.0-rc.1
docker.io/kubeflownotebookswg/volumes-web-app:v1.10.0-rc.1
docker.io/seldonio/mlserver:1.5.0
gcr.io/cloudsql-docker/gce-proxy:1.25.0
gcr.io/knative-releases/knative.dev/net-istio/cmd/controller@sha256:e70bc675f97778da144157f125b3001124ba7a5903b85dab9e77776352fea1c7
gcr.io/knative-releases/knative.dev/net-istio/cmd/webhook@sha256:7d76a6d42d139ed53aae3ca2dfd600b1c776eb85a17af64dd1b604176a4b132a
gcr.io/knative-releases/knative.dev/serving/cmd/activator@sha256:24c19cbee078925b91cd2e85082b581d53b218b410c083b1005dc06dc549b1d3
gcr.io/knative-releases/knative.dev/serving/cmd/autoscaler@sha256:5e9236452d89363957d4e7e249d57740a8fcd946aed23f8518d94962bf440250
gcr.io/knative-releases/knative.dev/serving/cmd/controller@sha256:5fb22b052e6bc98a1a6bbb68c0282ddb50744702acee6d83110302bc990666e9
gcr.io/knative-releases/knative.dev/serving/cmd/queue@sha256:c61042001b1f21c5d06bdee9b42b5e4524e4370e09d4f46347226f06db29ba0f
gcr.io/knative-releases/knative.dev/serving/cmd/webhook@sha256:0fb5a4245aa4737d443658754464cd0a076de959fe14623fb9e9d31318ccce24
gcr.io/ml-pipeline/application-crd-controller:20231101
gcr.io/ml-pipeline/minio:RELEASE.2019-08-14T20-37-41Z-license-compliance
gcr.io/ml-pipeline/mysql:8.0.26
gcr.io/ml-pipeline/workflow-controller:v3.4.17-license-compliance
gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/controller:v0.53.2@sha256:2cab05747826e7c32e2c588f0fefd354e03f643bd33dbe20533eada00562e6b1
gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/events:v0.53.2@sha256:0cf6f0be5319efdd8909ed8f987837d89146fd0632a744bf6d54bf83e5b13ca0
gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/resolvers:v0.53.2@sha256:6578d145acd9cd288e501023429439334de15de8bd77af132c57a1d5f982e940
gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/webhook:v0.53.2@sha256:1e8f8be3b51be378747b4589dde970582f50e1e69f59527f0a9aa7a75c5833e3
gcr.io/tfx-oss-public/ml_metadata_store_server:1.14.0
ghcr.io/dexidp/dex:v2.41.1
ghcr.io/kubeflow/kfp-api-server:2.4.0
ghcr.io/kubeflow/kfp-cache-deployer:2.4.0
ghcr.io/kubeflow/kfp-cache-server:2.4.0
ghcr.io/kubeflow/kfp-frontend:2.4.0
ghcr.io/kubeflow/kfp-inverse-proxy-agent:2.4.0
ghcr.io/kubeflow/kfp-metadata-envoy:2.4.0
ghcr.io/kubeflow/kfp-metadata-writer:2.4.0
ghcr.io/kubeflow/kfp-persistence-agent:2.4.0
ghcr.io/kubeflow/kfp-scheduled-workflow-controller:2.4.0
ghcr.io/kubeflow/kfp-viewer-crd-controller:2.4.0
ghcr.io/kubeflow/kfp-visualization-server:2.4.0
ghcr.io/metacontroller/metacontroller:v2.6.1
kserve/huggingfaceserver:v0.14.1
kserve/kserve-controller:v0.14.1
kserve/kserve-localmodel-controller:v0.14.1
kserve/kserve-localmodelnode-agent:v0.14.1
kserve/lgbserver:v0.14.1
kserve/models-web-app:v0.14.0-rc.0
kserve/paddleserver:v0.14.1
kserve/pmmlserver:v0.14.1
kserve/sklearnserver:v0.14.1
kserve/storage-initializer:v0.14.1
kserve/xgbserver:v0.14.1
kubeflow/model-registry-storage-initializer:latest
kubeflow/model-registry:v0.2.14
kubeflow/training-operator:v1-5170a36
mysql:8.0.29
mysql:8.0.3
mysql:8.0.39
nvcr.io/nvidia/tritonserver:23.05-py3
postgres:14.5-alpine
postgres:14.7-alpine3.17
python:3.9
pytorch/torchserve-kfs:0.9.0
quay.io/aipipeline/pipelineloop-controller:1.9.2
quay.io/aipipeline/pipelineloop-webhook:1.9.2
quay.io/aipipeline/tekton-exithandler-controller:2.0.5
quay.io/aipipeline/tekton-exithandler-webhook:2.0.5
quay.io/aipipeline/tekton-kfptask-controller:2.0.5
quay.io/aipipeline/tekton-kfptask-webhook:2.0.5
quay.io/brancz/kube-rbac-proxy:v0.13.1
quay.io/brancz/kube-rbac-proxy:v0.18.0
quay.io/brancz/kube-rbac-proxy:v0.8.0
tensorflow/serving:2.6.2

[juliusvonkohout]

I checked the manifests manually for OCI images hosted on Dockerhub that will then probably break for many users in March

• Istio: image: busybox:1.28 we should use registry.k8s.io/busybox as KFP does @juliusvonkohout
• Istio: docker.io/istio/proxyv2:1.24.2 and docker.io/istio/pilot:1.24.2 We can probaly use https://console.cloud.google.com/artifacts/docker/istio-release and https://console.cloud.google.com/artifacts/docker/istio-release/us/gcr.io/pilot?inv=1&invt=Abqa_w via istioctl profile dump default --set global.hub=gcr.io/istio-release > profile.yaml in https://github.com/kubeflow/manifests/blob/master/common/istio-1-24/README.md and the CNI version @juliusvonkohout @tarekabouzeid @akagami-harsh
• python:3.9 for PPC should anyway be updated to 3.12 in KFP multitenancy Chore: dockerhub introduces a stricter rate limit pipelines#11664 @juliusvonkohout @hbelmiro @HumairAK
• all kubeflownotebookswg/ images @thesuperzapper
• tensorboard tensorflow/tensorflow:2.5.1 @thesuperzapper
• VOLUME_VIEWER_IMAGE filebrowser/filebrowser:v2.25.0 @thesuperzapper
• docker.io/kubeflowkatib/ images and mysql:8.0.29 @andreyvelich @Electronic-Waste
• kserve kserve/ images and docker.io/seldonio/mlserver, tensorflow/serving, pytorch/torchserve-kfs @yuzisun
• kubeflow/training-operator:v1-5170a36 @andreyvelich @Electronic-Waste

@kubeflow/kubeflow-steering-committee