docs/sso/

[giscus[bot]]

docs/sso/

How to configure Single-Sign-On (SSO) in our Pi Kubernetes cluster.

https://picluster.ricsanfre.com/docs/sso/

[i5Js]

Interesting: Following it, I'm getting the following error, after login to the oauth2 console:

Error redeeming code during OAuth2 callback: token exchange failed: oauth2: "unauthorized_client" "Invalid client or Invalid client credentials"

However, if I disable "client authentication", I can login, but getting a 404 error:

10.244.7.215:42592 - 39395285-dd23-4475-b516-89a65da11707 - <user_email> [2024/03/30 14:46:19] [AuthSuccess] Authenticated via OAuth2: Session{email:user_email user:ce458eac-2a49-4a80-aa54-bb66ef545aad PreferredUsername:<user> token:true id_token:true created:2024-03-30 14:46:19.212878477 +0000 UTC m=+2760.130704669 expires:2024-03-30 14:51:19.189199437 +0000 UTC m=+3060.107025628 groups:[role:editor role:viewer role:offline_access role:admin role:uma_authorization role:default-roles-home role:account:manage-account role:account:manage-account-links role:account:view-profile]}
10.244.7.215:42592 - 39395285-dd23-4475-b516-89a65da11707 - - [2024/03/30 14:46:19] proxy.home.dominio.com GET - "/oauth2/callback?state=LCZx-8ct628slwX9Tgi6D9_vHIvK6TOWSTiuR1F7ImU%3A%2F&session_state=0d2be827-3d06-41b0-be04-7eca54adf6a7&iss=https%3A%2F%2Fsso.home.dominio.com%2Frealms%2Fhome&code=b3fb1b30-a272-4d1f-8e43-bedb35fd1680.0d2be827-3d06-41b0-be04-7eca54adf6a7.bebb575b-cd59-4822-a2fb-7d2fe70f0731" HTTP/1.1 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.15" 302 24 0.151
10.244.7.215:42592 - 16314e6e-8182-4044-9c54-33e6aca8756a - user_email[2024/03/30 14:46:19] proxy.home.dominio.com GET - "/" HTTP/1.1 "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.15" 404 19 0.002

[ricsanfre]

Client authentication has to be enabled. Otherwise, the client credentials are not created. Once the client is added to keycloak its client credentials: client_id and client_secret need to be used in the configuration: oauth2-proxy config.

[i5Js]

Hello Ricardo, thanks,

But I don't think I have understood it. I have created a user and a password in keycloak and I'm using that credentials to login to oauth2. am I doing wrong?. I mean, when I login to proxy.home.dominio.com it has for user and a password. What should I use ?

[i5Js]

Very true, I'm giving the last chance... if it doesn't work, I will move all to Nginx.

[i5Js]

Moved everything to Nginx Ingress Controller, and worked at the first shoot.
Bye bye traefik.

[i5Js]

Strange, I don't know if it's expected, but, I can login to longhorn, but, if I close the session in Keycloak, longhorn doesn't ask for a new login if I refresh it.

[ricsanfre]

Hi Javi,

Oauth2-Proxy Sign out functionality is not implemented yet. It is implemented in Grafana's SSO Keycloak integration but not in Oauth-proxy's. I have open an issue to keep track of this enhancement: (#378)

Regards
Ricardo

[i5Js]

Great, thanks Ricardo!!

[i5Js]

Yes, I saw it, it’s not encoded but “invalid credentials :(“ and it’s strange because I don’t see any error message on the pod. Enviado desde dispositivo móvil

…

________________________________ De: Ricardo Sanchez ***@***.***> Enviado: Sunday, March 31, 2024 7:05:32 PM Para: ricsanfre/pi-cluster ***@***.***> Cc: Javi ***@***.***>; Comment ***@***.***> Asunto: Re: [ricsanfre/pi-cluster] docs/sso/ (Discussion #369) If you use a Secret to store credentials (GitOps installation), you have to base64 encode them. — Reply to this email directly, view it on GitHub<#369 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAIFKG6V6YY33XAII56ETLLY3A65ZAVCNFSM6AAAAABFPTPYSCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DSNRVHE4TI>. You are receiving this because you commented.Message ID: ***@***.***>

[i5Js]

Finally I found a message on the keycloak pod

2024-03-31 18:02:12,535 WARN [org.keycloak.events] (executor-thread-77) type="LOGIN_ERROR", realmId="66b4fa57-c607-4d1f-8b48-4a0a1d72c464", clientId="oauth2-proxy", userId="null", ipAddress="10.244.2.0", error="user_not_found", auth_method="openid-connect"

Time to google it.

[benbourner]

I'm just getting endless crash loops trying to install this, it fails to connect to postgres...?
Pod error:

WARN  [io.agroal.pool] (agroal-11) Datasource '<default>': FATAL: password authentication failed for user "keycloak"

Postgres error:

2024-05-23 08:48:03.301 GMT [1681] FATAL:  password authentication failed for user "keycloak"
2024-05-23 08:48:03.301 GMT [1681] DETAIL:  Role "keycloak" does not exist.
	Connection matched file "/opt/bitnami/postgresql/conf/pg_hba.conf" line 1: "host     all             all             0.0.0.0/0               md5"

[benbourner]

Uninstalled and reinstalled and it worked ok, guess it was down to a timing issue somehow

[benbourner]

One more issue, maybe i've missed something along the line... I find the oauth-proxy is not able to resolve the external address oidc_issuer_url="https://sso.picluster.ricsanfre.com/realms/picluster". What might I have missed that enables the DNS resolution of sso.picluster.ricsanfre.com (and other ingresses) by the oath-proxy pod? Where should it be resolved? neither coredns nor dnsmasq know about it.

ERROR: Failed to initialise OAuth2 Proxy: error initialising provider: could not create provider data: error building OIDC ProviderVerifier: could not get verifier builder: error while discovery OIDC configuration: failed to discover OIDC configuration: error performing request: Get "https://sso.picluster.ricsanfre.com/realms/picluster/.well-known/openid-configuration": dial tcp: lookup sso.picluster.ricsanfre.com on 10.43.0.10:53: no such host

[ricsanfre]

Hi @benbourner, sso.picluster.ricsanfre.com dns is part of dnsmasq configuration. It is pointing to Ingress Gateway Load Balancer IP, so pods are able to resolve it and reach it.

See ansible variables defined for gateway node: https://github.com/ricsanfre/pi-cluster/blob/master/ansible/host_vars/gateway.yml#L40-L43

[benbourner]

Ah I see now yes thanks. So they have to be explicitly defined in dnsmasq, there is no mechanism for updating them on the fly from ingress definitions. I was trying to do that but had another issue - the name resolution on my gateway host was not pointing to dnsmasq, thus coredns was also not pointing to dnsmasq, thus the pods could not resolve the definitions. Thanks for the useful guide. There are problems to solve at every step but it all helps gain a deeper understanding.

[ricsanfre]

Services like external-dns can be used for automate the DNS configuration for exposed services. For simplicity, since I am exposing only a few services, i am not currently using it, so I need to manually configure dnsmasq.d.

[Elias-Hr]

I got readiness prob 500 on aouth2-proxy pod after 24h of working. Is really embarrassing since i cannot reproduce the error.
Can you help me please
Thanks for this awesome end to end tutorial

[ricsanfre]

Hi @eliohg,

Do you have the oauth2-proxy pod logs?

Regards
Ricardo

[Elias-Hr]

Thank you for your quick response.

here is the logs of Oauth2-proxy (showing this two lines inftinly):
100.64.3.252 - 324b8fef-5de5-4ceb-9408-7e54c57f9ad7 - - [2024/07/11 09:46:44] 100.64.3.89:4180 GET - "/ping" HTTP/1.1 "kube-probe/1.29" 200 2 0.000
100.64.3.252 - 5f47c73f-7076-46e1-a6e9-9564e6f688f0 - - [2024/07/11 09:46:44] 100.64.3.89:4180 GET - "/ready" HTTP/1.1 "kube-probe/1.29" 500 68 0.003

the event of pod is :
Unhealthy
Readiness probe failed: HTTP probe failed with statuscode: 500

Note that the redis pod showing no logs traffic just this message :
│ 1:M 11 Jul 2024 01:03:28.692 * Done loading RDB, keys loaded: 0, keys expired: 0. │
│ 1:M 11 Jul 2024 01:03:28.692 * DB loaded from base file appendonly.aof.1.base.rdb: 0.001 seconds │
│ 1:M 11 Jul 2024 01:03:28.696 * DB loaded from incr file appendonly.aof.1.incr.aof: 0.004 seconds │
│ 1:M 11 Jul 2024 01:03:28.696 * DB loaded from append only file: 0.005 seconds │
│ 1:M 11 Jul 2024 01:03:28.696 * Opening AOF incr file appendonly.aof.1.incr.aof on server start │
│ 1:M 11 Jul 2024 01:03:28.696 * Ready to accept connections tcp

thanks a lot

[valkiriaaquatica]

Hi,
great tutorial introduction to confgiruation of keyclack and outh2 on kubernetes!
This is my first time on this auth things.. So the problem is when I tried to place ouath before a app to get authentification (in my case i'm testing with a simple nginx app (just nginx cannonical nginx docker image), using the annotations of the service name in auth-url and ingress domain on auth-signin, nginx is skipping to redirect to the oauth2 proxy... Just debuged all kind of things, connections between pods, logs, rules... but all seems correct the problem is that I see 0 logs or interaction between the nginx controller pod and the ouath service, just requests are straight to 200, without any authorization in the middle. Also testes with differente browsers and pods , but all requests to the nginx app are 200, no redirection is needed never. :( :( :(

• nginx.ingress.kubernetes.io/auth-url: http://oauth2-proxy.test-ns.svc.cluster.local/oauth2/auth
• nginx.ingress.kubernetes.io/auth-signin: https://oauth2-proxy.picluster.ricsanfre.com/oauth2/start?rd=https://$host$request_uri

[valkiriaaquatica]

Btw oauth2 in completely accesible and up in https://oauth2-proxy.picluster.ricsanfre.com/oauth2/ so the problem should be in why nginx is not redirecting to oauth2 (my ngnx controller is the last of heml chrart)

[valkiriaaquatica]

Thnik I solved, network policy cilium error

[saashqdev]

This is the only Oauth2-proxy configuration that worked for me. Great write up. Do you have anything that redirects you to K8s dashboard? (or would you be willing to add to this article)...Cheers, Dave

[ricsanfre]

Thanks Dave
I suppose you refer to Kubernetes dashboard. I am not using it. I use FluxCD to deploy any workload in the cluster and Prometheus/Grafana dashboards for monitoring.

[saashqdev]

Ah OK, thanks. FluxCD is nice. I use it as well. I've been noodling around with multi-tenancy in K3s and I thought I'd give the k8s dashboard a try with multi-tenant stuff. Basically, a user hitting the dashboard url, getting redirected to keycloak for auth then off to the dashboard itself.