Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Key cloak User Management #378

Open
wants to merge 45 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
781c973
add token, register, ban, unban
eelcovdw Oct 8, 2024
6a9addd
merge dev
eelcovdw Oct 9, 2024
1756718
add email
eelcovdw Oct 9, 2024
aa8c8c0
merge settings
eelcovdw Oct 9, 2024
7f8f2de
update endpoint, use settings
eelcovdw Oct 9, 2024
9fc970d
add jwt auth
eelcovdw Oct 9, 2024
a9b5ccf
save progress
teo-milea Nov 8, 2024
b44fabb
syftbox/
teo-milea Nov 12, 2024
11c0272
Merge branch 'main' of github.com:OpenMined/syft into eelco/user-mana…
teo-milea Nov 13, 2024
20286e7
moved some functions after merge
teo-milea Nov 13, 2024
52e2e5b
remove redundant variables
abyesilyurt Nov 13, 2024
cfd1a41
add authentication to sync endpoints
abyesilyurt Nov 13, 2024
cfb0f56
fix api tests
abyesilyurt Nov 13, 2024
81c2b9e
fix unit tests
abyesilyurt Nov 13, 2024
368015c
client sends email on every request
abyesilyurt Nov 13, 2024
ff2de73
test can run against the keycloak server
abyesilyurt Nov 13, 2024
020df53
add email verification checks
abyesilyurt Nov 13, 2024
f8fe205
refactor email verification check
abyesilyurt Nov 13, 2024
81efcc2
clean up, new ux
teo-milea Nov 13, 2024
c4b65a2
added just args and remove some old code
teo-milea Nov 13, 2024
f173915
justfile run client args
teo-milea Nov 13, 2024
b422210
save uv lock
teo-milea Nov 14, 2024
a9c344a
Merge branch 'main' of github.com:OpenMined/syft into eelco/user-mana…
teo-milea Nov 14, 2024
c8b7e12
Merge branches 'aziz/keycloak' and 'eelco/user-management' of github.…
abyesilyurt Nov 14, 2024
153ef29
fix token
teo-milea Nov 14, 2024
1bd17b9
Merge branch 'eelco/user-management' of github.com:OpenMined/syft int…
abyesilyurt Nov 14, 2024
2f2e1a2
client auth working
abyesilyurt Nov 14, 2024
b45b0f9
Merge pull request #381 from OpenMined/aziz/keycloak
abyesilyurt Nov 14, 2024
3ee03e7
remove unintentional changes
abyesilyurt Nov 14, 2024
cbf4678
fix duplicate pre-commit entry
abyesilyurt Nov 14, 2024
bb79b40
remove submodules
abyesilyurt Nov 14, 2024
25287fc
remove unused code
abyesilyurt Nov 14, 2024
d687921
remove unused code
abyesilyurt Nov 14, 2024
f5f0a1b
working register
teo-milea Nov 15, 2024
8e476d4
Merge branch 'eelco/user-management' of github.com:OpenMined/syft int…
teo-milea Nov 15, 2024
2e8e80b
fix path
teo-milea Nov 15, 2024
2a6e0c5
add password
abyesilyurt Nov 15, 2024
3d7a3b2
added scope=openied
teo-milea Nov 18, 2024
622fb3d
add token validation to client
abyesilyurt Nov 18, 2024
8a90d73
moved auth check to somewhere else
abyesilyurt Nov 18, 2024
e37e445
fix email auth
abyesilyurt Nov 18, 2024
ff9fcc5
finished ux/removed password from config file
teo-milea Nov 18, 2024
ab92a8f
Merge branch 'eelco/user-management' of github.com:OpenMined/syft int…
teo-milea Nov 18, 2024
8d1091c
fix
abyesilyurt Nov 18, 2024
bcc372b
fix register
abyesilyurt Nov 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,13 @@ run-server port="5001" uvicorn_args="":

# ---------------------------------------------------------------------------------------------------------------------

# # Reset password
[group('client')]
reset-password email server="https://syftbox.openmined.org"
uv run syftbox/client/cli.py reset-password -e email --server={{ server }}

# ---------------------------------------------------------------------------------------------------------------------

# Run a local syftbox client on any available port between 8080-9000
[group('client')]
run-client name port="auto" server="http://localhost:5001":
Expand All @@ -59,12 +66,12 @@ run-client name port="auto" server="http://localhost:5001":
DATA_DIR=.clients/$EMAIL
mkdir -p $DATA_DIR

echo -e "Email : {{ _green }}$EMAIL{{ _nc }}"
echo -e "Client : {{ _cyan }}http://localhost:$PORT{{ _nc }}"
echo -e "Server : {{ _cyan }}{{ server }}{{ _nc }}"
echo -e "Data Dir : $DATA_DIR"
echo -e "Email : {{ _green }}$EMAIL{{ _nc }}"
echo -e "Client : {{ _cyan }}http://localhost:$PORT{{ _nc }}"
echo -e "Server : {{ _cyan }}{{ server }}{{ _nc }}"
echo -e "Data Dir : $DATA_DIR"

uv run syftbox/client/cli.py --config=$DATA_DIR/config.json --data-dir=$DATA_DIR --email=$EMAIL --port=$PORT --server={{ server }} --no-open-dir
uv run syftbox/client/cli.py --config=$DATA_DIR/config.json --data-dir=$DATA_DIR --email=$EMAIL --port=$PORT --server={{ server }} --no-open-dir

# ---------------------------------------------------------------------------------------------------------------------

Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ version = "0.2.4"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.9"

# add using uv add <pip package>
dependencies = [
"fastapi>=0.114.0",
"uvicorn>=0.30.6",
Expand Down
31 changes: 31 additions & 0 deletions syftbox/client/cli.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from pathlib import Path

import requests
from rich import print as rprint
from typer import Context, Exit, Option, Typer
from typing_extensions import Annotated
Expand Down Expand Up @@ -59,6 +60,16 @@
is_flag=True,
help="Enable verbose mode",
)
REGISTER_OPTS = Option(
"--register",
is_flag=True,
help="Register new user",
)
RESET_PASS_OPTS = Option(
"--reset_password",
is_flag=True,
help="Register new user",
)

# report command opts
REPORT_PATH_OPTS = Option(
Expand Down Expand Up @@ -105,6 +116,26 @@ def client(
raise Exit(code)


def server_reset_password(email, server):
response = requests.post(f'{server}/users/reset_password_email', params={'email': email})
return response

@app.command()
def reset_password(
email: Annotated[str, EMAIL_OPTS] = None,
server: Annotated[str, SERVER_OPTS] = DEFAULT_SERVER_URL,
):
from syftbox.client.cli_setup import prompt_email
if email is None:
email = prompt_email()

resp = server_reset_password(email=email, server=server)
if resp.status_code == 200:
rprint("[bold]Email for password reset sent![/bold]")
else:
rprint(f"[red]Error[/red]: {resp.text}")


@app.command()
def report(
output_path: Annotated[Path, REPORT_PATH_OPTS] = Path(".").resolve(),
Expand Down
61 changes: 60 additions & 1 deletion syftbox/client/cli_setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,34 @@

from pathlib import Path

import httpx
import requests
from rich import print as rprint
from rich.prompt import Confirm, Prompt

from syftbox.lib.client_config import SyftClientConfig
from syftbox.lib.constants import DEFAULT_DATA_DIR
from syftbox.lib.exceptions import ClientConfigException
from syftbox.lib.keycloak import get_token
from syftbox.lib.validators import DIR_NOT_EMPTY, is_valid_dir, is_valid_email

__all__ = ["setup_config_interactive"]


def setup_config_interactive(config_path: Path, email: str, data_dir: Path, server: str, port: int) -> SyftClientConfig:
def user_exists(email, server) -> bool:
response = requests.get(f"{server}/users/check_user", params={"email": email})
print(response.text)
response.raise_for_status()
return response.text == "true"


def setup_config_interactive(
config_path: Path,
email: str,
data_dir: Path,
server: str,
port: int,
) -> SyftClientConfig:
"""Setup the client configuration interactively. Called from CLI"""

config_path = config_path.expanduser().resolve()
Expand Down Expand Up @@ -51,6 +67,35 @@ def setup_config_interactive(config_path: Path, email: str, data_dir: Path, serv
if port != conf.client_url.port:
conf.set_port(port)

if conf.access_token is None:
register = not user_exists(email, server)
pwd = register_password() if register else login_password()
if register:
payload = {
"email": email,
"password": pwd,
"firstName": "",
"lastName": "",
}
response = requests.post(f"{server}/users/register", json=payload)
response.raise_for_status()
conf.access_token = get_token(email, pwd)

while True:
response = httpx.post(
f"{conf.server_url}sync/datasites",
headers={"Authorization": f"Bearer {conf.access_token}", "email": conf.email},
)
if response.status_code == 401:
rprint("[bold red]Wrong email or password![/bold red]")
pwd = login_password()
conf.access_token = get_token(conf.email, pwd)
continue

# crash on other errors, break on success
response.raise_for_status()
break

# DO NOT SAVE THE CONFIG HERE.
# We don't know if the client will accept the config yet
return conf
Expand Down Expand Up @@ -85,3 +130,17 @@ def prompt_email() -> str:
rprint(f"[bold red]Invalid email[/bold red]: '{email}'")
continue
return email


def register_password() -> str:
while True:
password = Prompt.ask("[bold]Enter your password[/bold]", password=True)
verify_password = Prompt.ask("[bold]Verify your password[/bold]", password=True)
if password == verify_password:
break
rprint("[bold red]Passwords don't match! Please try again.[/bold red]")
return password


def login_password() -> str:
return Prompt.ask("[bold]Password[/bold]", password=True)
37 changes: 12 additions & 25 deletions syftbox/client/client2.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@
from syftbox.client.utils import error_reporting, file_manager, macos
from syftbox.lib.client_config import SyftClientConfig
from syftbox.lib.datasite import create_datasite
from syftbox.lib.exceptions import SyftBoxException
from syftbox.lib.ignore import IGNORE_FILENAME
from syftbox.lib.workspace import SyftWorkspace

Expand Down Expand Up @@ -54,7 +53,17 @@ def __init__(self, config: SyftClientConfig, log_level: str = "INFO", **kwargs):

self.workspace = SyftWorkspace(self.config.data_dir)
self.pid = PidFile(pidname="syftbox.pid", piddir=self.workspace.data_dir)
self.server_client = httpx.Client(base_url=str(self.config.server_url), follow_redirects=True)

self.server_client = httpx.Client(
base_url=str(self.config.server_url),
follow_redirects=True,
# We are sending email along with the bearer token
# to support fallback to email based authentication in case of keycloak failure
# or local development without keycloak
# To configure the server to use bypass keycloak authentication
# set the environment variable SYFTBOX_NO_AUTH=1
headers={"email": self.config.email, "Authorization": f"Bearer {self.config.access_token}"},
)

# kwargs for making customization/unit testing easier
# this will be replaced with a sophisticated plugin system
Expand Down Expand Up @@ -85,7 +94,7 @@ def app_runner(self):
@property
def is_registered(self) -> bool:
"""Check if the current user is registered with the server"""
return bool(self.config.token)
return bool(self.config.access_token)

@property
def datasite(self) -> Path:
Expand All @@ -108,7 +117,6 @@ def start(self):

self.config.save() # commit config changes (like migration) to disk after PID is created
self.workspace.mkdirs() # create the workspace directories
self.register_self() # register the email with the server
self.init_datasite() # init the datasite on local machine

# start plugins/components
Expand Down Expand Up @@ -151,21 +159,6 @@ def init_datasite(self):
return
create_datasite(self.workspace.datasites, self.config.email)

def register_self(self):
"""Register the user's email with the SyftBox cache server"""
if self.is_registered:
return
try:
token = self.__register_email()
# TODO + FIXME - once we have JWT, we should not store token in config!
# ideally in OS keychain (using keyring) or
# in a separate location under self.workspace.plugins
self.config.token = str(token)
self.config.save()
logger.info("Email registration successful")
except Exception as e:
raise SyftBoxException(f"Failed to register with the server - {e}") from e

@lru_cache(1)
def as_context(self) -> "SyftClientContext":
"""Return a implementation of SyftClientInterface to be injected into sub-systems"""
Expand All @@ -184,12 +177,6 @@ def __run_local_server(self):
)
return self.__local_server.run()

def __register_email(self) -> str:
# TODO - this should probably be wrapped in a SyftCacheServer API?
response = self.server_client.post("/register", json={"email": self.config.email})
response.raise_for_status()
return response.json().get("token")

def __enter__(self):
return self

Expand Down
5 changes: 5 additions & 0 deletions syftbox/client/env.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ class SyftEnvVars(BaseSettings):
CLIENT_CONFIG_PATH: Path = Field(default=DEFAULT_CONFIG_PATH)
"""Path to the client configuration file."""

ACCESS_TOKEN: str = Field(default="")
"""Access token for the datasite."""

KEYCLOAK_ADMIN_TOKEN: str = Field(default="")

model_config = SettingsConfigDict(env_file=".env", env_prefix="SYFTBOX_")


Expand Down
16 changes: 14 additions & 2 deletions syftbox/lib/client_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

from syftbox.lib.constants import DEFAULT_CONFIG_PATH, DEFAULT_DATA_DIR, DEFAULT_SERVER_URL
from syftbox.lib.exceptions import ClientConfigException
from syftbox.lib.keycloak import get_user_from_token
from syftbox.lib.types import PathLike, to_path

__all__ = ["SyftClientConfig"]
Expand Down Expand Up @@ -49,8 +50,13 @@ class SyftClientConfig(BaseModel):
email: EmailStr = Field(description="Email address of the user")
"""Email address of the user"""

token: Optional[str] = Field(default=None, description="API token for the user")
"""API token for the user"""
token: Optional[str] = Field(
default=None, description="Depracated: Use access_token instead. API token for the user", deprecated=True
)
"""Depracated: Use access_token instead. API token for the user"""

access_token: Optional[str] = Field(default=None, description="Access token for the user")
"""Access token for the user"""

# WARN: we don't need `path` to be serialized, hence exclude=True
path: Path = Field(exclude=True, description="Path to the config file")
Expand All @@ -62,6 +68,12 @@ def port_to_url(cls, val):
return f"http://127.0.0.1:{val}"
return val

@property
def user_id(self):
if self.access_token:
return get_user_from_token(self.access_token)["sub"]
return None

@field_validator("token", mode="before")
def token_to_str(cls, v):
if not v:
Expand Down
Loading