Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement authentication client #4

Merged
merged 49 commits into from
Feb 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
66e0332
Add first version of the client auth for sdk
TiagoOpenCosmos Jan 22, 2025
21d0ee3
Run ruff format and ruff check fix
TiagoOpenCosmos Jan 22, 2025
1efe6ed
Make some changes regarding code formatting
TiagoOpenCosmos Jan 22, 2025
0f1479f
Add some unit tests to datacosmos client
TiagoOpenCosmos Jan 23, 2025
df656da
Add more unit tests
TiagoOpenCosmos Jan 24, 2025
d13ce9b
Start fixing some of the issues reported by the pipeline
TiagoOpenCosmos Jan 24, 2025
319536e
Apply black
TiagoOpenCosmos Jan 24, 2025
e1bee71
Revert changes in pyproject.toml
TiagoOpenCosmos Jan 24, 2025
1e40bd6
Apply pydocstyle
TiagoOpenCosmos Jan 24, 2025
0f7da25
Apply changes to workflow file
TiagoOpenCosmos Jan 27, 2025
03fe4d5
Attempt to fix line too long error
TiagoOpenCosmos Jan 27, 2025
311b789
another attempt to fix line too long error
TiagoOpenCosmos Jan 27, 2025
eb2d7c6
rollback changes
TiagoOpenCosmos Jan 27, 2025
424f914
Make bandit ignore B105
TiagoOpenCosmos Jan 27, 2025
dcaed8f
Another attempt to make bandit ignore B105
TiagoOpenCosmos Jan 27, 2025
d9c207b
Make bandit skip more unrelevant check
TiagoOpenCosmos Jan 27, 2025
dcfce90
Fix syntax main.yaml
TiagoOpenCosmos Jan 27, 2025
57d3c7a
Attempt to fix line too long for client.py
TiagoOpenCosmos Jan 27, 2025
2dc23a5
Attempt to exclude files from flake8
TiagoOpenCosmos Jan 27, 2025
0051dc3
Fix syntax in pyproject.toml
TiagoOpenCosmos Jan 27, 2025
2c0ac0a
Another attempt to exclude dirs from flake8
TiagoOpenCosmos Jan 27, 2025
b3326a0
Another attempt to exclude dirs from flake8
TiagoOpenCosmos Jan 27, 2025
5da1516
Another attempt to exclude dirs from flake8
TiagoOpenCosmos Jan 27, 2025
c20be93
Make flake8 not deal with line too long errors
TiagoOpenCosmos Jan 27, 2025
ee823b6
Attempt to make flake8 run from main.yaml taking in consideration pyp…
TiagoOpenCosmos Jan 27, 2025
bb372d9
Another attempt to make flake8 ignore E501
TiagoOpenCosmos Jan 27, 2025
34d9c14
Attempt to make flake8 job ignore E501
TiagoOpenCosmos Jan 27, 2025
3bf0038
Attempt to make flake8 job ignore E501
TiagoOpenCosmos Jan 27, 2025
e1bbed8
Attempt to make flake8 job ignore E501
TiagoOpenCosmos Jan 27, 2025
cbde259
Add requests package to dependencies
TiagoOpenCosmos Jan 27, 2025
101e186
Add oauthlib to the dependencies
TiagoOpenCosmos Jan 27, 2025
a7e17b5
Add requests-oauthlib to dependencies
TiagoOpenCosmos Jan 27, 2025
c625a54
Refactor of authentication client
TiagoOpenCosmos Jan 31, 2025
1464484
Add new line in end of files
TiagoOpenCosmos Jan 31, 2025
0170cd3
Add pydantic and pydantic-settings to dependencies
TiagoOpenCosmos Jan 31, 2025
2c1256e
Fix pipeline
TiagoOpenCosmos Jan 31, 2025
fcb80da
Remove hardcoded credentials from config.py
TiagoOpenCosmos Feb 4, 2025
16558a2
Allow sdk user to say what log levels he/she wants to see when using …
TiagoOpenCosmos Feb 4, 2025
6b355f5
remove useless comment from m2m_authentication_config model
TiagoOpenCosmos Feb 4, 2025
7a3c8b5
Fix CI linting problems
TiagoOpenCosmos Feb 4, 2025
7aafba3
Apply black
TiagoOpenCosmos Feb 4, 2025
2808200
Address comments in MR
TiagoOpenCosmos Feb 5, 2025
cc80330
Apply isort
TiagoOpenCosmos Feb 5, 2025
84ea9da
Apply changes in the pipeline
TiagoOpenCosmos Feb 5, 2025
982197d
Add changes in pyproject.toml to make isort behave similarly to black
TiagoOpenCosmos Feb 5, 2025
d26e181
Allow default values for type, token_url and audience
TiagoOpenCosmos Feb 5, 2025
84e077f
decrease cognitive load
TiagoOpenCosmos Feb 5, 2025
5e2135c
Attempt to decrease cognitive complexity even more
TiagoOpenCosmos Feb 5, 2025
9e3d7d1
use prod values as default values; remove logging config
TiagoOpenCosmos Feb 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[flake8]
ignore = E501
88 changes: 51 additions & 37 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,93 +15,107 @@ jobs:
steps:
- uses: actions/checkout@v3
- name: set up python
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: "3.10"
- name: set up poetry
run: curl -sSL https://install.python-poetry.org | python3 -
- name: configure gitlab auth
run: poetry config http-basic.gitlab __token__ ${{ secrets.GITLAB_PYPI_TOKEN }}
- name: install dependencies
run: pip install .[test]
- name: lint
uses: wearerequired/lint-action@v2
with:
continue_on_error: false
black: true
flake8: true
run: poetry install --no-interaction
- name: run isort before black
run: |
poetry run isort . --check-only # Run isort first
poetry run black . --check # Then black
poetry run flake8 .

bandit:
name: bandit
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: set up python
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: "3.10"
- name: set up poetry
run: curl -sSL https://install.python-poetry.org | python3 -
- name: configure gitlab auth
run: poetry config http-basic.gitlab __token__ ${{ secrets.GITLAB_PYPI_TOKEN }}
- name: install dependencies
run: pip install .[test]
run: poetry install --no-interaction
- name: bandit
run: bandit -r -c pyproject.toml .
run: poetry run bandit -r -c pyproject.toml . --skip B105,B106,B101

cognitive:
name: cognitive
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: set up python
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: "3.10"
- name: set up poetry
run: curl -sSL https://install.python-poetry.org | python3 -
- name: configure gitlab auth
run: poetry config http-basic.gitlab __token__ ${{ secrets.GITLAB_PYPI_TOKEN }}
- name: install dependencies
run: pip install .[test]
run: poetry install --no-interaction
- name: cognitive
run: flake8 . --max-cognitive-complexity=5
isort:
name: isort
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: set up python
uses: actions/setup-python@v1
with:
python-version: "3.10"
- name: install dependencies
run: pip install .[test]
- name: isort
uses: isort/isort-action@v1.1.0
run: poetry run flake8 . --max-cognitive-complexity=5 --ignore=E501

pydocstyle:
name: pydocstyle
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: set up python
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: "3.10"
- name: set up poetry
run: curl -sSL https://install.python-poetry.org | python3 -
- name: configure gitlab auth
run: poetry config http-basic.gitlab __token__ ${{ secrets.GITLAB_PYPI_TOKEN }}
- name: install dependencies
run: pip install .[test]
run: poetry install --no-interaction
- name: pydocstyle
run: pydocstyle .
run: poetry run pydocstyle .

test:
name: test
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
needs: [bandit, cognitive, isort, lint, pydocstyle]
needs: [bandit, cognitive, lint, pydocstyle]
steps:
- uses: actions/checkout@v3
- name: set up python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
python-version: "3.10"
- name: set up poetry
run: curl -sSL https://install.python-poetry.org | python3 -
- name: configure gitlab auth
run: poetry config http-basic.gitlab __token__ ${{ secrets.GITLAB_PYPI_TOKEN }}
- name: install dependencies
run: pip install .[test]
run: poetry install --no-interaction
- name: test
run: python -m pytest
run: poetry run pytest

release:
name: tag, changelog, release, publish
runs-on: ubuntu-latest
needs: test
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v3
- name: set up poetry
run: curl -sSL https://install.python-poetry.org | python3 -
- name: configure gitlab auth
run: poetry config http-basic.gitlab __token__ ${{ secrets.GITLAB_PYPI_TOKEN }}
- name: install dependencies
run: poetry install --no-interaction
- name: version
uses: paulhatch/semantic-version@v5.0.0
id: version
Expand Down
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -132,3 +132,9 @@ dmypy.json

# Cython debug symbols
cython_debug/

# Ignore config.yaml
config/config.yaml

# Ignore .vscode
.vscode/
10 changes: 10 additions & 0 deletions config/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
"""Configuration package for the Datacosmos SDK.

This package includes modules for loading and managing authentication
configurations.
"""

# Expose Config class for easier imports
from .config import Config

__all__ = ["Config"]
166 changes: 166 additions & 0 deletions config/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
"""Configuration module for the Datacosmos SDK.

Handles configuration management using Pydantic and Pydantic Settings.
It loads default values, allows overrides via YAML configuration files,
and supports environment variable-based overrides.
"""

import os
from typing import ClassVar, Optional

import yaml
from pydantic import field_validator
from pydantic_settings import BaseSettings, SettingsConfigDict

from config.models.m2m_authentication_config import M2MAuthenticationConfig
from config.models.url import URL


class Config(BaseSettings):
"""Centralized configuration for the Datacosmos SDK."""

model_config = SettingsConfigDict(
env_nested_delimiter="__",
nested_model_default_partial_update=True,
)

authentication: Optional[M2MAuthenticationConfig] = None
stac: Optional[URL] = None

DEFAULT_AUTH_TYPE: ClassVar[str] = "m2m"
DEFAULT_AUTH_TOKEN_URL: ClassVar[str] = "https://login.open-cosmos.com/oauth/token"
DEFAULT_AUTH_AUDIENCE: ClassVar[str] = "https://beeapp.open-cosmos.com"

@classmethod
def from_yaml(cls, file_path: str = "config/config.yaml") -> "Config":
"""Load configuration from a YAML file and override defaults.

Args:
file_path (str): The path to the YAML configuration file.

Returns:
Config: An instance of the Config class with loaded settings.
"""
config_data: dict = {}
if os.path.exists(file_path):
with open(file_path, "r") as f:
yaml_data = yaml.safe_load(f) or {}
# Remove empty values from YAML to avoid overwriting with `None`
config_data = {
key: value
for key, value in yaml_data.items()
if value not in [None, ""]
}

return cls(**config_data)

@classmethod
def from_env(cls) -> "Config":
"""Load configuration from environment variables.

Returns:
Config: An instance of the Config class with settings loaded from environment variables.
"""
authentication_config = M2MAuthenticationConfig(
type=os.getenv("OC_AUTH_TYPE", cls.DEFAULT_AUTH_TYPE),
client_id=os.getenv("OC_AUTH_CLIENT_ID"),
client_secret=os.getenv("OC_AUTH_CLIENT_SECRET"),
token_url=os.getenv("OC_AUTH_TOKEN_URL", cls.DEFAULT_AUTH_TOKEN_URL),
audience=os.getenv("OC_AUTH_AUDIENCE", cls.DEFAULT_AUTH_AUDIENCE),
)

stac_config = URL(
protocol=os.getenv("OC_STAC_PROTOCOL", "https"),
host=os.getenv("OC_STAC_HOST", "app.open-cosmos.com"),
port=int(os.getenv("OC_STAC_PORT", "443")),
path=os.getenv("OC_STAC_PATH", "/api/data/v0/stac"),
)

return cls(authentication=authentication_config, stac=stac_config)

@field_validator("authentication", mode="before")
@classmethod
def validate_authentication(
cls, auth_data: Optional[dict]
) -> M2MAuthenticationConfig:
"""Ensure authentication is provided and apply defaults.

Args:
auth_data (Optional[dict]): The authentication config as a dictionary.

Returns:
M2MAuthenticationConfig: The validated authentication configuration.

Raises:
ValueError: If authentication is missing or required fields are not set.
"""
if not auth_data:
cls.raise_missing_auth_error()

auth = cls.parse_auth_config(auth_data)
auth = cls.apply_auth_defaults(auth)

cls.check_required_auth_fields(auth)
return auth

@staticmethod
def raise_missing_auth_error():
"""Raise an error when authentication is missing."""
raise ValueError(
"M2M authentication is required. Provide it via:\n"
"1. Explicit instantiation (Config(authentication=...))\n"
"2. A YAML config file (config.yaml)\n"
"3. Environment variables (OC_AUTH_CLIENT_ID, OC_AUTH_CLIENT_SECRET, etc.)"
)

@staticmethod
def parse_auth_config(auth_data: dict) -> M2MAuthenticationConfig:
"""Convert dictionary input to M2MAuthenticationConfig object."""
return (
M2MAuthenticationConfig(**auth_data)
if isinstance(auth_data, dict)
else auth_data
)

@classmethod
def apply_auth_defaults(
cls, auth: M2MAuthenticationConfig
) -> M2MAuthenticationConfig:
"""Apply default authentication values if they are missing."""
auth.type = auth.type or cls.DEFAULT_AUTH_TYPE
auth.token_url = auth.token_url or cls.DEFAULT_AUTH_TOKEN_URL
auth.audience = auth.audience or cls.DEFAULT_AUTH_AUDIENCE
return auth

@staticmethod
def check_required_auth_fields(auth: M2MAuthenticationConfig):
"""Ensure required fields (client_id, client_secret) are provided."""
missing_fields = [
field
for field in ("client_id", "client_secret")
if not getattr(auth, field)
]
if missing_fields:
raise ValueError(
f"Missing required authentication fields: {', '.join(missing_fields)}"
)

@field_validator("stac", mode="before")
@classmethod
def validate_stac(cls, stac_config: Optional[URL]) -> URL:
"""Ensure STAC configuration has a default if not explicitly set.

Args:
stac_config (Optional[URL]): The STAC config to validate.

Returns:
URL: The validated STAC configuration.
"""
if stac_config is None:
return URL(
protocol="https",
host="app.open-cosmos.com",
port=443,
path="/api/data/v0/stac",
)
return stac_config
23 changes: 23 additions & 0 deletions config/models/m2m_authentication_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
"""Module for configuring machine-to-machine (M2M) authentication.

Used when running scripts in the cluster that require automated authentication
without user interaction.
"""

from typing import Literal

from pydantic import BaseModel


class M2MAuthenticationConfig(BaseModel):
"""Configuration for machine-to-machine authentication.

This is used when running scripts in the cluster that require authentication
with client credentials.
"""

type: Literal["m2m"]
client_id: str
token_url: str
audience: str
client_secret: str
34 changes: 34 additions & 0 deletions config/models/url.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
"""Module defining a structured URL configuration model.

Ensures that URLs contain required components such as protocol, host,
port, and path.
"""

from common.domain.url import URL as DomainURL
from pydantic import BaseModel


class URL(BaseModel):
"""Generic configuration model for a URL.

This class provides attributes to store URL components and a method
to convert them into a `DomainURL` instance.
"""

protocol: str
host: str
port: int
path: str

def as_domain_url(self) -> DomainURL:
"""Convert the URL instance to a `DomainURL` object.

Returns:
DomainURL: A domain-specific URL object.
"""
return DomainURL(
protocol=self.protocol,
host=self.host,
port=self.port,
base=self.path,
)
Loading