Skip to content

Commit

Permalink
Make datacosmos sdk not rely on private external packages; update rea…
Browse files Browse the repository at this point in the history
…dme file
  • Loading branch information
TiagoOpenCosmos committed Feb 14, 2025
1 parent eb36654 commit a3d8781
Show file tree
Hide file tree
Showing 13 changed files with 201 additions and 1,059 deletions.
36 changes: 12 additions & 24 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install GEOS
run: sudo apt update && sudo apt install libgeos-dev
- name: Install uv
run: pip install uv
- name: Set up uv environment
run: uv venv
- name: Install dependencies with GitLab token
run: uv pip install --index-url "https://__token__:${{ secrets.GITLAB_PYPI_TOKEN }}@git.o-c.space/api/v4/projects/689/packages/pypi/simple" .[dev]
- name: Install dependencies
run: uv pip install -r pyproject.toml .[dev]
- name: Activate virtual environment & run linters
run: |
source .venv/bin/activate
Expand All @@ -33,14 +31,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install GEOS
run: sudo apt update && sudo apt install libgeos-dev
- name: Install uv
run: pip install uv
- name: Set up uv environment
run: uv venv
- name: Install dependencies with GitLab token
run: uv pip install --index-url "https://__token__:${{ secrets.GITLAB_PYPI_TOKEN }}@git.o-c.space/api/v4/projects/689/packages/pypi/simple" .[dev]
- name: Install dependencies
run: uv pip install -r pyproject.toml .[dev]
- name: Activate virtual environment & run bandit
run: |
source .venv/bin/activate
Expand All @@ -51,14 +47,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install GEOS
run: sudo apt update && sudo apt install libgeos-dev
- name: Install uv
run: pip install uv
- name: Set up uv environment
run: uv venv
- name: Install dependencies with GitLab token
run: uv pip install --index-url "https://__token__:${{ secrets.GITLAB_PYPI_TOKEN }}@git.o-c.space/api/v4/projects/689/packages/pypi/simple" .[dev]
- name: Install dependencies
run: uv pip install -r pyproject.toml .[dev]
- name: Activate virtual environment & run ruff
run: |
source .venv/bin/activate
Expand All @@ -69,14 +63,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install GEOS
run: sudo apt update && sudo apt install libgeos-dev
- name: Install uv
run: pip install uv
- name: Set up uv environment
run: uv venv
- name: Install dependencies with GitLab token
run: uv pip install --index-url "https://__token__:${{ secrets.GITLAB_PYPI_TOKEN }}@git.o-c.space/api/v4/projects/689/packages/pypi/simple" .[dev]
- name: Install dependencies
run: uv pip install -r pyproject.toml .[dev]
- name: Activate virtual environment & run pydocstyle
run: |
source .venv/bin/activate
Expand All @@ -88,14 +80,12 @@ jobs:
needs: [bandit, cognitive, lint, pydocstyle]
steps:
- uses: actions/checkout@v3
- name: Install GEOS
run: sudo apt update && sudo apt install libgeos-dev
- name: Install uv
run: pip install uv
- name: Set up uv environment
run: uv venv
- name: Install dependencies with GitLab token
run: uv pip install --index-url "https://__token__:${{ secrets.GITLAB_PYPI_TOKEN }}@git.o-c.space/api/v4/projects/689/packages/pypi/simple" .[dev]
- name: Install dependencies
run: uv pip install -r pyproject.toml .[dev]
- name: Activate virtual environment & run tests
run: |
source .venv/bin/activate
Expand All @@ -108,14 +98,12 @@ jobs:
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v3
- name: Install GEOS
run: sudo apt update && sudo apt install libgeos-dev
- name: Install uv
run: pip install uv
- name: Set up uv environment
run: uv venv
- name: Install dependencies with GitLab token
run: uv pip install --index-url "https://__token__:${{ secrets.GITLAB_PYPI_TOKEN }}@git.o-c.space/api/v4/projects/689/packages/pypi/simple" .
- name: Install dependencies
run: uv pip install -r pyproject.toml
- name: Version
uses: paulhatch/semantic-version@v5.0.0
id: version
Expand Down
72 changes: 24 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,21 @@
# DataCosmos SDK

## Overview
The **DataCosmos SDK** allows Open Cosmos' customers to interact with the main APIs required for creating applications that integrate with **DataCosmos**. This SDK provides authentication handling, HTTP request utilities, and a client for interacting with the **STAC API** (SpatioTemporal Asset Catalog).
The **DataCosmos SDK** allows Open Cosmos' customers to interact with the **DataCosmos APIs** for seamless data management and retrieval. It provides authentication handling, HTTP request utilities, and a client for interacting with the **STAC API** (SpatioTemporal Asset Catalog).

## Installation

### Prerequisites
The SDK uses **uv** as a dependency manager. To install it:
### Install via PyPI
The easiest way to install the SDK is via **pip**:

```sh
pip install uv
```

### Setting Up the SDK
To install the SDK and its dependencies:

```sh
uv venv
uv pip install -r pyproject.toml
uv pip install -r pyproject.toml .[dev]
source .venv/bin/activate
pip install datacosmos
```

## Authentication
The SDK requires authentication via a token stored in a `.netrc` file.

### Setting Up Authentication
Create the `.netrc` file:
## Getting Started

```sh
touch ~/.netrc
```

Add the following content:

```
machine git.o-c.space
login __token__
password {provided_token}
```

You will need to **request** the token from Open Cosmos.

### Configuration Methods
To instantiate the `DatacosmosClient`, it is **recommended** to pass a `Config` object directly:
### Initializing the Client
The recommended way to initialize the SDK is by passing a `Config` object with authentication credentials:

```python
from datacosmos.client import DatacosmosClient
Expand All @@ -64,19 +36,8 @@ Alternatively, the SDK can load configuration automatically from:
- A YAML file (`config/config.yaml`)
- Environment variables

## Using the SDK

### Initializing the Client
The SDK provides the `DatacosmosClient` class to manage authentication and interact with the API.

```python
from datacosmos.client import DatacosmosClient

client = DatacosmosClient()
```

### STAC Client
The **STACClient** allows interaction with the STAC API, enabling search, retrieval, creation, updating, and deletion of STAC items.
The **STACClient** enables interaction with the STAC API, allowing for searching, retrieving, creating, updating, and deleting STAC items.

#### Initialize STACClient

Expand Down Expand Up @@ -167,12 +128,28 @@ stac_client.update_item(item_id="new-item", collection_id="example-collection",
stac_client.delete_item(item_id="new-item", collection_id="example-collection")
```

## Configuration Options
- **Recommended:** Instantiate `DatacosmosClient` with a `Config` object.
- Alternatively, use **YAML files** (`config/config.yaml`).
- Or, use **environment variables**.

## Contributing
If you would like to contribute:
1. Fork the repository.
2. Create a feature branch.
3. Submit a pull request.

### Development Setup
If you are developing the SDK, you can use `uv` for dependency management:

```sh
pip install uv
uv venv
uv pip install -r pyproject.toml
uv pip install -r pyproject.toml .[dev]
source .venv/bin/activate
```

Before making changes, ensure that:
- The code is formatted using **Black** and **isort**.
- Static analysis and linting are performed using **ruff** and **pydocstyle**.
Expand All @@ -187,4 +164,3 @@ pydocstyle .
bandit -r -c pyproject.toml . --skip B105,B106,B101
pytest
```

3 changes: 2 additions & 1 deletion config/models/url.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@
port, and path.
"""

from common.domain.url import URL as DomainURL
from pydantic import BaseModel

from datacosmos.utils.url import URL as DomainURL


class URL(BaseModel):
"""Generic configuration model for a URL.
Expand Down
12 changes: 8 additions & 4 deletions datacosmos/stac/stac_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,13 @@

from typing import Generator, Optional

from common.sdk.http_response import InvalidRequest, check_api_response
from pystac import Item

from datacosmos.client import DatacosmosClient
from datacosmos.exceptions.datacosmos_exception import DatacosmosException
from datacosmos.stac.models.item_update import ItemUpdate
from datacosmos.stac.models.search_parameters import SearchParameters
from datacosmos.utils.http_response import check_api_response


class STACClient:
Expand Down Expand Up @@ -173,9 +174,12 @@ def _extract_pagination_token(self, next_href: str) -> Optional[str]:
Optional[str]: The extracted token, or None if parsing fails.
Raises:
InvalidRequest: If pagination token extraction fails.
DatacosmosException: If pagination token extraction fails.
"""
try:
return next_href.split("?")[1].split("=")[-1]
except (IndexError, AttributeError):
raise InvalidRequest(f"Failed to parse pagination token from {next_href}")
except (IndexError, AttributeError) as e:
raise DatacosmosException(
f"Failed to parse pagination token from {next_href}",
response=e.response,
) from e
1 change: 1 addition & 0 deletions datacosmos/utils/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Http response and url utils for datacosmos."""
1 change: 1 addition & 0 deletions datacosmos/utils/http_response/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Validates an API response."""
34 changes: 34 additions & 0 deletions datacosmos/utils/http_response/check_api_response.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
"""Validates an API response and raises a DatacosmosException if an error occurs."""

from pydantic import ValidationError
from requests import Response

from datacosmos.exceptions.datacosmos_exception import DatacosmosException
from datacosmos.utils.http_response.models.datacosmos_response import DatacosmosResponse


def check_api_response(response: Response) -> None:
"""Validates an API response and raises a DatacosmosException if an error occurs.
Args:
resp (requests.Response): The response object.
Raises:
DatacosmosException: If the response status code indicates an error.
"""
if 200 <= response.status_code < 400:
return

try:
response = DatacosmosResponse.model_validate_json(response.text)
msg = response.errors[0].human_readable()
if len(response.errors) > 1:
msg = "\n * " + "\n * ".join(
error.human_readable() for error in response.errors
)
raise DatacosmosException(msg, response=response)

except ValidationError:
raise DatacosmosException(
f"HTTP {response.status_code}: {response.text}", response=response
)
1 change: 1 addition & 0 deletions datacosmos/utils/http_response/models/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Models for validation of API response."""
26 changes: 26 additions & 0 deletions datacosmos/utils/http_response/models/datacosmos_error.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
"""Structured API error message for Datacosmos."""

from pydantic import BaseModel


class DatacosmosError(BaseModel):
"""Structured API error message for Datacosmos."""

message: str
field: str | None = None
type: str | None = None
source: str | None = None
trace_id: str | None = None

def human_readable(self) -> str:
"""Formats the error message into a readable format."""
msg = self.message
if self.type:
msg += f" (type: {self.type})"
if self.field:
msg += f" (field: {self.field})"
if self.source:
msg += f" (source: {self.source})"
if self.trace_id:
msg += f" (trace_id: {self.trace_id})"
return msg
11 changes: 11 additions & 0 deletions datacosmos/utils/http_response/models/datacosmos_response.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
"""Structured response for Datacosmos handling multiple API errors."""

from pydantic import BaseModel

from datacosmos.utils.http_response.models.datacosmos_error import DatacosmosError


class DatacosmosResponse(BaseModel):
"""Structured response for Datacosmos handling multiple API errors."""

errors: list[DatacosmosError]
37 changes: 37 additions & 0 deletions datacosmos/utils/url.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
"""URL utility class for building and handling URLs in the SDK."""


class URL:
"""Class to represent and build URLs in a convenient way."""

def __init__(self, protocol: str, host: str, port: int, base: str):
"""Creates a new basis to build URLs.
Args:
protocol (str): Protocol to use in the URL (http/https).
host (str): Hostname (e.g., example.com).
port (int): Port number.
base (str): Base path (e.g., /api/v1).
"""
self.protocol = protocol
self.host = host
self.port = port
self.base = base

def string(self) -> str:
"""Returns the full URL as a string."""
port = "" if self.port in [80, 443] else f":{self.port}"
base = f"/{self.base.lstrip('/')}" if self.base else ""
return f"{self.protocol}://{self.host}{port}{base}"

def with_suffix(self, suffix: str) -> str:
"""Appends a suffix to the URL, ensuring proper formatting.
Args:
suffix (str): The path to append.
Returns:
str: Full URL with the suffix.
"""
base = self.string()
return f"{base.rstrip('/')}/{suffix.lstrip('/')}"
8 changes: 0 additions & 8 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ classifiers = [
]
dependencies = [
"pydantic_settings>=2.7.0",
"python-common==1.4.0",
"requests==2.31.0",
"oauthlib==3.2.0",
"requests-oauthlib==1.3.1",
Expand Down Expand Up @@ -51,10 +50,3 @@ include_trailing_comma = true
force_grid_wrap = 0
use_parentheses = true
line_length = 88

[tool.uv.sources]
python-common = { index = "gitlab" }

[[tool.uv.index]]
name = "gitlab"
url = "https://git.o-c.space/api/v4/projects/689/packages/pypi/simple"
Loading

0 comments on commit a3d8781

Please sign in to comment.