Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core/cli): add cli for interacting with fabric api #37

Merged
merged 7 commits into from
Dec 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions docs/core/fabric-api.md → docs/core/fabric-api/index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
# Fabric API

A collection of functions to interact with the Fabric API. Automatically handles pagination and authentication.
The functions can either be called with an `id` or `name` parameter, however it is recommended to use the `id` as using the name requires more API requests, and is thus slower.

::: msfabricutils.core.workspace
::: msfabricutils.core.lakehouse
::: msfabricutils.core.sql_endpoint
The functions can either be called with an `id` or `name` parameter, however it is recommended to use the `id` as using the name requires more API requests, and is thus slower.
3 changes: 3 additions & 0 deletions docs/core/fabric-api/lakehouse.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Lakehouse

::: msfabricutils.core.lakehouse
3 changes: 3 additions & 0 deletions docs/core/fabric-api/notebook.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Notebook

::: msfabricutils.core.notebook
3 changes: 3 additions & 0 deletions docs/core/fabric-api/sql_endpoint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# SQL Endpoint

::: msfabricutils.core.sql_endpoint
3 changes: 3 additions & 0 deletions docs/core/fabric-api/workspace.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Workspace

::: msfabricutils.core.workspace
26 changes: 26 additions & 0 deletions docs/usage/cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Command Line Interface

The CLI is a way to interact with the Microsoft Fabric REST API. It includes commands for workspaces, lakehouses, and notebooks creation, deletion and updating.

For complete documentation, run `msfu --help`.

## Examples

### Workspace

```bash
msfu workspace create --name "My Workspace" --description "My workspace description"
```

### Lakehouse

```bash
msfu lakehouse create --name "My Lakehouse" --workspace-id "beefbeef-beef-beef-beef-beefbeefbeef" --enable-schemas
```

### Notebook

```bash
msfu notebook create --path "path/to/notebook.Notebook" --workspace-id "beefbeef-beef-beef-beef-beefbeefbeef"
```

2 changes: 1 addition & 1 deletion docs/usage/fabric-api.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Fabric API

A collection of helper functions for working with the Fabric API.
A collection of helper functions for working with the Fabric API. See the [API Reference](../core/fabric-api.md) for more details.

## List workspaces

Expand Down
11 changes: 9 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,23 +49,30 @@ plugins:
show_root_heading: false
filters: ["!^_", "^__init__$"]
show_object_full_path: false
heading_level: 1
heading_level: 2
members_order: source
separate_signature: true
show_signature_annotations: true
docstring_section_style: table
show_if_no_docstring: false

nav:
- Home:
- index.md
- Usage:
- usage/installation.md
- usage/cli.md
- usage/etl.md
- usage/fabric-api.md
- API Reference:
- Core:
- core/authentication.md
- core/fabric-api.md
- Fabric API:
- core/fabric-api/index.md
- core/fabric-api/workspace.md
- core/fabric-api/lakehouse.md
- core/fabric-api/notebook.md
- core/fabric-api/sql_endpoint.md
- ETL:
- etl/index.md
- etl/read.md
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,9 @@ dev = [
"python-dotenv>=1.0.0",
]

[project.scripts]
msfu = "msfabricutils.cli.cli:main"

[tool.setuptools_scm]

[tool.ruff]
Expand Down
Empty file.
256 changes: 256 additions & 0 deletions src/msfabricutils/cli/cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,256 @@
import argparse
import logging
import sys
from typing import Callable

from msfabricutils import __version__
from msfabricutils.cli.lakehouse import create_lakehouse_command, delete_lakehouse_command
from msfabricutils.cli.notebook import (
bulk_create_notebook_command,
create_notebook_command,
delete_notebook_command,
)
from msfabricutils.cli.workspace import create_workspace_command, delete_workspace_command


def create_parser():
"""Creates the main parser and subparsers."""
examples = """
Examples:
Create a workspace:
msfu workspace create --name "My Workspace" --description "My Workspace Description" --capacity-id "beefbeef-beef-beef-beef-beefbeefbeef" --on-conflict "update"

Create a lakehouse:
msfu lakehouse create --name "My Lakehouse" --description "My Lakehouse Description" --workspace-id "beefbeef-beef-beef-beef-beefbeefbeef" --on-conflict "update"

Create a single notebook:
msfu notebook create --path "path/to/notebook.Notebook" --workspace-id "beefbeef-beef-beef-beef-beefbeefbeef"

Create multiple notebooks:
msfu notebook create --path "directory/of/notebooks" "path/to/notebook.Notebook" --workspace-id "beefbeef-beef-beef-beef-beefbeefbeef"
"""

parser = argparse.ArgumentParser(
prog="msfabricutils",
description="Utility CLI for Microsoft Fabric REST API operations",
epilog=examples,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("--version", "-v", action="version", version=__version__)
parser.add_argument(
"--log-level",
"-l",
type=str,
default="INFO",
choices=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
help="The log level to use. Defaults to INFO.",
)
parser.add_argument(
"--show-azure-identity-logs",
action="store_true",
default=False,
help="Show Azure Identity logs. Defaults to False.",
)

subparsers = parser.add_subparsers(dest="command", help="Subcommands")

register_workspace_commands(subparsers)
register_lakehouse_commands(subparsers)
register_notebook_commands(subparsers)

return parser


def register_workspace_commands(subparsers: argparse._SubParsersAction):
"""Registers the workspace commands."""
workspace_parser = subparsers.add_parser("workspace", help="Workspace commands")
workspace_subparsers = workspace_parser.add_subparsers(
dest="workspace", help="Workspace commands"
)
add_subcommand(
subparsers=workspace_subparsers,
name="create",
handler=create_workspace_command,
required_args=["--name"],
choices_args={"--on-conflict": ["error", "ignore", "update"]},
optional_args=["--description", "--capacity-id"],
)
add_subcommand(
subparsers=workspace_subparsers,
name="delete",
handler=delete_workspace_command,
mutually_exclusive_args=["--id", "--name"],
choices_args={"--on-conflict": ["error", "ignore"]},
)


def register_lakehouse_commands(subparsers: argparse._SubParsersAction):
"""Registers the lakehouse commands."""
lakehouse_parser = subparsers.add_parser("lakehouse", help="Lakehouse commands")
lakehouse_subparsers = lakehouse_parser.add_subparsers(
dest="lakehouse", help="Lakehouse commands"
)

add_subcommand(
subparsers=lakehouse_subparsers,
name="create",
handler=create_lakehouse_command,
required_args=["--name", "--workspace-id"],
has_long_running_operation=True,
choices_args={
"--on-conflict": ["error", "ignore", "update"],
},
optional_args=["--description"],
flags=["--enable-schemas"],
)
add_subcommand(
subparsers=lakehouse_subparsers,
name="delete",
handler=delete_lakehouse_command,
required_args=["--workspace-id"],
mutually_exclusive_args=["--id", "--name"],
choices_args={"--on-conflict": ["error", "ignore"]},
)


def register_notebook_commands(subparsers: argparse._SubParsersAction):
"""Registers the notebook commands."""
notebook_parser = subparsers.add_parser("notebook", help="Notebook commands")
notebook_subparsers = notebook_parser.add_subparsers(dest="notebook", help="Notebook commands")

add_subcommand(
subparsers=notebook_subparsers,
name="create",
handler=create_notebook_command,
required_args=["--workspace-id", "--path"],
optional_args=["--name", "--description"],
has_long_running_operation=True,
choices_args={"--on-conflict": ["error", "ignore", "update"]},
)
add_subcommand(
subparsers=notebook_subparsers,
name="bulk-create",
handler=bulk_create_notebook_command,
required_args=["--workspace-id"],
nargs=["--path"],
has_long_running_operation=True,
choices_args={"--on-conflict": ["error", "ignore", "update"]},
)
add_subcommand(
subparsers=notebook_subparsers,
name="delete",
handler=delete_notebook_command,
required_args=["--workspace-id"],
mutually_exclusive_args=["--id", "--name"],
choices_args={"--on-conflict": ["error", "ignore"]},
)


def add_subcommand(
subparsers: argparse._SubParsersAction,
name: str,
handler: Callable,
required_args: list[str] | None = None,
nargs: list[str] | None = None,
choices_args: dict[str, list[str]] | None = None,
mutually_exclusive_args: list[str] | None = None,
optional_args: list[str] | None = None,
has_long_running_operation: bool = False,
flags: list[str] | None = None,
):
"""Adds a subcommand to the parser.

Args:
subparsers (argparse._SubParsersAction): The subparsers to add the subcommand to.
name (str): The name of the subcommand.
handler (Callable): The handler function to call when the subcommand is invoked.
required_args (list[str] | None): The required arguments for the subcommand.
nargs (list[str] | None): The nargs arguments for the subcommand.
choices_args (dict[str, list[str]] | None): The choices arguments for the subcommand. The default choice is the first in the list.
optional_args (list[str] | None): The optional arguments for the subcommand.
"""

if not required_args:
required_args = []

if not choices_args:
choices_args = {}

if not optional_args:
optional_args = []

if not nargs:
nargs = []

if not flags:
flags = []

create_parser = subparsers.add_parser(name, help=f"{name.capitalize()} commands")

for arg in required_args:
create_parser.add_argument(
arg, required=True, help=f"The {arg.lstrip('-')} of the {subparsers.dest} to {name}."
)

for arg in nargs:
create_parser.add_argument(
arg, nargs="+", help=f"The {arg.lstrip('-')} of the {subparsers.dest}s to {name}."
)

for arg in optional_args:
create_parser.add_argument(
arg, required=False, help=f"The {arg.lstrip('-')} of the {subparsers.dest} to {name}."
)

for flag in flags:
create_parser.add_argument(
flag, action="store_true", default=False, help=f"{flag.lstrip('-')} flag for the {subparsers.dest} to {name}."
)

if has_long_running_operation:
create_parser.add_argument(
"--no-wait", action="store_true", default=False, help="Do not wait for the long running operation to complete."
)

if mutually_exclusive_args:
argument_group = create_parser.add_mutually_exclusive_group(required=True)
for arg in mutually_exclusive_args:
argument_group.add_argument(
arg, help=f"The {arg.lstrip('-')} of the {subparsers.dest} to {name}."
)

for arg, choices in choices_args.items():
create_parser.add_argument(
arg,
type=str,
choices=choices,
default=choices[0],
help=f"The {arg.lstrip('-')} of the {subparsers.dest} to {name}. Defaults to `{choices[0]}`.",
)

create_parser.set_defaults(func=handler)


def main():
parser = create_parser()
args = parser.parse_args()

logging.basicConfig(
level=args.log_level,
format='{"timestamp": "%(asctime)s", "level": "%(levelname)s", "message": "%(message)s"}',
)

try:
azure_log_level = args.log_level if args.show_azure_identity_logs else logging.CRITICAL
logging.getLogger("azure").setLevel(azure_log_level)
args.func(args)
except Exception as e:
logging.error(e)
sys.stderr.write(str(e))
sys.exit(1)

sys.exit(0)


if __name__ == "__main__":
main()
Loading
Loading