Skip to content

Commit

Permalink
feat: merged aws and azure external location module
Browse files Browse the repository at this point in the history
  • Loading branch information
MyroslavLevchyk authored and MyroslavLevchyk committed Nov 19, 2024
1 parent d502845 commit efa2f97
Show file tree
Hide file tree
Showing 5 changed files with 41 additions and 43 deletions.
15 changes: 7 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Azure Databricks External Location Terraform module
Terraform module for creation Azure Databricks External Location
# Databricks External Location Terraform module
Terraform module for creation Databricks External Location

## Usage
```hcl
Expand Down Expand Up @@ -76,13 +76,13 @@ module "databricks_locations" {
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >=1.0.0 |
| <a name="requirement_azurerm"></a> [azurerm](#requirement\_azurerm) | >= 4.0.1 |
| <a name="requirement_databricks"></a> [databricks](#requirement\_databricks) | >=1.48.3 |
| <a name="requirement_databricks"></a> [databricks](#requirement\_databricks) | ~>1.0 |

## Providers

| Name | Version |
|------|---------|
| <a name="provider_databricks"></a> [databricks](#provider\_databricks) | >=1.48.3 |
| <a name="provider_databricks"></a> [databricks](#provider\_databricks) | ~>1.0 |

## Modules

Expand All @@ -95,15 +95,14 @@ No modules.
| [databricks_external_location.this](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/external_location) | resource |
| [databricks_grants.credential](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/grants) | resource |
| [databricks_grants.locations](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/grants) | resource |
| [databricks_storage_credential.azure](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/storage_credential) | resource |
| [databricks_storage_credential.gcp](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/storage_credential) | resource |
| [databricks_storage_credential.this](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/storage_credential) | resource |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_external_locations"></a> [external\_locations](#input\_external\_locations) | List of object with external location configuration attributes | <pre>list(object({<br> index = string # Index of instance, for example short name, used later to access exact external location in output map<br> name = string # Custom whole name of resource<br> url = string # Path URL in cloud storage<br> owner = optional(string) # Owner of resource<br> skip_validation = optional(bool, true) # Suppress validation errors if any & force save the external location<br> read_only = optional(bool, false) # Indicates whether the external location is read-only.<br> force_destroy = optional(bool, true)<br> force_update = optional(bool, true)<br> comment = optional(string, "External location provisioned by Terraform")<br> permissions = optional(set(object({<br> principal = string<br> privileges = list(string)<br> })), [])<br> isolation_mode = optional(string, "ISOLATION_MODE_OPEN")<br> }))</pre> | `[]` | no |
| <a name="input_storage_credential"></a> [storage\_credential](#input\_storage\_credential) | Object with storage credentials configuration attributes | <pre>object({<br> azure_access_connector_id = optional(string, null) # Azure Databricks Access Connector Id<br> cloud = optional(string, "azure")<br> name = string # Custom whole name of resource<br> owner = optional(string) # Owner of resource<br> force_destroy = optional(bool, true)<br> comment = optional(string, "Managed identity credential provisioned by Terraform")<br> permissions = optional(set(object({<br> principal = string<br> privileges = list(string)<br> })), [])<br> isolation_mode = optional(string, "ISOLATION_MODE_OPEN")<br> })</pre> | n/a | yes |
| <a name="input_external_locations"></a> [external\_locations](#input\_external\_locations) | List of object with external location configuration attributes | <pre>list(object({<br> index = string # Index of instance, for example short name, used later to access exact external location in output map<br> name = string # Custom whole name of resource<br> url = string # Path URL in cloud storage<br> credentials_name = optional(string)<br> owner = optional(string) # Owner of resource<br> skip_validation = optional(bool, true) # Suppress validation errors if any & force save the external location<br> read_only = optional(bool, false) # Indicates whether the external location is read-only.<br> force_destroy = optional(bool, true)<br> force_update = optional(bool, true)<br> comment = optional(string, "External location provisioned by Terraform")<br> permissions = optional(set(object({<br> principal = string<br> privileges = list(string)<br> })), [])<br> isolation_mode = optional(string, "ISOLATION_MODE_OPEN")<br> }))</pre> | `[]` | no |
| <a name="input_storage_credential"></a> [storage\_credential](#input\_storage\_credential) | Object with storage credentials configuration attributes | <pre>object({<br> azure_access_connector_id = optional(string, null) # Azure Databricks Access Connector Id<br> cloud = optional(string, "")<br> name = optional(string, null) # Custom whole name of resource <br> owner = optional(string) # Owner of resource<br> force_destroy = optional(bool, true)<br> comment = optional(string, "Managed identity credential provisioned by Terraform")<br> permissions = optional(set(object({<br> principal = string<br> privileges = list(string)<br> })), [])<br> isolation_mode = optional(string, "ISOLATION_MODE_OPEN")<br> })</pre> | n/a | yes |

## Outputs

Expand Down
36 changes: 17 additions & 19 deletions main.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,35 @@ locals {
}
}

resource "databricks_storage_credential" "gcp" {
count = var.storage_credential.cloud == "gcp" ? 1 : 0
resource "databricks_storage_credential" "this" {
count = var.storage_credential.cloud != "" ? 1 : 0

name = var.storage_credential.name
owner = var.storage_credential.owner

databricks_gcp_service_account {}

force_destroy = var.storage_credential.force_destroy
comment = var.storage_credential.comment
}

resource "databricks_storage_credential" "azure" {
count = var.storage_credential.cloud == "azure" ? 1 : 0

name = var.storage_credential.name
owner = var.storage_credential.owner
# Dynamic block for Azure
dynamic "azure_managed_identity" {
for_each = var.storage_credential.cloud == "azure" ? [1] : []
content {
access_connector_id = var.storage_credential.azure_access_connector_id
}
}

azure_managed_identity {
access_connector_id = var.storage_credential.azure_access_connector_id
# Dynamic block for GCP
dynamic "databricks_gcp_service_account" {
for_each = var.storage_credential.cloud == "gcp" ? [1] : []
content {}
}

force_destroy = var.storage_credential.force_destroy
comment = var.storage_credential.comment
isolation_mode = var.storage_credential.isolation_mode
isolation_mode = var.storage_credential.cloud == "azure" ? var.storage_credential.isolation_mode : null
}

resource "databricks_grants" "credential" {
count = length(var.storage_credential.permissions) != 0 ? 1 : 0
count = var.storage_credential.cloud != "" ? 1 : 0

storage_credential = coalesce(try(databricks_storage_credential.azure[0].id, null), try(databricks_storage_credential.gcp[0].id, null))
storage_credential = try(databricks_storage_credential.this[0].id, null)
dynamic "grant" {
for_each = var.storage_credential.permissions
content {
Expand All @@ -58,7 +56,7 @@ resource "databricks_external_location" "this" {
name = each.value.name
owner = each.value.owner
url = each.value.url
credential_name = coalesce(try(databricks_storage_credential.azure[0].id, null), try(databricks_storage_credential.gcp[0].id, null))
credential_name = coalesce(try(databricks_storage_credential.this[0].id, null), each.value.credentials_name)
comment = each.value.comment
skip_validation = each.value.skip_validation
read_only = each.value.read_only
Expand Down
6 changes: 3 additions & 3 deletions outputs.tf
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
output "storage_credential_name" {
value = coalesce(try(databricks_storage_credential.azure[0].name, null), try(databricks_storage_credential.gcp[0].name, null))
value = try(databricks_storage_credential.this[0].name, null)
description = "Storage Credential name"
}

output "storage_credential_metastore_id" {
value = coalesce(try(databricks_storage_credential.azure[0].metastore_id, null), try(databricks_storage_credential.gcp[0].metastore_id, null))
value = try(databricks_storage_credential.this[0].metastore_id, null)
description = "Storage Credential metastore id"
}

Expand All @@ -18,6 +18,6 @@ output "external_locations" {
}

output "databricks_gcp_service_account" {
value = try(databricks_storage_credential.gcp[0].databricks_gcp_service_account[0].email, null)
value = try(databricks_storage_credential.this[0].databricks_gcp_service_account[0].email, null)
description = "The email of the GCP service account created, to be granted access to relevant buckets"
}
25 changes: 13 additions & 12 deletions variables.tf
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
variable "storage_credential" {
type = object({
azure_access_connector_id = optional(string, null) # Azure Databricks Access Connector Id
cloud = optional(string, "azure")
name = string # Custom whole name of resource
owner = optional(string) # Owner of resource
cloud = optional(string, "")
name = optional(string, null) # Custom whole name of resource
owner = optional(string) # Owner of resource
force_destroy = optional(bool, true)
comment = optional(string, "Managed identity credential provisioned by Terraform")
permissions = optional(set(object({
Expand All @@ -17,15 +17,16 @@ variable "storage_credential" {

variable "external_locations" {
type = list(object({
index = string # Index of instance, for example short name, used later to access exact external location in output map
name = string # Custom whole name of resource
url = string # Path URL in cloud storage
owner = optional(string) # Owner of resource
skip_validation = optional(bool, true) # Suppress validation errors if any & force save the external location
read_only = optional(bool, false) # Indicates whether the external location is read-only.
force_destroy = optional(bool, true)
force_update = optional(bool, true)
comment = optional(string, "External location provisioned by Terraform")
index = string # Index of instance, for example short name, used later to access exact external location in output map
name = string # Custom whole name of resource
url = string # Path URL in cloud storage
credentials_name = optional(string)
owner = optional(string) # Owner of resource
skip_validation = optional(bool, true) # Suppress validation errors if any & force save the external location
read_only = optional(bool, false) # Indicates whether the external location is read-only.
force_destroy = optional(bool, true)
force_update = optional(bool, true)
comment = optional(string, "External location provisioned by Terraform")
permissions = optional(set(object({
principal = string
privileges = list(string)
Expand Down
2 changes: 1 addition & 1 deletion versions.tf
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ terraform {
}
databricks = {
source = "databricks/databricks"
version = ">=1.48.3"
version = "~>1.0"
}
}
}

0 comments on commit efa2f97

Please sign in to comment.