prepare new release

docs update
databrickslabs · Feb 4, 2025 · fb40b2d · fb40b2d
1 parent 234b213
commit fb40b2d
Show file tree

Hide file tree

Showing 4 changed files with 40 additions and 21 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,13 @@
 # Version changelog
 
+## 0.1.10
+
+*  Support datetime arguments for column range functions (#142) [View](https://github.com/databrickslabs/dqx/pull/142)
+*  DQX engine refactor and docs update (#138) [View](https://github.com/databrickslabs/dqx/pull/138)
+*  Add column functions to check for valid date strings (#144) [View](https://github.com/databrickslabs/dqx/pull/144)
+*  Generate rules for DLT as Python dictionary (#148) [View](https://github.com/databrickslabs/dqx/pull/148)
+*  Make DQX compatible with Serverless (#147) [View](https://github.com/databrickslabs/dqx/pull/147)
+
 ## 0.1.9
 
 * New dashboard query, Update to demos and docs [#133](https://github.com/databrickslabs/dqx/pull/133)

diff --git a/docs/dqx/docs/dev/contributing.mdx b/docs/dqx/docs/dev/contributing.mdx
@@ -101,23 +101,23 @@ You can use any Unity Catalog enabled Databricks workspace.
 
 #### Using terminal
 
-If you want to run integration tests and code coverage from your local machine, you need to set up the following environment variables:
+If you want to run the tests from your local machine in the terminal, you need to set up the following environment variables:
 ```shell
 export DATABRICKS_HOST=https://<workspace-url>
 export DATABRICKS_CLUSTER_ID=<cluster-id>
 
-# either use PAT token for authentication
-export DATABRICKS_TOKEN=<pat-token>
-
-# or use service principal OAuth token for authentication
-export DATABRICKS_CLIENT_ID=<client-id>
-export DATABRICKS_CLIENT_SECRET=<client-secret>
+# Authenticate to Databricks using OAuth generated for a service principal (recommended)
+export DATABRICKS_CLIENT_ID=<oauth-client-id>
+export DATABRICKS_CLIENT_SECRET=<oauth-client-secret>
 
 # Optionally enable serverless compute to be used for the tests.
 # Note that we run integration tests on standard and serverless compute clusters as part of the CI/CD pipelines
 export DATABRICKS_SERVERLESS_COMPUTE_ID=auto
 ```
 
+We recommend using [OAuth access token](https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html) generated for a service principal to authenticate with Databricks as presented above.
+Alternatively, you can authenticate using [PAT token](https://docs.databricks.com/en/dev-tools/auth/pat.html) by setting the `DATABRICKS_TOKEN` environment variable. However, we do not recommend this method, as it is less secure than OAuth.
+
 Run integration tests with the following command:
 ```shell
 make integration
@@ -139,13 +139,17 @@ Create the `~/.databricks/debug-env.json` with the following content, replacing
 ```json
 {
   "ws": {
-    "DATABRICKS_TOKEN": "<PAT-token>",
+    "DATABRICKS_CLIENT_ID": "<oauth-client-id>",
+    "DATABRICKS_CLIENT_SECRET": "<oauth-client-secret>",
     "DATABRICKS_HOST": "https://<workspace-url>",
     "DATABRICKS_CLUSTER_ID": "<databricks-cluster-id>"
   }
 }
 ```
-You must provide an existing cluster, but it will be auto-started for you as part of the tests.
+You must provide an existing cluster which will be auto-started for you as part of the tests.
+
+We recommend using [OAuth access token](https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html) generated for a service principal to authenticate with Databricks as presented above.
+Alternatively, you can authenticate using [PAT token](https://docs.databricks.com/en/dev-tools/auth/pat.html) by providing the `DATABRICKS_TOKEN` field. However, we do not recommend this method, as it is less secure than OAuth.
 
 **Running Tests on Serverless Compute**
 
@@ -154,7 +158,8 @@ To run integration tests on serverless compute, add the `DATABRICKS_SERVERLESS_C
 ```json
 {
   "ws": {
-    "DATABRICKS_TOKEN": "<PAT-token>",
+    "DATABRICKS_CLIENT_ID": "<oauth-client-id>",
+    "DATABRICKS_CLIENT_SECRET": "<oauth-client-secret>",
     "DATABRICKS_HOST": "https://<workspace-url>",
     "DATABRICKS_CLUSTER_ID": "<databricks-cluster-id>",
     "DATABRICKS_SERVERLESS_COMPUTE_ID": "auto"

diff --git a/docs/dqx/docs/reference.mdx b/docs/dqx/docs/reference.mdx
@@ -176,17 +176,17 @@ If you want to run the tests from your local machine in the terminal, you need t
 export DATABRICKS_HOST=https://<workspace-url>
 export DATABRICKS_CLUSTER_ID=<cluster-id>
 
-# either use PAT token for authentication
-export DATABRICKS_TOKEN=<pat-token>
-
-# or use service principal OAuth token for authentication
-export DATABRICKS_CLIENT_ID=<client-id>
-export DATABRICKS_CLIENT_SECRET=<client-secret>
+# Authenticate to Databricks using OAuth generated for a service principal (recommended)
+export DATABRICKS_CLIENT_ID=<oauth-client-id>
+export DATABRICKS_CLIENT_SECRET=<oauth-client-secret>
 
 # Optionally enable serverless compute to be used for the tests
 export DATABRICKS_SERVERLESS_COMPUTE_ID=auto
 ```
 
+We recommend using [OAuth access token](https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html) generated for a service principal to authenticate with Databricks as presented above.
+Alternatively, you can authenticate using [PAT token](https://docs.databricks.com/en/dev-tools/auth/pat.html) by setting the `DATABRICKS_TOKEN` environment variable. However, we do not recommend this method, as it is less secure than OAuth.
+
 #### Setting up Databricks workspace client authentication in an IDE
 
 If you want to run the tests from your IDE, you must setup `.env` or `~/.databricks/debug-env.json` file
@@ -199,21 +199,27 @@ Create the `~/.databricks/debug-env.json` with the following content, replacing
 ```json
 {
   "ws": {
-    "DATABRICKS_TOKEN": "<PAT-token>",
+    "DATABRICKS_CLIENT_ID": "<oauth-client-id>",
+    "DATABRICKS_CLIENT_SECRET": "<oauth-client-secret>",
     "DATABRICKS_HOST": "https://<workspace-url>",
     "DATABRICKS_CLUSTER_ID": "<databricks-cluster-id>"
   }
 }
 ```
-You must provide an existing cluster, but it will be auto-started for you as part of the tests.
+
+You must provide an existing cluster which will be auto-started for you as part of the tests.
+
+We recommend using [OAuth access token](https://docs.databricks.com/en/dev-tools/auth/oauth-m2m.html) generated for a service principal to authenticate with Databricks as presented above.
+Alternatively, you can authenticate using [PAT token](https://docs.databricks.com/en/dev-tools/auth/pat.html) by providing the `DATABRICKS_TOKEN` field. However, we do not recommend this method, as it is less secure than OAuth.
 
 **Running Tests on Serverless Compute**
 
 To run the integration tests on serverless compute, add the `DATABRICKS_SERVERLESS_COMPUTE_ID` field to your debug configuration:
 ```json
 {
   "ws": {
-    "DATABRICKS_TOKEN": "<PAT-token>",
+    "DATABRICKS_CLIENT_ID": "<oauth-client-id>",
+    "DATABRICKS_CLIENT_SECRET": "<oauth-client-secret>",
     "DATABRICKS_HOST": "https://<workspace-url>",
     "DATABRICKS_CLUSTER_ID": "<databricks-cluster-id>",
     "DATABRICKS_SERVERLESS_COMPUTE_ID": "auto"
@@ -224,7 +230,7 @@ When `DATABRICKS_SERVERLESS_COMPUTE_ID` is set the `DATABRICKS_CLUSTER_ID` is ig
 
 ### Local testing with DQEngine
 
-If workspace-level access is unavailable in your unit testing environment, you can perform local testing by installing the latest `pyspark` package and mocking the workspace client.
+If workspace-level access is unavailable in your testing environment, you can perform local testing by installing the latest `pyspark` package and mocking the workspace client.
 
 **Note: This approach should be treated as experimental!** It does not offer the same level of testing as the standard approach and it is only applicable to selected methods.
 We strongly recommend following the standard testing procedure outlined above, which includes proper initialization of the workspace client.

diff --git a/src/databricks/labs/dqx/__about__.py b/src/databricks/labs/dqx/__about__.py
@@ -1 +1 @@
-__version__ = "0.1.9"
+__version__ = "0.1.10"