[Feature] Implement U2M Authentication in the Go SDK

[mgyucht]

What changes are proposed in this pull request?

This PR moves logic about U2M OAuth login from the CLI to the Go SDK. This eliminates a cyclic dependency between the SDK and CLI for interacting with the OAuth token cache and enables U2M support directly in the Go SDK without need for the CLI to be installed.

Most of this code is carried over from the CLI, but I have made specific refactors to generalize it where needed.

Currently, the token cache key follows a specific structure:

• https://<accounts-host>/oidc/accounts/<account-id> for account-based sessions
• https://<workspace host> for workspace-based sessions
  This can be generalized to allow callers to cache tokens in other manners. For example, users may want to cache tokens per principal or per set of OAuth scopes for their own OAuth applications.

Additionally, products should be able to use an isolated token cache or a token cache backed by a different storage medium, like a database. This is simple to do by introducing a token cache interface. Implementers can define their own credential strategy and reuse the PersistentAuth type to handle the negotiation.

How is this tested?

Carried over tests from the CLI.

More tests are forthcoming, once a decision is made on this approach.

[renaudhartert-db]

Thanks for the PR — always good to get the SDK more self-contained.

[renaudhartert-db]

Move in cache.go?

[renaudhartert-db]

I realize you didn't write these comments. Though, let's remain consistent by writing them as full sentences (with majuscule and period). Ref: https://go.dev/wiki/CodeReviewComments#comment-sentences

[renaudhartert-db]

Maybe explain a little more what problem this is solving? It's not super clear from looking at this definition.

[renaudhartert-db]

What is auth:",sensitive"? I don't remember seeing this annotation before.

[mgyucht]

In some cases (like failed authentication), debug logs will include configuration parameters. Config attributes marked as sensitive are redacted in these logs. I don't think that also applies to this struct though. I'll take a closer look here.

[renaudhartert-db]

Use oauth2.Token instead?

[mgyucht]

oauth2.Token has an "Expiry" field, which is computed based on the "expires_in" field. This mimics the unexported tokenJSON type from that library.

[renaudhartert-db]

[nit] What location was doing wasn't obvious to me reading this line. We could make that clearer with a better name. For example tokenCacheFilepath(). Though, I think it might just be simpler to inline the code since it results in the same number of line minus the location method.

Suggested change

	loc, err := c.location()
	if err != nil {
	return err
	}
	home, err := os.UserHomeDir()
	if err != nil {
	return fmt.Errorf("could not access home dir: %w")
	}
	c.fileLocation = filepath.Join(home, tokenCacheFileName)

[renaudhartert-db]

Could we briefly explain what this cache does? Something like:

// FileTokenCache caches tokens in "~/.databricks/token-cache.json". FileTokenCache
// implements the TokenCache interface.

[mgyucht]

Great, will use this description. Sorry, most of this is copied directly from the CLI without carefully reviewing for style or comments, but thanks for pointing these out, and I'm happy to correct them here.

[renaudhartert-db]

[optional] This is slightly more idiomatic to test for nil error first.

Suggested change

	if errors.Is(err, fs.ErrNotExist) {
	dir := filepath.Dir(c.fileLocation)
	err = os.MkdirAll(dir, ownerExecReadWrite)
	if err != nil {
	return fmt.Errorf("mkdir: %w", err)
	}
	} else if err != nil {
	return fmt.Errorf("load: %w", err)
	}
	if err != nil {
	if !errors.Is(err, fs.ErrNotExist) {
	return fmt.Errorf("load: %w", err)
	}
	dir := filepath.Dir(c.fileLocation)
	if err := os.MkdirAll(dir, ownerExecReadWrite); err != nil {
	return fmt.Errorf("mkdir: %w", err)
	}
	}

[renaudhartert-db]

Naively, I would have expected the lock to be managed by this cache implementation as an internal detail. Having the lock managed by the client sounds a little bug prone — especially given that the client does not have a programmatic way to know in what file the cache is storing the tokens. Am I missing something?

[mgyucht]

Good question. It needs to be at least locked in PersistentAuth: the Load() method will read from the cache and then, if the access token is expired, refresh it and update the cache, which replaces it entirely. That requires some form of user-wide mutual exclusion that lasts longer than an individual method call, thus the lock file. I don't see any issues with adding an in-memory mutex within FileTokenCache as well in case an application is directly managing and updating tokens, considering what that would be protecting against.

[renaudhartert-db]

It looks like locker is never locked. Am I missing something?

[mgyucht]

You're not missing anything, I forgot to add it before, but I have it locally and will include it in my next update.

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/sdk-go

Inputs:

• PR number: 1108
• Commit SHA: 356a7c9aefe20a753a62b4f4d7fa6654529d4feb

Checks will be approved automatically on success.