Authenticate access to Databricks with a service principal using OAuth (OAuth M2M) #51276
Description
Is your feature request related to a problem? Please describe.
The current implementation of the Databricks drivers were inspired by the implementation of relferreira
metabase-sparksql-databricks-driver. However, the problem with this implementation, in contrast to Brigad is that it makes use of User Access Tokens. Brigad's implementation, which follows Databricks recommendations, uses Machine 2 Machine authentication.
Note
We make the assumption that no Database connection should be set up using individual user credentials. The connection method, which should be somewhat abstracted from the underlying drivers of the current Databricks implementation should not be affected.
Describe the solution you'd like
We expect that the Databricks Drivers utilise OAuth client id and client secrets as recommended by Databricks and implemented by Brigad. Minimal changes to the underlying drivers are likely needed, only how the authentication works.
Databricks recommends using client unified authentication, however if you must manually generate, refresh, or use Databricks OAuth access tokens, follow the instructions in this section. (they must be updated every hour)
Describe alternatives you've considered
There are no serious alternatives to do M2M authentication with Metabase and databricks as of today. Thus, the current implementation forces users to utilise non-recommended methods by the vendor.
How important is this feature to you?
Let's just say that using individual credentials to connect to Databases isn't exactly something I dream of......
Additional context
There is an implementation of this already. Maybe we can just lift and shift the authentication part?
Activity