-
'Mount ADLS Gen2 or Blob Storage in Azure Databricks' from 2023-06:
-
Mount example:
storageAccountName = "<storage-account-name>"
storageAccountAccessKey = "<access-key>"
sasToken = "<sas-token>"
blobContainerName = "<blob-container-name>"
mountPoint = "<mount-point-path>"
if any(mount.mountPoint == mountPoint for mount in dbutils.fs.mounts()):
dbutils.fs.unmount(mountPoint)
try:
dbutils.fs.mount(
source = f"wasbs://{blobContainerName}@{storageAccountName}.blob.core.windows.net",
mount_point = mountPoint,
# using access keys: extra_configs = {'fs.azure.account.key.' + storageAccountName + '.blob.core.windows.net': storageAccountAccessKey}
# using sas tokens: extra_configs = {'fs.azure.sas.' + blobContainerName + '.' + storageAccountName + '.blob.core.windows.net': sasToken}
)
print("mount succeeded!")
except Exception as e:
print("mount exception", e)
-
Unmount example:
dbutils.fs.unmount(mountPoint)
-
List mounts:
dbutils.fs.mounts()
-
One faced issue example:
shaded.databricks.org.apache.hadoop.fs.azure.AzureException: java.lang.IllegalArgumentException: The String is not a valid Base64-encoded string.
- Solution: add token into Azure Key Vault and access it using dbfs, like
dbutils.secrets.get(scope = "<secret-scope>", key = "<key-vault-secret-name>")
- Solution: add token into Azure Key Vault and access it using dbfs, like
-
Additional documentation: https://learn.microsoft.com/en-us/azure/databricks/connect/storage/azure-storage