Skip to content

Instantly share code, notes, and snippets.

@chgeuer

chgeuer/AzureStorageClientSideEncryptionAndRewrap.cs

Last active August 28, 2024 08:50
Show Gist options
  • Save chgeuer/e51aca40f55b9158da065e489b5e0242 to your computer and use it in GitHub Desktop.
Save chgeuer/e51aca40f55b9158da065e489b5e0242 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
AzureStorageClientSideEncryptionAndRewrap.cs
/*
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Identity.Client" Version="4.64.0" />
<PackageReference Include="Azure.Identity" Version="1.12.0" />
<PackageReference Include="Azure.Security.KeyVault.Keys" Version="4.6.0" />
<PackageReference Include="Azure.Storage.Blobs" Version="12.21.2" />
</ItemGroup>
</Project>
*/
using Azure.Core;
using Azure.Core.Cryptography;
using Azure.Identity;
using Azure.Security.KeyVault.Keys;
using Azure.Security.KeyVault.Keys.Cryptography;
using Azure.Storage;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Specialized;
using System.Text;
var (accountName, containerName, blobName) = ("deadbeef1", "container1", "myblob.txt");
var (keyVaultName, (keyWrapAlgorithm, rsaKeyNamesInKeyVault)) = (
"chgeustorkv",
(KeyWrapAlgorithm.RsaOaep, new[] { "keywrapkey1", "keywrapkey2", "keywrapkey3" }));
var (tenantId, clientId, clientSecret) = (
"workentra.geuer-pollmann.de",
Environment.GetEnvironmentVariable("CLIENT_ID"),
Environment.GetEnvironmentVariable("CLIENT_SECRET"));
ClientSecretCredential credential = new(tenantId, clientId, clientSecret,
options: new ClientSecretCredentialOptions
{
AdditionallyAllowedTenants = { "*" }
}
);
Func<string, ClientSideEncryptionOptions> enc_opts(string keyVaultName, TokenCredential credential) =>
kekName =>
new(ClientSideEncryptionVersion.V2_0)
{
KeyEncryptionKey = new MyKeyEncryptionKey(
keyVaultName: keyVaultName,
keyName: kekName,
credential: credential),
KeyResolver = new MyKeyEncryptionKeyResolver(
keyVaultName: keyVaultName,
credential: credential),
KeyWrapAlgorithm = keyWrapAlgorithm.ToString()
};
var kek = enc_opts(keyVaultName, credential);
#region Upload
BlobServiceClient bsc = new(
serviceUri: new Uri($"https://{accountName}.blob.core.windows.net"),
credential: credential,
options: new SpecializedBlobClientOptions
{
ClientSideEncryption = kek(rsaKeyNamesInKeyVault[0])
}
);
BlobContainerClient containerClient = bsc.GetBlobContainerClient(containerName);
await containerClient.CreateIfNotExistsAsync();
BlobClient blob = containerClient.GetBlobClient(blobName);
var encoding = Encoding.UTF8;
blob.Upload(new MemoryStream(encoding.GetBytes("Hallo")), overwrite: true);
#endregion
#region Re-wrap content encryption keys with a different key encryption key
await blob.UpdateClientSideKeyEncryptionKeyAsync(kek(rsaKeyNamesInKeyVault[1]));
await blob.UpdateClientSideKeyEncryptionKeyAsync(kek(rsaKeyNamesInKeyVault[2]));
#endregion
#region Download the most recent version
// Not that for the download, we don't need to specify the key, only the IKeyEncryptionKeyResolver
BlobServiceClient bsc2 = new(
serviceUri: new Uri($"https://{accountName}.blob.core.windows.net"),
credential: credential,
options: new SpecializedBlobClientOptions
{
ClientSideEncryption = new(ClientSideEncryptionVersion.V2_0)
{
KeyResolver = new MyKeyEncryptionKeyResolver(
keyVaultName: keyVaultName,
credential: credential),
KeyWrapAlgorithm = keyWrapAlgorithm.ToString()
}
}
);
BlobContainerClient containerClient2 = bsc2.GetBlobContainerClient(containerName);
await containerClient2.CreateIfNotExistsAsync();
BlobClient blob2 = containerClient.GetBlobClient(blobName);
// Download and decrypt the encrypted contents from the blob.
MemoryStream outputStream = new();
blob2.DownloadTo(outputStream);
Console.WriteLine($"Downloaded blob. Contents: {encoding.GetString(outputStream.ToArray())}");
#endregion
public class MyKeyEncryptionKey(string keyVaultName, string keyName, TokenCredential credential) : IKeyEncryptionKey
{
private readonly string keyVaultName = keyVaultName;
private readonly string keyName = keyName;
private readonly TokenCredential credential = credential;
string IKeyEncryptionKey.KeyId => keyName;
byte[] IKeyEncryptionKey.WrapKey(string algorithm, ReadOnlyMemory<byte> key, CancellationToken cancellationToken)
=> ((IKeyEncryptionKey)this).WrapKeyAsync(algorithm, key, cancellationToken).Result;
byte[] IKeyEncryptionKey.UnwrapKey(string algorithm, ReadOnlyMemory<byte> encryptedKey, CancellationToken cancellationToken) =>
((IKeyEncryptionKey)this).UnwrapKeyAsync(algorithm, encryptedKey, cancellationToken).Result;
async Task<byte[]> IKeyEncryptionKey.WrapKeyAsync(string algorithm, ReadOnlyMemory<byte> key, CancellationToken cancellationToken)
{
CryptographyClient cryptoClient = await GetCryptographyClient(cancellationToken);
WrapResult wrapResult = await cryptoClient.WrapKeyAsync(algorithm, key.ToArray(), cancellationToken);
return wrapResult.EncryptedKey;
}
async Task<byte[]> IKeyEncryptionKey.UnwrapKeyAsync(string algorithm, ReadOnlyMemory<byte> encryptedKey, CancellationToken cancellationToken)
{
CryptographyClient cryptoClient = await GetCryptographyClient(cancellationToken);
var unwrapped = await cryptoClient.UnwrapKeyAsync(algorithm, encryptedKey.ToArray(), cancellationToken);
return unwrapped.Key;
}
private async Task<CryptographyClient> GetCryptographyClient(CancellationToken cancellationToken)
{
KeyClient keyClient = new(
vaultUri: new Uri($"https://{this.keyVaultName}.vault.azure.net/"),
credential: credential);
KeyVaultKey rsaKey = await keyClient.GetKeyAsync(name: this.keyName, cancellationToken: cancellationToken);
CryptographyClient cryptoClient = new(keyId: rsaKey.Id, credential: credential);
return cryptoClient;
}
}
public class MyKeyEncryptionKeyResolver(string keyVaultName, TokenCredential credential) : IKeyEncryptionKeyResolver
{
private readonly string keyVaultName = keyVaultName;
private readonly TokenCredential credential = credential;
IKeyEncryptionKey IKeyEncryptionKeyResolver.Resolve(string keyId, CancellationToken cancellationToken) =>
((IKeyEncryptionKeyResolver)this).ResolveAsync(keyId, cancellationToken).Result;
async Task<IKeyEncryptionKey> IKeyEncryptionKeyResolver.ResolveAsync(string keyId, CancellationToken cancellationToken)
{
await Console.Out.WriteLineAsync($"Resolving key #{keyId}");
return new MyKeyEncryptionKey(this.keyVaultName, keyId, this.credential);
}
}
@chgeuer
Copy link
Author

chgeuer commented Aug 22, 2024 •
edited
Loading

Simply speaking, you can re-wrap the content encryption key with a different key encryption key, without having to download the full ciphertext, locally re-wrap, and re-upload.

Have a look here: https://gist.github.com/chgeuer/e51aca40f55b9158da065e489b5e0242

For this to work, I created a storage account and a KeyVault, and I have 3 (RSA) keys in KeyVault, called “keywrapkey1”, “…2” and “…3”. I upload a blob and request encryption under kek1, then I call UpdateClientSideKeyEncryptionKeyAsync once with kek2, and then with kek3, so that the content encryption key is wrapped with kek1 -> kek2 -> kek3 over time.

image

The UpdateClientSideKeyEncryptionKeyAsync function does what you would expect it does, below is a fiddler trace screenshot. This is the UpdateClientSideKeyEncryptionKeyAsync(kek2) call, i.e. where the blob currently protected with kek1.

  • In Request 1, the SDK fetches the encryption metadata (the JSON stored in the x-ms-meta-encryptiondata field):
  • The JSON is this
{
    "EncryptionMode":"FullBlob",
    "WrappedContentKey":{
       "KeyId":"keywrapkey1",
       "EncryptedKey":"....",
       "Algorithm":"RSA-OAEP"},
    "EncryptionAgent":{
        "Protocol":"2.0",
        "EncryptionAlgorithm":"AES_GCM_256"
    }, 
    ...
}
  • So as a result, the SDK knows that “keywrapkey1” is required to decrypt the content encryption key
  • Request 2 lists the versions of that key in KV
  • Request 3 fetches metadata about the most recent version
  • Request 4 is the POST to /keys/keywrapkey1/…/unwrapKey?api-version=7.5 to unwrap the content encryption key under keywrapkey1’s most recent version
  • Requests 6 and 7 fetch the most recent public key material (modulus exponent) of keywrapkey2
  • The encryption of the content encryption key unwrapped in step 4, under keywrapkey2, happens on the client side, i.e. no keyvault involved
  • Request 8 is PUT /container1/myblob.txt?comp=metadata which updates the blob’s metadata with the new EncryptedKey and indicating that it’s wrapped using keywrapkey2
{
    ...,
    "WrappedContentKey":{
    	"KeyId":"keywrapkey2",
    	"EncryptedKey": "…new stuff",
	},
	...
)

So long story short, you can achieve your whole thing just by rolling to a new KEK by calling UpdateClientSideKeyEncryptionKeyAsync, no need to mess around storing key material and stuff in some database.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment