Practice - Azure storage blob

Azure storage types

Azure currently support storage types include: Blob, File, Table and Queue.

Snip20180611_1.png

Azure blob structure

Blob was used to store unstructured objects such as files, images and videos.

blob_storage.jpg

You can think it's three level hierarchy: storage account -> container -> blob.

storage account: It's a name space which was used for access control and payment.

Container: silimar to AWS bucket in S3, you can think it's directory to orgnize your files.

Access URL:

http://<account>.blob.core.windows.net/<container>/<filename>

example:
http://nick.blob.core.windows.net/pictures/image1.png

Upload files to blob

All steps go through Azure CLI.

Create storage account

az storage account create -g resourc_group_name -n storage_account_name

$ az storage account create -g RoyResourceGroup -n royrepoblob
{
  "accessTier": null,
  "creationTime": "2018-06-08T07:08:56.652151+00:00",
  "customDomain": null,
  "enableHttpsTrafficOnly": false,
  "encryption": {
    "keySource": "Microsoft.Storage",
    "keyVaultProperties": null,
    "services": {
      "blob": {
        "enabled": true,
        "lastEnabledTime": "2018-06-08T07:08:56.714690+00:00"
      },
      "file": {
        "enabled": true,
        "lastEnabledTime": "2018-06-08T07:08:56.714690+00:00"
      },
      "queue": null,
      "table": null
    }
  },
  "id": "/subscriptions/xxx/resourceGroups/RoyResourceGroup/providers/Microsoft.Storage/storageAccounts/royrepoblob",
  "identity": null,
  "kind": "Storage",
  "lastGeoFailoverTime": null,
  "location": "japanwest",
  "name": "royrepoblob",
  "networkRuleSet": {
    "bypass": "AzureServices",
    "defaultAction": "Allow",
    "ipRules": [],
    "virtualNetworkRules": []
  },
  "primaryEndpoints": {
    "blob": "https://royrepoblob.blob.core.windows.net/",
    "file": "https://royrepoblob.file.core.windows.net/",
    "queue": "https://royrepoblob.queue.core.windows.net/",
    "table": "https://royrepoblob.table.core.windows.net/"
  },
  "primaryLocation": "japanwest",
  "provisioningState": "Succeeded",
  "resourceGroup": "RoyResourceGroup",
  "secondaryEndpoints": {
    "blob": "https://royrepoblob-secondary.blob.core.windows.net/",
    "file": null,
    "queue": "https://royrepoblob-secondary.queue.core.windows.net/",
    "table": "https://royrepoblob-secondary.table.core.windows.net/"
  },
  "secondaryLocation": "japaneast",
  "sku": {
    "capabilities": null,
    "kind": null,
    "locations": null,
    "name": "Standard_RAGRS",
    "resourceType": null,
    "restrictions": null,
    "tier": "Standard"
  },
  "statusOfPrimary": "available",
  "statusOfSecondary": "available",
  "tags": {},
  "type": "Microsoft.Storage/storageAccounts"
}

Create container

az storage container create -n container_name —account-name storage_account_name

$ az storage container create -n src --account-name royrepoblob
{
  "created": true
}

Upload single file

az storage blob upload -f path_to_file --account-name storage_account_name -c container_name -n file_name_in_blob

[centos@roy-ansible src]$ az storage blob upload -f ./jdk-8u162-linux-x64.tar.gz --account-name royrepoblob  -c src -n jdk-8u162-linux-x64.tar.gz
Finished[#############################################################]  100.0000%
{
  "etag": "\"0x8D5CD10DA4C33B8\"",
  "lastModified": "2018-06-08T07:24:20+00:00"
}

Set access permission for container

az storage container set-permission --name container_name --account-name storage_account_name --public-access public_option

$ az storage container set-permission --name src --account-name royrepoblob --public-access blob
{
  "etag": "\"0x8D5CD17CE5743B7\"",
  "lastModified": "2018-06-08T08:14:07+00:00"
}

Note for public_option:

--public-access

Specifies whether data in the container may be accessed publically. By default, container data is private ("off") to the account owner. Use "blob" to allow public read access for blobs. Use "container" to allow public read and list access to the entire container.

accepted values: blob, container, off

[Note] container permission: anonymously read and list only be available for application not for web browser.

Verify if the new uploaded file can be access

The default endpoint for Blob storage is storage-account-name.blob.core.windows.net

Try:

wget http://royrepoblob.blob.core.windows.net/src/jdk-8u162-linux-x64.tar.gz

Upload batch files

az storage blob upload-batch -s local_path -d blob_container —account-name storage_account_name

az storage blob upload-batch -s ./ -d local --account-name royrepoblob

Change blob file

Files in blob cannot be edit directly, you should re-upload the file to overwrite it. (the same command as upload single file)

Further more: access private file in blob

Files in blob are set private access permission by default, you can access it by using shared access signatures(SAS).

You can generate SAS in storage account page via Azure web portal as below:

SAS_azure_set.png

Verify

You will fail to download the file in http://royrepoblob.blob.core.windows.net/src/test.txt since it's private file. You should change it to use https and add SAS token at end of the url. (get SAS in above web page)

It looks like:

 https://royrepoblob.blob.core.windows.net/src/test.txt?sv=2018-03-28&ss=b&srt=co&sp=rl&se=2019-06-08T10:44:53Z&st=2018-12-28T02:44:53Z&spr=https&sig=7oEcPe7FtwatpNYlkepy0DJF0pdb7TBUTaMG97n5pUs%3D

Reference:

Quickstart: Upload, download, and list blobs using the Azure CLI
Using the Azure CLI with Azure Storage

Generate SAS Token for Blob in Azure Storage

Using shared access signatures (SAS)

Authorizing access to Azure Storage

az storage blob command