Note: This needs updating for Ocean V3.
This tutorial is for publishers who want to get started using Azure to store some of their data assets. (Some data assets could also be stored in other places.)
Publishers must run Provider to mediate consumer access to data assets stored in Azure Storage. Provider needs the following Azure credentials from the publisher:
AZURE_ACCOUNT_NAME: Azure Storage Account Name (for storing files)
AZURE_ACCOUNT_KEY: Azure Storage Account key
AZURE_RESOURCE_GROUP: Azure resource group
AZURE_LOCATION: Azure Region
AZURE_CLIENT_ID: Azure Application ID
AZURE_CLIENT_SECRET: Azure Application Secret
AZURE_TENANT_ID: Azure Tenant ID
AZURE_SUBSCRIPTION_ID: Azure Subscription ID
If you go through this tutorial, then you will get all the Azure credentials listed above.
If you already have data assets stored in Azure, then you might already have, or be able to get, the above information. You could use this tutorial to get a sense of where to look (but don’t create anything new).
To give the above Azure credentials to Provider, you either put them in a Provider config file or in environment variables with the above names. Environment variables should be used if you’re running Provider inside a container. If you want to use the config file option, see Provider README.
If you’re using Barge to run Provider and other Ocean Protocol components, then the above Azure credentials should go in the file
barge/provider.env. (That file gets used to set environment variables.)
Note: Azure is constantly changing. For that reason, we give try to give links to official Azure documentation, since it should stay up-to-date.
If you don’t already have an Azure account, then you will have to create one. Go to the Microsoft Azure website and follow the links.
Once you have an Azure account, go to https://portal.azure.com/ and sign in.
The Azure docs say, “A subscription is a logical grouping of Azure services that is linked to an Azure account. A single Azure account can contain multiple subscriptions.”
If you see Subscriptions in the left sidebar of Azure Portal, then click that. If you don’t see it, just type “Subscriptinos” into the search bar at the top, then click on Subscriptions under the SERVICES heading.
You should see a list of one or more subscriptions. Click on the one you want to use for Azure storage. Remember to use that one for the rest of this tutorial (whenever you are asked for a subscription name).
Subscription ID. That’s what Provider calls
AZURE_SUBSCRIPTION_ID. You now have one of the Azure credentials!
# Example AZURE_SUBSCRIPTION_ID (Azure Subscription ID) 479284be-0104-421a-8488-1aeac0caecaa
See the Azure docs page:
The first step there is to Create an Azure Active Directory application. Do that.
Sign-on URL can be totally made up. The URL doesn’t need to be real.
Once the app is created, copy the
Application ID: that’s what Provider calls the
AZURE_CLIENT_ID. It should look something like this:
# Example AZURE_CLIENT_ID (Application ID) 5d25ee8a-da2c-4e6f-8fba-09b6dd091038
On the same Azure docs page, find the section titled Get application ID and authentication key or similar. You already have your application ID, but you still need generate an authentication key by following the instructions in that section.
You can make up whatever you like for the key’s
Once the application key is generated, copy its value: that’s what Provider calls the
AZURE_CLIENT_SECRET. It should look something like this:
# Example AZURE_CLIENT_SECRET (Application key) RVJ1H5gYOmnMitikmM5ehszqmgrY5BFkoalnjfWMuDM
On the same Azure docs page, find the section titled Get tenant ID or similar. Follow the instructions.
The tenant ID is what Provider calls
# Example AZURE_TENANT_ID (tenant ID, Directory ID) 2a4a3887-4e2e-4a31-8006-6e2b5877640e
See the Azure docs page:
That page says how to create a new empty resource group. Do that. You can make up whatever name you like, but it’s good practice to avoid special characters and to include:
- some words to indicate what it’s for, e.g.
- your name
- the month and year it was created, e.g.
to help you and others manage it. The Resource group name is what Provider calls the
AZURE_RESOURCE_GROUP and the Resource group location is what Provider calls the
AZURE_LOCATION. Here are examples of both:
# Example AZURE_RESOURCE_GROUP (Resource group name) StorageCreatedNov2018ByTroy
# Example AZURE_LOCATION (Resource group location) West Europe
Inside your new resource group:
- click Access control (IAM)
- click + Add role assignment
- In the
Contributor. See the note below.
- Assign access to
Azure AD user, group, or service principal
- In the
Selectfield, begin entering the name of your AD application (created earlier). When it appears in the list, click on it there. It should now be listed as one of the “Selected members”.
- Click Save
Note: You might want to give your application fewer permissions than what a
Contributor role gets. The Azure docs have a list of all the built-in roles for Azure resources.
Follow the instructions in the Azure docs page:
except you should use the existing resource group you created earlier, i.e. don’t create a new one.
The Storage account name you choose is what Provider calls the
# Example AZURE_ACCOUNT_NAME (Storage account name) troystorageaccount1
Use the same
Location as your resource group.
The other fields can be left with their default values unless you want to change them.
Wait for it to say, “Your deployment is complete.”
See the Azure docs page:
Go to the subsection about access keys and follow the instructions to view your new storage account’s credentials.
Copy the value of one of the keys (e.g. key1, not the connection string). That’s what Provider calls
# Example AZURE_ACCOUNT_KEY (Storage account access key) 93uKDkbjfnSUNPKw2tpe0LOM+3Wk+OSkNmgwhzjvzDw1d3sKVhMRTC5ikvN0r3zsx8eQrmT9Wgjz22iLPu3aGw==
You now have all the Azure credentials Provider needs. See the instructions near the top of this page about how to give those Azure credentials to Provider.
You now have a storage account, but you don’t have any data stored under it yet. To get some data stored in Azure Storage, the easiest option is to use Azure Storage Explorer, a free desktop app that works on Windows, macOS and Linux.
Azure Storage can store blobs, files, queues and tables. To work with Ocean Network, you should store your files in Azure Blob storage (also called object storage), not Azure Files.
Besides Azure Storage Explorer, there are many other Azure Storage APIs, libraries and tools.