Microsoft Windows Azure Development Cookbook: Realize the full potential of Windows Azure with this superb Cookbook that has over 80 recipes for building advanced, scalable cloud-based services. Simply pick the solutions you need to answer your requirements immediately.

The Windows Azure Blob Service is the Windows Azure Storage Service feature that manages the storage of file-based entities referred to as blobs. In this chapter, we will focus on the Blob service. In the next two chapters, we will look at the other storage services: the Table service and the Queue service.

The definitive way to access the storage service is through the Windows Azure Storage Service REST API. The Windows Azure Storage Client library is a high-level managed library, included in the Windows Azure SDK, which abstracts the RESTful interface into a set of classes. The Windows Azure Storage Client Protocol classes contained in the Storage Client library expose functionality not provided in the core library classes. Specifically, the protocol classes provide access to the underlying HttpWebRequest object used to invoke the RESTful operations.

Nearly all the recipes in this chapter use the Storage Client library. Scattered through the recipes are examples of how to use...

The Windows Azure Blob Service allows metadata and properties to be associated with a blob. The metadata comprises a sequence of developer-defined, name-value pairs. The properties comprise HTTP request headers including: Cache-Control, Content-Encoding, Content-MD5, and Content-Type. The Blob service also supports metadata and properties for blob containers. Requesting a download of the blob attributes—its metadata and properties—is an efficient way of checking blob existence as only a small amount of data is downloaded.

The Blob service uses the Content-MD5 property to validate blob upload. When specified, the Content-MD5 property must be set to the base64-encoded value of the MD5 hash of the blob data. The Blob service returns an error if this value does not match the value it calculates from the uploaded blob content.

In this recipe, we will learn how to set and get blob properties and metadata. We will also learn how to calculate the Content...

The Windows Azure Blob Service uses a simple organizational structure for containers and blobs. A storage account has zero or more containers each of which contains zero or more blobs. Containers contain only blobs and may not contain other containers. There is no hierarchy for containers.

The Windows Azure Storage Service REST API provides support for a simulation of a hierarchical directory structure through an ability to parse blob names containing a special delimiter character and navigate the list of blobs while taking account of that delimiter. This delimiter is the forward-slash symbol (/). The Windows Azure Storage Client library exposes this feature through the CloudBlobDirectory class.

The CloudBlobDirectory class provides methods allowing blobs to be enumerated in a way that takes account of the directory structure built into the naming convention used. A blob name can include multiple levels of directory.

A CloudBlobDirectory object can be created using either...

The Windows Azure Blob Service supports the creation of read-only snapshots of a blob. A storage account is billed only for those blocks and pages in a snapshot that differ from those in the underlying blob. A blob snapshot is useful in providing a backup for a blob as it can be used to reset the blob to an earlier state. Indeed, multiple snapshots can be made over time allowing a historic record to be kept of changes to a blob.

An important use case for blob snapshots is provided by the Azure Drive feature. An Azure Drive is a page blob comprising an NTFS-formatted VHD that can be attached to a hosted service role instance and accessed as an NTFS drive. To avoid read-write conflicts, a single VHD page blob can be attached to only one instance at a time. However, a blob snapshot is read-only, allowing no possibility of write contention, so an arbitrary number of instances can attach a VHD snapshot simultaneously.

A blob snapshot is created using the CloudBlob...

The Windows Azure Blob Service supports a simple two-level hierarchy for blobs. There is a single level of containers, each of which may contain zero or more blobs. Containers may not contain other containers.

In the Blob service, a blob resource is addressed as follows:

http://{account}.blob.core.windows.net/{container}/{blob}

{account}, {container}, and {blob} represent the name of the storage account, container, and blob.

This addressing convention works for most uses of blobs. However, when using Silverlight the runtime requires that a cross-domain policy file reside at the root of the domain and not beneath a container, as would be the case with the standard addressing for blobs. The cross-domain policy file allows a web client to access data from more than one domain at a time. (http://msdn.microsoft.com/en-us/library/cc197955(VS.95).aspx) Microsoft added support for a root container, named $root, to the Blob service, so that it could...

The Windows Azure Blob Service supports two types of blobs: block blobs optimized for streaming, and page blobs optimized for random access. Block blobs are so named because they comprise blocks and can be updated by either replacing the entire blob or by replacing individual blocks. Page blobs can be updated by either replacing the entire blob or by modifying individual pages.

A block blob can be up to 200 GB, and comprises blocks that can be up to 4 MB. Block blobs larger than 64 MB must be uploaded in blocks and then a list of uploaded blocks must be committed to create the blob. The various upload methods in the CloudBlob class handle this two-phase process automatically. However, there are times when it is worthwhile taking direct control of the block upload and commit process. These include: uploading very large blobs, performing parallel uploads of blocks, or updating individual blocks.

At any given time, a block blob can comprise a set of committed...

An instance of a Windows Azure role comprises several virtual disks (VHD) deployed into a virtual machine. One VHD contains the Guest OS; another contains the role package, while the third is a writable VHD containing the local resources of the instance.

Windows Azure provides the Azure Drive feature whereby an arbitrary VHD, contained in a page blob, can be attached to an instance and then accessed as an NTFS drive. This VHD must be an NTFS-formatted, fixed-size VHD. The VHD can be created directly as a page blob or uploaded like any other page blob.

The Disk Management snap-in for the Microsoft Management Console (MMC) can be used to create and format a VHD on a local system. Once attached to the local system, files can be copied to, or created on the file system of the VHD just as they can with any hard drive. The VHD can then be detached from the local system and uploaded as a page blob to the Windows Azure Blob Service.

A page blob, which may have a maximum...

The Windows Azure Storage Client library provides synchronous and asynchronous versions of nearly all the methods that access the Windows Azure Storage Service.

The asynchronous methods follow the common language runtime (CLR) Asynchronous Programming Model (APM). In this model, asynchronous methods for an action are defined as a pair named BeginAction and EndAction. The asynchronous operation is initiated through a call to BeginAction and is cleaned up by a call to EndAction. BeginAction has a parameter that is a callback delegate and EndAction must be invoked in that delegate.

This apparent complexity can be greatly simplified through the use of a lambda expression to represent the callback delegate. Furthermore, local variables defined in the method containing the lambda expression are available inside the lambda expression. This removes any difficulty caused by a need to pass variables into the delegate. Using a lambda expression, instead of a callback...

Large blobs need to be uploaded to the Windows Azure Blob Service, either in blocks for block blobs or in pages for page blobs. Similarly, blobs can be downloaded in byte ranges. These operations can be implemented in parallel using different threads to upload or download different parts of the blob.

The Blob service has a scalability target for throughput of about 60 MB per second for an individual blob. This creates a limit to how much performance improvement can be achieved through parallelizing operations on a blob. When contemplating parallelization, it is always worth testing the actual workload to ensure that any expected performance gain is in fact realized.

.NET Framework 4 includes the Parallel Extensions to .NET that simplifies the task of parallelizing operations. These provide parallel versions of the traditional for and foreach statements. The Parallel.For and Parallel.ForEach methods can be dropped in almost as replacements for for and foreach...

A storage operation accessing the Windows Azure Storage Service can fail in various ways. For example, there could be an unexpected timeout if the storage service is moving a partition for performance reasons. It is advisable, therefore, to code defensively in the assumption that failure could occur unexpectedly.

The Windows Azure Storage Client library supports defensive coding by providing a retry policy for operations to the storage service. This is done by default, but the retry policy classes support parameterization and customization of the process.

CloudBlobClient has a RetryPolicy property. A storage operation on a CloudBlob object has a retry policy associated with it through the RetryPolicy property of its BlobRequestOptions parameter. These RetryPolicy properties provide access to a RetryPolicy delegate that returns a ShouldRetry delegate which specifies whether or not a retry should be attempted.

The RetryPolicies class provides several...

RESTful APIs have become a common way to expose services to the Internet since Roy Fielding first described them in his Ph.D. thesis (http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm). The basic idea is that a service exposes resources that can be accessed through a small set of operations. The only operations allowed are those named for and which behave like the HTTP verbs—DELETE, GET, HEAD, MERGE, POST, and PUT. Although this appears to be very simplistic, RESTful interfaces have proven to be very powerful in practice.

An immediate benefit of a RESTful interface is cross-platform support as regardless of platform an application capable of issuing HTTP requests can invoke RESTful operations. The Windows Azure Platform exposes almost all its functionality exclusively through a RESTful interface, so that it can be accessed from any platform.

In particular, the Windows Azure Storage Service REST API provides the definitive...

The Windows Azure Storage Service REST API provides the definitive way to access the Windows Azure Storage Service. It provides platform independence for the storage service allowing it to be accessed from any platform capable of using a RESTful interface.

Microsoft also provides the Windows Azure Storage Client Library that simplifies access to the storage service from a managed .NET environment. The high-level classes in this library are in the Microsoft.WindowsAzure.StorageClient namespace. These classes hide from the complexity of dealing with the raw HttpWebRequest and HttpWebResponse classes used with the Storage Service REST API. However, this simplicity comes at the cost of hiding some features of the storage service.

The Storage Client library also provides a set of lower-level managed classes in the Microsoft.WindowsAzure.StorageClient.Protocol namespace that expose the HttpWebRequest and HttpWebResponse...

The Windows Azure Blob Service is hosted in a small number of Windows Azure datacenters worldwide. The Windows Azure Content-Delivery Network (CDN) is a service that enhances end user experience by caching blobs in more than 20 strategic locations across the World.

After the CDN is enabled for a storage account, a CDN endpoint can be used, instead of the storage-account endpoint, to access a cached version of publicly accessible blobs in the storage account. The CDN endpoint is location aware, and a request to the CDN endpoint is directed automatically to the closest CDN location. If the blob is not currently cached there, then the CDN retrieves the blob from the Blob service endpoint and caches it before satisfying the request.

The cache-control property of the blob can be used to specify a time-to-live in the cache. Otherwise, the CDN uses a heuristic based on how old the blob is and caches the blob, for the shorter of 72 hours or 20...