Are you considering (or actively building) applications in Windows Azure?  If so, how are you building and architecting your solution?  Tomorrow, Sep 7 between 3pm-5pm CDT, join me for a presentation to the Windows Azure User Group (a virtual user group that meets on-line) on 10 Architectural/Design Considerations for Running in the Azure Cloud.  To sign up and attend click here.  If you can't make it and would like the slides from the event, click here.

As a consultant I am expected to be up to speed, if not an expert, on a wide range of technologies.  With the pace that technology changes, staying current can be an exhausting and seemingly never ending challenge.  One of my favorite avenues for satisfying these expectations is attending local user groups and conferences. 

My favorite local conference is easily Twin Cities Code Camp.  This fall?s camp takes place October 9th and 10th and will be held at the University of Minnesota (yep, a weekend but the Vikings don?t play until Monday so that?s no excuse not to attend).  The organizers do a fantastic job of lining up quite a few locally and nationally recognized speakers and (as always) it is a free event!  The schedule, except for a few slots, has just been published.  There are 45 different sessions scheduled and topics include good old ASP.Net, mobile app development, security, Silverlight, WCF, ALM, Windows Azure and about 10 other subjects (some of which I haven?t even heard of yet).  I?ll be speaking at this event for the first time this year, I?m doing the Intro to Azure session at 3:45 on Saturday so come on out and join me, heckle me, quiz me, grill me ? whatever.

A couple other upcoming events I?ll be attending include this week?s Azure (virtual) Users Group talk on 10 Architectural/Design Considerations for Running in the Azure Cloud and the talk on Expression Blend for Developers at the September 15 MN Visual Studio user group.

I hope to see you out at one or more of these events.

By Jim White (Director of Training and instructor)

I have some exciting news for those trying to learn Windows Azure!  I have been working on our Complete Windows Azure class over the summer.  It is just about done (it should be finalized by Labor Day 2010).

If you or your organization wants to learn Windows Azure, I encourage you to check out the class description and outline on Intertech's Web site here.  Our first public offering is scheduled for October 2010.  If your company needs training sooner than that, please contact Dan McCabe (dmccabe@intertech.com) at Intertech.

Intertech also offers consulting services in Windows Azure.  If your organization needs some assistance in understanding the why/how of Azure, please contact Ryan McCabe (rmccabe@intertech.com).

By Jim White (Intertech Instructor and Director of Training)

If you are a Web developer, you are probably very familiar with managing user state - that is you are familiar with tracking user activity and actions across several request-response exchanges that occur in Web applications.  Since HTTP is a stateless protocol, developers over the years have developed all sorts of means to manage state.  You'll even find an MSDN page providing alternatives and recommendations for state management here.  Cookies, hidden fields, and query strings are some client-side options to tracking user state.    When it comes to managing that state on the server-side, most Web developers rely on session objects.

Session Objects in Azure

There are sessions in Azure just as you would find them in ASP.NET applications.  For example, the following lines of code would add and retrieve a Customer from the current HttpSessionState object.

Session["currentCustomer"] = new Customer("Joe", "Smith");

Customer customer = (Customer)Session["currentCustomer"];

So, the Web server in Azure provides you the same session object you are accustomed to working with today.  However, the sessions in Azure are not ?sticky.?  In other words, if you have multiple Web role instances, Azure does not guarantee to route a user?s requests to the same Web role instance.  In fact, the load balancer in Azure works to the opposite effect. The load balancer tries to route traffic evenly to all instances regardless of the request's origination.  Since each Web role instance has its own session, there is effectively no shared session data across Web roles.

To help understand the implications of this issue, say the user represented by the "client" box above is working with a retail Web site.  If "Request 1" was a request to add an item to the shopping cart, when the user goes to check out per "Request 2", she will be very disappointed to find the item missing from her cart.  That is because the item is in a shopping cart session object associated to the first Web Role and not the shopping cart object associated to the second Web Role.

In-memory Web Server Session Management

Session objects, when managed by the Web server, are often stored in memory on the Web server.  Of course in Windows Azure, each Web role has its own Web server.  You could remove this issue by having only one Web role.  By having just one Web role, you remove load balancing from the picture.  Limiting your application to one Web role also limits your application's scalability - presumably one of the reasons you chose to move your application to Azure in the first place.  If you eventually need to scale your application to involve multiple instances, you now need to rewrite parts of your application. 

Even if scalability is not a chief concern, there are other reasons you should hesitate before using standard Web server sessions to manage user state.  In-memory sessions use a precious commodity: memory.  If your application puts a lot of data in session, your application can experience out of memory errors.  Also, if your Web role fails, Azure establishes your instance on another server.  However, the in-memory session information - still back on the other Web server - is lost.

Session State in Azure Storage

What's the solution to this session dilemma?  Persist your session objects in Azure Storage versus in-memory on the Web server.  Putting session information in table storage has several advantages.  It scales.  It allows all instances of a Web role (or any role) to share session data.  It is always available and stored in triplicate. So, if your role goes down, session data is not lost.

Setup to Use Table Storage for Sessions

To use table storage to persist sessions, you must first modify the Web.config file in your Web role project to indicate you are using a different session provider.  Add the following element to the <system.web> element, replacing ?yourWebAppName? with the name of your Web application.

<sessionState mode="Custom"

       customProvider="TableStorageSessionStateProvider">

    <providers>

      <clear/>

        <add name="TableStorageSessionStateProvider" type=  "Microsoft.Samples.ServiceHosting.AspProviders.TableStorageSessionStateProvider"

             applicationName="yourWebAppName"/>

    </providers>

</sessionState>

Note: you may find Visual Studio editors provide false warnings on this entry indicating type and applicationName attributes are not allowed.

Secondly, get a copy of the Windows Azure Platform Training Kit (the June 2010 release is available for download here).  In the kit, look for an AspProviders Visual Studio project (its in the WindowsAzureDeployment lab under Source\Assets).  This project contains a TableStorageSessionStateProvider class that implements the Table Storage Service Provider.

You can either include the AspProviders project into your solution or compile the DLL and reference the DLL.  The AspProviders project uses the DataConnectionString configuration setting to get all the information needed to access table storage for your account.  Make sure you establish your DataConnectionString and point it to use development storage or your storage in the cloud as appropriate.

Using Sessions Persisted in Table Storage

With these adjustments, you use the same API calls as those listed at the top of this blog entry to add and retrieve data to/from session.  However, now user state is not held in memory on the Web server.  It is, instead, persisted to table storage.  The TableStorageServiceProvider actually creates a ?Sessions? table in table storage by default.  So you don't even have to set up or manage the actual table schema in table storage.  After you test and run your application, you can use a tool like Cerebrata?s Cloud Storage Studio to explore the "Sessions" data table.

When exploring the Sessions table in table storage, be aware that the data is not stored in a fashion for easy viewing.  In fact, the actual session data is packed away in an associated blob storage record.  Also, don?t forget that objects must be serializable if they are to be stored in session.

By Jim White (Director of Training and Instructor)

This is the second of a two part blog post on considering Windows Azure for your applications.  See the 1st post here.

Cloud computing, and in particular Microsoft Windows Azure are hot topics in IT circles today.  A report on CNN has even suggested that Internet users will live in the cloud by 2020 (see the report here).  While you and your organization is probably investigating Azure and other cloud computing technologies, here are a few things you should know before you jump into running your applications and data in Azure.

1. Migration is not free.

Azure is a different environment. Despite what some in the industry might lead you to believe, your existing .NET applications are going to need work before they can run in the Azure cloud. You can?t just flop an existing ASP.NET application on to the cloud without some thought. Architecture and design are still important steps on the way to cloud migration. Sriram Krishnam, a member of the Windows Azure program management team and author of Programming Windows Azure (O?Reilly) said it best in this passage from his book.

?you?ll often find yourself needing to rewrite/modify parts of your application. Though platforms such as Windows Azure try to make it as seamless as possible to port applications, there are some inherent differences, and you?ll need to make at least some small changes.?

Yes, you can leverage existing .NET knowledge and code, but you still have many things that operate a bit differently in the Azure environment. File storage/access, logging, diagnostics, security, and data storage/access are just a few aspects of your application that you?ll want to look at in migrating to the cloud.

2. Scaling is not automatic

One of the key selling points of Azure, and cloud computing in general, is that it allows organizations to scale their applications and data storage on demand without provisioning new hardware, bandwidth, software, etc. Azure does provide as near-to-infinite set of resources to host your applications and data as can be thought of in the real world. Moreover, these resources can be requisitioned within minutes or hours, versus days or months in normal provisioning cycles. However, the IT community often misunderstands that the scaling in Azure does not happen automatically. That is, if your application suddenly starts to see a huge spike in requests, Azure does not detect that and automatically requisition more hardware and virtual environments for application to run in to meet the onslaught of demand. You must either manually, or through code your write or purchase, detect the need to scale and then provision Azure resources (and deploy your apps/data) to answer the blitz of new requests.

If this doesn?t sound easy, you are right. In fact, one of the reasons that Microsoft does not yet provide automatic scaling and provisioning in Azure is that even they are not yet sure how to determine a scaling need. What does ?busy? or ?stressed? mean? Without a way to effectively measure and configure a response to measures, it is difficult to scale automatically.

3. Backups aren?t there.

Just one incident of not backing up your hard drive and losing sensitive data is usually enough to teach most computing resource users to backup early and often. We in the industry go to great lengths to backup crucial data; often contemplating and dealing with the unthinkable just to ensure the survival of our organization?s data. Many that explore Azure are glad to hear that all data stored in Azure, no matter what the form (Azure Storage or SQL Azure) is replicated three times (on three different physical servers). A common misconception is that this should suffice for ?backups.? The triplicate replicas of data stored in Azure is really meant for high availability of the data. It is not for backups! Backups are still your responsibility. When something happens to one of the three replicas, the other two take over while Azure rebuilds the third data replicate. Therefore, your applications and customers that use the data are never worse for wear. However, if your application (or a malicious user) creates bad data, or if the whole data center goes up in smoke (unthinkable ? maybe ? but remember backup is about contemplating the unthinkable), your data is gone, corrupt, unusable, etc? Backups in the cloud are not any picnic either. You do not have access to the servers that host your data and today there are no Azure tools that provide backup management activity. You must either write or buy code and tools to backup your data out of the cloud and put it somewhere it is safe.

4. Logging/Diagnostics require some thinking.

As developers, you cannot debug applications in the cloud. You debug your applications in a development environment and then deploy them to the cloud where you do not have very good vision of what is happening. Azure provides the means to capture all the standard log and diagnostic data you want. However, you must code and configure your applications to request and capture this data. You must also code or buy a solution for extracting this data to an environment near you where you can make sense of the information. In the case of some problematic event, you want to make sure your Azure logging/diagnostic solution can report back to you in a timely manner with enough data that you can react to the current situation. If this sounds like you have to put some planning and design into bug and downtime events (and the handling of those) you are correct again!

5. Savings are negated by poor architecture.

Azure and other cloud platforms are about saving money. Instead of buying and operating a whole data center yourself, you purchase a slice of a data center and share in the cost of its operation. Further, you ?pay for play.? That is, you provision the computing resources you need as/when you need them and you pay for exactly what you use ? nothing more and nothing less. All true, but because you are now charged for each computing hour, each transaction, GB of data stored, and GB of data bandwidth, you can write applications that exceed traditional data centers. Software engineers, as Azure engineers, need to think like an MBA graduate as much as they think like a CS major. Writing applications that are extremely (and often needlessly) chatty increase costs. Data stored too long or in the wrong place can inflate costs. Azure offers multiple data storage options for a reason. Some are cheaper and more scalable in certain situations. As you are charged for every hour of computing time, underutilized application instances meant to handle the occasional spikes in demand cost you money in Azure. Design and architect your applications and data storage to leverage the pay-as-you-go model without evaporating your savings on the data center.

If you would like to learn more about Windows Azure, check out our training course here. If your consultants can help you ?navigate in the clouds?, please contact Ryan McCabe for assistance. Also, I invite you to join the Windows Azure User Group ? it?s a virtual users group ? where you?ll find some great educational material and virtually meet people discussing Azure.

By Jim White (Director of Training and Instructor)

This is the first of a two part blog post on considering Windows Azure for your applications.  See the 2nd posting next week.

Cloud computing, and in particular Microsoft Windows Azure are hot topics in IT circles today.  A report on CNN has even suggested that Internet users will live in the cloud by 2020 (see here).  Why is cloud computing ?the next big thing? in IT? Here are five reasons you should explore Windows Azure ? Microsoft?s cloud computing platform.

1. Save money

In fact, I really wanted to name this blog post 10 reasons to consider moving an application to the cloud and have #1 ? 5 be about money. As that would be unfair to the reader, I kept it to five, but the importance of saving money via cloud computing and Azure should not be understated. Buying, building, and operating a data center, no matter how small it is, is an expensive endeavor. The initial costs for hardware, network, and software give managers sticker shock. Operating costs (electricity, heating/cooling, and personnel to operate it) are growing expenses. Just about the time you have it up and running efficiently, it (hardware, networks, OS, etc.) need to be upgrade. Because the demands on the data center do not stay constant, most of the time your data center is either over or under utilized. If your business takes off, you may not have enough computing resources to meet the need. If stay in front of resource demand, you have long periods of excess capacity.

Windows Azure affords organizations data center resources (and more) in a pay-for-use model. Cloud computing is called utility computing because like water or electricity, you pay for what you use and not what you think you might use ? or for having to build the entire water pump house or electric generator all on your own. Don?t take my word for the savings. Prove it for yourself. In fact, so confident are Microsoft and others about the savings in Azure that there are two return on investment (ROI) calculators that allow you to size up your savings. Microsoft?s own, very detailed and thorough ROI calculator is here. You can also check out Neudesic?s calculator (itself a cloud application) out for a simpler-big-picture ROI report here.

2. Scaling

Most organizations have a hard time building an application or data store that must ultimately support incredible scale. Most applications and data stores start out relatively small, and through success take on more and larger needs. How do you plan to take that garage web site to the top ten most visited sites of the Internet? Azure allows you to scale by adding more resources in support of your application on demand. Whereas it might take days or months to provision new hardware, networks, software, etc. to support increases of demand on your applications and data stores today, you can provision what for all practical purposes is an infinite supply of computing resources in Azure in a matter of minutes or hours. Remember, scaling computing resources is often a two way street. You may need extra storage and computing resources today, but you do not want to pay for that extra forever do you? Say you have an application that you know is going to achieve a spike in utilization during a short period (think Dominos Pizza on Super Bowl weekend ? they are in fact a user of Azure ? see here) but after that, it?s back to normal computing. Azure allows you to take down those same resources as fast as you put them up.

3. High Availability

No one likes to have to answer the beeper or cell phone go off in the middle of the night ? especially when it?s to learn that a server, network device, database, or application has gone haywire. Software engineers like to code and solve business problems. Baby sitting equipment typically comes with the territory, but it can hardly be classified as satisfying work. Let someone else manage the environments that run and support your applications and data store! Azure provides a service level agreement (SLQ) backed environment (computational resources are guaranteed at 99.95% up time ? see here). If you doubt the SLA is being achieved, you can see an Azure data center dashboard that tells you what the up-time of the Microsoft cloud is today (see here).

In Azure, if the server box running your application or data store dies or an application running on an Azure virtual machine falls over, the Azure Fabric Controller brings it back up automatically. Furthermore, all data stored in Azure (Azure storage or SQL Azure) is replicated three times. So, applications and more importantly users should never see unavailable applications or data. Geographical redundancy is something Azure developers have requested and may be available in future releases. That is, your application and/data may be made available in multiple Microsoft data centers located across the world to provide expanded availability. Even if one data center were down, the other would still be up and take on the requests. With Azure, organizations can focus on their core business solutions and not on keeping a data center operational.

4. Rapid Application Development/Testing

If you have ever worked in a relatively large organization and started on a new project, you have probably experienced the team down-cycles at the beginning of a project when new equipment, development environments, etc. have to be approved, then ordered, setup and made operational. What a drag on the ?rapid? part of RAD development. If someone in your organization has an idea about how to make the company more money or improve operations, how long does it take to get that idea in front of customers or decision makers? If the answer is too long, perhaps you need a better way to provision the RAD environments and get product to market faster. Azure offers a ready and waiting environment with developer tools to allow teams to get prototypes and early versions of applications up and running quickly and cheaply. Moreover, since Azure is .NET based, a lot of the tools and existing .NET skills can be applied and reused in cloud computing. If the project fails or needs to be moved to a more tightly managed environment, the cloud resources can be shed just as quickly with no wasted and unused equipment standing around. Azure can serve as an incubator to hot ideas that can help businesses move and react quicker.

5. Geographical Distribution

Who are your customers today? Are they all in your city? Your state? Your country? Increasingly, applications, content and data is shrinking the world. However, your applications and data are still probably located where you are. Is this a problem? Maybe. If your customers are around the world, you may be required to have your applications and data where the customers are and not where you are. Increasing legal and regulatory demands from governing bodies (local, national and international) make it necessary to put applications and data within certain geopolitical boundaries. Tax laws may encourage you to host your application and data in certain areas of the world while avoiding others. And even though the speed of light that pushes data through fiber optic cable is good, it may not be enough when that data has to travel half way across the world in a time-critical transaction. As many organizations ?go global?, they need to find a way to distribute applications and data geographically (without the expense and trouble of building their own data centers in several locations across the world). Microsoft has already done that for you. Azure is hosted in data centers around the world. When you build your application for any one Azure site, you can easily deploy it to an ever-increasing set of Azure data centers with no changes ? free (except for the actual resources you use in the cloud) geographical distribution.

If you would like to learn more about Windows Azure, check out our training course here. If your consultants can help you ?navigate in the clouds?, please contact Ryan McCabe for assistance. Also, I invite you to join the Windows Azure User Group ? it?s a virtual users group ? where you?ll find some great educational material and virtually meet people discussing Azure.

By Jim White (Director of Training and Instructor)

Last week, my fellow Intertech colleague and Microsoft MVP, Tim Star, and I presented the Windows Azure Bootcamp for the Twin Cities.  According to Microsoft reps, we broke the record for the most attended bootcamp in the US with nearly a hundred people at the event.

A common question that I received during and after the bootcamp was "Why would I want to use Windows Azure Table Storage versus Windows SQL Azure to store my application data?"  A good question with the answer dependent on your application and data needs.

Table Storage and SQL Azure Defined

First, allow me to backup and set the stage a little for this discussion.  SQL Azure is essentially SQL Server in the Microsoft cloud computing environment known as Azure.  That is not quite true in that SQL Azure currently has a number of limitations and unsupported features that SQL Server 2008 has (here is a starter list of limitations:  http://msdn.microsoft.com/en-us/library/ee336245.aspx).  I like to say that SQL Azure is either SQL Server 2008 minus or SQL Express plus depending on how you want to view it.  Table Storage is one of three alternate storage mechanisms built into Azure that is collectively called Windows Azure Storage Services.  The other two being queues and blobs.  Table Storage allows you to store serialized entities in a table, but the term table here  is not a relational database table.  To provide people with some analogy they can use to get their arms around Table Storage, I like to tell people to think of Table Storage as a fancy spreadsheet.  You can store the state of your entities in the columns of the spreadsheet.  However, there is no linkage or relationship (therefore joins) between entities - at least none that is automatically managed and maintained by Azure.  There are no custom indexes - at least not today.

Interestingly, when Azure was first introduced in 2008, SQL Server was not part of the original picture.  Because of developer negative reactions, SQL Azure was added to the next preliminary release in 2009.  There is a growing faction that is trying to get the software community to look at SQL alternatives.  The "No-SQL" community (see here and here), to some extent, has influenced part of the Azure cloud computing platform through the Table Storage option, but not enough to eliminate it from the Microsoft cloud.

While both SQL Azure and Azure Table Storage provide data persistence via table structure, there are a number of differences between them.  The sections below outline some of the key differences and factors you want to weigh before building an application for Azure that requires some form of table persistence.

Scale and Performance

When looking at sheer volume, Table Storage is today far more scalable than SQL Azure.  Given a storage account (storage accounts hold blobs, queues and tables) is allowed to be 100TB in size, in theory your table could consume all 100TB.  At first glance, a 100TB chunk of data may seem overwhelming.  However, Table Storage can be partitioned.  Each partition of Table Storage can be moved to a separate server by the Azure controller thereby reducing the load on any single server.  As demand lessens, the partitions can be reconsolidated.  Reads of Azure Table Storage are load balanced across three replicas to help performance.

Entities in Table Storage are limited to 1MB each with no more than 255 properties (3 of which are required partition key, row key, and timestamp).  That seems like an absurd number, and it is, but remember that there are no relationships and joins in Table Storage.  Therefore, you might need some wide tables to handle associated data.

Today, SQL Azure databases are limited to 1GB or 10GB.  However, sometime this month (June 2010), a 50GB limit is supposed to be available.  What happens if your database is larger than 10GB today (or 50GB tomorrow)?  Options include repartitioning your database into multiple smaller databases or sharding (Microsoft's generally recommended approach).  Without getting into the database details of both of these database design patterns, both of these approaches are not without issue and complexity, some of which must be resolved at the application level.

Data Access

Data in the cloud, be it in SQL Azure or Azure Table Storage, can be accessed from in or out of the cloud.  To access data in SQL Azure, all the standard tools and APIs apply that work with SQL Server.  Meaning, your existing .NET/SQL Server knowledge and experience can be heavily leveraged.  ADO.NET and ODBC APIs can be used by application code to access the SQL Azure database.  Tools like SQL Server Management Studio and Visual Studio can be pointed to the SQL Azure instance and manipulate the schema and data just as you do today with SQL Server. 

Access to Azure Table Storage is accomplished either via REST API or Storage Client Library provided with the Windows Azure SDK.  Using the REST API allows client applications to communicate and use data from Table Storage without having detailed and specific knowledge of an Azure API, but it is more complex and difficult to work with.  The Storage Client Library (which leverages LINQ to Objects) provides a layer of convenience but requires the application reference the Storage Client Library APIs.  REST and the Storage Client Library incur a learning curve that is typically not there when using SQL Azure.

Portability

As mentioned, data in SQL Azure and Table Storage can be accessed from applications in and out of the cloud.  That means applications can be moved in or out of the cloud and still deal with the data in the cloud in the same way.  However, one question that may need to be considered is whether the data must always live in the cloud?  Applications generally view data in SQL Azure similar enough to data in a normal SQL Server database as to allow the data to migrate back and forth between the cloud and on-premise databases.  In fact, there are even migration tools to help move data between instances of SQL Server and SQL Azure. 

However, given the unique nature and access APIs of Table Storage, portability of the data is not as straight forward.  Table Storage tightly couples your data to the cloud.  Moving the data out of the cloud would require an on-premise data storage alternative, a data migration strategy, and likely require application code changes.

Transactions and Concurrency

SQL Azure supports typical ACID transactions for work within the same database.  Transactions across databases are not supported.  SQL Azure allows for typical optimistic and pessimistic concurrency strategies.

Table Storage supports transactions for entities in the same table and table partition, but not across tables or partitions.  Additionally, only 100 operations or less (what is called a batch in Azure Table Storage) can be part of the transaction.  Only one operation can be performed on each entity in the batch, and the batch must be limited to 4MB.  Table Storage abides strictly by an optimistic concurrent strategy.  If, on commit of the transaction, data has been changed by another process the whole transaction must be rolled back and retried.  Due to this single concurrency strategy, a built-in retry option is provided with the Storage Client Library.

Queries

Using Table Storage, queries are limited to 1000 entities by default.  If more than 1000 entities are found, a continuation token is returned and must be used by the application to retrieve the next set of entities.  Queries that take longer than 5 seconds also return a continuation token.  Queries that take longer than 30 seconds are cancelled.  Data in Table Storage is organized by partition key and indexed by row key.  Because there are no custom indexes in tables, queries by partition key and row key are fast, but queries that do not use partition key and row key are slow.

Generally speaking, SQL Azure has no limitations, issues or special programming requirements to work with large queries.  Good database and index design can help improve performance of queries; especially large ones.

Column types

Columns in Table Storage are limited to the types in the table below.

Cost

Perhaps the most unique aspect to designing and architecting applications for the cloud is that it requires developers to think like businessmen.  Each technical choice often has direct costs associated with it when developing for the cloud.  The choice in data storage can have a huge impact on the cost of running an application.

Azure Table Storage costs 15? per GB of storage per  month.  Additionally, you pay 1 cent per 10,000 transactions with Table Storage.  SQL Azure costs are $9.99 for 1 GB of storage per month ($99.99 for 10GB). 

See Microsoft's sight for more details and specifics on costs here.

Bottom Line

Chris Hay and Brian Prince in their forth coming book Azure in Action (published by Manning - see here) provide a synopsis of the SQL Azure vs. Table Storage in a few paragraphs.  "If size is the issue, that would be the first flag that you might want to consider Azure Tables. As long as the support Tables has for transactions and queries meets your needs. The size limit surely will, at 100TB."  Further they suggest sophisticated transactions, or a complex authorization model might require the services of SQL Azure.  And as shown by the cost table above, "The final consideration is cost. I can store a lot of data in Azure Tables for a lot less money than I can in SQL Azure. SQL Azure is giving me a lot more features to use (joins, relationships, etc.), but it does cost more."

Future

We are given every indication by Microsoft that SQL Azure will have far more capability in the future - akin to the SQL Server you might find in your data centers today.   So some of the comparison above may be moot or less important over time.  Additional functionality is also being proposed to Table Storage as well.  For example, support of secondary (non-key) indexes is already been suggested for a future release (see here).  However, key architectural differences between SQL Azure and Table Storage will remain and leave application designers having to pick the best option for their systems.  Welcome to cloud computing.  There is a lot of ROI to be had by running in the cloud, but only with proper application design and architecture.

If Intertech can help you  negotiate the issues of cloud computing, please contact Ryan McCabe at ryan.mccabe@intertech.com.

By Jim White (Intertech Director of Training and Instructor)

In general, Microsoft has tried to make writing Web applications for the Azure platform akin to writing for your local IIS environment.  In fact, Microsoft lists "use your existing development skills to build cloud applications" on the Azure Web site as one of its major selling points.  I think, to a large extent, .NET developers will find building or moving an application to Azure straightforward.  However, to be fair, there are also a number of areas where the application will change to deal with the reality of this new platform and, more generally, cloud computing.  One of those areas is simple file access and/or local storage.

File Access in the "Normal" ASP.NET world

For example sake, say you needed to read/write from a simple text file.  Perhaps you need to read some configuration information from the file, or perhaps some SQL that you need to execute was in the file.  Whatever the reason, accessing this file is pretty straightforward.  In a "normal" ASP.NET application, accessing a file that is stored on a virtual path on the Web server is quite simple and might look like the code below (in C#).

    //----------------------- read config contents --------------------------------
    try
    {
        string s = System.IO.File.ReadAllText(Server.MapPath("myConfigs") + "myFile.txt");
        //... do your work with s
    }
    catch (Exception myException)
    {
       ...    }

    //----------------------- write config contents --------------------------------
    if (!text.Equals(""))
    {
        System.IO.File.WriteAllText(Server.MapPath("myConfigs") + "myFile.txt", text);
    }
    else
    {
        System.IO.File.WriteAllText(Server.MapPath("myConfigs") + "myFile.txt", "no data");
    }

File Access in the Cloud

Now, how would that look in the cloud?  Well, first you must step back and consider that local storage in the cloud is not the same as that under our local IIS environment.  Local storage, is assumed under an ASP.NET application running on a local IIS server.  In the Azure world, you must first configure your role to request local storage as it is deployed.  This can be done with the Role Editor in VS 2008.  The Local Storage tab on the editor allows you to specify the name, size and clean/role policy for the storage as shown below.

The information from this editor is stored in the service definition file of the role and used to configure the Azure environment when the application is published to the cloud.

Local Storage Location

So now you have local storage, but where is it?  When you deploy your application (consisting of either a web or worker role) to Azure, a storage area is assigned to your application.  The exact location is at the Azure servers discretion.  The physical path of the folder for your assigned storage can be located through the Azure API.  In this case, request the location of your local storage through, Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.  The RoleEnvironment class represents the Azure environment that your instance of your role (web or worker) is running in.  So, reading and writing to that file might look something like the following code in an Azure world.

    //----------------------- read config contents in Azure -----------------------
    try
    {

            LocalResource myConfigsStorage = RoleEnvironment.GetLocalResource("myConfigs");

        string s = System.IO.File.ReadAllText(myConfigStorage.RootPath + "myFile.txt");
        //... do your work with s
    }
    catch (Exception myException)
    {
        ...    }

    //----------------------- write config contents in Azure------------------------
    if (!text.Equals(""))
    {

           LocalResource myConfigsStorage = RoleEnvironment.GetLocalResource("myConfigs");

        System.IO.File.WriteAllText(myConfigStorage.RootPath + "myFile.txt", text);
    }
    else
    {
        System.IO.File.WriteAllText(myConfigStorage.RootPath + "myFile.txt", "no data");
    }

Easy enough you say.  Yes, the API is straightforward enough so as to appease most .NET developers.  However, a bigger issue with regard to this local storage must be considered.

Local Storage is Temporary and Instance Relative in the Cloud

Local storage, like that just used in the example to read/write text to myFile.text is temporary in Azure!  So, if the virtual machine supporting your role dies and cannot recover, your local storage is lost!  Therefore, Azure developers will tell you, only volatile data should ever be stored in local storage of Azure.  Furthermore, I eluded earlier to the fact that the storage was per instance of a role.  Importantly, local storage is not shared across multiple role instances.  Usually, people move applications to the cloud to take advantage of the scalability and redundancy multiple instances provide.  But these instances each have their own local storage and cannot access each others.  If the data in the local storage is changing and must be shared, you need to rethink the application design.

Local Storage Alternatives

If you need to read/write data in the cloud in a non-volatile and shared way , take advantage of Azure's storage options:  blobs, table storage, queues and SQL Azure databases.  My co-worker Tim Star (Microsoft MVP) has described them in his blog post here.  This may require some application re-architecting, but in the end, your application will run better, stronger, faster in the Azure cloud.

In my last post I described how to configure your machine to get started developing Windows Azure applications.  We built a simple web role (web site host) and took a look at what role the development fabric played in developing a Windows Azure application.  As soon as we try to build more than the most basic web application we are going to have to start considering our storage options.  Following is a high level overview of the four main storage options.  More details for each option will follow in separate posts.

Table Services : The table service is designed to store non-relational entity type data, serialized DTOs for example.  Redundancy is built in. The size of each row of data is limited to 1 MB and there are some fairly restrictive rules around what types of data may be stored in the table.  We should consider using the Table Service rather than a relational database to persist our data.  Not everything has to be stored in a relational database.  If you can keep your entity relationships simple, live without the joins and manage the relationships yourself, Table Storage looks to be a more simple, cost effective, and scalable approach.  Consider using table storage instead of a traditional database and there is even an ASP.Net session provider available in the SDK that uses table storage.

Blob Storage : There are two kinds of blob storage.  Block Blobs are designed to store up to 200GB of binary data and are optimized for streaming.  A Page Blob can store up to 1 TB of binary data and is designed for random access.  Like table storage, redundancy is built in.  This storage mechanism is ideal for Streaming videos, waves, images, or working with random access files such as fixed length field flat file

Queue Service : The queue service supports 8 KB XML messages and also has redundancy built in.  We are not going to let the size limitation scare us, the intention is not to allow us to put large binary objects on the queue for processing rather the intent is that the message would contain some sort of instruction and if need be the message can reference some object already in storage via a key (the message just contains a pointer to the object we want the worker role to act upon).  The key with using a Queue is this is not a chatty interface this should be used more as a  fire and forget.  Queue messages should be communicating a task the sender wants performed, not performing some type of query that requires a response.

SQL Azure : For the sake of this post we can say SQL Azure is similar to the Sql Server we know and love and that it is designed to store and mange relational data.

By Jim White (Director of Training and Instructor)

I just finished a review of Azure in Action by Chris Hay and Brian Prince (Manning Publications).  This book is to be formally released later this summer (August 2010).  You can get an early electronic copy at www.manning.com/hay.

If you are exploring or actively developing applications for Windows Azure, this is a must have book.  Windows Azure is Microsoft's public platform as a service (PAAS) entry in the cloud computing arena.  I have already written in this blog (see here) about the potential importance and impact of cloud computing and platforms like Windows Azure on our industry.  I believe this is where our industry is headed and Azure in Action will help you navigate the "clouds"in Windows Azure form of this new atmosphere.

While this book does provide instruction on the basics of Windows Azure, the Azure SDK and API, potential readers will be happy to find that its real assistance is in getting you to think about real world design and architectural aspects of your applications running in the cloud.  Hay and Prince challenge the reader to think about how running in the Azure cloud impacts performance, organization of data and costs of operating in the cloud.  They do a good job of laying out options and providing you with the pros/cons of those options as they might apply in your application setting.

For example, in Chapter 6 (called "Scaling web roles"), they cover session management.  While you may find that porting a typical ASP.NET application to Azure is straight forward, Hay and Prince inform you that with regard to something like persisting data across requests you have to consider potential issues such as the fact that "Windows Azure does not consistently implement sticky sessions."  As you scale, session management becomes a bit more complicated.  They provide information about what session management is and how it works and then provide alternative implementation options (such as In-Process and Table Storage session management).

Azure in Action also peels back the covers on how Azure works under the covers - in the Microsoft data centers.  I found chapter 3 (titled "How it works") particularly fascinating.  The information they provide on the lengths and efforts Microsoft have taken to provide the computing foundation and data centers in which Azure runs should give all of us writing applications for this platform a bit of confidence in the platform's future.

A word of warning, the Manning Early Access Program (MEAP) version of the text is still in the midst of the editing process.  So there are still quite a few typos, grammatical issues, and even a couple of code samples that need to be corrected.  One would assume those would be taken care of before the book formally comes out this summer.

If you are looking for an API reference on Windows Azure, dial up the MSDN site on your local browser.  But if you are looking at how to design/write real world applications that scale and perform well in the Windows Azure cloud, help yourself out and download the MEAP version of Azure in Action today.