Nuno Filipe Godinho

All about WebDevelopment (ASP.NET & Silverlight)

Sponsors

Microsoft Windows Azure Development Cookbook by Neil Mackenzie

For those of you interested in a good book about Microsoft Windows Azure Development you should really read the “Microsoft Windows Azure Development Cookbook” in http://www.packtpub.com/microsoft-windows-azure-development-cookbook/book.

Currently I’m still reading but I’ll post a review about it as soon as I finish my reading.

Things to take into account when you deliver applications for Connected Devices – Windows Azure to the rescue

Currently one very important space is the development of Applications for Connected Devices like Windows Phones, Windows Slates and so on.

One important elements is that we are currently going more and more to Wireless, and so it’s important to understand the current state of the art on Wireless.

According to AT&T “We will deliver as much data over our network in the first 5 weeks of 2015 as we did all year in 2010”.

Also currently there are several identified congestion factors that are affecting Wireless, like:

  • More subscribers with smartphones consuming data, uploading and downloading photos and videos
  • Spectrum + cell towers + antennas + home users
  • Mobile Operator bandwidth caps
  • Latency of all communications between the server and the destination device, passing by the Cell Towers, Backhaul network, internet, firewalls and so on

And so having this is mint, it’s really important to understand that we as developers cannot continue thinking that we’re going to continue developing our apps and services the same way, without thinking about these network related topics, because this will affect greatly the User Experience of our Solutions. And one important note is that UX is not only related to the User Interface (UI), like some developers try to think, it’s more then that. It’s really about having a great experience while using our applications.

So in order to do great applications for connected devices, and support millions of those devices, we need to think about scale.

Some examples of how we can scale our applications are:

  • Making Database that almost nobody touches, since we really shouldn’t let every single device connect and work on our databases, instead we should really be doing multi-tier application to abstract those devices from the Database, and by doing that we are reducing the stress of our databases and at the same time enabling the possibility of scaling the services that really give us access to our data
  • Place thousands of Web Servers on commodity hardware, by scaling out their app
  • Replicate Data
  • Scale horizontally
  • Use NoSQL databases at the edge
  • Shard our data using small tables for fast access
  • Place Load-Balancing at every tier
  • Use Hundreds of terabytes of data in an in-RAM distributed cache (For example Facebook uses MemCache and has something like 300 TB of data in memcache)
  • Use pre-compile elements, since compiling slow (For example Facebook uses PHP because they think that is very productive for them, but it didn’t work very well in terms of compiling, so they made a team to create a PHP to C++ compiler in order to get more performance)
  • use MapReduce pattern for parallel analysis of “Big Data”

Some of the apps that are doing this are, Bing, Facebook, Twitter, Google, Zynga, and so on.

So how in order to achieve that kind of scale we have Windows Azure. But what should we do to take the best out of Windows Azure for Connected Devices?

  • Scale-out. This is easy in Windows Azure, because we can define the number of instance we want of a particular role
  • SQL Azure should be never touched by the Devices. Instead we should use:
    • In terms of Outbound Data - Worker roles that replicate outbound data from SQL Azure to NoSQL Azure Table Storage
    • In terms of inbound Data - Worker Role move inbound data from Azure queues to SQL Azure
  • Use Distributed caching, that in Windows Azure is handled by AppFabric Caching
  • Do wireless efficient Services. For example using Web Roles that expose REST + JSON WCF Services is very interesting, because it saves message size, and by doing so saves time and money
  • Also important is making our devices cache data and work on it offline. Not every thing should be made “live” and on top of “live” data. Don’t stress the wireless connection

I hope this helps you think better about your Connected Device Apps.

Session Review - Migrating Apps into Windows Azure

Simon Davies @ Microsoft

  • Not all applications, services and data will mode to or be created in Windows Azure
    • Technology Fit – Windows Azure Service / Programming Model
    • Data – some data will remain on premise
    • Economics – not worth moving some applications
    • Timing – cannot move everything at once
    • SaaS – some applications will be provided by SaaS vendors
  • What capabilities does Windows Azure provide to help migrate and connect?

Rules/Considerations about Windows Azure Applications

  • Windows Azure apps is built from one or more roles
  • A Windows Azure apps runs multiple IDENTICAL STATELESS instances of each role
    • Stateless
      • This is true but you can always use the Role File System is needed, what happens is that it won’t be permanent since if the machine fails the state in that machine won’t be maintained.
  • A Windows Azure apps behaves correctly when any role fails

Roles

  • Web Role : the role is a web app hosted in IIS on Microsoft’s Windows Image
  • Worker Role: the role is an app hosted on Microsoft’s Windows Machine
  • VM Role: The role is a pre-loaded app hosted on YOUR Windows Image
    • don’t forget that it’s not a persistent VM, is transient
  • Types of usage:
    • Web Role
    • Worker Role
    • Admin Web or Worker Role (same roles as previously but with some more Startup Tasks that allows you to customize the Microsoft’s Windows Image)
    • VM Role

Why VM Role?

  • Long running application installations
  • Error-prone application installations
  • Application installations require manual interaction

How to Build a VM Role?

  1. Take a Windows Server 2008 R2 Enterprise Image
  2. Make it your BASE.VHD
  3. Boot from the machine
  4. Customize your machine
    • Install your packages
    • Install your apps
  5. Install the Windows Azure Integration components
  6. Run the “sysprep /generalize” command in order to generalize it and you’ll get an Diff.VHD
    • generalize the name of the machine
  7. Upload both Base and Diff to the Blob Storage
    • “csupload add-vmimage –literalpath <base vhd path> –location <Chosen Data Center>” command compresses the VM and uploads
  8. Add a VM Role in Visual Studio and choose the VHD in the Blob Storage

Example Application Scenarios

  • Stateless ASP.NET Web Application or Service
    • Ideal candidate
  • Stateful ASP.NET application (either use or instance state)
    • Moves with some work
  • Highly parallel HPC application
    • Ideal Candidate
  • Windows Client Apps
    • Not suitable
  • A single instance sever application with local state (eg. Database server)
    • May work with additional design
    • Virtual Machine Role will not help (over and above web\worker role)

Windows Azure Connectivity

  • Data Sync : SQL Azure Data Sync
  • Application Layer Connectivity & Messaging: Service Bus
  • Security: Federated Identity & Access Control
  • Secure Network Connectivity: Windows Azure Connect

Windows Azure Connect

  • Secure network connectivity between on-premise and Cloud
    • Support standard IP protocols
  • Enable Azure Roles for external connectivity via service model
  • Enable computers for connectivity by installing connect agent
    • Only available in Windows Vista SP1, Windows 7, Windows Server 2008, Windows Server 2008 R2
  • Network policy managed thought portal
  • Automatic setup of secure IPv6 network between connected role instances and external computers
    • Tunnel firewalls/NAT’s thought hosted SSL-based relay service
    • Secure via end-to-end IPSec
  • Domain-joints

Identity Federation

  • WIF – Windows Identity Foundation
  • Windows Access Control

Service BUS

  • Extend reach of applications securely though the cloud
  • Enabled multi-tenant apps to integrate with tenants on-premise services
  • Securely integrate partners outside of organization boundaries
  • Extend reach of on-premises web services layer
  • Usage Patterns
    • Connectivity
      • Service Remoting
      • Cloud Eventing
      • Protocol Tunneling
    • Messaging
      • Load Leveling
      • Multicast Messaging
Session Review - Windows Azure In Depth

Simon Davies ( World Wide Technical Support Professional @ Microsoft )

Windows Azure Overview

  • Is “An Operating System for the Data Center”
    • Treat the data center as a machine
  • Compute
    • Virtualized compute environment based o Windows Server
  • Storage
    • Durable (Automatically Managed by Microsoft. Always have 3 copies for Disaster Recovery purposes)
    • Scalable
    • High Availability
  • Network
    • Automated network provision and management
    • On Premise Connectivity
    • Global Content Delivery Network (CDN)
  • Management
    • Automated
    • Model Driven Management as a Service
    • Resources
    • Management
    • Provisioning
    • Monitoring
  • Allows Developers to think their app and data only

Windows Azure In Depth

  • Modeling Cloud Applications
    • A could app is typically made up of different components
      • Front End
      • Middle Tier
      • Backend storage
      • Multiple instances of each for scalability and availability
  • Windows Azure Service
    • Windows Azure Hosts “Services”
    • A service is
      • An isolated boundary
      • A set of component roles, each within endpoints
      • At runtime numbered, identical instances of each role are created, each instance is a Virtual Machine
    • All of this specified declaratively in the service model and configured at runtime using a service configuration
  • What is a role?
    • Definition
      • Role Name
      • Role Type
      • VM Size
      • Network Endpoints
    • Code
      • Web/Worker: Hosted DLL and other executable
      • VM Role: VHD
    • Configuration
      • Number of instances
      • Number of update and fault domains
    • Example:
      • Front-End Role
        • Definition
          • Type: Web
          • VM Size: Small
          • Endpoints: External 1
        • Configuration
          • Instances:2
          • Update Domains: 2
          • Fault Domains: 2
  • Service Model Files
    • Service Definition in in ServiceDefinition.csdef (Definition Part of the Role)
    • Service Configuration is in ServiceConfiguration.cscfg (Configuration Part of the Role)
    • CSPack, program that exists in the SDK, will zips a service binary into a package (Code Part of the Role)
  • Compute Runtime Environment
    • Fabric Controller talks to the Fabric Agent in order to configure and run the App
    • Fabric Agent runs inside the Host OS and talks with the Windows Azure Agend in order to manage their configurations
    • Each instance runs inside a Guest OS managed by the Host OS and is composed by
      • Windows Azure Agent
      • Windows Azure Compute
      • YOUR APP
  • Service Isolation
    • Your Services are isolated from other services
  • The Fabric Controller
    • The “kernel” of the cloud operating system
      • Manages datacenter hardware
      • Manages Windows Azure Services
    • Some of the  main responsibilities
      • Datacenter resource allocation
      • Datacenter resource provisioning
  • Managing Availability
    • Upgrade Domains
      • Default and Max Fault Domain is 5
      • Vertical concept
    • Fault Domains
      • Default 2 Fault Domains
  • Provisioning a role instance
    • 3 VHDs are created
      • Guest OS Image (D:\)
      • Resource VHD (C:\)
      • Role VHD (E:\ or the next available letter) = contains the package we deployed
  • Fabric Controller Security
    • The VM is the security boundary upon which Windows Azure security is based
      • the host OS and FC agent are trusted
      • The guest agent is untrusted
  • Updating the OS
    • Initiated by Windows Azure Team
    • Goal: update all machines as quickly as possible
    • Constraint: must not violate the SLA and so we need at least 2 instances in order not to have a outage of the service
Session Review - Windows Azure - From Startup to the Cash Cow

Luis Alves Martins (Architect Evangelist @ Microsoft Portugal)

How do we Start?

  • Think BIG / Globally
    • Multi-Language
    • Multi-Currency
  • Start SMALL
    • Start building things step by step, instead of trying to conquer all at once
    • Think only on the Application and Data instead of all of those and also Hardware, Hardware Management, and so on.

How to Get Windows Azure?

    • Microsoft BizSpark Program (Startup with less than 3 years)
    • MSDN Subscriptions (Microsoft Partner Benefit)
    • Windows Azure Subscriptions
    • Windows Azure Pay-as-you-Go

Why Azure?

  • Cost
  • Agility
  • Risk Mitigation
  • Expertise
    • Join the expertise that you already have about developing solutions and Managing Data and connect that with the Expertise by Microsoft on Managing Data Centers
  • Symmetry
      Cloud SQL Azure Windows Azure
      On-Premise SQL Server Microsoft Windows Server 2008 or greater
  • Platform Continuum
    • The same code in Traditional DataCenters, Public or Private Windows Server, Windows Azure, Azure Appliance
      • This can happen if we have some considerations prior to beginning the development
  • It’s a natural Evolution, not Revolution
    • Step forward in terms of:
      • Costs
      • Economy of Scale
      • Don’t think about the Machines and the Hardware

Business Model

  • Without any change:
    • Customer pays for the software
    • Customer buys its own Windows Azure Account
  • Provide SaaS
    • Customer pays for the solution that already includes the Windows Azure costs

Windows Azure Internals Interesting facts

Pricing is easy (Example)

  • Compute       =       42,52 €/month (1 role – no Licensing fees)
  • Data             =       7,085 €/GB/month  (SQL)
  • Transfer       =        0,1064 €/GB/month

How to Start with Azure?

Some Q&A about Windows Azure AppFabric Caching

Today someone asked me about Windows Azure AppFabric Caching, and how it works and so I thought it would be interesting to do a blog post about it, with the Questions and Answers for everyone to see and comment and even ask more, to complete it.

Question: What are the Cache Sizes available in the Windows Azure AppFabric Caching Service?

Answer: 128 MB, 256MB, 512MB, 1GB, 2GB, 4GB

 

Question: What happens when provisioning Windows Azure AppFabric Caching in the Portal?

Answer: When you do the provisioning of the Caching Service, you define the Cache Size you want to use, and so that Cache Size is Reserved for you in the Internal Cluster of Cache Machines that are associated with the Service.

Question: Is Windows Azure AppFabric Caching Cache Size Always Available?

Answer: Yes, the complete size is always Reserved for you only.

Question: Is Windows Azure AppFabric Caching Cache Size all stored in the same Machine?

Answer: No. Caching Service has a Distributed Architecture and so it spreads your cache size across several machines.

 

Question: Security is very important and it’s important to understand how does Windows Azure AppFabric Caching Service works in terms of Security? How can we define the security of the Data?

Answer: Windows Azure AppFabric Caching Service is highly integrated with the Windows Azure AppFabric Access Control Service (ACS), and so that is the way to define the security that will be used to access the cache.

 

Question: What about the Princing? What is the pricing associated with Windows Azure AppFabric Caching Service?

Answer: The Price depends on the Caching Size you choose.

  • 128 MB cache for $45.00/month
  • 256 MB cache for $55.00/month
  • 512 MB cache for $75.00/month
  • 1 GB cache for $110.00month
  • 2 GB cache for $180.00/month
  • 4 GB cache for $325.00/month

But you have it free until August 1st, 2011. More about this check here.

 

If you have any more questions just ask away and I’ll try to answer them.

MIX11 - Session Review - Building Windows Phone 7 Applications with the Windows Azure Platform

Wade Wegner ( Technical Evangelist @ Windows Azure Team )

Why Phone + Cloud?

  • The cloud levels the playing field
  • The cloud provides larger pool of resources from which to pull
  • The cloud provides a way to reach across device platforms

Why Windows Phone 7 and Windows Azure?

  • Because Windows Azure is PaaS. you build, Windows Azure runs it
  • Automatic OS patching
  • Scalable
  • Utility billing model
  • Additional services
    • Access Control Services (Provides Identity Federations)
    • Traffic Manager (Provides an easy way to manage the location from which Data Center the app is served)
    • Caching ( Provides a Caching Service that allows us to easily create cache in our apps)
    • CDN (Content Delivery Network)
  • Common Development tools
    • Visual Studio
    • Languages
    • Emulators for development
  • Demo Source Code: http://www.wadewegner.com/

What was shown in the Demo

  • Identity
    • Identity Options
      • Create your own (eg. username + password, token, …)
        • Custom Model
        • ASP.NET Membership Providers
      • Use a single existing identity system (eg. Live Id, Facebook, Twitter, …)
      • Outsource identity management (eg. Access Control Service)
      • More information about this:
        • Authenticating Users in a Windows Phone 7 App via ACS, OData and Windows Azure - http://bit.ly/wp7acs
  • Storage
    • Storage Options
      • SQL Azure
        • Relational Database
        • Highly available
        • Managed for you as a Service
        • Pattern
          • WebRole that exposes an OData Service
            • Create a Web role with an OData Service that will expose your SQL Azure data to your clients
      • Windows Azure Tables
        • Non-relational structured Storage
        • Massive scale-out
        • OData
        • Pattern
          • Using Public Blobs
            • Client sends data to the Web Role
            • Web Role stores data in blobs
            • Client fetches public blobs directly
          • Shared Access Signatures (SAS)
            • Client gets Shared Access Signature from the Web Role
            • Client inserts the SAS in the URI and stores data in blobs
            • Client fetches public blobs and even private containers as long as the SAS allows it
      • Windows Azure Blobs
        • Big files
        • REST
        • Pattern:
          • Proxy Calls
            • Client sends data to the Web role
            • Web role stores data in Blobs
  • Services
    • Web Role vs Worker Role
    • Web Role has IIS
    • Worker role does not
      • think about it as being a DLL with a Main()
    • Both implement the RoleEntryPoint
    • Scaling Work in Windows Azure
      • Traditionally
        1. Web role receives message
        2. Web role enqueues work
        3. Worker role pools queue
        4. Worker role sends notifications
        • Interesting because it splits the IIS and Services parts allowing you to scale them independently
    • Multitenancy
      • You can have more than one application running in your role instance
      • based on the Sites tag in the ServiceDefinition.csdef
    • Content Delivery Network (CDN)
      • Currently at least 24 nodes that will cache data locally in order to provide the data faster
    • Traffic Manager
      • Intelligently routes the request to the most appropriate Data Center according to the policies defined in it
  • Communications
    • Two primary communication models
      • Phone-initiated
        • Options
          • Http-based, request/response
          • Can be based on pooling of User Initiated requests
          • Frameworks choises (WCF, OData, WebRequest, …)
          • Wire Formats (SOAP, JSON, POX, …)
      • Cloud-initiated
        • Push Notifications
          • Single connection between phone and Microsoft Push Notification Service
            • The cloud cannot connect directly to the phone, just passing by the Microsoft Push Notification
          • There’s no warranty of delivery
          • Bandwidth and battery friendly
        • There kinds of push notifications
          • Raw – send a message to an app
          • Toast – send a message to the user
            • Blue bar at the top
          • Tile – Update an image, title or count
        • How does this works?
          1. Phone opens a channel in the Microsoft Push Notifications Service
          2. Phone sends the URL to the cloud
          3. Cloud pushes notifications via URL
          4. Microsoft Push Notification Service notifies the phone

Important Best Practice: DO NOT STORE SECRETS ON YOUR PHONE

What is there to make this easier?

  • Announcing: Windows Azure Toolkit for Windows Phone 7 (v1.1.0)
  • Make it easier for phone developers to use Windows Azure
  • Toolkit includes:
    • Client libraries
    • Sample apps
    • Source code
    • Documentations
  • Available on http://watoolkitwp7.codeplex.com
  • The phone will only communicate with a Service over HTTPs if
    • The cert is installed on the device
    • The cert is signed by a Trusted Authority
  • In order to use the self signed certs you need to install it on the phone
MIX11 - Session Review - Powering Data on the Web and Beyond with SQL Azure

David Robinson (Senior Lead Program Manager)

SQL Azure Relational Database

  • Based on SQL Server 2008 R2 engine
  • Use same tools and data access frameworks
  • Six global datacenters
  • High Availability & Redundancy
    • Reads are completed at the primary
    • Writes are replicated to a quorum of secondaries

SQL Azure Roadmap

  • MIX08
    • Introduced SQL Server Data Services (ACE type)
  • MIX09
    • Introduce SQL Data Services (Real Relational Database)
  • MIX10
    • New Features announced
      • 50 GB Database
      • PHP Support
      • MARS support
      • OData Endpoint Preview
      • Geo Spatial Data Types

ASP.NET MVC 3

  • Razor view Engine which provides a very streamlined syntax for writing clean and concise views.
  • Improved support for Dependency Injection
  • Global Action Filters
  • jQuery based Unobtrusive Ajax and Client Validation
  • ViewBag property for dynamic access to ViewData
  • Support for view engine selection in the New Project and Add View dialog

Extending the Reach of Data

  • This is achieved using ODATA protocol
  • Cloud Data Services
    • Windows Azure Tables
    • SQL Azure
    • Windows Azure MarketPlace DataMarket
  • Powering Web/AJAX Experiences
    • Consuming ODATA via jQuery & JSON
  • Powering Windows Phone 7
    • ODATA libraries available for Windows Phone 7
  • Powering Interoperability on Other platforms

Why does Performance matters?

  • More responsive applications
  • Faster page load times
    • 8 seconds vs 3 seconds
  • Higher interactivity – new type of applications
  • Better User Experience – more $$$$

SQL Azure Data Sync

  • Sync within the Cloud
  • Read scale-out via multiple copies
    • reporting
    • web site reference data
  • Read-write scale-out via multiple copies:
    • ex. geo located web apps
  • Same or different data centers
  • Key features
    • No-Code sync configuration
      • Easily define data to be synchronized and locations
      • Choose how often data is synchronized
    • Full Data Synchronization Capabilities
      • Two-way sync of same data, as well as one-way sync
    • Conflict Handling
      • Detect and resolve conflicts caused by the same data being changed in multiple locations
    • Logging and Monitoring
      • Administration capabilities for tracking usage
    • Scale
      • Service scales as resources requirements grow
  • Currently there are 2 CTPs available
    • CTP 1 publicly available
    • CTP 2 invitation only
    • CTP 3 will be available by the end of the summer and will be feature complete

Geo Available Applications

  • Windows Azure Traffic Manager
    • This will help handle geo-locating SQL Azure Databases
    • This will manage the traffic
  • SQL Azure Data Sync
    • Syncs the SQL Azure Databases between regions

CY11 Investment Themes

  • Managed Services
    • Data Protection / Recovery
      • Local Backup/Point in Time Restore: to recover from user/apps errors
      • Geo Backup/Disaster Recovery from data center/geographical disasters
    • New Data Movement Capabilities
      • DB Import/Export of data and schema between SQL Azure and SQL Server
      • Migration Support to SQL Azure from Sybase, in addition to existing support for Access, MySQL and Oracle
    • Enhanced Management Experiences
      • Enhanced portal based experiences
      • Multiple DB administrators per subscriptions to support enterprise scenarios
  • Scale on Demand
    • Scale-out with Federations
      • Scale to virtually unlimited size to host very large databases across multiple servers
      • Better support for multi-tenant applications with numerous tenants
    • Scale-up and Scale-down Databases
      • Scale beyond 50GB and less than 1GB
    • Performance Predictability
      • Reserve compute capacity with Resource SLA
    • Enhanced multi-tenancy support
      • API level support for multiple DB server per subscription, provisioning & de-provisioning
  • Faster Innovation
    • Rich Insights with Reporting as a Service
      • Ability to author rich reports, host in Azure and view in on-premise or cloud apps
    • Spanning Apps with Sync Service
      • Synchronize data between SQL Azure DBs between SQL Azure & SQL Server
    • Rich Developer Experiences
      • Professional Devs: VS “Juneau”
      • Department Devs: LightSwitch
      • Java Devs: supported JDBC driver
      • International Devs: Localized Azure Portal
    • Data Market Enhancements
      • International rollout
      • Enhanced Content Management
      • Integration with Denali, Office and CRM

Resources

MIX11 - Session Review - Windows Azure Storage - Getting Started and Best Practices

Haridas (Software Developer Lead)

  • Windows Azure Storage
    • What is it?
      • Scalable, Durable, Highly Available Cloud Storage System
      • Pay for what you use
      • Abstractions
        • Blobs – Provides a simple interface for storing named files along with metadata for the file
        • Drives – Provides durable NTFS volumes for Windows Azure Applications to use – based on Page Blobs
        • Tables – Provides structured storage. A Table is a set of entities which contains a set of properties
        • Queues – Provides reliable storage and delivery of messages for an Application
    • Data Storage Concepts
      • Based on the Account
        • Container for Blobs (Blob Storage)
          • https://<account>.blob.windows.net/<container>
        • Table for Entities (Table Storage)
          • https://<account>.table.windows.net/<table>
        • Queue for Messages (Queue Storage)
          • https://<account>.queue.windows.net/<queue>
    • Blobs
      • Provides a highly scalable, durable and available file system in the cloud
      • An account can create many containers
        • No limit on number of blobs in a container
        • Limit of 100TB per account
      • Associate metadata with Blobs
      • Upload / Download Blobs
        • Allows range reads
        • Conditional operations – If-Match, Id-Not-Modified-Since, …
        • Sharing – Public containers, Shared Access Signatures (SAS)
          • SAS – pre-authenticated url
      • Storage client uses by default a timeout of 90 seconds. This can be changed using the BlobRequestOptions class to set the timeout to be according to the type of blob you’re uploading
      • Types of Blobs
        • Block Blobs
          • Targeted at streaming workloads
          • Each Blob consists of a sequence of blocks
            • 2 phase commit: Blocks are uploaded and them separately committed
            • Efficient continuation and retry
            • Send multiple out of order blocks in parallel and decide the block order during commit
            • Random range reads possible
          • Size limit is 200GB per blob
          • What to do?
            1. File has variable sized blocks
            2. Upload blocks in Parallel using PutBlock
            3. Retry failed blocks
            4. Commit blob using PutBlockList
        • Page Blobs
          • Targeted at random write workloads
          • Each blob consist of an array of pages
          • Size limit 1TB per blob
          • Page
            • Each page range write is committed on PUT
            • Page is 512 byte in size
            • Write boundary aligned at multiple of 512 byte
            • Range reads possible
            • Pages that do not have data are zeroed out
          • How?
            1. Write 5K bytes – PutPage
            2. Clear starting at a particular offset – ClearPage
            3. Overwrite bytes – PutPage
            4. Truncate Blob – SetMaxBlobSize
      • Sharing
        • Every blob request must be signed with the account owner’s key
        • Share your files options
          • The container must be public – read-only rights whenever you make a container public
          • Shared Access Signatures (SAS) – share pre-authenticated URLs with users
            • You decided who you’d like to share it with
            • You can give variable permition
              • Delete Blob
              • Write Blob
              • Read or Listing Blob
            • Two ways to do that
              • Everything embedded using the URL and doing that signing it with your owner key
              • Create an access policy that will contain all the parameters that normally are in the url
                • Advantage is that this will make it possible to change the policy after giving the URL so someone.
        • SAS
          • Use container level access as it allows access to be easily revoked
      • Snapshots
        • Point in time read-only copy of blob
        • Every snapshot creates a new read only point in time copy
        • Charged only for unique blocks or pages (ex. reuse blocks or pages)
          • For reuse, use WritePages or PutBlock & PutBlock
        • Restore snapshots using copy blob
        • Remember to cleanup your snapshots
      • Best Practices
        • Use parallel blocks upload count to reduce latency when uploading photo
        • Client Library uses a default of 90 seconds timeout – use size based timeout
        • Snapshots – For block or page reuse, issue block and page uploads in place of UploadXXX methods in Storage Client
        • Shared Access Signatures
          • Use container level policy as it allows revoking permissions
          • Share SAS Url using Https
        • Create new container for blobs like log files that have retention period
          • Delete logs after 1 month – create new containers every month
        • Container recreation
          • Garbage Collection can take time until which time container with same name cannot be created (this might make that you cannot create a new container immediately with exactly the same name after deleting the previous one)
          • Use unique names for containers
    • Drive
      • Provides a durable NTFS volume for Windows Azure Applications
        • Use existing NTFS APIs
        • Easy migration path to the cloud
        • Durability and survival of data on application failover or hardware failure
          • All flushed and un-buffered writes to drive are made durable
      • A Windows Azure Drive is a Page Blob
        • Mounts Page Blob as an NTFS drive
        • Mounted by one VM at a time for read/write
        • A VM can dynamically mount up to 16 drives
        • Drives can be up to 1TB
    • Tables
      • Provides Structured Storage
        • Massively Scalable and Durable Tables
          • Billions of entities (rows) and TBs of data
          • A storage account can contain many tables
          • No limit on number of entities (aka rows) in each table
          • Provides flexible schema
        • Familiar and Easy to use API
          • WCF Data Services – .NET classes and LINQ
          • REST (OData Protocol) – with any platform and language
      • Best Practices
        • Use the context.SaveChangesWithRetries(SaveChangesOptions.Batch)
          • SaveChangesOptions.Batch gives you transactional semantics to the operation
          • Transactions on entities are only allowed if the partition key is the same
        • CloudTableQuery<> handles ContinuationToken
        • Use clustered index in queries for performance (PartitionKey)
        • Limit large scans and expect continuation tokens for queries that scan
          • Split “OR” on keys as individual queries
        • Entity Group Transactions – Batch to reduce costs and get transaction
        • Do not reuse DataServiceContext across multiple logical operations
        • Discard DataServiceContext on failures
        • AddObject/AttachTo can Throw exceptions if entity is already being tracked
        • Point query throws an exception if resource does not exist. Use IgnoreResourceNotFoundException property
    • Queue
      • Queue are highly scalable, available and provide reliable message delivery
        • Simple, asynchronous work dispatch
        • A storage account can create any number of queues
        • 8K message size limit and default expiry of 7 days
        • Programming semantics ensures that a message can be processed at least once
          1. Get message to make the message invisible
          2. Delete message to remove the message
      • Access is provided via REST
      • Best Practices
        • Make message processing idempotent
        • Do not rely on order – invisibility time can result in out of order
        • Messages > 8K => use blobs or tables to store and message contains the blob or table entity key
        • Use message count to dynamically increase/decrease workers. Example:
          • Retain one instance that polls once every X time period
          • One instance polling every second result in 2.678.400 calls which cost around $2.67
          • Spawn more instances when you detect backlog
        • Use dequeue count to detect
          • Visibility expiry time needs to increase
          • Poison messages
  • Partitioning & Scalability
    • Know the scalability Targets
      • Single Blob Partition
        • Throughput up to 60 MB/s
      • Single Queue/Table Partition
        • up to 500 transactions (entities or messages) per second
      • Storage account
        • SLA – 99,9% availability
        • Capacity – Up to 100 TBs
        • Transactions – Up to 5000 entities per second
        • Bandwidth – Up to 3 gigabits per second
      • Scale above the limits
        • Partition between multiple storage accounts and partitions
        • When limit is hit, app may see ‘503 server busy’. Apps should implement exponential back-off
    • Storage Partition – How to Scale?
      • Every data object has a partition key
        • Different for each data type (blobs, tables, queues)
      • Partition Key is unit of scale
        • A partition can be served by a single server
        • System load balances partitions based on traffic
        • Controls entity locality
      • Systems load balances
        • Load balancing can take a few minutes to kick in
        • Can take a couple of seconds for partition to be available on a different server
      • Server busy
        • Use exponential back-off on “Server Busy”
        • Our system load balances to meet your traffic needs
        • Single partition limits have been reached
    • Automatic Load Balancing
      • Assignment
        • Process:
          1. When a request is made the Load Balancer delivers it to one of the Front-Ends and it’s delivered to the appropriate Back-Ends that is serving that partition
            • Each server has multiple partitions and the loads can be different
          2. If the Master System recognizes that a single service has to many request, then it will offload those partitions and reassign then to the systems that have less load
    • Partition Keys in each abstraction
      • Blobs
        • “Container Name + Blob Name” is the partition key
        • Every blob and its snapshot are in a single partition
      • Tables
        • “Table Name” + Partition Key is the partition
        • Entities with same partition key value are served from the same partition
      • Queues
        • Queue Name is the Partition Key
        • All messages for a single queue belong to the same partition

 

  • Interesting Tools to understand what’s happening with the Storage
    • Fiddler
    • Wireshark
    • NetMon

 

MIX11 - Session Review - What’s New in the Windows Azure Platform

James Conard (Sr. Director – @jamescon)

What is the Windows Azure Platform?

  • Provide the best and most comprehensive Platform as a Service offering for building, deploying, and running cloud applications
  • Open & Flexible
    • Using several technologies and not only the .NET Framework
  • Leverage Existing Investments
    • Use the known technologies and ways to develop that exist already in the market in order to reduce the difficulty of entering in the cloud
  • Ready for next Generation apps
    • Prepare for the next types of apps like mobile, and high available

Windows Azure Core Services

  • Compute
    • Scalable environment for running code
    • Enables .NET, C++, PHP, Ruby, Python
      • By default .NET Framework is installed but you can deploy the technology you’d like and use it
    • Automate Service Management
      • Abstract you from managing the infrastructure where your apps is running on
  • Storage
    • Scalable and highly available cloud storage
    • Blobs, Tables, Queues, Drives
    • REST APIs and several client libraries
  • Database
    • SQL Relational Database (SQL Azure)
      • Use the same knowledge that exist in using SQL Server and make them available in the cloud
    • Familiar programming model & Tools

Windows Azure Platform Roadmap

  • October 2008
    • Announced the Windows Azure Platform
    • First CTP of Windows Azure Platform
  • March 2009
    • Announced SQL Azure Relational DB
  • November 2009
    • Updates Windows Azure CTP
    • Announced VMRole, Project Sydney and Windows Azure Platform pricing and SLAs
    • Enable FullTrust & PHP, Java etc.
    • Project Dallas CTP
  • February 2010
    • Windows Azure Platform generally available
  • June 2010
    • Windows Azure Update
      • .NET Framework 4
      • OS versioning
    • SQL Azure Update
      • 50 GB databases
      • Spatial data Support
      • DAC support
  • November 2010 – Big Release
    • New Windows Azure Platform Management Portal
    • Multiple Service Administrators (co-Admins)
    • Remote Desktop
    • Full IIS
      • Web Roles previously using Hosted Web Core
        • Only supported a single HTTP or HTTPs endpoing
      • Web Roles now supporting Full IIS 7.0/7.5
      • Enables new scenarios
        • Multiple IIS websites
        • Multiple virtual machines
        • Configure IIS extensions
        • Web Deploy for Rapid development
    • Windows Server 2008 R2 & IIS 7.5
    • Elevated Privileges
    • Windows Azure Connect (CTP)
    • Windows Azure Virtual Machine Role (Beta)
    • Extra Small Instances
    • Startup Tasks & Admin Mode
      • Enables short, unattended setups on role startup
      • Silent MSIs, COM components, Registry Keys, Configuring Windows Server Roles, etc
      • Configuration Sample for startup tasks in the ServiceConfiguration.cscfg
        • <WebRole name=”test”>
          • <Startup>
            • <Task commandline=”<commandToBeRunned>” executingContext=”limited|elevated” taskType=”simple|foreground|background”/>
          • </Startup>
        • </WebRole>

Windows Azure Content Delivery Network (CDN)

  • Enabled from Windows Azure Platform Portal
  • Integrated with Storage (Blob Storage)
  • Recent Enhancements
    • Delivery from Windows Azure Compute instances
    • Https support
  • CTP of Smooth Streaming – End of April 2011

Windows Azure AppFabric Access Control Service v 2.0 (ACS)

  • Provides AuthN support using multiple identity providers
    • Easily integrate Live ID, Facebook, Yahoo, Google and Active Directory
    • Support for WS-Federation, WS-Trust, OpenID 2.0, OAuth 2.0
  • Familiar & Consistent .NET Programming Model
    • Use existing Windows Identity Foundations SDK & Assemblies
  • REST-based Service Management API
    • Integrate into your existing apps and control panels
  • Now Available
    • No charge during promotion period ending January 1st, 2012
    • Low cost after the promotion: $1.99 per 100.000 transactions

Windows Azure AppFabric Caching

  • Distributed in-memory cache for Windows Azure apps
    • Integrated directly with the cache using .NET client library
    • Session State Provider for Windows Azure applications
  • Caching provided as a building block service
    1. Configure an AppFabric Namespace
    2. Choose your cache size
    3. Write code against that service
  • Familiar & Consistent APIs
    • Same APIs as Windows Service AppFabric
  • Available commercially by the end of April
  • In the CTP only the Data Center of South Central US has the Caching Service
    • supported cache sizes: 128 MB, 256MB
  • After the end of April (commercial Availability) you’ll have it available in every data center with supported cache sizes from 128MB to 4GB

Cloud Data Services

  • SQL Azure Reporting (Currently in a Private CTP)
    • SQL Server Reporting provided as a Service
    • Reports authored using existing tools (BIDS) and deployed to SQL Azure Reporting
  • SQL Azure Data Sync
    • Data synchronization provided as a Service
    • Sync to/from SQL Azure Database
    • Sync between SQL Server and SQL Azure
  • Windows Azure DataMarket
    • Information marketplace for ISVs and IWs that provide trusted public domain & premium commercial data
    • Now commercially available
    • Firstly announced as codename “Dallas”

Performance Matters

  • The new Windows Azure Traffic Manager solved this problem
  • Load Balancing across multiple Hosted Services
  • Now Available in CTP
  • Three scenarios that are currently in the CTP
    • Performance
      • Directs the user to the best / closest deployment
    • Fault Tolerance
      • Redirect Traffic to another deployment based on availability
    • Round Robin
      • Traffic routed to deployments based on a fixed ration
  • Also allow to keep a status of all your apps in order to understand how to rout the traffic

Windows Azure Platform Access

  • Windows Azure Pass
    • Free 30-Day access to the platform
    • Includes:
      • Compute: 3 small instances
      • Storage: 3 GB + 250K transactions
      • Data Transfer: 3GB in & 3GB out
      • Database: 2x1GB Web Edition
    • Signup: http://windowsazurepass.com
    • Enter code: MIXABC
  • Introductory Trial
    • Available through September 30th, 2011
    • Includes:
      • Compute: 750 extra small hours + 25 small
      • Storage: 20GB + 50K transactions
      • Data Transfer: 20GB in & 20 GB out
      • Database: 1 GB Web Edition (90 Days)
    • Sign-up: http://windowsazure.com
    • Note: Requires Credit Card Information but allows you to access CTP services
  • Ultimate
    • Compute: 1.500 hours of small instances
    • Storage: 30 GB
    • Transactions: 2M
    • Bandwidth: 35 GB in / 35 GB out
    • Databases: 5 GB Web Edition
  • Premium
    • Compute: 1.500 hours of extra-small instances
    • Storage: 25 GB
    • Transactions: 1M
    • Bandwidth: 30 GB in / 30 GB out
    • Databases: 1 GB Web Edition
  • Professional
    • Compute: 750 hours of extra-small instances
    • Storage: 20 GB
    • Transactions: 250K
    • Bandwidth: 25 GB in / 25 GB out
    • Databases: 1 GB Web Edition

Resources

Interesting topics

  • You can select which OSFamily you want to use by editing the ServiceConfiguration.cscfg file with the values:
    • 1 – Windows Server 2008
    • 2 – Windows Server 2008 R2
  • Currently ASP.NET MVC 3 isn’t available on Windows Azure and so you need to install it using a Startup Task
  • Release of some Extensions for ACS 2.0 that allow ACS to be plugged directly in Umbraco allowing it to configured without going to the Management Portal
More Posts Next page »