A weblog post with info on two amazing services of Amazon: S3 (Amazon Simple Storage Service) and EC2 (Amazon Elastic Compute Cloud), virtualized computing power. Could Microsoft deliver comparable functionality using their Hypervisor technology code-named Viridian, the new virtualization technology from Microsoft?
A few weeks ago I attended a presentation by Werner Vogels, CTO of Amazon.com. He stated that Amazon is more than just an online bookshop, it is an IT company. He told about the possibility to utilize Amazon’s computing power at 10 dollarcents an hour. After some browsing on the Amazon site I found the two amazing services he was mentioning:
Amazon Simple Storage Service (Amazon S3)
Amazon S3 is storage for the Internet. It is designed to make web-scale computing easier for developers.
Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers.
Amazon S3 Functionality
Amazon S3 is intentionally built with a minimal feature set.
- Write, read, and delete objects containing from 1 byte to 5 gigabytes of data each. The number of objects you can store is unlimited.
- Each object is stored and retrieved via a unique, developer-assigned key.
- Authentication mechanisms are provided to ensure that data is kept secure from unauthorized access. Objects can be made private or public, and rights can be granted to specific users.
- Uses standards-based REST and SOAP interfaces designed to work with any Internet-development toolkit.
- Built to be flexible so that protocol or functional layers can easily be added. Default download protocol is HTTP. A BitTorrent(TM)protocol interface is provided to lower costs for high-scale distribution. Additional interfaces will be added in the future.
- Pay only for what you use. There is no minimum fee, and no start-up cost.
- $0.15 per GB-Month of storage used.
- $0.20 per GB of data transferred.
Amazon Elastic Compute Cloud (Amazon EC2) - Limited Beta
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.
Just as Amazon Simple Storage Service (Amazon S3) enables storage in the cloud, Amazon EC2 enables "compute" in the cloud. Amazon EC2's simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your computing resources and lets you run on Amazon's proven computing environment. Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. Amazon EC2 changes the economics of computing by allowing you to pay only for capacity that you actually use.
Amazon EC2 Functionality
Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to requisition machines for use, load them with your custom application environment, manage your network's access permissions, and run your image using as many or few systems as you desire.
To use Amazon EC2, you simply:
- Create an Amazon Machine Image (AMI) containing your applications, libraries, data and associated configuration settings. Or use our pre-configured, templated images to get up and running immediately.
- Upload the AMI into Amazon S3. Amazon EC2 provides tools that make storing the AMI simple. Amazon S3 provides a safe, reliable and fast repository to store your images.
- Use Amazon EC2 web service to configure security and network access.
- Use Amazon EC2 web service to start, terminate, and monitor as many instances of your AMI as needed.
- Pay for the instance hours and bandwidth that you actually consume.
Amazon EC2 enables you to increase or decrease capacity within minutes, not hours or days. You can commission one, hundreds or even thousands of server instances simultaneously. Of course, because this is all controlled with web service APIs, your application can automatically scale itself up and down depending on its needs.
- Completely Controlled
You have complete control of your instances. You have root access to each one, and you can interact with them as you would any machine. Each instance predictably provides the equivalent of a system with a 1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth.
- Designed for use with Amazon S3
Amazon EC2 works in conjunction with Amazon Simple Storage Service (Amazon S3) to provide a combined solution for computing and storage across a wide range of applications.
Amazon EC2 offers a highly reliable environment where replacement instances can be rapidly and reliably commissioned. The service runs within Amazon's proven network infrastructure and datacenters.
Amazon EC2 provides web service interfaces to control network security. You define groups of instances and their desired accessibility.
Amazon EC2 passes on to you the financial benefits of Amazon's scale. You pay a very low rate for the compute capacity you actually consume. Compare this with the significant up-front expenditures traditionally required to purchase and maintain hardware, either in-house or hosted. This frees you from many of the complexities of capacity planning, transforms what are commonly large fixed costs into much smaller variable costs, and removes the need to over-buy "safety net" capacity to handle periodic traffic spikes.
- Pay only for what you use.
- $0.10 per instance-hour consumed (or part of an hour consumed).
- $0.20 per GB of data transferred outside of Amazon (i.e., Internet traffic).
- $0.15 per GB-Month of Amazon S3 storage used for your images (charged by Amazon S3).
Data transferred within the Amazon EC2 environment, or between Amazon EC2 and Amazon S3, is free of charge (i.e., $0.00 per GB).
Amazon S3 usage is billed separately from Amazon EC2; charges for each service will be billed at the end of the month.
It is my feeling that solutions like S3 and EC2 will be the future of software development and deployment. Especially startup companies can benefit from these kind of solutions: at $72/month you have a server up and running, and you can scale up the number of servers when needed. Especially very useful if you need huge computing power during small amounts of time, like for example a three days online action.
After some reading I found out that the EC2 virtualization technology is Xen. Currently only Linux images can be hosted on EC2.
Viridian, Microsoft’s new virtualization technology
Although I have been using Microsoft’s virtualization technology for a few years with Virtual Server and Virtual PC, I was never very impressed by its performance when you compare it to VMWare. I mostly use it to run SharePoint in a virtual machine, and as you can see in this weblog post by Todd Baginski VMWare is the better option with respect to performance.
Wouldn’t it be great if we could have EC2 like functionality using Microsoft Windows Server 2003 images? VMWare has such capabilites using the VMWare ESX Server. Microsoft is currently working on similar technology with their Hypervisor technology based product code-named Viridian, which architecture seems to be similar to the architecture of Xen, a technology Microsoft Research contributed to.
And now someone please build a similar hosted virtual server model to EC2 where we can build our applications using the Microsoft .Net platform technology at comparable prices to Amazon!
For more info on Viridian: http://www.google.com/search?num=100&hl=en&q=Viridian+virtualization
On the official blog of the SharePoint product group a link to a spreadsheet containing a comparison of the features of MOSS 2007 to the features of SPS 2003 is provided. It provides a feature comparision between the following products:
- SharePoint portal Server 2003
- Windows SharePoint Services 3.0
- Office SharePoint Server 2007 for Search
- Office Forms Server 2007
- Office SharePoint Server 2007 Standard CAL
- Office SharePoint Server 2007 Enterprise CAL or for Internet Site
It is an extensive overview subdivided in the following categories:
- Content Management
- Business Process and Forms
- Business Inteligence
The Search category definitly proves me wrong on the rumors I had heard with respect to the search functionality available in the Standard CAL versus the Enterprise CAL: Only the BDC search functionality is part of the Enterprise CAL.
Like WSS2, WSS3 will be free of charge, covered by the standard Windows Client Access License (CAL).
In the new version of Microsoft Office SharePoint Server 2007 (MOSS 2007) there will be two different CALs: Standard CAL and Enterprise CAL.
Users of SharePoint Portal Server 2003 (SPS) will be “migrated” to the MOSS 2007 Standard CAL. Extra money has to be payed for the Enterprise CAL.
No sweat until this point. But rumours have reached me…
Standard CAL: Workflow, Document Management, Web Content Management, Site Model and Security
Enterprise CAL: Data Integration (BDC), E-Forms (Forms Server), Excel Services and…. Search
Although the search in WSS3 (the basis for MOSS 2007) is way better than the current WSS2 search (which is completely different from SPS search) and can now searchover complete site collections, I’m more than surprised that the full Search is no longer part of Standard CAL.
Current customers of SPS 2003 using features of the full Search like indexing Exchange, Web Sites and the Filesystem, and defining custom properties for search will be not amused (to say the least).
Is there someone out there who can confirm this?
UPDATE: Dustin Miller pointed me to a weblog entry by Arpan Shah, a Group Product Manager for SharePoint Products and Technologies who has more details on Microsoft Enterprise Search and SKU breakdown.
The following quote is important:
It's important to note that there are different SKUs available that contain SharePoint search. Depending on your business needs, you want to explore the following.
1. SharePoint Server for Search. This is a "Search SKU" and the licensing model is dollars/server. It comes in two versions (Standard and Enterprise) and depending on the number of documents, you want to choose the appropriate one. This SKU builds on top of Windows SharePoint Services (WSS) v3 and has all the extensibility that SPS 2003 has today. There's also an upgrade path from the search SKU to the Standard and Enterprise versions of SharePoint Server.
2. SharePoint Server w/ Standard CAL. This is a Server/CAL licensing model and provides a lot more functionality across the board than the "search SKU". This is also built on WSS v3 and gives you portal and enterprise content management features (for specifics on what SharePoint Server provides, read this post). From a search perspective, this adds the ability to search people and introduces a new extensibility and UI experience known as the Search Center. People search can be enhanced with Knowledge Network for SharePoint which is an add-on if you own this SKU.
3. SharePoint Server w/ Enterprise CAL. This is a Server/CAL liecnsing model and in addition to the capabilities provided w/ Standard, this provides rich Business Intelligence and Forms capabilities. From a search perspective, above and beyond what Standard provides, this introduces a new feature known as the Business Data Catalog (BDC) that allows you to easily search structured LOB systems without writing code. Out of the box, we plan to provide integration with SAP and Seibel as well as any database via ADO.NET. Any LOB system that exposes information via XML Web Services can be connected to.
Please note: While #2 and #3 provide search functionality, they provide rich Information management features that helps information workers share, collaborate, find and retain information end-to-end. It's more than search!
In recap (all will be available in the Office 2007 timeframe):
1. Office SharePoint Server for Search follows a per server licensing model, is extensibile, crawls file shares, sharepoint sites, web sites, exchange pfs, lotus notes databases out of the box. it comes in to versions: standard and enterprise - you choose depending on how many docs. it upgrades to the other office sharepoint server skus.
2. Office SharePoint Server Standard follows a CAL/server licensing model. It provides much richer features than the search SKU. From a search perspective, it provides people search and the Search Center.
3. Office SharePoint Server Enterprise follows a CAL/server licensing mode. It provides more features than Standard such as BI and Forms. From a search perspective, it introduces the Business Data Catalog (BDC) that allows you to connect to LOB systems without writing code.
Lets hope this SKU approach is true, it would mean that the only thing that SharePoint Search will privide in the Enterprise version is search through the BDC in LOB systems.