During the last year, we (Tellago) have been involved in various business intelligence initiatives that leverage some emerging BI techniques such as self-service BI or complex event processing (CEP). Specifically, in the last few months, we have partnered with Microsoft to deliver a series of events across the country where we present the different technologies of the SQL Server 2008 R2 BI stack such as PowerPivot, StreamInsight, Ad-Hoc Reporting and Master Data Services. As part of those events, we try to go beyond the traditional technology presentation and provide a series of best practices and lessons we have learned on real world BI projects that leverage these technologies.
Now that SQL Server 2008 R2 has been released to manufacturing, we have launched a series of quick adoption programs that are designed to help customers understand how they can embrace the newest additions to Microsoft's BI stack as part of their IT initiatives. The programs are also designed to help customers understand how the new SQL Server features interact with established technologies such as SQL Server Analysis Services or SQL Server Integration Services. We try to keep these adoption programs very practical by doing a lot of prototyping and design sessions that will give our customers a practical glimpse of the capabilities of the technologies and how they can fit in their enterprise architecture roadmap.
Here is our official announcement (you can blame my business partner, BI enthusiast, and Tellago's CEO Elizabeth Redding for the marketing pitch ;)):
Tellago Marks Microsoft's SQL Server 2008 R2 Launch With
Business Intelligence Quick Adoption Program
Microsoft launched SQL Server 2008 R2 last week, which delivers several breakthrough business intelligence (BI) capabilities that enable organizations to:
- Efficiently process, analyze and mine data
- Improve IT and developer efficiency
- Enable highly scalable and well-managed Business Intelligence on a self-service basis for business users
The release offers a new feature called PowerPivot, which enables self service BI through connecting business users directly to enterprise data sources and providing improved reporting and analytics.
The release also offers Master Data Management which helps enterprises centrally manage critical data assets company-wide and across diverse systems, enabling increased integrity of information over time.
Finally, the release includes StreamInsight, which is a framework for implementing Complex Event Processing (CEP) applications on the Microsoft platform. With StreamInsight, IT organizations can implement the infrastructure to process a large volume of events near real time, execute continuous queries against event streams and enable real time business intelligence.
As a thought leader in the Business Intelligence community, Tellago has recognized the occasion by launching a series of quick adoption programs to enable the adoption of this new BI technology stack in your enterprise.
Our Quick Adoption programs are designed to help you:
- Brainstorm BI solution options
- Architect initial infrastructure components
- Prototype key features of a solution
As a 2-3 day program, our approach is more efficient and cost effective than a traditional Proof of Concept because it allows you to understand the new SQL Server 2008 R2 feature set while seeing directly how you can leverage it for your business intelligence needs.
If you are interested in learning more about the BI capabilities of Microsoft's Business Intelligence stack, including SQL Server 2008 R2, we can help. As industry experts and software content advisers to Microsoft, Tellago is the place where ideas meet technology expertise.
Let us help you see for yourself the advantages that you can gain from Microsoft's
SQL Server 2008 R2.
Email or call for more information - info@tellago.com or 847-925-2399.
One of the fundamental differentiators of Microsoft's StreamInsight compared to other Complex Event Processing (CEP) technologies is its flexible deployment model. In that sense, a StreamInsight solution can be hosted within an application or as a server component. This duality contrasts with most of the popular CEP frameworks in the current market which are almost exclusively server based. Whether it's undoubtedly that the ability of embedding a CEP engine in your applications opens new possibilities for CEP scenarios such as Web analytics or mobile CEP it is also unquestionable that the majority of CEP scenarios still rely on a server-centric model. This blog post provides an initial overview to the server hosting capabilities of Microsoft's StreamInsight.
Instead of providing a brand new server product, the current version of Microsoft's StreamInsight relies on Windows Communication Foundation (WCF) for its hosting capabilities. This model allows to reuse familiar and proven techniques to scale and improve the performance of StreamInsight solutions. The key element of StreamInsight's server-based model is the Microsoft.ComplexEventProcessing.Server class. This class abstracts the basic functionalities of a CEP Server such as storing application's definition, binding and controlling the execution of queries, among many others. In order to expose a CEP server as a WCF service we should invoke the CreateManagementService operation which exposes an implementation of the IManagementService service contract that contains the following operations.
1: [ServiceContract(
Namespace = "http://schemas.microsoft.com/ComplexEventProcessing/2009/05/Management")]
2: public interface IManagementService
3: { 4: [OperationContract(...)]
5: ChangeQueryStateResponse ChangeQueryState(ChangeQueryStateRequest request);
6:
7: [OperationContract(...)]
8: void ClearDiagnosticSettings(ClearDiagnosticSettingsRequest request);
9:
10: [OperationContract(...)]
11: CreateResponse Create(CreateRequest request);
12:
13: [OperationContract(...)]
14: DeleteResponse Delete(DeleteRequest request);
15:
16: [OperationContract(...)]
17: EnumerateResponse Enumerate(EnumerateRequest request);
18:
19: [OperationContract(...)]
20: GetResponse Get(GetRequest request);
21:
22: [OperationContract(...)]
23: GetDiagnosticSettingsResponse GetDiagnosticSettings(
GetDiagnosticSettingsRequest request);
24:
25: [OperationContract(...)]
26: GetDiagnosticViewResponse GetDiagnosticView(
GetDiagnosticViewRequest request);
27:
28: [OperationContract(...)]
29: void SetDiagnosticSettings(SetDiagnosticSettingsRequest request);
30: }
In order to enable persists the definition of CEP queries, applications and events, StreamInsight uses a SQLCE-based database that uses the schema defined by the CepMetadata.sdf template included in the StreamInsight bits. The following code illustrates the techniques used to create a new StreamInsight's WCF service host.
1: private static void StartHost()
2: { 3: Console.ForegroundColor = ConsoleColor.Yellow;
4: ServiceHost host = null;
5: try
6: { 7: Server server = GetServer();
8: host = new ServiceHost(server.CreateManagementService());
9: host.Open();
10: if (!server.Applications.Keys.Contains("sampleapp") ) 11: registerSampleQuery();
12: EventWaitHandle adapterStopSignal = new EventWaitHandle(false,
13: EventResetMode.ManualReset, "StopAdapter");
14: Console.ReadLine();
15: }
16: catch (Exception innerException)
17: { ... } 18: finally
19: { 20: if ((host != null) && (host.State != CommunicationState.Faulted))
21: host.Close();
22: if (server != null)
23: server.Dispose();
24: }
25: }
26:
27:
28: private static Server GetServer()
29: { 30: string str = ConfigurationManager.AppSettings["SQLCEMetadataFile"];
31: bool flag = false;
32: if (new List<string>(ConfigurationManager.AppSettings.AllKeys).
33: Contains("CreateSqlCeMetadataFileIfMissing")) 34: { 35: try
36: { 37: flag = bool.Parse(ConfigurationManager.
38: AppSettings["CreateSqlCeMetadataFileIfMissing"]);
39: }
40: catch (FormatException exception)
41: {... } 42: }
43: if (str != null)
44: { 45: SqlCeMetadataProviderConfiguration config =
46: new SqlCeMetadataProviderConfiguration();
47: config.DataSource = str;
48: config.CreateDataSourceIfMissing = flag;
49: return Server.Create(config);
50: }
51: return Server.Create();
52: }
The previous code uses the following WCF configuration.
1: <system.serviceModel>
2: <services>
3:
4: <service
5: name="Microsoft.ComplexEventProcessing.ManagementService.ManagementService"
6: behaviorConfiguration="MtdBehavior">
7:
8: <host>
9: <baseAddresses>
10: <add baseAddress="http://localhost:9999/StreamInsight"/>
11: </baseAddresses>
12: </host>
13:
14: <endpoint
15: address="/cephost" binding="basicHttpBinding"
16: contract=
"Microsoft.ComplexEventProcessing.ManagementService.IManagementService" />
17:
18: </service>
19: </services>
20: <behaviors>
21: <serviceBehaviors>
22: <behavior name="MtdBehavior">
23: <serviceMetadata httpGetEnabled="true"/>
24: </behavior>
25: </serviceBehaviors>
26: </behaviors>
27: </system.serviceModel>
Notice that our WCF service implements the Microsoft.ComplexEventProcessing.ManagementService.IManagementService contract as its main interface.
After implementing our CEP server, client applications can interact with it at the URIs specified in the service endpoint definition. It is important to notice that the definition of the CEP artifacts such as applications, queries or events remains stored in the server database. A client application can connect to the server and trigger the execution of queries as illustrated in the following code:
1: private static void ExecuteRemoteQuery()
2: { 3: Server cepServer = Server.Connect(
4: new EndpointAddress("http://localhost:9999/StreamInsight/cephost"), 5: new BasicHttpBinding());
6: Application cepApplication= cepServer.Applications[my cep application...];
7: Query cepQuery= cepApplication.Queries[my cep query...];
8: cepQuery.Start();
9: Console.ReadLine();
10: }
Notice that our client application simply connects to the remote CEP server and executes the query.
I will be the first person to tell you that the StreamInsight’s server model requires more work in terms of the tooling and the underlying infrastructure. However, I believe we can agree that the fact that the host model is base on a highly flexible and scalable model like WCF takes us a large way. At the end, its always easier to build tooling that infrastructure, isn’t it?
Last week I had the pleasure of presenting two sessions at Microsoft's Dutch DevDays at Den Hague. On Tuesday I presented a sessions about how to implement real world RESTFul services patterns using WCF, WCF Data Services and ASP.NET MVC2. During that session I showed a total of 15 small demos that highlighted how to implement key aspects of RESTful solutions such as Security, LowREST clients, URI modeling, Validation, Error Handling, etc. As part of those demos I used the OAuth implementation created by my colleague Pablo Cibraro.
On Wednesday (March 31st ) I delivered a session that introduces a refactored version of the traditional Enterprise Integration Patterns from a cloud computing perspective. Specifically, I spent a lot of time explaining how the use of the Windows Azure Platform AppFabric Service Bus can change some of the fundamental patterns of enterprise integration solutions. I had a full house on both sessions and got a lot of interesting feedback and questions from a very enthusiastic audience.
This is first time presenting at Dutch DevDays and I was really impressed by the impeccable organization of the event and the energy of the attendees. Big kudos to Arie Leeuwesteijn and his team for putting together a great conference.
PS: The demos for my sessions will soon be available on this weblog.