In the last post i said about LinqtExtender implementing necessary property and injecting specific settings for entity objects. The issue i have is that it works fine under full / high trust settings but when running in medium trust it gives the following error:

image

Ouch.. it is only happening while i am using the extender from medium trust environment. As to add a little prologue, actually it creates a proxy around your original entity and if you look closely it is giving the error in .ctor(). Now lets dig in.

   1: ILGenerator ilGenerator = constructorBuilder.GetILGenerator();
   2:  
   3: ConstructorInfo baseConstructor = typeof(object).GetConstructors()[0];
   4:  
   5: ilGenerator.Emit(OpCodes.Ldarg_0);
   6: ilGenerator.Emit(OpCodes.Call, baseConstructor);
   7: ilGenerator.Emit(OpCodes.Ret);

The IL code is pretty simple actually , its the method body of the proxy class constructor that calls default constructor of the base class. Now, this will work fine in full/high trust as here i require the entity class to have a default constructor. But, as i switch the mode to medium trust it will throw me the above exception. The reason why, in medium trust the framework verifies the IL generated and though it may seem from my point of view that its a valid IL as it is expecting default constructor , to framework’s context its invalid as the base object might never have any default constructor (it should throw error in high trust). In that case the call is absolutely suspicious duh…

So the valid block for this scenario would be:

   1: ILGenerator ilGenerator = constructorBuilder.GetILGenerator();
   2:  
   3: ConstructorInfo baseConstructor = null;
   4: ConstructorInfo[] constructorInfos = parent.GetConstructors();
   5:  
   6: bool containsDefaultConstructor = false;
   7:  
   8: foreach (ConstructorInfo info in constructorInfos)
   9: {
  10:     if (info.GetParameters().Length == 0)
  11:     {
  12:         baseConstructor = info;
  13:         containsDefaultConstructor = true;
  14:         break;
  15:     }
  16: }
  17:  
  18: if (!containsDefaultConstructor)
  19:     throw new Exception(Properties.Messages.MustHaveADefaultConstructor);
  20:  
  21: ilGenerator.Emit(OpCodes.Ldarg_0);
  22: ilGenerator.Emit(OpCodes.Call, baseConstructor);
  23: ilGenerator.Emit(OpCodes.Ret);

The code is similar , just it checks for a default constructor , if not throws a valid exception and to medium trust everything looks fine.  I have added a patch for extender with this as well, do check it out.

Finally, while you are doing Reflection.Emit do check this and more like, you should not emit any unmanaged code and even not play around with private stuffs , etc to avoid pitfalls.

Hope that helps

Shout it
Posted Sunday, June 07, 2009 4:14 PM by mehfuzh | with no comments
Filed under: ,

While developing LinqExtender, i have come across various scenarios that people don’t want to extent any query class or even implement any interface. Also, it is almost absurd when they have to add new extender specific attributes to their entity class. It is though not very important for people creating new provider with small codebase but with large codebase it soon becomes a pain to modify each class.

The solution is to make Extender as a container , which is people will write their logic to act on their specific entity or list of entities, also define their settings via fluent interface. To wrap it around , there is a basic ORM that comes with the extender pack and i have a made LibraryContext on top of SqlQueryContext<T>. Inside, the constructor i wrote the following code to define the extender specific settings.

 

   1: Extender
   2:     .Settings
   3:         .For<Base>()
   4:         .Begin
   5:             .Property(x => x.Id).MarkAsUnique
   6:         .End
   7:         .For<Book>()
   8:         .MapToEntity("book")
   9:         .Begin
  10:             .Property(x => x.Id).MapToAttribute("Bk_Id")
  11:             .Property(x => x.ShelveId).MapToAttribute("Shelve_Id")
  12:             .Property(x => x.Title).MapToAttribute("Bk_Title")
  13:             .Property(x => x.BookInfo).MarkToIgnore()
  14:         .End
  15:         .For<Library>()
  16:         .MapToEntity("Bk_Library")
  17:     .InstantiateIn(this);    

Note that there is a base settings that will be applied to all the common properties under my project and is a local class inherited from LinqExtender.Configruation.All. During the instantiation it will be applied to the current container instance.

Now, as i have said extender should be used a container so that there should be no strings attached. In my entity class, i can implement IQueryObject to mark it as a query class, but if not then extender should be smarter enough to implement it internally when the entity is used in LINQ query via extender query class. Actually , under the hood it dynamically creates a proxy via Reflection.Emit that has the settings and ensures IQueryObject. I will in days time come out with a open source project where i have practiced all these reflection stuffs and which i will extensively blog about , so stay tuned.

To make it more clear i have re-implemented the LINQ to Twitter post using the new extender. So few things like implementing IQueryObject and defining unique identifier is no longer required. You can get the new copy at the end of the post. Also, before i forget, there is a tiny catch which is you can’t left the extender auto generate settings, if you are using it in medium-trust as Reflection.Emit does not work in such environment. In that case you will have to implement IQueryObject by hand and add the attributes manually to make it work so i am leaving LinqToTwitter post untouched [See bottom]. I know most of the good hosting providers these days support high trust and soon i think medium trust will become classic with dynamic proxy generation and cross domain request is becoming growing common (Just my two cents, don't want to get in war with these ;)).

In previous release you can access the Fluent implementation of Bucket just by Bucket.Instance, as it seems that i have somewhere statically stored the bucket object and the syntax bit more sexy but considering multiple threads , there is a possibility of data corruption. Therefore Bucket instance is passed to Query<T> overrides as IBucket and new syntax for Bucket.Instance follows

   1: FluentBucket fluentBucket = FluentBucket.As(bucket);
   2: // do your work.

Down to acknowledgement , Microsoft Germany has published two new tutorials around LinqExtender using LinqToFlickr and LinqToTwitter that also led me fix the VB method call issue and thanks Lars and Tim. In German way i would say, “Danke[Dang – ko]”. You can find the recordings in the following URLs

http://www.microsoft.com/germany/msdn/solve/codeclips/library.aspx?id=msdn_de_33389

http://www.microsoft.com/germany/msdn/solve/codeclips/library.aspx?id=msdn_de_33369

Finally, while we are preparing our new Sitefinity release and making some LINQ extensions, bob’s comments were helpful shaping the release as well and nonetheless all community reviews and feedbacks are great. Try the new release here.

Linq To Twitter Source : LinqToTwitter.zip

Enjoy !!!

June 7, 2009 => Correction ,Reflecton.Emit does not work in medium trust for unmanaged IL codes and  possible IL that might destabilize the runtime that it finds by verifying the assembly, will show more on it in upcoming post. Added a patch of extender, this will work in medium trust with auto implementaion and fluent configuration.

Shout it

In a post few month back , i showed how can i simulate a callback using JQuery and ASP.NET with my experimental FlickrXplorer project. More detail on this can be found at the following URL

http://weblogs.asp.net/mehfuzh/archive/2008/10/13/using-jquery-to-do-ajax-form-posts-in-asp-net-mvc.aspx

Now, what i have done here is basically, i did an AJAX call to the controller and rendered the view result in a div.  One thing about this way is that i am able to make the view strongly typed as i first rendered view using standard way with all the strongly type ViewData.Model. It is nice for rendering views with small amount of html in it (Like , tag lists), but slow for big outputs. The best practice for all these is to use the JSONResult. Here, i will show a small example of how i render the comment list in FlickrXplorer using JSON result and JQuery $.get in conjunction to make the AJAX request.

First, let’s see how the controller looks like. Assume that there is a controller named CommentController and we have an action named List

[AcceptVerbs(HttpVerbs.Get)]
public ActionResult List(string photoId, int ? index, int? page)
{
    int skip = index == null ? 0 : index.Value*page.Value;
    int take = page == null ? int.MaxValue : page.Value;

    PhotoCommentViewData comments = new PhotoCommentViewData();

    try
    {
        comments = model.GetComments(photoId, skip, take, false);
        return Json(comments);
    }
    catch (Exception ex)
    {
         /// do some exception tracking
    }
}

As it seems, it gets the list of comment from its underlying model and then pass it through JSON result. Optionally, i limited that the action should only be accepting GET calls as it is a list request.

Now, coming back to JavaScript we first need to do a GET request to this action and syntax for doing it in JQuery style:

$.get(url, { index: index, page: page }, function(result) {

   processComments(url, result, index, page, loaderTarget, contentTarget); [strike though is not covered in the post]

 }, "json");

The first argument is the comment list URL, this is constructed in server side using Url.Action and then passed in the JS that also ensures the routing setup, second is the JSON string for params, third is the callback for the result it fetches and final is the type of response it should contain.

Inside processComments , i basically do some DOM operations to build the necessary html block

$("#layout")
    .append($("<tr/>")
        .css({'vertical-align' : 'top'})
        .append($("<td/>")
        .attr("class", "said")
        .append($("<strong/>")
        .append($("<a/>")
                .attr("href", "/Photo/NsId/" + comment.Author)
                .css({ 'color': '#0063DC', 'text-decoration': 'none'})
                .append(comment.AuthorName))
            .append("&nbsp;says: "))
        .append($("<br/>"))
        .append(comment.Text)));

So, there is a “layout” panel in the comment container where i am to add the dynamic html. jQuery has this nice way of adding html in a chain and elements under it. For example i want to render a html link with some custom CSS and i want to point that to some URL as well.

Using jQuery , i would do

$("<a/>")
 .attr("href", "/Photo/NsId/" + comment.Author)
 .css({ 'color': '#0063DC', 'text-decoration': 'none' })
 .append(comment.AuthorName)

This will render a html tag something like

<a href=”/photo/NsId/x@y” style=”color:#0063DC, text-decoration:none”>Mehfuz </a>

The rule is that you start a element with $(…) in XHTML style then add necessary attributes, styles and text and it will automatically give you a well-formed html,  moreover you can nest it as you like with other elements.

The above block finally produces a look similar to

image

There are few other JS that  is responsible for generating the pager control which is not covered in this post, for this and more you can take a look at the latest bits of FlickrXplorer under www.codeplex.com/flickrXplorer .That’s so far and if you have some cool ideas to share on this,  let me know.

Hope that helps.

Shout it

In this post, i will show how you can generate SQL programmatically from M. Now, so far i have learnt that MGrammer is a contract that converts user’s input into MGraph. Now, Oslo by default comes with MSchema. Through MSchema you can define a type and extend it with MGraph to populate your repository. Here, i will use MSchema to define an entity object , then MGraph and finally run this through a custom SQL generator in C# to get my DDL statement similar to what you can generate by using intellipad that comes with Oslo SDK.

Let’s define a Contacts M

module Contacts {
    type Person {
        Id : Integer32;
        Name: Text;
        Age: Integer32;
    } where identity Id;
    
    People : Person*
    {    
        {Id = 1, Name="Steve",Age=36}, 
        {Id = 2, Name="Kazi",Age=30}
    };
    Employee : Person*
    {    
        {Id = 1, Name="Nike",Age=29} 
    };
}

So, “Person” is a type with ID,Name and Age respectively that is under “Contacts” module with two different MGraph which are “People” and “Employee” that will generate separate create table statements of “Person” type.

Internally , M is  represented somewhat like

image

Parsing the M all starts with Microsoft.M.Parser.SourceParser. In my sample M to SQL project, i used it like

SqlGenerator generator = SqlGenerator.Load(new StringReader(code));
string sql = generator.Generate();

Inside Load…

SourceParser parser = new SourceParser();
syntaxTerm = (CompilationUnit)parser.ParseTerm(reader, "buffer.m");
...
...

/// check if the parse was successful
if (!parser.LastParseSuccessful)
{
    /// do some error processing for invalid schema.
}

Now, once we have a valid grammar, our next step is to get the modules from ISytaxTerm implementation, for each module we have to process the MGraph for the specified type.

foreach (IModuleDeclaration declaration in syntaxTerm.Modules)
{
    builder.AppendSetExtactAbort();
    builder.AppendGo();

    builder.AppendBeginTransaction();
    builder.AppendGo();

    builder.AppendSetANSINulls();
    builder.AppendGo();

    builder.AppendCreateSchema(declaration.Name.Value);
    builder.AppendGo();
    builder.Append(ProcessMembers(declaration.Name.Value, declaration.Members));

    builder.AppendCommitTransaction();
    builder.AppendGo();
}

This shows the basic skeleton of how the output will look like. ProcessMembers is the actual place that generates the entity from type and associates Insert statements. Only, during the MGraph initialization type is realized. Therefore, the table name is equal to the extent name not to the type name.

Inside ProcessMembers …

/// There will be three declaration 
/// -TypeDeclaion
///- ExtentDeclaration
///- ExtnetDeclaration
foreach (IDeclaration declaration in declarations)
{
    if (declaration is TypeDeclaration)
    {
        typeDeclaration = (TypeDeclaration) declaration;
    }
 
    if (declaration is ExtentDeclaration)
    {
       /// use the type declaration from previous step.
        builder.Append(ProcessExtent(name, typeDeclaration, (ExtentDeclaration) declaration));
    }
}

TypeDeclaration is narrowed down to ParameteriedExpression , which has different operations for the declaring type, which is tracked by  DeclartionReference.Name (“$Operator$Where”, “$Operator$Identity” , etc). Therefore, to process constraint and data types separately, we need to switch them for operation type.  Now, the question is how the MGraph is referred from the type specified. The JSON like string  is basically translated to GraphExpression. Like for the “Person” type , we have three properties. GraphExpression.Successors.Count will be equal to numeric value 3 and each of those which are GraphExpression themselves will have only one successor which is a M.Literal. This will actually contain the property name, type and value for parent type ref.

GraphExpression graphExpression = (GraphExpression) extentDeclaration.InitialValue;

if (extentDeclaration.InitialValue != null)
{
foreach (GraphExpression successor in graphExpression.Successors)
{
    int index = 0;

    foreach (var expression in successor.Successors)
    {
        if (expression is GraphExpression)
        {
            GraphExpression node = (GraphExpression) expression;

            Literal literal = (Literal) (node).Successors[0];
           /// process fragment
        }

        index++;
    }
   /// process the final statement
}

That’s all to get started. All the code has been taken from a MSqlProvider that i created while learning M. The technology is itself in bleeding edge so things will change and i will share my updated knowledge forward. You can further enhance MSQLProvider to run the result in configured database along with the mx.exe that comes with Oslo SDK.

You can download the source here to play around and see it live.

Enjoy!!

kick it on DotNetKicks.com

Just released LinqExtender 2.0. Over previous release , it contains generally bugs fixes. Overall, I have focused on striping out complexity as much as possible to keep you focused on your business logic. You can see the full list of features in the documentation that comes with it.

Now, while creating the LinqToTwitter example, I have shown that creating custom providers with LinqExtender requires two steps, first you have to define a query object by implementing IQueryObject and then you have made a Query<T> successor and override few methods. In LinqExtender, object/ entity equivalent is Bucket. Starting from 2.0, there is a Fluent interface implementation of it that works as an internal DSL and gives you a single entry point for all query and update , inserts and delete, thus making things more declarative and one way.

In this post, I will show you how to generate SQL statement from LINQ query with nested where clause, using the fluent interface implementation. To start, let's consider the following query

var query = from book in bookContext
where ((book.Author == "Mehfuz" && book.ISBN == "2") || book.Id == books[2].Id) || 
(book.Author == "Paolo Pialorsi" && (book.ISBN == "100-11-777" || book.ISBN == "1"))
select book;

It's taken from the Entrypoint.cs test class that comes with LinqExtender project along with OpenLinqToSql sample ORM. Now, to do things in a very basic way. Let's first create a string builder and append the initial select.

StringBuilder builder = new StringBuilder("SELECT * FROM [" + Bucket.Instance.Entity.Name + "]");

To be precise, Bucket.Instance is the entry point for all query and object details. Moving to where statement , LinqExtender uses simplified expression tree that is exposed by Bucket.Instance.ExpressionTree. All we need to setup how the output will look like and the rest will be covered by the toolkit.

builder.Append("WHERE");
builder.Append("\r\n");
Bucket
.Instance
.ExpressionTree
.DescribeContainerAs(builder)
.Root((containter, operatorType) => containter.Append(" " + operatorType + " "))
.Begin(container => container.Append("("))
.EachLeaf((container, item) =>
 {
     string value = GetValue(item.Value);
     container.Append(item.Name + RelationalOperators[item.RelationType] + value);
 })
.End(container => container.Append(")"))
.Execute();

Here, let assume that GetValue(..) will return a formatted SQL string and ReleationalOperators is a dictionary that has mappings for item.RealtionType ( GreaterThan -> ">" , LessThan => "<", etc). Basically, as the implementation shows, we are defining the way output will be stored in the container (In this case, StringBuilder). All these code should be placed in Query<T>.Process(IModify<T> items) method and on execute it will just return the following SQL block

Select * from book 
 WHERE
(((Author='Mehfuz' AND ISBN='2') OR Bk_Id='1734') 
OR (Author='Paolo Pialorsi' AND (ISBN='100-11-777' OR ISBN='1'))) 

Extender basically knows how to get though this nested where clause. From basic computer science, it builds linked objects based on a Syntax Tree, with each parenthesis adding diversion to the tree like shown below

 image

So far, this is a very rudimentary example. With 2.0 it is also possible to build your own reusable format provider for building literals that saves repetitive code blocks for same kind of task (updating a database). This is actually just a schema builder that defines how the output will look like using the same Bucket.Instance calls. Out of the box, TSqlFormatProvider is provided.Depending under which Query<T> method it is used; it will generate insert, update, delete or select statement where all you have to call Bucket.Instance.Translate(...).

string sql = Bucket.Instance.Translate(new TSqlFormatProvider());

This enables you to build not only your own LINQ to Anything format providers but also share it with the community to reuse what is already out there. More information on how to get started can be found at the project documentation.

In a word, Bucket.Instance is all you need to go through and build your own LINQ provider. Optionally, you can override the following methods (under Query<T>) to give your provider various OTS (Object tracking service) support .

1. AddItem () - called during SubmitChanges() call for new object.
2. RemoveItem () - called during SubmitChanges() call for Delete.
3. UpdateItem () – called during SubmitChanges() call for update.

I have updated Creating LinqToTwitter using LinqExtender post with the latest release. You can get a copy of the latest 2.0 release from www.codeplex.com/linqextender,  and of course as usual all your feedbacks are really helping to shape the toolkit.

Hope that helps,

Update on 28th Feb 2009, with new patch for Bucket.Instance.ExpressionTree

kick it on DotNetKicks.com

In my last post, I showed how to turn on Amazon S3 support in Sitefinity. In this post, I will show how to create a simple video site using the video library support that is bundled with 3.6 release. It is often required that you want to add a streaming video content to your site, either it could be a product demonstration or a getting started tutorial. Using third party video provider is one way to get the job done but sometimes it requires a lot of manual work, thus headache for managing them as the library grows bigger. 3.6 release breaks you free from all these and provides an easy way for creating , uploading and managing video content.

Now, uploading video content and publishing them requires a few easy steps. You can either decide to store / stream it from your hosting server or from Amazon S3, typically which can be divided into following.

    1. Configure the data provider
    2. Create a video library
    3. Upload videos
    4. Create a page and add the video content control onto it.
    5. Publish it.

Sitefinity uses RadMediaPlayer (comes as part of Telerik Rad controls suit), a cool Silverlight media player made on top of the basic MediaPlayer with out of the box features that can get you on the groove right away. As I mentioned already, Sitefinity is a combination of content, modules and pages. Modules can have their own data providers, own handlers, whereas each page can control its content presentation by using the revolutionary control designer.

image

Coming back, to start creating our simple video site first we need to create a module (which is a video library). We can either choose to use the default data provider "Libraries" or we can define our own custom provider. To define one by ourselves we must ensure the following

  • Create an entry under <cmsEngine> that looks like "<add name=Libraries..."  or like the one described in Amazon S3 post.
  • Add related meta fields under <metaFields> that are same as the default ones with only prefix = name, you have given to the data provider
    <add key="Libraries.Description" valueType="ShortText" visible="True" searchable="True" sortable="True" defaultValue="" />

    This converters to the following where name = "MyLib". Similar goes for the rest.

    <add key="MyLib.Description" valueType="ShortText" visible="True" searchable="True" sortable="True" defaultValue="" />
  • Map it with "Libraries" module under libraries.genericContentproviders
     <libraries defaultGenericProvider="Libraries">
          <genericContentProviders>
            <add providerName="MyLib" publicationDateField="Publication_Date" />
      .........
      .........
  • Under cmsEngine.providers.add (where name="MyLib/Libraries") there is a key called applicatinName, which is set to "/Libraries" but we can set it to "/MyOwnApplciation" to ensure that it does not mess data up with the default one.

For multiple data providers you will see a dropdown in the library create wizard, to upload under the specific one, we just need to select and start uploading.

 MultipleLIbrary

So, once we have created a video library it will automatically take us to the upload view and once it is done, we can see the preview right away. Next,  we just need to drag and drop the video control onto the page. To further customize it, we can play around with designer that is tied with it (click on the "edit" link) we can set things like the max width for the thumbnail or video player and tweak labels and meta datas as we go.

VideoEdit 

We have already seen how to set up multiple data providers and select specific libraries from designer. Moreover, we can set custom provider from advance mode of the content control. This applies to all Sitefinity content controls like Events, Blogs, etc that implements the same generic content base class.

advanced

Finally, if all are set, we will see a video page like the one shown just below. While we are uploading video, each will have a thumbnail generated from the video itself that will be shown in the list view. Each items in list view is tied with a detail view (similar to video sites), where you can play the video, leave comments and bookmark it in your favorite social networking site.

Video 

We have already seen in my previous post that we can set the urlRewriteFormat in the data provider declaration. For local libraries , when urlRewriteFormat="~/{LibraryName}/[Name].sflb.ashx" the video detail permalink will be http://<hostname>/<applicationPath>/<pagename>/MyLib/ButterFly.aspx, that means the video content control (detail view) will have the same mirror URL terminating with "*.aspx". So, if you change the URL format in library data provider it will reflect in the content control as well.

So far, that's all for the simple video site. Let us know, how you like to shape it for you and what features you really want to add in it. You can grab a free community edition  and of course any feedback will really make us feel better.

Hope that helps

kick it on DotNetKicks.com

Here at Telerik, Sitefinity team is preparing for the 3.6 release next week. 3.6 comes with lot of core level enhancements and cool new features. Of which , I can't but mention one feature that will make most ISV vendors who want the power of CMS but want to let go their pain of managing large content. As, the title suggests its external storage support via Amazon S3. The data plan for hosting and transferring data to/from S3 server is pretty cheap these days in return to world class service with almost zero downtime that it offers which of course makes the most of your money. If you are not familiar with Sitefinity to add it is a product by Telerik  that comes with full content management features along with blogs, news, events, list and few other pre-made modules with easy setup and management via unique control designer.

In this post, I will show how to turn on S3 support in Sitefinity 3.6 that comes out of the box.In Sitefinity, files are kept in Library. Every file which can be audio, video, document or anything else is stored as Library entries, which are consumed by modules for example images in blog post. These library entries are dependant on data providers which can be extended from internal to cloud. S3 support is just a plugin to that extensibility.

Now, to configure S3 support let's start with a simple example. I want to create a personal blog and I want all the images and attachments go to Amazon S3 server. To do so , we need to ensure the following

  1. Turn on the S3 data provider in web.config (that comes with the installation)
  2. Add your S3 API and secret key to the specific place holder.
  3. Create your blog module
  4. Drag and drop your module over a page.
  5. Publish it
  6. Use Admin or Live writer to do your post with Images and attachments.

Steps 3- 6 are pretty basic and are beyond the scope of this post. I will deeply encourage you to grab a community version from the project's download page(link provided at the end) and see the step by step documentation that comes right with it.

To start setting up, first we need to navigate to the cmsEngine node. If amazon block is there then un-comment it and remove the default "Libraries" data provider or less add the following data provider.

<add name="Libraries" urlRewriteFormat="~/{LibraryName}/[Name].s3lb" 
    thumbnailExtension=".tmb"
    urlDateTimeFormat="yy-MM-dd" urlWhitespaceChar="_" visible="False" 
    defaultMetaField="Name" applicationName="/Libraries" allowVersioning="True" 
    allowLocalization="False"
    localizationProviderName="" allowWorkflow="False" 
    securityProviderName="" versioningProviderName=""
    connectionStringName="GenericContentConnection" 
    type="Telerik.Libraries.Data.AmazonProvider, Telerik.Libraries.Data"
    tagEditorTemplate="~/Sitefinity/Admin/ControlTemplates/Libraries/BatchTagsEditor.ascx"/>

Now, inside the data provider you can do plenty of tweaks like, modify the URL format , turn on/off localization, have multiple providers of same type (with different name) and map it from admin, etc. The most important thing for libraries or any other module is the urlRewriteFormat. You can set up any format and the detail (permalink) url will generate accordingly. Here, [Name] = name of the file but you can change it to any meta fields that the module supports or add any constants you want.

The amazon data provider is different in two ways from local one

  1. It has different content extension(.s3lb).
  2. Separate data provider dedicated to communicate with S3. Default type = Telerik.Libraries.Data.DefaultProvider

As, we are pretty much done with the first step. Next, we have to put the right API and secret access key that we have purchased from Amazon. All the transfers will be made against the key associated to a user's account. Under the telerik node we have a section called storageSetting which looks like

<storageSetting defaultProvider="Amazon">
      <providers>
        <add name="Amazon"           
          type="Telerik.Libraries.AmazonStorage.AmazonStorageProvider, Telerik.Libraries" 
          downloadUrlPrefix="http://s3.amazonaws.com" 
          accessKey="YOUR ACCESS KEY"
          secretAccessKey="YOUR SECRECT KEY"
          bucketName="Sitefinity" />
      </providers>
</storageSetting>

Just Fill in the gaps and you are through. In Sitefinity module is the heart of every content and a page can contain 1..N modules but each module can have its own specific settings like data provider, URL format, etc. As we have finished setting up the data provider and associated configuration to talk with S3 server. we can now play the way we like. We can create gallery with image library or setup  a download list for providing documents and pdfs. Even we can setup a video site(which I will cover in later post) easily. But for now , let's create a blog, write a post, add some images and text and publish it.

SiteFinity Admin page edit 

The above is the snap of edit mode of the blog page along with the post that I have done using Live Writer. Now, to be more sure (we haven't done any wrong) if you monitor using firebug Net tab your list will look like the pasted

Fiirbug stat 

Down in the bottom there are two request for images posted by Live Writer which are directly forwarded to S3 server (302->200). Thus, the whole load goes to Amazon and gives you peaceful night sleep.

This is basically shortly all for now, I will cover more things from Sitefinity like creating video site , working with Sitefinity designers to configure your site without touching any of the html code and more that is interesting in my future posts.

Till then, you can get a free copy from http://www.sitefinity.com/product/community-edition.aspx (you need to create an account if you haven't. You can use it for other Telerik products and to log any issues as well.

Hope that helps

kick it on DotNetKicks.com

In this post, I will show how to cook an LINQToTwitter library with LinqExtender. My target is to create it easily and over existing twitter API. I used Yedda Twitter library. Yedda library basically returns response in XML/JSON/RSS/ATOM format depending on the option provided. I used XML format and serialized it to objects.

Now, with our LINQToTwitter we will be able to get public/user/friend's timeline, sort them via date, screen name. Finally, we will be able to update our status. Twitter response format for status looks like

<statuses>
     <status>
          ....
          ....
         <user>
             ....
             ....
         </user>
     </status>
</statuses>

For this, I have created a class named Twit that maps to the status and a class name User that is used as an auto property in Twit. These two classes contain some Xml Serialization attributes  by which response is serialized to respective properties.

We have our Twitter API in place, the next step is to make them LINQ-able. Prior to the latest release to make an object queryable, it is required to inherit from QueryObjectBase. Using this approach the feed back I got from users that they need to try LINQ in their existing data objects, which is already inherited from a base class and they need to change a lot for using the abstract approach. The new release just solves that with a blank IQueryObject interface, which makes it more easier to port LINQ into existing data object in no time.

Building LINQ providers with LinqExtender requires few easy steps. First, Create or convert an object to be queryable by implementing IQueryObject interface. With our Twit object it looks like

image

Here, note that I added UniqueIdentifier attribute on top of Id property that makes the object trackable during add, update or delete. Every object should have property that distinguishes entities from each other and should be marked Unique. There is another attribute called Ignore. If we want properties not to be processed by the toolkit we can just put the attribute on top. Once, it is done next we have to create query provider that will be used for making LINQ queries. Let's name it TwitterContext and it needs to inherit Query<Twit> and override few methods depending on support that the library is going to provide.

image

Now, as I said earlier that we will be doing some queries by timeline and update our status. While adding new items we need to override Query<Twit>.AddItem where we will add object to repository and update any property that we need to get and finally return true/ false depending on the execution status.

Inside Query<Twit>.AddItem it looks like (Partially).

string username = (string)Bucket.Instance.For.Item(QueryItems.USERNAME).Value;
string password = (string)Bucket.Instance.For.Item(QueryItems.PASSWORD).Value;

Validate(username, password);

string statusText = (string)Bucket.Instance.For.Item("Text").Value;
string source = (string)Bucket.Instance.For.Item("Source").Value;

Twitter twitter = new Twitter();

if (string.IsNullOrEmpty(source))
{
    twitter.Source = source;
}

string response = twitter.Update(username, password, statusText, Twitter.OutputFormatType.XML);

Twit twit = XmlToObject<Twit>.Deserialize(response);
Type twitType = twit.GetType();

Bucket.Instance.For.EachItem
.Process(delegate(BucketItem item)
 {
     item.Value = twitType.GetProperty(item.Name).GetValue(twit, null);
 });

The code is pretty simple as we can see that we are getting the required values from bucket object , setting them into Twitter API, updating the result object (Twit) and finally returning TRUE that will tell LinqExtender to update it's collection. For any FALSE or exception toolkit will remove the partial object. There are other methods that we can override like UpdateItem, RemoveItem, GetItem. Due to the API constraint we will skip them now. But you can check them out at the project documentation or in the provided OpenLinqToSql ORM.

Next, we need to override the Query<Twit>.Process which will be handling the LINQ queries we make.

protected override void Process(LinqExtender.Interface.IModify<Twit> items)
{
    string response = string.Empty;

    Twitter twitter = new Twitter();

    /// general select statement with no where clause.
    if (!Bucket.Instance.IsDirty)
    {
        response = twitter.GetPublicTimeline(Twitter.OutputFormatType.XML);
    }
    else
    {
            string username = (string)Bucket.Instance.For.Item(QueryItems.USERNAME).Value;
            string password = (string)Bucket.Instance.For.Item(QueryItems.PASSWORD).Value;
            object obj = (Timeline)Bucket.Instance.For.Item(QueryItems.TIMELINE).Value;

            if (obj == null)
            {
                throw new Exception("Must provide a valid timeline");
            }

            Timeline timeline = (Timeline) obj;

            if (timeline == Timeline.Friends)
            {
                Validate(username, password);
                response = twitter.GetFriendsTimeline(username, password, Twitter.OutputFormatType.XML);
            }
            else if (timeline == Timeline.User)
            {
                response = twitter.GetUserTimeline(username, password, Twitter.OutputFormatType.XML);
            }
        }

        Twits twits = XmlToObject<Twits>.Deserialize(response);

        /// do some extra in-memory stuff which are not supported by API
        IList<Twit> list = GetList(twits);

        items.AddRange(list, true);
}

Here, we can see that Bucket.Instance.IsDirty is used for public timelines. When Bucket.Instance.IsDirty = false, we can be sure that no where clause is used and as such we can decide to handle the way we want. IModify<T>.AddRange with true tells the toolkit to do in-memory sort when orderby is used. There is another way to do sort (natural sort) by which we can pass in the sort data to source (if supported) and in that case we can either pass false or use the other overload of AddRange or even add items one by one.

This is our simple LINQToTwitter library so far. To start let's do some queries. First, I want to get my friend's timeline sorted by last updated date.

TwitterContext context = new TwitterConext();
var query = from twit in context
             where twit.Username == "myuser"
                       && twit.Password == "mypass"
                       && twit.Timeline == Timeline.Friends
             orderby twit.LastUpdated descending
             select twit;

foreach (var t in query)
{
     /// do something useful.
}

I also want to update my status. To do so, the following needs to be done.

Twit twitObject = new Twit()
{
  Username = USERNAME,
  Password = PASSWORD,
  Text = "Happy new year to all"
};
context.Add(twitObject);
context.SubmitChanges();

if (!string.IsNullOrEmpty(twitObject.Id))
{
    /// do something
}

There are few other queries that can be made. You can download the source here. I have shown here very few of the options in LinqExtender, to know more check out the word document at project's release page. Also, check the OpenLinqToSql ORM that is made on it and comes out of the box.

Enjoy!!

Updated on Feb 24th 2009 with LinqExtender 2.0

kick it on DotNetKicks.com

There could be plenty of reason that you might need to do a cross domain web request in your application. One could be let's say you want to divert the resource pressure from your server to some third party content provider like Amazon S3. In my last post I have mentioned a bit about uploading content using WSE to S3 server. I also mentioned about the simple library located at www.codeplex.com/threesharp that does not necessarily require you to work in full trust mode. This is also true if you have a personalized startpage where you want your wideget developers to restrict to partiuclar URLs.

Now, before starting it is worth mentioning that in medium trust mode any cross domain request is disabled by default. In other words, you can do calls only for your local endpoints.

How to get around this ?

When you install .net framework under %installed dir%\framework\version\config you will find a web_(high or medium or low)trust.config file that defines some policy about how your application will behave in a particular security level specified.

trustLevel

Now, to customize the default medium trust settings best is to copy it to your application directory so that you can tweaks thing only for your application and don't end up making it unstable in global scope. To show it as an example I have created a sample project that tries to do a web request, gets the data out and finally stuffs it to the response stream. To simulate, first let's put the following line in web.config which is pretty general for those of you works with it daily to strictly support medium trust in your app.

<trust level="Medium"  /> // this is only for simulating in localhost

Next is to define your security level and config that will look for policies

<securityPolicy>
    <trustLevel name="Medium" policyFile="web_mediumtrust.config"/>
</securityPolicy>

The above line will come into action only when medium trust is enabled. But here to note that if you dont have control over your hosting provider or your hosting has a <location allowOverride="false"> node in the root config then this approach won't work. Either you have to tell your hosting provider to add the requested URL for you or if you are doing things with your own hosting then you have to do it by yourself :-).

If you generally try to do a web request without any predefined policy defined in medium trust then you might see the following error

Request for the permission of type 'System.Net.WebPermission, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed.

To overcome, once you have medium trust config in place you need to override the connectAccess property of WebPermission class. To do so , let's say I want to grant request for search.live.com and its descendant URLs, so the changes that I need to make looks something like the pasted.

<IPermission class="WebPermission" version="1">
  <ConnectAccess>
     <URI uri="http://www\.live\.com/.*"/>
  </ConnectAccess>
</IPermission>

The default URL for requests in medium trust config is set to $OriginHost$ which says, "Can't let you do request to domains other than your own :-)" . The URL property takes regular expression and I can grant as many as URLs I want just by adding a new URL element and associating it with proper endpoint that my API or application might call.  You can download the sample here and hope that helps a bit.

Update : As most of the shared hosting are with <location allowOverride="false"> the approach is useful only if you are a system administrator or want to have full control over your site by restricting your widget developers to a limited URLs

Enjoy !!!

kick it on 
DotNetKicks.com

Recently, I was bit playing around with amazon S3. This is said to be the cheapest storage for people who don't want to spend much of their brain cells thinking about DB issues. Now, Amazon S3 primarily supports Web service (SOAP) to store and query data out of the S3 server. The concept behind S3 is that you have to create a bucket and under it you can store as many objects you want. Generally for a particular account 100 is the threshold value for buckets. In this post, I will basically focus on configuring your proxy for uploading large files to amazon using WSE and optionally I will point out a cool browser extension for analyzing your S3 space and third party library alternative.

Amazon depends on WSE 2.0 if you want to upload files larger than 1MB. The large file is transferred in the form of DIME (Direct Internet Message Encapsulation) attachment. Microsoft has released WSE 3.0 that uses MTOM which is a W3C recommendation for replacing DIME. But the time of writing this post it is not yet supported by Amazon.

How to prepare your proxy to support DIME  ?  Well, first you have to get the WSE 2.0 SP3 from Microsoft downloads. Once the step is completed, you can copy the Microsoft.Web.Services2.dll to your local folder. Basically, once the WSE 2.0 pack is installed the DLL is added to the GAC so it is not a required step, but if you want to ship it along with your distribution then you can consider doing that.

Now once you have created the proxy from http://s3.amazonaws.com/doc/2006-03-01/AmazonS3.wsdl. You need to do some manual tweaks before your project is ready to go with WSE.

From the Diff for Reference.cs we can see that we have to add a reference to Microsoft.Web.Services2 and inherit from WebServicesClientProtocol instead of SoapHttpClientProtocol

ReferenceDiff

Pretty easy but hang on a bit. While you are instantiating the proxy class you have to add/remove some standard WS filters. Therefore, if we have a method called CreateProxyInstance , with the OutputFilters it will look like

private AmazonS3 CreateProxyInstance()
{
    AmazonS3 proxy = new AmazonS3();

    /// Remove the standard WSE soap headers.
    proxy.Pipeline.OutputFilters.Remove(typeof(Microsoft.Web.Services2.Security.SecurityOutputFilter));
    proxy.Pipeline.OutputFilters.Remove(typeof(Microsoft.Web.Services2.Referral.ReferralOutputFilter));
    proxy.Pipeline.OutputFilters.Remove(typeof(Microsoft.Web.Services2.Policy.PolicyEnforcementOutputFilter));

    /// Add our custom filter to remove the unwanted WSE soap headers.
    proxy.Pipeline.OutputFilters.Add(new HeaderOutputFilter("wsa:"));

    return proxy;
}

Here the HeaderOuputFilter is a inherited from SoapOutputFilter , where we build the SoapEnvolop header without wsa node. Pretty much that's it for setting up the proxy , now let's upload something to the S3 server.

MetadataEntry[] metadataEntries = new MetadataEntry[2];

metadataEntries[0] = mContentType;
metadataEntries[1] = contentLength;

DateTime timestamp = GetCurrentTimeInMilliseconds();
string signature = GenerateSignature("PutObject", timestamp);

using (AmazonS3 proxy = CreateProxyInstance())
{
    MemoryStream stream = new MemoryStream(content);
    Microsoft.Web.Services2.Dime.DimeAttachment dimeAttachment = new 
    Microsoft.Web.Services2.Dime.DimeAttachment("S3Object",   
    Microsoft.Web.Services2.Dime.TypeFormat.Unknown, stream);
    dimeAttachment.ContentType = contentType; 
    /// add the object in wire.
    proxy.RequestSoapContext.Attachments.Add(dimeAttachment);
    proxy.PutObject(_bucketName, key, metadataEntries, content.LongLength, grants, 
    StorageClass.STANDARD, true, _accessKey, timestamp, true, signature, null);
}

From the fragmented code block we can see that "PutObject" is the method for large content not "PutObjectInLine" and along passing the stream, we have to pass the content type as well or less it wont recognized properly even you specify it in the MimeDataEntry array.

So far that is all. I have joined in a tiny class library project that will be helpful if you are getting started with S3 using SOAP. You can download it HERE. About the tool there is a cool firefox addon called S3Fox that gives you an explorer look for analyzing and even let you to add/delete items directly from S3.

s3Fox

Finally, WSE 2.0 does not work in medium trust mode which is an absolute show stopper for CMS and blog solutions. There is a nice S3 library at codeplex can be found at www.codeplex.com/ThreeSharp that don't requires any of these and can be a good alternative for medium trust problem.

Enjoy !!!

kick it on DotNetKicks.com
More Posts Next page »