Fredrik Normén

ASP.NET, AJAX, Silverlight, RIA, Architecture, Clean Code

  • Azure Mobile Service and manual database migrations

    In my previous blog post I wrote about my experience with Azure Mobile Service and Database migrations. In this blog post I will explain step by step how to get manual migration in place.

    I like to have my migrations close to the domain model which I often create as a separate class library. Then I enable database migration for the Domain project using Package Manager in Visual Studio (VS) (Tools -> NuGet Package Manager -> Package Manager Control):

    Enable-Migrations

    By default Azure Mobile Services uses automatic migration with code first. It’s also configured to no support data loss during migrations so we need to turn this off (if you don’t care about data loss).

    When database migration is enabled a Configuration.cs file will be added to the Migrations folder. To add the following to its ctor (constructor) we will disable automatic migrations and data loss is allowed during a migration:

      public Configuration()
      {
    AutomaticMigrationsEnabled = false;
    AutomaticMigrationDataLossAllowed = true;
    MigrationsAssembly = this.GetType().Assembly;
    }

     

    Note: The MigrationsAssembly is where the assembly where the migrations are added. In my case the Migrations are added to the Domain project.

    Next step is to make sure the correct database schema is used. By default Entity Framework (EF) database migration will use “dbo” as the default database schema. When creating an Azure Mobile Service from the Visual Studio project template, mobile service will create a DbContext and use the name of the service as the default database schema for EF (so “dbo” will not be the default schema).

    If you locate the class that inherits the DbContext in your Mobile Service project you will see the following code:

       protected override void OnModelCreating(DbModelBuilder modelBuilder)
       {
          var schema = ServiceSettingsDictionary.GetSchemaName();
       
          if (!string.IsNullOrEmpty(schema)
             modelBuilder.HasDefaultSchema(schema);
       }

     
    The code will make sure the EF will use the name of the mobile service as the default schema.

    Note: Something to have in mind is that the name of the mobile service will be the name of the project you have created, but in Azure the name will be the name of the azure mobile service you have specified when you created the service in the Azure portal.

    This key (MS_MobileServiceName) is added to appSettings in the web.config to set the name of the mobile service:

        <add key="MS_MobileServiceName" value="MySampleMobileSerivceProjectName " />

    If you created your project in VS with a different name than the name you gave the Azure Mobile Service, you will get some problems when running your migrations. Because of the code in the OnModelCreating method that sets the EF default schema to mobile service name, all migrations you are added with the “Add-Migration” command will use the value of the key MS_MobileServiceName as the schema. That will cause a mismatch when you publish the code to Azure. If you have more than one Mobile Service environment in Azure, like Test and Prod, you can remove the code from the OnModelCreating and let EF use the default schema “dbo”. If you only have one Mobile Service environment, just make sure the MS_MobileServiceName is set to the name you gave the Azure Mobile service in Azure and skip down to where you see the code Database.SetInitializer in this blog post.

    When creating a Mobile Service in Azure, it will add a database schema to the associated database. The schema will have the name of the Mobile Service. Permissions will also be granted to this schema. The database schema that is created need to be added as part of EF database migrations, if not migrations will fail.

    Note: I will recommend you to have two Azure Mobile Service environment in Azure, one for test and one for production.

    If you have two environments, just remove the code that sets the default schema in the OnModelCreating.

       var schema = ServiceSettingsDictionary.GetSchemaName();

       if (!string.IsNullOrEmpty(schema)
          modelBuilder.HasDefaultSchema(schema);

     

    Next step is to give the Azure Mobile Service database user permission to perform actions against the “dbo” schema. When a Mobile Service is created in the Azure portal, a random database use will be created and added to the associated database. This user will have its default schema set to the name of the Mobile Service. We need to make sure that user will have its default schema change to “dbo” and some other permission against the “dbo” schema, like ALTER, SELECT, UPDATE and DELETE etc.

    To make sure the database user created by Azure Mobile service gets the correct permission against the “dbo” schema, connect to the database (for example through the Microsoft SQL Server Management) and go to Users under Security of your database. You will see a user with a lot of random character and end with “LoginUser”. That is the user created by the Mobile Service. Create a new Query and write (replace <UserName> with the created Azure Mobile service user):

       ALTER USER <UserName> WITH DEFAULT_SCHEMA = dbo; 

    Then the following to give the user permissions to perform action against the “dbo” schema:

       GRANT INSERT ON SCHEMA :: dbo TO <UserName>
       GRANT SELECT ON SCHEMA :: dbo TO <UserName>
       GRANT ALTER ON SCHEMA :: dbo TO <UserName>
       GRANT DELETE ON SCHEMA :: dbo TO <UserName>
       GRANT UPDATE ON SCHEMA :: dbo TO <UserName>
       GRANT CONTROL ON SCHEMA :: dbo TO <UserName>
       GRANT EXECUTE ON SCHEMA :: dbo TO <UserName>


    The last step is to make sure the Database Migration will be added while the Azure Mobile service starts. By doing that, add the following code to the WebApiConfig.cs in the  App_Start folder of your Mobile Service project:

       Database.SetInitializer<YourEntityContext>(null);

       var migrator = new DbMigrator(new Configuration());
       migrator.Update();


    Note: There is probably already a Database.SetInitializer in you code, just replace it with the code above, just make sure you specify your name of the EntityContext.

    The SetInializer method will null as an argument will make sure no migrations will take place while there are some changes to your model the first time your code touches the EF DbContext. The code will disable that. Instead the DbMigrator will be used to update the database while the Azure Mobile Service is started.

    If everything is done correctly, you can now use manual database migrations, the key thing to get it working is to give the correct user the right permissions to a database schema.

    If you want to know when I will post a new blog post, please follow me on twitter @fredrikn

    Read more...

  • Database Migration and Azure Mobile Service adventure

    During the last months I have worked with an API backend for a mobile application. Azure Mobile Service with a .Net backend is our infrastructure platform.  There was a lot of struggling to get Entity Framework Database Migration work the way I wanted it to work. But the solution was quite easy, but needed some few changes that I think shouldn’t be needed at all.

    By default when creating an Azure Mobile Service Backend with Entity Framework, the Automatic migration is turned on. In my case I don’t want the model and the tool take control over my database schema and migration, so I turned off that feature. It was now the adventure started ;)

    How Entity Framework Database Migration handles database schema


    By default Entity Framework uses “dbo” as the default database schema. So the user that logins to the database when a Migration should be applied need permissions to that schema. When creating an Azure Mobile Service a random SQL Server user will be created and added to the associated database. That user doesn’t have permissions to the “dbo” schema, but to another schema that is based on the name of the Azure Mobile Service name. If you create an Azure Mobile Service with the name “mymobile-test”, the SQL Server user will have permissions to a schema with the name “mymobile_test”. That means that you need to change the Entity Framework default schema to “mymobile_test”. This can be done by overriding your DbContext’s OnModelCreating method, and use the DbModelBuilder’s (an argument to the OnModelCreating method) HasDefaultSchema method:

     

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
    var schema = ServiceSettingsDictionary.GetSchemaName();

    if (!string.IsNullOrEmpty(schema)
    modelBuilder.HasDefaultSchema(schema);
    }

     

    The code above will be part of a DbContext created by the Azure Mobile Service project template in Visual Studio. The ServiceSettingsDictionary.GetSchemaName will take the value of the appSettings “MS_MobileSeriveName” and use it as the default schema.

    Note: If you name your Azure Mobile Service project in Visual Studio to a different name than your Azure Mobile Service in Azure, your local code will use the project name as your mobile service name, but in Azure the name you gave the Azure Mobile Service. So when you add a migration with the “Add-Migration” command, the Entity Framework will use your local service name as the default schema. It will give you permission denied when you publish your code to Azure and run the migrations.

    I named the Azure Mobile Service in Azure with the name “mymobile-test”, but in Visual Studio I call my project “My.Mobile”. When I added a migration it added the My.Mobile as a schema to the migration file and also set the DefaultSchema in the migration’s .resx file to “My.Mobile”. The migration worked fine on my dev machine, but when my code was published to the Azure, I got permission denied against the schema “My.Mobile”. I removed the schema from my migration files but left the DefaultSchema in my migration’s .resx file. Now things started to work better, BUT! Because the default schema set to the DbContext is different from the one in the .resx file, the migrations will fail because Entity Framework notice a different between the schemas, it want me to run “Add-Migration” again to get the new model changes. In this case a schema change. The reason why I can’t use the “mymobile-test” as the schema name in my migrations files, is because I have two environment, test and production, they uses different names and the SQL server user for those have permissions to its own schemas. So that gives me more problem when it comes to migrations. The main problem is the migrations .resx file and its DefaultSchema key. If Entity Framework will just ignore it everything should work great by just setting the DbModelBuilder’s default schema to the name of the Azure Mobile Service.

    How did I fix the problems with database migrations?

     

    I removed the code that will set the DbModelBuilder’s default schema to the Azure Mobile Service name from the DbContext OnMdelCreating method:


    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
    //var schema = ServiceSettingsDictionary.GetSchemaName();

    //if (!string.IsNullOrEmpty(schema)
    //modelBuilder.HasDefaultSchema(schema);
    }

     
    Now Entity Framework will use “dbo” as default. But the Azure Mobile Service user against the SQL Server doesn’t have permissions to the “dbo” schema, so I just gave the user permission to it.

    GRANT CONTROL ON SCHEMA::[dbo] TO [The Generated User Login]

    I did the same thing to the production SQL Server for my Azure Mobile Service too.  I can now add migrations without worry about permission problems and schemas based on the Azure Mobile Service name.

    My next blog post will be about how to disable Automatic Migrations and take control over the migrations.

    Please feel free to follow me on twitter if you want to know when I publish more blog posts. @fredrikn

    Read more...

  • Merge hell and Feature toggle

    For about three years ago we use feature branches in our current project. It took about a half to one day to merge those branches before deployment. The reason why feature branches was used was more or less because most team members have worked like that in earlier projects and they thought it was the way to work. At that time some of them also wanted to introduce bug branches! That was the right branch strategies for them, because that was what they have learned.

    Read more...

  • Stop saying "but it works on my computer"

    Often developers and testers are using their own machines for developing and testing software. The local environment can look different, have different tools installed, even different libraries, sometimes some may use technical previews (like me using Visual Studio 2013 preview ;)). When a new team member is joining the team, it may take a whole day or more until he or she has an up and running local environment for the project. Sometimes several developers and testers works with different projects, that requires different versions of tool, libraries etc, all installed on their own local machine. Can result to a lot of mess and problems.

    It's not so strange to hear someone say "but it works on my computer".

    What if we could check-in all en tools and stuffs we need for a specific project into our source control, and when we work with a project we only gets all the tools and stuffs needed, and have an isolated environment configured and installed for the specific project? Wouldn't hat be something?. I read about teams that adds the compiler they use with other stuffs into source control, and also setup scripts to solve this problem.

    Here is one solution to make it much faster to setup a new environment for both new members and existing members based on the project they are working on, and also remove the words "but it works on my computer".

    Create a Virtual Machine for each projects. Configure and install it with the correct tools and all the things the project requires. Then make sure each team members copy those Virtual Machines to their local computer and start the correct Virtual Machine for the project they should work with. By doing so everyone will have the exact environment for a specific project. It requires of course a good computer with hyper threading. I live in a world where I hope the customer or employee understands that a computer with the correct performance, will reduce time and cost when it comes to develop software. A cheap computer can result in a higher cost compared to a more expensive computer with great performance (At the end of this blog post I will share my story about how I managed to get my former employee to invest in new computers for the whole team).

    Adding Virtual Machines to the source control will probably not be so good, each new branch or history will or can create a copy of a large Virtual Machine. So we need a new kind of "repository" to store the Virtual Machine for each projects, a "repository" everyone in the team have access to. It can be a network share, or on a server where it can be accessible over HTTP etc. Make sure you have a good backup plan for the "repository"!

    By adding a Virtual Machine to a "repository" it would be great to have some kind of pointer to the Virtual Machine to be used in our source control for a specific project. We can for example add a file to our source control that points to the Virtual Machine to be used (from where we can copy/download it, add then add it to our local Virtual Machine environment and start it up). But what if we can make this setup and start up easy for us. What if we could use a simple command like "start the Virtual Machine associated with the project"?

    There is a tool that can help us with this, Vagrant. Vagrant is a tool to that can help us copy/download, setup and start a Virtual Machine locally with just a few lines of commands. Vagrant uses by default Virtual Box but have providers for WMWare. I hope to see one plugin in the future for Hyper-V. Just type the command in your command line tool after Vagrant and VirtualBox is installed:

    vagrant init MyProjectDev http://.../MyProjectDev.box

    Vagrant will now create a local VagrantFile with the information about how to get and which virtual machine to be used. This file can then be checked in into your project source control.

    Then to start up and configure a guest machine, just type the following in the command line tool (make sure you have first navigate to the folder where the VagrantFile is located, probably where you have your local source code):

    vagrant up

    Every team member that should work with a specific project just need to get the latest VagrantFile from source control, and then type "vagrant up" and they have a guest environment started.

    When a new team member joins the team, just install Vagrant and Virtual Box on their machine. Then get the VagrantFile for the project they should work on, and then type "vagrant up". No need to install a lot of tools and stuffs that can take a lot of time before the new team member is up and running. Everyone in the team will use the exact environment for the current project.

    How did I managed to replace all computers with new better and faster computer?

    For several years ago I worked with a large enterprise product. I was frustrated every time I needed to run my code in debug mode. I hated my computer, it was so slow! I often heard team members say the same thing. One day I couldn't just stand it, so I went over to my boss and told him "we need new computers, ours are so slow". The answer was "no!". I tried with several arguments but failed. Then I remember something I read in a book, too many leaders have learned to just watch numbers. So I did the following:

    I opened the solution we where working on. I measured the time it took to build the solution on the local machine, and the time to start it up. I took an average number of times a team member will do this task every day. I took what I guess was the average developers salary and calculated the cost for just waiting to the project to start up, based on one year.

    I went home, did the same thing with my computer at home (a much faster one). Took the price of a new computer times number of developers. I went to my boss and showed him how much new computers will cost, and how much we will save with the new computers compared to the current computers during one year. When he notice how much money the current computer cost in waste compared to investment in new computer. He bought us new computers.

    So sometimes numbers must be included.

    Summary

    By using Virtual Machines, we can setup a development and testing environment for team members fast, and we can make sure everyone uses the correct version of tools for the specific project. By using tools like Vagrant we can make the setup much faster and in an easy way,

    If you want to know when I post a new blog post, please follow me on twitter: @fredrikn

    Read more...

  • Introducing Polymelia Deploy

    During the last month I have created different deployment tools, as a proof of concepts. The tools have change from push deploy into pull deploy, from an own XML workflow and environment definition into using Microsoft Workflow. Finally I decided to introduce to you the Polymelia Deploy tool. The goal of the tool is to make it open source. The code I have is still in a proof of concept level, and need some more work until it will be available.

    Read more...

  • Deployment tool

    During the last twelve months I have spent a lot of time on Continuous Delivery and a deep dive into Team Foundation Server 2012. A commit stage is set, we use TFS build to build, and NuGet as an Artifact Repository. Now my goal is to bring the deployment to the team, let them also be part of the deployment process, maintain and create deployment scripts etc. I have looked around among some deployment tools like Octopus and InRelease. InRelease looks promising. Instead of using one of the great tools, I decided to create my own. Why you may ask? The reason is that I want to be able to do any modifications and bring the code to the team, and to be honest, I needed something to do on my spare time ;)

    Read more...

  • Feature enabling when using Branch by Abstraction

    Something that I haven't liked for a long time is the use of branches in a projects. I don't like the waste when it comes to merge between branches, "Merge hell". In the beginning of the project I'm in now, it took hours to do merges before a release, both handling conflicts, but also selects different changeset that should include and not include in the merge before a release. The hard part was when a branch per feature was present. Later on this was changed into one single dev branch, but still sometimes two branches may exists. The team was used to work like this from previous projects. To handle this merge hell one dedicated person handle all the merge so the team can work undisturbed (not true, they were involved when conflicts occurs). Even with a dedicated person, the problem still exists, it was just moved. The time it takes to do the merge, was there, the problem with non-disciplines people that did not following the guidelines when it comes to fixing bugs in different branches, or forgot to merge into the main branch, created problems. Wouldn't it be nice to get rid of all the merge problems, let everyone just work in the same branch? There is a way, it may not work for every projects though (sometimes a bigger change may be useful to have in a single branch, but I think it should first be avoided). The solution to the problem is what Paul Hammant calls "Branch by Abstraction".

    Read more...