Announcing CSverter 1.0 -> .Text 0.95 to CommunityServer Conversion Wizard

First off, I wanted to thank Jayson Knight for his help with this project. Without the source for his DotTextCS project, this would not have been possible. Also thanks to Divelements, whose WizardControl WinForms component made it easier for me to build this app.

I'm pleased to announce that I've completed work on a wizard to automate the process of moving from .Text to CommunityServer. It has been tested on several different size installations, including a test run for LonghornBlogs.com, which will be live shortly.

Features:

  • Installs all script files and schemas for the CommunityServer destination database
  • Automatically creates new authors with the proper permissions
  • Automatically creates new blogs & blog directories for those authors
  • Automatically migrates entry statistics
  • Automatically migrates blog settings (news, secondarycss, etc)
  • Automatically creates custom blog groups based on your .Text settings.
  • Allows you to select which blogs to migrate
  • Allows you to save the DTS packages to the server for troubleshooting
  • Quick Install option assumes you're starting with a blank database and migrating all blogs, reducing the number of steps to beginning the process
  • Multithreaded to give you realtime feedback on the status of the migration
  • Heavy logging for help troubleshooting failed migrations.

As I've said, I've tested it on several sites, and it works well in all cases on my end. If you have any problems, please reply to this thread and I'll fix it as soon as I can.

Some things to remember:

  • Right now, the source and destination databases need to be on the same server. If enough people need to move between separate servers, I can add it later.
  • You need to run the wizard on a machine where the CS web application is installed. This is because the temporary web service we use to create users and blogs needs to run in the HttpContext of the application (more on my feelings about this later).
  • You will need to set up CS and change the web.config file to point to the destination database you will use to complete the migration. Again, this is because the web service will use those settings to create blogs, users, and blog groups.
  • Depending on the number of users, blogs, posts, referrers, etc, this process can take SEVERAL HOURS. DTS is rather fast, but it still took over seven hours to migrate LonghornBlogs.com
  • You may need to use the CS admin UI to change the blog theme after the migration. I'm looking into why this isn't working, but it's not a showstopper, so I'm releasing it anyway.

That being said, enjoy! Don't forget to back up your .Text database first. It doesn't have as much UI validation as I would like, so you can expect some minor improvements.

Download Now!

34 Comments

  • +1 vote to migrating between different source and destinations database servers.

  • Hmmm. This will work out for one set of blogs. However, the second set is a server-to-server install as well. Anyway to tweak it so the out puts are INSERT/UPDATE scripts that we can then run on the target server? Thanks!

  • Keith,



    I have no idea what you're talking about. Can you elaborate for me please? Thanks.

  • Any chance that we can get the source code to this tool, Robert? Or add in support directly, for running the process from a different machine? I need to be able to run the process from my home computer, and have it upgrade the database that is being hosted by WebHost4Life, for me. I'd also like to see the code account for me already having the same username in my new blog, and having existing posts in my new blog. Perhaps the tool could allow me to specify what username to transfer from / to? i.e. move from user "myname" to user "old-myname" or something like that?

  • I tried running it on a Windows 2003, MSSQL 2000 install of .Text 0.95... there were plenty of Object reference not set to an instance of an object and it just continued on happily until it went to execute a non-existant DTS package at that point it hangs.



    The only thing I can think is that I do not have SQL and Windows authentication turned on, just Windows authentication. I see the database, the users, I think I have all the scripts setup correctly.



    I'm at work now, going home, if you want, email me and I'll send you the log file.



    Other than that, it looks great! It was very easy to walk through, if only I could get the last step to work. :)

  • I'm posting a new build soon with better error handling and some UI fixes.

  • Thanks for all the effort you've put into this it looks like I'm soon going to be able to migrate my blogs across to CS 1.0.

    I had the same problem as Adam but Jayson's comment allowed me to get past that. Users and blog settings are being migrated but not the posts - It looks like the DTS packages are never created so therfore can't run. I had a similar problem with Jaysons code. I even switched off the "quick" option but still can't find them after having asked the app to save them.

    Do I need anything specific installed to get the DTS packages created?

  • Yes, you have to have SQL Client tools installed. I should probably add that to the list of requirements.

  • I have the client tools installed; however, I'm still not seeing any DTS packages getting created. Though it appeared to execute 6 of the DTS packages then got hung up on one of them. And I'm not sure why it says it executed each package twice ...



    6:04:12 PM: Step 6 Complete: Blogs created.

    6:04:12 PM: Executing DTS for 'adam'... (1 of 15)

    6:04:13 PM: Executing DTS for 'adam'... (1 of 15)

    6:04:13 PM: Executing DTS for 'eric'... (2 of 15)

    6:04:14 PM: Executing DTS for 'eric'... (2 of 15)

    6:04:14 PM: Executing DTS for 'am'... (3 of 15)

    6:04:14 PM: Executing DTS for 'am'... (3 of 15)

    6:04:14 PM: Executing DTS for 'nik'... (4 of 15)

    6:04:14 PM: Executing DTS for 'nik'... (4 of 15)

    6:04:14 PM: Executing DTS for 'humayun'... (5 of 15)

    6:04:14 PM: Executing DTS for 'humayun'... (5 of 15)

    6:04:14 PM: Executing DTS for 'paul'... (6 of 15)

    6:04:14 PM: Executing DTS for 'paul'... (6 of 15)

    6:04:14 PM: Executing DTS for 'indaloop'... (7 of 15)



    One other thing, I was manually going to setup all the bloggers, and noticed that the user "am" is too short (looks like CS requires 3 character minimum), how's your tool going to handle this, or will it just bypass the application level checks?

  • That was an error on the logging, I forgot to change the "success" string.



    RE: application checks, no. We're going through the CS API, but in a round-about way. The checks happen through the user creation server controls, not through the object model.

  • Same symptoms here Adam - I made sure I had all parts of SQL installed (including DTS tools like dtsrun, dtswiz, etc.) but still no DTS packages or Blog posts.



    The only other info I can add is although the Wizard completes successfully the "Finish" button on the last dialog box doesn't work - I have to press cancel. There also seems to be no step 7 in the log file?

  • Surely it's useful for many developers.

    Thanx.

  • I'm still having problems. The only other thing that could be different is that the SQL Server is not the default instance but a named instance.



    I'm going to try pulling the databases down to my local computer and migrating them on a default instance.

  • And another thing, I'm running the MSSQL service as a least privileged account. And only Windows authentication is allowed. I'm not sure if any of these are contributing factors or not, just hoping that one of them will set off some ideas.

  • Getting farther locally, however I get this error message set for the occassional authors.



    9:53:09 AM: Step 1failed. The number of failing rows exceeds the maximum specified. (Microsoft OLE DB Provider for SQL Server (80004005): Unspecified error)

    9:53:09 AM: Step 2failed.

    9:53:09 AM: Step 3failed.

    9:53:09 AM: Step 4failed.

    9:53:09 AM: Step 5failed.

    9:53:09 AM: Step 6failed.

    9:53:09 AM: Step 8failed.

    9:53:09 AM: Step 9failed.



    And on one blog, it looks like it is doing something, but never creates the DTS package and just hangs, no errors or anything.

  • Adam,

    My SQl server is also on a named instance and the CS is not of the default website but on a separate site that uses a host header.

  • Dam,

    One other thing. Did you try Jayson's original DTS engine? I had similar problems with that too.

  • Took a few times to get the settings just right but when i finally did, everything worked fine. The only thing i had to do once migration was complete was to maually assign a group id to the blogs in the database. Thanks for a great tool guys!

  • Sweet! WE've been waiting for a migration tool! thanks a lot! donny

  • Oy, now it doesn't do anything. :( I tried it with a community (without any users), and with a blank database.



    7:47:31 AM: Step 0: Cleaning up...

    7:47:37 AM: Temporary web service removed successfully.

    ---------------------------------------------------------------------------------------

    2/27/2005 7:47:37 AM

    CSverter Conversion Process Completed.

    ---------------------------------------------------------------------------------------

    7:50:06 AM: Step 0: Cleaning up...

    7:50:11 AM: Temporary web service removed successfully.

    ---------------------------------------------------------------------------------------

    2/27/2005 7:50:11 AM

    CSverter Conversion Process Completed.

    ---------------------------------------------------------------------------------------

  • It's the same for me... The wizard does nothing. I have the same log as exposed above this comment... I've tried on three installations I have but it performs the same operations in all of them.



    What a shame :(

  • Same for me - It will be great when this is working though as I'm sure it will. We all apreciate the effort you're putting in Robert!

  • First off, you need to drop the test CS database and re-create it for a "Quick Install" to work. Second, you need to clear out all CSverter-related DTS package from the server. The default settings save all packages to the server before they are executed.



    Second, how long did you wait? It takes several minutes to execute a migration, and you may or may not see CPU usage change. Some of the users on LonghornBlogs.com took over 20 minutes to migrate. You just need to give it time, it will more than likely tell you when it fails.



    And make sure you've downloaded the latest build from the link. I just posted one about 30 minutes ago.

  • Chris



    From my testing with my similarly small blog, the actual DTS package itself should take about a minute or so to run after it's created.



    Can you check for data in the aspnet_Users, cs_Groups, and cs_Sections tables (I'm going off of memory right now, not at my machine...but I'm pretty sure those are the tables)? Also, check your .Text blog_Config table and see if anything looks out of the ordinary (null values); my first reaction is that there could be something wrong there.

  • I'm able to convert again, but I still cannot get a DTS package generated for one blog. I pulled a dump of the application at the point where it was hung. With further analysis, the only thing odd I could find (with my limited knowledge of windbg and .net) is a null reference exception in one of the threads.



    I can send you the entire dump file (91mb uncompressed) if you want it.



    0:000> !Threads

    ThreadCount: 7

    UnstartedThread: 0

    BackgroundThread: 3

    PendingThread: 0

    DeadThread: 3

    PreEmptive GC Alloc Lock

    ID ThreadOBJ State GC Context Domain Count APT Exception

    0 1250 0015edc0 6020 Enabled 01382480:01383ff4 00146550 0 STA

    2 1680 00167508 b220 Enabled 00000000:00000000 00146550 0 MTA (Finalizer)

    XXX 0 001cb290 1800820 Enabled 00000000:00000000 00146550 0 MTA (Threadpool Worker)

    XXX 0 001a39a0 1800820 Enabled 00000000:00000000 00146550 0 Ukn (Threadpool Worker)

    XXX 0 001d0028 1820 Enabled 00000000:00000000 00146550 0 Ukn System.NullReferenceException

    8 bc 049a1f80 800220 Enabled 0134940c:01349ff4 00146550 0 MTA (Threadpool Completion Port)

    10 d50 049c8530 1800220 Enabled 01370128:01372024 00146550 0 MTA (Threadpool Worker)

  • I'm getting the same error as Chris. The migration tool just hangs. Last line in the log file was



    10:57:43 AM: Executing DTS for 'molson'... (1 of 381)



    From looking at the DB I can see that the cs_Urls table was updated with the links but I didn't see any user info in the aspnet_users table. There's one group named Administrators in the cs_Groups table and it has a GroupID = 1. This is the same as the GroupID value for all entries in the blog_config table of my .Text 0.95 database.



    I deleted all the records in the cs_Urls table and then ran the DTS package again. It ran successfully. So I think the DTS package was OK, what happens after the very first DTS package runs?

  • If there aren't any users in the aspnet_users table, and there aren't any blogs in the "cs_Sections" table, or the cs_weblog_weblogs" table, then the users and blogs are not being created. I'll take a look at the code again this evening and see if I can track down why it's not creating the users when it says that it does.

  • I am getting the same error. It stops at 97.3% and then nothing else...

  • I have this same issue, whereby a hang occurs at step 7 of 8 for the migration process, at 97.08% complete. At this point one DTS package is complete Url Conversions. Of the tables you mention above no users are addeds, neither blogs.



    Is it significant that my BlogID is 5 in my old .Text 95 blog?



    My log below:



    ---------------------------------------------------------------------------------------

    28/02/2005 23:30:20

    CSverter Conversion Process Starting.

    ---------------------------------------------------------------------------------------

    23:30:20: CSverter configuration file updated.

    23:30:20: Step 1 Starting: Install ASP.NET MemberRole schema.

    23:30:20: Executing 'InstallCommon.sql'...

    23:30:20: 'InstallCommon.sql' Completed.

    23:30:20: Executing 'InstallMembership.sql'...

    23:30:20: 'InstallMembership.sql' Completed.

    23:30:20: Executing 'InstallProfile.sql'...

    23:30:20: 'InstallProfile.sql' Completed.

    23:30:20: Executing 'InstallRoles.sql'...

    23:30:21: 'InstallRoles.sql' Completed.

    23:30:21: Step 1 Complete: ASP.NET MemberRole schema installed.

    23:30:21: Step 2 Starting: Install CommunityServer database schema.

    23:30:21: Executing 'cs_Schema.sql'...

    23:30:24: 'cs_Schema.sql' Completed.

    23:30:24: Executing 'cs_Procedures.sql'...

    23:30:29: 'cs_Procedures.sql' Completed.

    23:30:29: Executing 'cs_Default.sql'...

    23:30:29: 'cs_Default.sql' Completed.

    23:30:29: Step 2 Complete: CommunityServer database installed.

    23:30:29: Step 3 Starting: Create New Community

    23:30:30: Step 3 Complete: New community site created.

    23:30:30: Cast from type 'DBNull' to type 'Integer' is not valid.

    23:30:30: BlogGroups created successfully.

    23:30:30: Creating User 'jansonr'... (1 of 1)

    23:30:35: User: jansonr Created.

    23:30:35: Step 5 Complete: Users created.

    23:30:35: Creating Blog 'myworkblog'... (1 of 1)

    23:30:35: Cast from type 'DBNull' to type 'String' is not valid.

    23:30:35: Blog: myworkblog created.

    23:30:35: Step 6 Complete: Blogs created.

    23:30:35: DTS Package 'CSverter package for blogDB: Url Conversions' already exists with a different ID in this category.

    23:30:35: Step 7 starting: 8

    23:30:35: Executing DTS for 'myworkblog'... (1 of 1)

  • Ignore the line 23:30:35: DTS Package 'CSverter package for blogDB: Url Conversions' already exists with a different ID in this category.



    I had forgotten to clean out the DTS package between attmepts, but even with this cleaned out I still experience the hang.



    Thanks for your help on this, and I'm sure this is more to do with some 'quirky' .Text implementations (like my BlogID=5 perhaps!)

  • AH HA!!!!! See, that makes an ENORMOUS difference. I'll have a fix posted shortly.

  • Robert, I had a null BlogGroup in one row (out of 381) in the blog_config table. I fixed that (set it to a value of 1 like the rest of them) and then ran CSVerter again. It didn't make any difference. CSVerter still hangs after exectuting the first DTS package which inserts URLs into the cs_Urls table in the CS database. CSVerter shows that it is creating all the users and their blogs but then when I go into the CS database I don't see more than 2 rows in the aspnet_Users table and the cs_Sections table. CSVerter just hangs.





    Here are the last few lines of my log file.



    9:58:24 PM: Step 6 Complete: Blogs created.

    9:58:24 PM: Invalid authorization specification

    9:58:24 PM: Step 7 starting: 8

    9:58:24 PM: Executing DTS for 'molson'... (1 of 381)



    What is "Invalid authorization specification"?

  • There will be a new build soon that, if it doesn't solve the problem, will give a better idea of what has happened.

  • Hi Robert,

    Good to there thats where the problems appears to lie. Perhaps Mark has a slightly different issue. When I modifed the BlogGroup to 1 the tool completed successfully. So I guess there just needs to be a mod to make single .Text blog compliant?



    Log attached:



    ---------------------------------------------------------------------------------------

    01/03/2005 08:16:32

    CSverter Conversion Process Starting.

    ---------------------------------------------------------------------------------------

    08:16:33: CSverter configuration file updated.

    08:16:33: Step 1 Starting: Install ASP.NET MemberRole schema.

    08:16:33: Executing 'InstallCommon.sql'...

    08:16:34: 'InstallCommon.sql' Completed.

    08:16:34: Executing 'InstallMembership.sql'...

    08:16:35: 'InstallMembership.sql' Completed.

    08:16:35: Executing 'InstallProfile.sql'...

    08:16:35: 'InstallProfile.sql' Completed.

    08:16:35: Executing 'InstallRoles.sql'...

    08:16:36: 'InstallRoles.sql' Completed.

    08:16:36: Step 1 Complete: ASP.NET MemberRole schema installed.

    08:16:36: Step 2 Starting: Install CommunityServer database schema.

    08:16:36: Executing 'cs_Schema.sql'...

    08:16:40: 'cs_Schema.sql' Completed.

    08:16:40: Executing 'cs_Procedures.sql'...

    08:16:44: 'cs_Procedures.sql' Completed.

    08:16:44: Executing 'cs_Default.sql'...

    08:16:44: 'cs_Default.sql' Completed.

    08:16:44: Step 2 Complete: CommunityServer database installed.

    08:16:44: Step 3 Starting: Create New Community

    08:16:47: Step 3 Complete: New community site created.

    08:16:54: Step 4: BlogGroup for .Text GroupID 1 created.

    08:16:54: BlogGroups created successfully.

    08:16:54: Creating User 'jansonr'... (1 of 1)

    08:16:54: User: jansonr Created.

    08:16:54: Step 5 Complete: Users created.

    08:16:54: Creating Blog 'myworkblog'... (1 of 1)

    08:16:55: Blog: myworkblog created.

    08:16:55: Step 6 Complete: Blogs created.

    08:16:57: Step 7 starting: 8

    08:16:57: Executing DTS for 'myworkblog'... (1 of 1)

    08:16:59: Executing DTS for 'myworkblog'... (1 of 1)

    08:16:59: Updating Statistics...

    08:16:59: Statistics Updated

    08:16:59: Step 8: Cleaning up...

    ---------------------------------------------------------------------------------------

    01/03/2005 08:24:06

    CSverter Conversion Process Completed.

Comments have been disabled for this content.