A good version control system is about more than giving you access to old versions of your files. If that was all you needed, then periodic zip files could meet your needs perfectly well. Ideally, you should be able to use Vault to answer the question "How did my source file get into this state?" Vault should help tell the story of your source code. Like I said
earlier, in this series of articles, I hope to provide Vault users with tips and utilities to help leverage the history they have in Vault to answer complex questions and to analyze that history in an automated way that answers the questions before you can ask them.
Mmmm... Donuts
Here at Sourcegear, there are strict punishments for any developer who checks in a change which causes Vault compilation errors. The law states that if you break the build, then you must bring in donuts the next morning. This means that when the automated build system sends out an email that says "Vault could not be built", the sprinkle-deprived masses have but one question on their minds... Who's bringing donuts?
Before Vault 2.0, the process went something like this:
1. Read the output log in the email to determine which file is broken.
2. Load up the Vault client and look through that file's history to see who's been changing it recently.
3. Try to catch the guilty party before they can sneak out.
With the introduction of Blame in Vault 2.0, step 2 is now a lot faster, giving the hope that you may even be able to confront the guilty party in their office instead of the parking lot. That's great, but it would be better if the email from the automated build system said who broke the build. Then, it's just a matter of whose email client has the lowest refresh time.
Blame
The most familiar mode of blame one is the GUI mode, where each line in your source file is displayed with notations for the user and version in which that line last changed. Ihe mode that I'm showing here is also available in the Vault command line client, but I'm reimplementing it here so that I can discuss the steps that are being taken.
We'll start with the template that we created earlier, and add three steps that are necessary to get the user who broke the build.
1. Parse the broken file and line number from the Visual Studio output.
2. Convert the disk file path to a repository path.
3. Login to the Vault server and request the blame for that file, looping until we find the broken line.
Parsing the Output from a failed build.
The following assumes that your automated build calls Visual Studio like this:
"c:\Program Files\Microsoft Visual Studio .NET\Common7\IDE\devenv.exe" /rebuild release /out build.log Solution.sln
When a build fails, a line like this will appear in the build.log output file.
c:\blah\outputparser\class1.cs(16,4): error CS0246: The type or namespace name 'intt' could not be found ...
The two items that we care about are the first things on the line, the filename and the line number. Here's an excerpt from the
complete solution that parses the build log file to find the first line that failed.
string currentLine = null, brokenDiskFile = null;
int lineNumber = 0;
System.IO.StreamReader sr = new System.IO.StreamReader(logfile);
//This regex looks for lines like C:\Path\To\Class1.cs(16,4): error
System.Text.RegularExpressions.Regex reg = new System.Text.RegularExpressions.Regex(@".*\([0-9]+,[0-9]+\)\: error ");
while ((currentLine = sr.ReadLine()) != null)
{
if (reg.Match(currentLine).Length > 0)
{
int ParenPos = currentLine.IndexOf("(");
//Everything before the paren is the file name
brokenDiskFile = currentLine.Substring(0, ParenPos);
//The part between the paren and the comma is the line number.
int CommaPos = currentLine.IndexOf(",", ParenPos);
lineNumber = int.Parse(currentLine.Substring(ParenPos + 1, CommaPos - ParenPos - 1));
break;
}
}
sr.Close();
Requesting the blame for the file from VaultHere's another small excerpt that converts the disk path to a Vault repository path. I'm taking a shortcut and assuming that the disk folder that is the root of the source tree and the repository folder that corresponds to that disk folder are hard coded into the app, or passed in as command line arguments.
brokenDiskFile = brokenDiskFile.ToLower();
string brokenFileInRepository = brokenDiskFile.Replace(workingFolderPath, repositoryPath);
brokenFileInRepository = brokenFileInRepository.Replace("\\", "/");
Now given the path to the broken file in the repository, we just need to ask the server for the blame output for the file and loop over the output until we find the line that is broken
VaultBlameNode[] blames = null;
//Generate the blame from version 20 versions ago to the latest version.
myClient.Connection.Blame(myClient.ActiveRepositoryID, brokenFileInRepository, -20 /*shortcut to say "20 versions before the latest one"*/, -1 /*shortcut to say "the latest version"*/, ref blames);
foreach (VaultClientNetLib.ClientService.VaultBlameNode bn in blames)
{
//Blame lines are zero based, so we have to bump the firstline for every blame
if ((bn.FirstLine + 1) <= lineNumber && ((bn.FirstLine + 1) + (bn.CountLines - 1)) >= lineNumber)
{
Console.Out.WriteLine("User " + bn.UserName + " last changed line " + lineNumber + " of " + brokenFileInRepository + " with the comment:\r\n" + bn.Comment);
break;
}
}
Now that we have an automated way of determining who broke the build, all we need is a donut store that has a web service for ordering donuts. :).
Darren Sargent wrote me today to share his enhancement to my
recursive find sample. You can read his announcement
here and get the code from
vaultpub.sourcegear.com. It has search support with regular expressions and version number/modified date in the output.
We've had lots of requests from customers who want to be able to hook into the
Vault server to catch events. Since
Sourcegear can't meet everyone's need's exactly, I wanted to provide a small example plugin, which could be modified to do more useful work, such as notifying an email address, or starting an automated build. Two things to note:
- Sourcegear still plans on writing plugins to ship with Vault and those will be fully supported. We're not going to ask everyone to write their own email plugin, for example. This is just a kick start for those Vault users who know what they want and want it now.
- As we write our plugins, we may change the plugin API and architecture. Don't count on this API staying constant. When or if we do change the API, expect me to update this example with the new way of doing things.
The template can be found at
here.
Setting this up is a bit complicated, as it is a Web Service.
- Unzip the template to somewhere on disk.
- In IIS Manager, create a virtual directory named PluginServiceTemplate that points to the directory that holds the PluginServiceTemplate.asmx file.
- Load the solution file. Build the solution. If there are any problems building, verify that the references point to the correct locations on disk. By default, the VaultPluginLib reference points to c:\inetpub\wwwroot\VaultService\bin, and all others point to c:\program files\Sourcegear\Vault Client.
- Start debugging. When the Web Service page comes up, click on UpdateVaultConnectionInformation. Fill in the hostname, username, and password that you wish to use to connect to your Vault server. Hit the Invoke button will set the connection information.
- Go back to the Web Service page. Click on UpdateRepositoryInformation. Fill out the repositoryName and pathToWatch fields. Hit Invoke again.
- Go back to the Web Service page. Click on RegisterWithServer. Type in the password for the user "admin" on the Vault Server that you specified above. Hit Invoke. If no longer wish to have the Vault server call the plugin, you must call UnRegisterWithServer.
Now that you're set up, you should be able to put a breakpoint in the OnEndTx method in PluginServiceTemplate.asmx.cs, and when a change is made to the repository that you're watching while you're debugging, the breakpoint should be hit. There are four methods that a plugin can define.
- OnLogin
- OnLogout
- OnBeginTx
- OnEndTx
The example only overrides OnEndTx, since that's the most interesting of the four. When the plugin is informed that a transaction has finished, OnEndTx walks through the following steps.
- Log in to the Vault server and refresh the tree cache.
- Use FindFolderRecursive (Remember that from last time?) to find the path to watch.
- If the version number on that object is different that the last version number we saw, run a history query to find out what changed in the last transaction.
- Loop through all of the VaultHistoryItems. Right now that loop is empty.
Note that if all you want is a plugin that kicks off a new build, you can probably skip the history query.
One of the most common things that people ask for in Vault is a way to find every file in their repository that matches some pattern and do something. Some of the requests are:
- Search for a file based on name to see if it has already been added to the repository.
- Search for any files named AssemblyInfo.cs, check them out, edit them and check them in.
I'll be showing you how to do the second, since lots of people have questions on the best way to edit a file and check it in. Once again, we'll be modifying the Vault client
template that we created a long time ago. You can find the complete project
here.
Searching through the repository The first thing that we need to do is find the topmost node that we want to search. In most cases, you don't want the top of your search to be $/, because then you will be finding any AssemblyInfo.cs files in branches, etc. That's where the handy "FindFolderRecursive" function comes in. Also note that there is a corresponding FindFileRecursive function.
VaultClientFolder searchFolder = myClient.Repository.Root.FindFolderRecursive(searchRoot);
FindFolderRecursive looks through the repository to find a folder with the path that you give it. Once you get a VaultClientFolder, there are a few interesting things you can do to it, such as pass it as an argument to ClientInstance.Get, ClientInstance.CheckOut, etc. For this example, we're going to walk down the folder looking for files with a certain name. I do this with a function called RecursivelySearchForFileName.
public static void RecursivelySearchForFileName(VaultClientFolder folder, string fileName, ref ArrayList returnArray)
{
if (folder.Files != null)
{
foreach (VaultClientFile subfile in folder.Files)
{
if (subfile.Name == fileName)
{
returnArray.Add(subfile);
break;
}
}
}
if (folder.Folders != null)
{
foreach (VaultClientFolder subfolder in folder.Folders)
{
RecursivelySearchForFileName(subfolder, fileName, ref returnArray);
}
}
}
Get the files, Modify the files, Check in the files
Once we have the array of VaultClientFile, we need to check them out and get them
myClient.CheckOut((VaultClientFile[])returnArray.ToArray(typeof(VaultClientFile)), 2 /*request exclusive checkout. Use 1 for non-exclusive*/, "Checkout Comment");
myClient.Get((VaultClientFile[])returnArray.ToArray(typeof(VaultClientFile)), true, MakeWritableType.MakeAllFilesWritable, SetFileTimeType.Current, MergeType.OverwriteWorkingCopy, null);
myClient.Refresh();
I'm going to skip the actual editing of the files here, but the key from the Vault side of things is using the following line to get the path on disk that corresponds to the VaultClientFile.
string diskFile = myClient.TreeCache.PhysicalPath(foundFile);
Now we can use myClient.Commit() to commit the changes. Now you should be able to write your own API programs to edit files.
There are lots of Vault users out there. Most are content to just use it to check in changes to their files and occasionally get an old version. Then there are others who want to get the most out of Vault. In this series, I'm going to try to help out power users by showing them how to write simple programs that connect to Vault and extract some interesting bits of information. Here's a list of some of the samples I want to guide you through
1. A small program to parse the log file from a failed build, which connects to Vault to determine the last user who changed the line that caused the build failure.
2. A small program to generate a changelog to track the differences between builds.
3. A Vault client built in VB to display the Vault tree and track who's making changes.
4. A simple plugin that will start a build after every change to the tree.
I have more ideas, and would love to get feedback from the power users out there who have ideas.
The template
This all starts with a template project that can connect to Vault. In the rest of this post, I'll walk you through a simple template. I'll just be addressing the most interesting bits here, but you can download the full solution here. This is assuming that you've already downloaded and installed the Vault ClientAPI.
ClientInstance myClient = new ClientInstance();
This line creates a new ClientInstance, which is the highest level object you can use to talk to the Vault server. The client instance holds the connection, tree cache, working folder data, pretty much everything. If there's something that you want to do, you probably want to go through client instance to do it.
myClient.Init(VaultClientNetLib.VaultConnection.AccessLevelType.Client);
myClient.Login(hostname, username, password);
These two lines log you on to the Vault server. To understand what these two functions do, it's best to think of them in terms of how the Vault GUI client works. Init is called before the login dialog comes up, to set up the web service object. Login is called after you hit ok on the login dialog. Note that Init is where you declare if you are an admin client or a regular client. Certain methods are only available to admin clients.
VaultRepositoryInfo[] reps = null;
//List all the repositories on the server.
client.ListRepositories(ref reps);
//Search for the one that we want.
foreach (VaultRepositoryInfo r in reps)
{
if (String.Compare(r.RepName,repositoryName, true) == 0)
{
//This will load up the client side cache files and refresh the repository structure.
//See http://support.sourcegear.com/viewtopic.php?t=6 for more on client side cache files.
client.SetActiveRepositoryID(r.RepID, client.Connection.Username, r.UniqueRepID, true, true);
break;
}
}
This loop is necessary due to a deficiency the Vault API. We never got around to implementing a nice function to connect to a repository based on the repository name. This means that (until we fix it) every program that uses the Client API will need to jump through this particular hoop. To explain what is going on, the client is requesting a list of all of the repositories on the Vault server, then looping through them to find the one that we want. When we find it, we use the clientInstance's SetActiveRepositoryID to load it. SetActiveRepositoryID does a lot. It loads the client side cache files and refreshes all the repository information from the server.
Once you've jumped through the four necessary hoops, you're finally connected and ready to do something. To recap, the hoops are:
1. Construct ClientInstance
2. Init
3. Login
4. SetActiveRepositoryID
The next post will actually do something with your Vault connection.
More Posts