Cleaning invalid characters from SharePoint

11 Comments

• __CookieMonster fails :)
  
  use: name.TrimStart('_');
  
  I don't understand why the removing of periods is different then the other character replacements that are above it.
  
  Surely, a regex to remove those 'unwanted' chars would be far better, even if you can't do the entire thing.
  
  Regex.Replace(name, Regex.Escape(@"#%&*:?\/{|}~+-,()."), string.Empty);
  
  With a little more elbow grease, the whole thing apart from the space replacement and the length trimming could be put into a regex.
  

• This should work for you
  
  ([$#%&*:?\/{}|~+-,().-\])
  

• @Marc: Thanks, I knew there was an easier way.
  

• @hexy: Yes, it could be and your pattern does replace all those calls to .Replace. However if you remove the while(cleanName.Contains(".")) which strips out all periods it fails. There probably needs a "match all '.'" regex thingy here. As for the underline, I see the double underscore fails, but the rule is to only check it at the beginning of the name and not throughout. Again, a regex here could probably work. It's just beyond my meager skills.
  

• I'm also a fan of System.Web.HttpUtility.UrlEncode() and/or HtmlEncode() - use at the end to catch any odd characters you've missed (e.g. ñ)?
  
  Anyway, neat.
  

• There are couple of problems with your code.
  
  1. You should do trim before length check, not after. This way, you can fit more characters. Try following input for your test.
  
  string name = " testInput".PadRight(60, 'C');
  string expected = "testInput".PadRight(50, 'C');
  Assert.AreEqual(expected, Helper.RemoveInvalidCharacters(name));
  
  By trimming first, you can fit in more non-space characters in your name.
  
  2. The length limit should be enforced at the end. Because you are converting space to %20 after checking for length, your function may return a string with length more than 50.
  Try following input for your test.
  
  string invalidName = "Cookie Monster".PadRight(50, 'C');
  Assert.AreEqual(50, Helper.RemoveInvalidCharacters(invalidName).Length);
  
  
  3. Why the special treatment to remove the period (.)?
  
  4. The following code removes a valid character from the index 0.
  if(cleanName.Length > 50)
  cleanName = cleanName.Substring(1, 50);
  
  You should do
  cleanName = cleanName.Substring(0, 50);
  
  I have re-written the code to use Regex and covered the edge cases.
  Hope that helps,
  Jd
  
  
  using System;
  using System.Text.RegularExpressions;
  
  namespace Misc
  {
  public class Helper
  {
  public static string RemoveInvalidCharacters(string name)
  {
  if(name == null)
  throw new ArgumentNullException("name");
  
  //We should trim the input before we do length check.
  //This way, we could be able to fit in more characters in
  //our output if there are spaces in the beginning.
  name = name.Trim();
  
  string[] invalidCharacters =
  new string[]
  {
  "#", "%", "&", "*", ":", "", "?", "\\", "/", "{", "}", "~", "+", "-", ",", "(", ")", "|",
  "."
  };
  
  Regex cleanUpRegex = GetCharacterRemovalRegex(invalidCharacters);
  
  string cleanName = cleanUpRegex.Replace(name, string.Empty);
  
  cleanName = cleanName.Replace(" ", "%20");
  
  if (cleanName.StartsWith("_"))
  cleanName = cleanName.Substring(1);
  
  if (cleanName.Length > 50)
  cleanName = cleanName.Substring(0, 50);
  
  return cleanName;
  }
  
  private static Regex GetCharacterRemovalRegex(string[] invalidCharacters)
  {
  if(invalidCharacters == null)
  throw new ArgumentNullException("invalidCharacters");
  
  if(invalidCharacters.Length == 0)
  throw new ArgumentException("invalidCharacters can not be empty.", "invalidCharacters");
  
  string[] escapedCharacters = new string[invalidCharacters.Length];
  
  int index = 0;
  foreach (string input in invalidCharacters)
  {
  escapedCharacters[index] = Regex.Escape(input);
  index++;
  }
  
  return new Regex(string.Join("|", escapedCharacters));
  }
  }
  }
  
  
  
  using MbUnit.Framework;
  using Misc;
  
  namespace TestMisc
  {
  [TestFixture]
  public class HelperTest
  {
  [Test]
  [RowTest]
  [Row("_Monster", "Monster")]
  [Row("...period...removal", "periodremoval")]
  [Row(" test ", "test")]
  [Row("Cookie Monster", "Cookie%20Monster")]
  [Row("...", "")]
  public void CharacterRemovalTest(string name, string expected)
  {
  Assert.AreEqual(expected, Helper.RemoveInvalidCharacters(name));
  }
  
  [Test]
  public void SpecialCharacterRemovalTest()
  {
  string invalidName = @"#%&*:?\/{|}~+-,().";
  Assert.AreEqual(string.Empty, Helper.RemoveInvalidCharacters(invalidName));
  }
  
  [Test]
  public void NameLengthTest1()
  {
  string invalidName = "CookieMonster".PadRight(51, 'C');
  Assert.AreEqual(50, Helper.RemoveInvalidCharacters(invalidName).Length);
  }
  
  [Test]
  public void NameLengthTest2()
  {
  string invalidName = "Cookie Monster".PadRight(50, 'C');
  Assert.AreEqual(50, Helper.RemoveInvalidCharacters(invalidName).Length);
  }
  
  [Test]
  public void NameLengthTest3()
  {
  string name = " testInput".PadRight(60, 'C');
  string expected = "testInput".PadRight(50, 'C');
  Assert.AreEqual(expected, Helper.RemoveInvalidCharacters(name));
  }
  }
  }
  
  
  
  
  

• @Peter: The url encoding isn't the issue. For example create a document library in SharePoint using the API, Web Service, or from the UI called "my-library" and the created library will have a display name of "my-library" but a url addressable name of "mylibrary".
  
  @JD: Thanks for the RegEx and edge cases. The 50 character limit before the "%20" replacement is done because of a SharePoint quirk, but it makes more sense to trim it afterwards in any case.
  

• SPEncode.IsLegalCharInUrl(c)
  

• @Red: Thanks and that could be useful, however it ties me to the SharePoint library and frankly, SPEncode and all of it's static methods are highly untestable IMHO. I'll look into this though.
  

• The problem with trimming after ' '->%20 replacement is that you can end up with an invalid string.
  
  Suppose the pre-replacement string was 48 Xs, a space, and an X. This would after replacement convert to a string where the 48th-52nd characters are "X%20X". Truncating this would end in "X%2", causing problems.
  
  Darn those boundary conditions.
  

• Just wanted to drop in to say Great Post! and surprisingly good work by the other commentators : )