Jeff and .NET

The .NET musings of Jeff Putz

Sponsors

News

My Sites

Archives

October 2008 - Posts

Rebates for shiny new MacBook Pro's on Amazon

I know a lot of .NET devs are converts to Mac hardware, but if you haven't made the leap yet, Amazon has a $100 rebate on them now, and there's no sales tax. That's at least $200 saved versus buying in the Apple Store.

My trusty 2006 model is 2.5 years old, and I'm enormously tempted to snag one. I noticed in the store these run a little cooler, and it is the weirdest thing how they don't flex, bend or squeak. At the end of the day, I'm a simple man, and I just like shiny things!

Conquering Deep Zoom (Part 2), serving tiles, custom MultiScaleTileSource

When we last left our heroes, we talked a bit about cutting up a single image for the purpose of serving tiles to a MultiScaleImage (Deep Zoom) in Silverlight. The motivation in this case is to offer some other means of serving the images, like from a database, instead of the mess of files that Deep Zoom Composer generates.

I kind of glossed over the whole data aspect of this, as you can see from the last entry that the TileCreated handler was a little vague...

private void maker_TileCreated(object sender, TileCreatedEventArgs e)
{
    DataContext context = new DataContext();
    var tile = e.TileContainer;
    Tile tileData = new Tile { ImageID = _imageID, Level = tile.Level, TilePositionX = tile.TilePositionX, TilePositionY = tile.TilePositionY, ImageData = new System.Data.Linq.Binary(tile.ImageData) };
    context.Tiles.InsertOnSubmit(tileData);
    context.SubmitChanges();
}

(I also left out the TileContainer class, but you can infer what it looks like from the Tile Maker and how I assign its properties to the data Tile.) So to fill in the blanks a little, I wanted to share what the DataContext looks like in this case.

As you can see, this isn't very complicated. The Image table is something I'm still thinking about and messing with for integration into another project, so you probably only need to worry about the width, height and ID. The Tile table is a bit more useful. Note the multiple fields specified for the primary key. I'm not crazy about this arrangement, but LINQ won't allow you to insert without a primary key, and the only way to represent a unique combination is to specify this combination or add a totally useless new identity field. In an ideal world, I just assume have a clustered index on ImageID.

Given this data arrangement, hopefully the event handler listed above for the last entry's TileMaker class makes sense.

Now that we have images in the database, we need to pull them back out using an HttpHandler. Specifically, we can use an asynchronous HttpHandler so as not to tie up the thread pool with a bazillion image requests. If the database is indexed right and you've got a nice balance between tile size and number of tiles per image, I'm not sure this is totally necessary, but I suppose it's a good practice. Hopefully I'm doing it right!

public class TileHandler : IHttpAsyncHandler
{
    private AsyncTaskDelegate _asyncDelegate;
    protected delegate void AsyncTaskDelegate(HttpContext context);

    public void ProcessRequest(HttpContext context)
    {
        int id;
        int level;
        int x;
        int y;
        var queryString = context.Request.QueryString;
        if (int.TryParse(queryString["id"], out id)
            && int.TryParse(queryString["level"], out level)
            && int.TryParse(queryString["x"], out x)
            && int.TryParse(queryString["y"], out y))
        {
            var data = new Data.DataContext();
            var image = data.Tiles.Single(t => t.ImageID == id && t.Level == level && t.TilePositionX == x && t.TilePositionY == y);
            if (image == null)
                context.Response.StatusCode = 404;
            else
            {
                context.Response.ContentType = "image/jpeg";
                var bytes = image.ImageData.ToArray();
                context.Response.OutputStream.Write(bytes, 0, bytes.Length);
            }
        }
        else
            context.Response.StatusCode = 404;
    }

    public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
    {
        _asyncDelegate = ProcessRequest;
        return _asyncDelegate.BeginInvoke(context, cb, extraData);
    }

    public void EndProcessRequest(IAsyncResult result)
    {
        _asyncDelegate.EndInvoke(result);
    }

    public bool IsReusable
    {
        get { return false; }
    }
}

The meat of the code is of course the ProcessRequest method. It checks to see that all of the required query string parameters are there, and if so, continues to try and load the image using the same LINQ context we showed above. If the image is found, it is streamed out to the output stream of the handler. Again, hopefully I'm getting the async part of this rigth.

The MultiscaleImage by default wants to find a Deep Zoom XML file and a bunch of actual files, so we need to tell it how we want to load the tiles. To do this, we'll add a class to our Silverlight project called ViewerSource, and we'll define it like this:

public class ViewerSource : MultiScaleTileSource
{
    /// <summary>
    /// Constructs a new tile source for a multi scale image by indicating its overall size and a string used to generate
    /// URL's that will return the tile images.
    /// </summary>
    /// <param name="imageWidth">Width of the source image.</param>
    /// <param name="imageHeight">Height of the source image.</param>
    /// <param name="tileSize">Tile size (square on one side) in pixels.</param>
    /// <param name="imageID">ID of the image to pass into the URL.</param>
    /// <param name="urlToFormat">URL to be formatted with ID, tile level, tilePositionX and
    /// tilePositionY, using {0}, {1}, {2} and {3}, respectively.
    /// </param>
    public ViewerSource(int imageWidth, int imageHeight, int tileSize, int imageID, string urlToFormat)
        : base(imageWidth, imageHeight, tileSize, tileSize, 0)
    {
        ImageID = imageID;
        UrlToFormat = urlToFormat;
    }

    public int ImageID { get; private set; }
    public string UrlToFormat { get; private set; }

    protected override void GetTileLayers(int tileLevel, int tilePositionX, int tilePositionY, System.Collections.Generic.IList<object> tileImageLayerSources)
    {
        string source = String.Format(UrlToFormat, ImageID, tileLevel, tilePositionX, tilePositionY);
        Uri uri = new Uri(source, UriKind.Absolute);
        tileImageLayerSources.Add(uri);
    }
}

MultiScaleTileSource is an abstract class with one method you have to define, GetTileLayers. MultiScaleImage calls this method to obtain the location of the tiles, with numbers you may find familiar from the mess of Deep Zoom Composer generated files: the tileLevel (or zoom level), and the tile's X and Y positions in the image. We're formatting the URL based on whatever was passed in from the constructor, in this case a URL that our HttpHandler is going to recognize. So the calling code in your XAML code-behind will look something like this, where Viewer is the name of the MultiScaleImage control:

Viewer.Source = new ViewerSource(width, height, tileSize, id, urlToFormat);

So you're probably wondering where the URL comes from, and that'll be in the third part, where I wire up this stuff to Javascript in the browser. For now, know that the URL I'm passing to the ViewerSource constructor is:

http://localhost:2974/TileHandler.ashx?id={0}&level={1}&x={2}&y={3}

Even without the wiring in the actual Silverlight control, you can probably see where this is going. In the last part of this series, I'll go over the hell of figuring out how to center the image and reset its zoom level.

Conquering Deep Zoom (Part 1), making tiles, and other Silverlight 2 thoughts

Wow, I can't believe three weeks passed since my last post. This is clearly the problem with a day job that doesn't align with your interests!

Since that last post, Silverlight 2 has been released into the world, and it is good. I've been spending quality time with Deep Zoom, which is a strange name since it's a bit of a concept and not really a technology per se. MultiScaleImage is really the control we're talking about. The Deep Zoom Composer app does a nice job of letting you put together these sweet collections of images at crazy resolutions and various zoom levels, and it's all very cool. Once you get passed the Hard Rock demo, you start to think a little harder about the technology and how you might use it.

I'm still not entirely convinced that a bazillion thumbnails are the best way to navigate image collections. What I do think is useful is taking those 20+ megapixel images we can take with quasi-affordable cameras and make them available at full resolution. Reducing them to 2.5% of their original resolution seems to me like throwing away many stories captured in the moment. So my goal was to conquer Deep Zoom and purpose it for single-image viewing in an image library.

The requirements break down like this: Create a code library that handles database storage of the tiled images (because hundreds of photos times hundreds of tiles makes using the file system a nightmare), cuts up images you feed it and offers an HttpHandler to serve up the images to the MultiScaleImage control. The other side of it is the Silverlight app itself, which is relatively straight forward and uses the Deep Zoom Composer templates, plus a class to correctly handle the calls to score image tiles.

There's a lot to cover, so I won't try to do it all here. I want to start with a class that I wrote to take an image and make tiles. It uses WPF classes to accomplish this instead of System.Drawing. I don't have a good reason for this other than I wanted to see what was available.

public class TileMaker
{
    public TileMaker(BitmapImage bitmapImage, int tileSize, int quality)
    {
        _original = bitmapImage;
        _tileSize = tileSize;
        _quality = quality;
        Tiles = new List<TileContainer>();
        EnableTileContainerCollection = false;
        MaxZoom = CalculateMaxZoom();
    }

    private readonly BitmapImage _original;
    private readonly int _tileSize;
    private readonly int _quality;

    public List<TileContainer> Tiles { get; private set; }
    public int MaxZoom { get; private set; }
    public bool EnableTileContainerCollection { get; set; }
    public event TileCreatedEventHandler TileCreated;

    public void CreateTiles()
    {
        for (int z = MaxZoom; z >= 0; z--)
        {
            BitmapImage image = _original;
            if (z != MaxZoom)
            {
                int? newWidth = null;
                int? newHeight = null;
                int divFactor = MaxZoom - z;
                int divisor = (int)Math.Pow(2, divFactor);
                if (_original.PixelHeight > _original.PixelWidth)
                    newHeight = (int)Math.Ceiling((double)_original.PixelHeight / divisor);
                else
                    newWidth = (int)Math.Ceiling((double)_original.PixelWidth / divisor);
                image = _original.ToBytes().Resize(newWidth, newHeight);
            }
            int tilesHigh = (int)Math.Ceiling((double)image.PixelHeight / _tileSize);
            int tilesWide = (int)Math.Ceiling((double)image.PixelWidth / _tileSize);
            for (int x = 0; x < tilesWide; x++)
            {
                for (int y = 0; y < tilesHigh; y++)
                {
                    var tile = new TileContainer
                                   {
                                       TilePositionX = x,
                                    TilePositionY = y,
                                    Level = z
                                   };
                    int xoffset = x * _tileSize;
                    int yoffset = y * _tileSize;
                    int xsize = _tileSize;
                    int ysize = _tileSize;
                    if (xoffset + _tileSize > image.PixelWidth)
                        xsize = image.PixelWidth - xoffset;
                    if (xsize < 1)
                        xsize = 1;
                    if (yoffset + _tileSize > image.PixelHeight)
                        ysize = image.PixelHeight - yoffset;
                    if (ysize < 1)
                        ysize = 1;
                    tile.ImageData = image.Crop(xoffset, yoffset, xsize, ysize).ToJpegBytes(_quality);
                    if (EnableTileContainerCollection)
                        Tiles.Add(tile);
                    OnTileCreated(new TileCreatedEventArgs(tile));
                }
            }
        }
    }

    private int CalculateMaxZoom()
    {
        return (int)Math.Ceiling(Math.Max(Math.Log(_original.PixelWidth, 2), Math.Log(_original.PixelHeight, 2)));
    }

    protected virtual void OnTileCreated(TileCreatedEventArgs e)
    {
        if (TileCreated != null)
            TileCreated(this, e);
    }
}

public delegate void TileCreatedEventHandler(object sender, TileCreatedEventArgs e);

public class TileCreatedEventArgs : EventArgs
{
    public TileCreatedEventArgs(TileContainer tileContainer)
    {
        TileContainer = tileContainer;
    }

    public TileContainer TileContainer { get; private set; }
}

This class doesn't save the tiles anywhere, it only generates them. The most important math in this case is that it first calculates the maximum zoom level of the original image, meaning the greater of the number of tiles on either axis required to compose the entire image at its native resolution. It uses the Log function to find this. For example, 2 to the power of 10 is 1024, enough to cover a 1000 pixel wide image, so the maximum zoom level is 10. If the image is 1025 wide, then the next up is 2 ^ 11, for 2048, though those last tiles will only be one pixel wide. Does that make sense?

Each zoom level down, you'll divide the image's width and height by half, until you get to level 0, which will always be a 1x1 single tile. The loops, which could probably use some refactoring, cut up the resized image. There are two extension methods in here as well, to crop and make bytes for a JPEG:

public static BitmapSource Crop(this BitmapSource source, int x, int y, int width, int height)
{
    CroppedBitmap crop = new CroppedBitmap(source, new System.Windows.Int32Rect(x, y, width, height));
    return crop;
}

public static byte[] ToJpegBytes(this BitmapSource image, int qualityLevel)
{
    if (image == null)
        throw new Exception("Image parameter is null.");
    var encoder = new JpegBitmapEncoder();
    MemoryStream stream = new MemoryStream();
    encoder.Frames.Add(BitmapFrame.Create(image));
    encoder.QualityLevel = qualityLevel;
    encoder.Save(stream);
    int length = (int)stream.Length;
    byte[] imageData = new byte[length];
    stream.Position = 0;
    stream.Read(imageData, 0, length);
    return imageData;
}

Because I chose to make this event driven and not tied to a specific storage medium, I need to tie in some code to save the images.

TileMaker maker = new TileMaker(bitmapImage, 256, 60);
maker.TileCreated += maker_TileCreated;
maker.CreateTiles();

This simple code creates the TileMaker, assigns a handler to its TileCreated event, then calls the CreateTile() method to make it actually happen. Here are two examples of what the handler might look like.

private void maker_TileCreated(object sender, TileCreatedEventArgs e)
{
    DataContext context = new DataContext();
    var tile = e.TileContainer;
    Tile tileData = new Tile { ImageID = _imageID, Level = tile.Level, TilePositionX = tile.TilePositionX, TilePositionY = tile.TilePositionY, ImageData = new System.Data.Linq.Binary(tile.ImageData) };
    context.Tiles.InsertOnSubmit(tileData);
    context.SubmitChanges();
}

private void maker_TileCreated(object sender, TileCreatedEventArgs e)
{
    var tile = e.TileContainer;
    FileStream stream = new FileStream(String.Format(@"C:\Documents and Settings\Jeff\Desktop\test\{0}_{1}_{2}.jpg", tile.Level, tile.TilePositionX, tile.TilePositionY), FileMode.OpenOrCreate);
    stream.Write(tile.ImageData, 0, tile.ImageData.Length);
    stream.SetLength(tile.ImageData.Length);
    stream.Close();
}

The first assumes I have a LINQ to SQL definition, with a table called Tiles, and I'm feeding in an image ID which is part of some other code. The second one actually writes out files. I've run this code out of a Web page, which isn't ideal because it's somewhat CPU intensive. Really big, high resolution panoramas take a good 30 seconds.

Next time I'll get into how you feed the tiles from the database, to an HttpHandler and have the MultiScaleImage call the handler for tiles.

On a side note, I have to say that overall I'm ridiculously impressed with Silverlight 2. Sure, I wish more stuff would've made it into the distribution, but it's just stunning to me how much managed code you can now run right in the browser.

I'm not happy with the whole Blend thing. Apparently, according to a friend who was blocked from it, Blend is not available to the typical MSDN subscription levels. I'm not sure of what the criteria is. Blend is a pretty neat tool, but I'm absolutely annoyed that the designer surface in Visual Studio is useless. I'd love to know why this is, when you can use it in WPF.

Debugging simply works without any intervention, which I'm very happy about. It even debugs just fine while in Firefox. Getting scriptable methods to work right was a pain (because the MSDN docs are too vauge about the right way to do it), but I think I've got that licked.

I'm thinking about publishing a mini-Deep Zoom kit to facilitate some of these roll-your-own scenarios. Is there any interest on that?

EDIT: Part II is here.

Deep Zoom... so close!

I've been trying to put together a couple of articles on rolling your own Deep Zoom viewer, fed by a database with images all cut up for you. The image cutting I've got, the data I've got... but the MultiScaleImage control is not working as expected.

There are a couple of articles out there already that describe how to feed custom data to the control about image locations. Here's my version:

    public class ViewerSource : MultiScaleTileSource {
        public ViewerSource(int imageWidth, int imageHeight, int tileSize, int overlap, int imageID) : base(imageWidth, imageHeight, tileSize, tileSize, overlap)
        {
            ImageID = imageID;
        }

        public int ImageID { get; private set; }

        protected override void GetTileLayers(int tileLevel, int tilePositionX, int tilePositionY, System.Collections.Generic.IList<object> tileImageLayerSources)
        {
            string source = String.Format("http://localhost:2974/TileHandler.ashx?id={0}&level={1}&x={2}&y={3}", ImageID, tileLevel, tilePositionX, tilePositionY);
            Uri uri = new Uri(source, UriKind.Absolute);
            tileImageLayerSources.Add(uri);
        }
    }

If I step through the code, I can see that it's getting the URL's for the images right (I can cut and paste and see they work), but the requests are never made for them. In the constructor for the XAML page, I use this:

ViewerSource source = new ViewerSource(4372, 2906, 256, 0, 6);
msi.Source = source;

... where msi is the MultiScaleImage. For some reason, no go. Interestingly enough, when I put a break point on the .Add(uri), the IList<object> is empty, which doesn't seem right.

This is frustrating the heck out of me, because I feel like I'm pretty close. Does anyone else have experience with this? If I can't make it work, it might be an interesting exercise to try and do it in Javascript since I have the tile part all figured out.

Is LINQ to SQL too simple-minded?

I've been playing around with LINQ to SQL lately and I found an issue that I can't quite get my head around. Perhaps you can help.

As you know, there are times when you have database tables that don't have an identity field. The data typically is selected on a combination of conditions, often foreign keys and certain other data. While I'm not super anal about databases the way some people are, I don't see a need for the column or its index in these cases.

But LINQ to SQL insists on having it, or you can't even insert data. It tells you, "Can't perform Create, Update or Delete operations on 'Table(whatever)' because it has no primary key." Well, sure you can. I do all of the above on tables without primary keys. My initial suspicion is that this has to do with the concurrency checking, and I totally get that, but turning it off or rolling my own code has not changed anything. It still gives me this error. That, or I'm seriously missing something.

Yes, I know I could just add the column and move on, but it's not the right thing to do. Any suggestions?

More Posts