Why is photon mapping popular in gaming right now, and will it stay?
Photon mapping is a somewhat new technique for computing the flow of light enery within a scene (I will attempt to use layman's terms and won't be defining any of the domain specific terms in this entry if I can help it). I say somewhat new, because it has been around for a while in the research community, and only recently, has it acquired actual usage in real-time computer graphics in gaming. I was actually somewhat stunned that it was being used to some degree in Halo 2. Seems Adrian Perez, author of Advanced 3D Game Programming in DirectX 7.0 (later updated by Peter Walsh for 8/9), had a small chat with some college students, and the article was later published (thought I can't seem to find it, if someone wants to post it in the comments then please do). The results were that various tools used by the modelers used photon mapping so they could get a better feel for what their graphics would look like in game. Impressive to say the least.
Why was Adrian using photon mapping? Well, it is faster than radiosity. Radiosity was previously the number one lighting technique used in games. But it is very slow, and so you generally run a radiosity tool to generate radiosity maps, rather than in real-time. If they decided later to use radiosity instead, they still could, but the artists tools needed more immediate feedback if they were going to be useful and the artists were going to be productive.
The alternatives to photon mapping tend to be ray tracing and radiosity. Both methods are tried and true, and I'm sure everyone has seen the results of these techniques. They just take huge amounts of processing power that don't exist on end-user-desktops (at least not for real-time use). The largest issue with ray-tracing is the fact that it isn't a global illumination algorithm (it doesn't properly compute all of the features of shadows, namely the penumbra or soft edge that exists the further something is from the surface it is casting the shadow on). While the largest issue with radiosity is it's huge consumption of memory while processing.
Photon mapping overcomes these two issues by being lightweight (doesn't consume that much memory), and still providing a model for global illumination. Don't get me wrong, it still isn't perfect, but it definitely fills a slot in the graphics pipeline quite well. It achieves lightweight status by only storing primitive point information for each photon, namely direction, position, and some sort of color that demonstrates the power of the photon. This is much smaller than the equivalent form of storing light using subdivided mesh patches for the objects in the scene (aka radiosity).
As the photons interact with the geometry in the scene, they are stored in a global collection, generally a BSP or KD tree for fast access. They are accessed later to compute lighting while the scene is being rendered. This means photon mapping is always going to be a two step process. First you have to emit the light from the light-source, and then you have to fire rays from the observer's viewpoint to calcuate the effect of those photons on each pixel in the resulting image. The space partioning tree speeds up the search for these photons later, and photons are sampled by averaging them together to get a final lighting sample. Since we average photons together we get a softer shadow, and thus can simulate shadowing more accurately.
There are some issues with photon mapping:
- The more photons you shoot the more accurate the scene is, so you tend to want a large number of photons. The more photons the more processing power is used.
- You need a large number of photons to accurately sample the light at a given point. Marlon John points out in his book Focus on Photon Mapping, that he uses about 50.
- Current techniques don't take very many short-cuts. Photon mapping is meant to simulate the way photons traverse the natural world, but yet photon mapping doesn't intrinsically build in various laws of nature.
- The more lights you have, the more photons will be emitted into the scene to be processed.
- Materials have to be carefully designed so that photon interactions with the material are accurate. This produces accurate surfaces, while improper material design can really mess up a scene.
Some things I'll be talking about to help augment photon mapping:
- Photon ownership. In nature photons are constantly being processed by a massive, highly parralel computing system. Don't believe me do you? Well, each time a photon interacts with an atom, the atom individually processes that photon, and decides what to do with it. That means photons are individually being processed by their own CPU. Once an atom is done processing a photon it gets to decide whether or not to release a new one (it doesn't decide, rather the energy computations involved decide, aka the massive super computer known as the laws of nature) and in what direction.
- Locality of photons. This bridges from the first assertion. Once a photon hits an object that object determines the future of the photon. Current photon mapping techniques compute the bounce and continue going. I'll be talking about how you can gain performance by culling the processing of photons that wouldn't change the scene (note that you can compute the photon mapping for a scene once and re-use it, assuming nothing moves within the scene and nothing about the emission of photons changes).
- SOL Rendering Principles. Turn on a light in a room and it takes some fraction of time (even if this is really fast) for the room to *balance* the flow of light within the room. This isn't just our eyes adjusting (though that is one thing that is happening), but also the concepts that photons only travel so fast, and as they interact with things on the atomic level, it takes a while for the flow of light energy (flux, so I don't have to keep saying flow of light energy), to balance. This leads us to an optimization where we can slowly build the photon map over time, and re-use previous results. This will mean the quality of the scene will get better over time, depending on how much process power we have, this could be instantaneous, or it might take a seconds worth of frames to reach parity. The human eye will appreciate the increased detail in shadows and reflections.
- Lights or Emissive properties. Lights are generally defined in a 3D environment. But I hate lights, since they represent some pseudo-element. I prefer that the actual object be emissive (aka a lightbulb), rather than declare an overlapped light on top of the model of a light bulb. Or in most cases, you can't even see the light bulb model. I have some great optimizations that take into account dynamic lights such as flashlights, strobes, and other types of extremely dynamic lighting that can really add to the atmosphere of the scene. Currently in 3D gaming, the user doesn't appreciate when the scene is too interactive for them to be able to clearly interpret the image, but Hollywood makes use of this technique all the time to provide an atmosphere. One of my first photon mapping demos will be a nice techno club scene.
- Instance lights, continuous lights, and turning lights off. Changing the flux in an environment is easy. You just have to change the light source. Some light sources are extremely continous, some are constantly changing. Notifying the photon map of the origin of various photons is important. This lets us tell materials that the photon affecting their surface should either continue effecting (building up our LOD) the material in question or whether it should hold only a specific few photons from that source (a queue so that older photons fade off, and possibly a timed queue, so that if the source stops emitting, eventually all the photons are lost). My example for this will be a television set. While I can't render this example in real-time, I expect to be able to eventually. As the television changes picture, the entire scene oustide of the TV changes as well.
The biggest question is whether or not photon mapping is here to stay? I really think so. I mean look at all of the great optimizations that still exist for the algorithm (or at least the optimizations that I've noted from personal experience against currently published examples of the algorithm). Photon mapping for me is a passion, but I have a lot of them. And like most passions, they take a back burner to more realistic goals. Hopefully I find enough time to continue talking on this subject, since this was a thoroughly enjoyable article for me. Ask lots of questions in the comments, or make comments of your own. I'll definitely make sure to respond.