About two years ago, I bought myself a really neat looking camera. At this time, Canon's Digital Elph series was pretty new, and I just had to have one. I very quickly became the proud owner of a new PowerShot S110. It didn't take long for my digital camera directory to fill up with thousands of JPEG images.

One thing I loved to do in particular was take low- or natural-light images. I felt the harsh lighting from the camera's flash needed to be avoided at all costs. The S110 did a good job in low-light conditions overall, but my expectations grew too much. I became tired of the 1-second shutter limit the S110 had, and desired a digital camera with the ability do expose a shot for several seconds, if not dozens.

Of course, I didn't have a ton of money to spend on a new camera, so I decided to improvise...

I began to shoot several identical shots, 1 second exposure each, then recombined them in photoshop using the 'screen' (add) blending mode. This worked to an extent... but I wanted to see just how far I could push this idea.

So, one night, I rigged up a system where my camera would shoot in continuous mode, using a 1 second exposure, on a tripod, for as long as the battery, CF card, or my patience would last. This turns out to be about 200 images; my patience gave in first. In the dark of night from my back yard, I pointed the camera into the woods and let it shoot.

Now suddenly I had about 200 JPEG files on my hands. There was no way I was going to use photoshop to 'screen' them all together. So, of course, I wrote a program to do it for me. Using SDL_Image to decode each JPEG in sequence, I loaded all ~200 1600x1200 images into RAM. Of course, I didn't need to load them all at once, but I did so for coding simplicity.

Now that I had a large "volume" of images in RAM, I was able to perform operations on them in sequence. I considered each pixel to actually be a column of pixels. Each pixel in the column corresponded to light captured from one of the 200 frames. So, thus, there were 200 pixels per column, 1600x1200 columns. This means there were about 384,000,000 pixels to crunch. Simply adding all 200 pixels in each column together, to create one single 1600x1200 pixel image, produced some pretty nice results (Figure 1), compared to the same scene shot only once (Figure 2, Figure 3). Also notice Figure 1 is cropped differently from the other figures, so it does not contain the lawn chair at the lower right corner.

Figure 1 200 combined exposures

Figure 2 A single normalized exposure

Figure 3 A single original exposure

In order to avoid over-exposure of some areas, I decided to use a measure of transientness to determine how persistent the exposure in a certain column was being. If the exposure was being persistent, then light was steadily reflecting from that area and striking that pixel, so I did not need to integrate it as much. If the exposure wasn't being persistent, or was being very transient, then the light wasn't steadily striking the pixel. Since the camera was not in motion, this only meant that this area was extremely under-exposed, and required more integration to get a 'real' exposure value. I tried a form of Shanonn's Entropy equation, which would tell me how much 'information' was present in each column. This measure would roughly correspond to how much the value varied, but it didn't work as desired.

An even simpler equation to find the transientness of each column worked very well (Figure 4).

Figure 4 Exposure transience Blue - Low transience Red - Moderate Transience Yellow - High Transience

The equation is simply the sum of the absolute deltas between every two pixels in each column:

double signalTransience(double *a, int len)
{
	double transience = 0;
	
	for (int i=1; i < len; i++)
	{
		transience += fabs(a[i] - a[i-1]);
	}
	
	return(transience);
}

In Figure 4, which is false-colored to exaggerate features, the red areas correspond to high transience, whereas the blue areas correspond to low transience. Low transience is observed in the areas which are very exposed (bench in foreground, lawn chair in lower-right corner), but also in areas where almost no light struck the camera's sensor at all (deepest woods). The porch swing, and many of the trees, showed up as being very transient in their exposure, and thus required the most attention when integrating the 200 images together.

Pseudocode for the main loop would look sort of like this:

main()
{
    double st = 0;
    double result = 0;

    for (each column at X Y)
    {
        normalize(column);
        st = signalTransience(column);
        result = integrateColumn(column, st);
        image.pset(x, y, result);
    }
}

This took about 300 seconds to process a 1600x1200x200 volume on a MIPS R12000 @ 270MHz (2MB), using doubles.

I'm still experimenting with this technique. Major hurdles are thermal CCD noise and JPEG compression artifacts. Thermal CCD noise results from electrons bleeding between sensor elements, and can only be decreased by using a larger CCD. The camera I use is very small, and thus has a very small CCD. JPEG compression artifacts can easily be worked around by using RAW output from the digital camera. Unfortunately my camera doesn't have that capability. Also, at such low light, color differentation is nearly impossible. The values being captured are either 0 or 1, out of 255. This results in nearly grayscale images. A camera with a slightly longer shutter time (4-15 seconds) should correct this problem.