3D Photography

Introduction

3D OR NOT 3D ? THAT IS THE QUESTION . . .

Numerous 3D rendering software is now available to easily play the power of computers.
It allows more and more people, artists or engineers, to produce photo-realistic images.

Each time one needs to VIEW something unreachable with a camera, whether it is because it does not exist or is out of scale for human eyes, one can use a computer.

However, how realistic can those images be? Anyone looking at the computer screen can perfectly SEE that he is looking at an image, not directly at a real scene, or model.

This difference comes from the fact that in our three dimensional real world our two eyes give us two different images.
This is because they are in two different positions in space, separated by an horizontal 2.5 inches offset (~6.5 cm).
The brain accepts the small horizontal disparity between those two images, and in return gives a single image with accurate depth perception.
This ability is known as stereoscopy.

Due to stereoscopy, you can perfectly notice the difference between a model car in a box and the image of it on the top of the box despite both having the same dimensions.
Looking to the model, you see in stereo as each eye has its own image of the car model.
But when looking at the image on the top, you see a flat image as both eyes are focused on the same image.

Now, as we know the difference between "flat viewing" and "stereo viewing", let's see how to use the first to create the second.

Chapter I

Stereo 3D on computers

Creating a stereo image means first creating two flat images, i.e., a stereo pair: one image for the left eye and one for the right eye.
This is easy to achieve: you render one image with the observer in the left eye position, apply an horizontal offset to the observer position and then render the right eye image.
The offset is called the BASE in the stereoscopy vocabulary and is assumed to be the same as the inter-ocular distance (About 6.5 cm).
The base has to be increased or decreased relatively to the scale of the scene to have a significant stereo effect.
Obviously, you cannot use the inter-ocular distance to view in stereo a chemical molecule or a galaxy.
A typical average value for the base is 1/30 of the distance from the observer to the nearest object of a scene.

Why 1/30? If you stand in front of a window, which opens to a landscape to the horizon, you will notice that you cannot see clearly both the horizon AND the window itself if you stand within two meters away from the window.
When you are two or more meters away from the window, you can view all the scene comfortably from the nearest point (The window) to the infinity (The horizon).
This value of two meters depends on the person but is a statistical value.
The fact is that 6.5 cm (Inter-ocular distance) is about 1/30 of two meters.
So, if you take for the base the 1/30 of the distance from the observer to the nearest object of the scene, you're sure that you will see the full stereo image comfortably from the first point until the last.
You will also be able to see it with enough stereo sensation.
When the base is larger than the average inter-ocular distance, the resulting stereo is called hyper-stereo.
It gives you the sensation of looking at reduced models, as if you were a giant.
On the other hand, when the base is smaller than the average inter-ocular distance, the resulting stereo is called hypo-stereo.
It gives you the sensation of looking at enlarged models, as if you were a Lilliputian.

An error that needs to be avoided is making a stereo pair with converging viewing axes.
It appears natural to use convergence since eyes converge while they are looking at something, although it is not the right way.
When your eyes converge, the point at which they converge appears perfectly clear.
The fact is that everything else appears blurry but you don't notice it because you are used to it.
However, due to the accomodation reflex, when you look at something blurry your eyes will naturely adjust to it.
In a stereo image, all the image has to be clear to be viewed clearly at whatever point you look in the image. Converging on one point would make the image comfortable for all points before the converging point. However, this would be difficult for points after it to fuse.
By converging at the infinity, i.e., keeping viewing axes parallel, all the image will be easy to fuse.

Things become a little more complex when you want to see in stereo a stereo pair . . .
To fuse the two images you've produced in a stereo one, each eye must see only its own image.
Different solutions have been found over the years, mainly a result of the use of stereo pairs from stereo cameras used during the 50's and 60's.
You can use a lens stereoscope but you will have to transform your two images onto slides.
You can also use a mirror stereoscope (If you can find one . . .) but you will have to print your images.
It's possible to directly use the screen but that will divide the usuable surface on the screen into two; as you will have to display the two images side by side.
If you do not have stereoscope, you could train to "free-view" by crossing your eyes with the right view on the left and the left view on the right as many stereo enthusiasts do.
You will need time and patience as it is not totally obvious.

You can have a try with the stereo pair below : viewed in stereoscopy you'll notice that there is one ring not connected to any of its neighbours.

The best known solution is to write your own "SoftStereo" code.
Then, use LCD shutter glasses.

The trouble is that this solution is not adapted for your aim if you just want to make some stereo images yourself.
You can do this out of curiosity to see what it looks like by using your own computer and software you are accustomed to.

To do so in a cheap and quick way is absolutely possible, but, (Of course there is a but) there will be some restrictions about the kind of images you will be able to convert properly into stereo.
However, that will give you the opportunity to verify by yourself the interest to escape "flatland".

Chapter II

Andy Warhol, creature of the black lagoon and comics

They have all used a stereo process called anaglyph. Andy Warhol produced a 3D anaglyph Frankenstein in 1970.
Creature of the black lagoon is one of the most popular 3D anaglyph movie.
From time to time, comics use that process too.
(Notice that, in that last case, each stereo pair is hand made . . .)
Unfortunately, most of those realizations suffer from terrible defects.
They often have as a result an audience that is disgusted from stereo.
The fact is that the anaglyph process by itself is not a bad one but is difficult to apply with optical systems.
Let's analyze those old results and have a close look to the "theory".
Let's see how to apply it correctly with a computer.

In comics, left and right images are printed, one with blue ink and the other with red ink.
Looking with red-blue anaglyph glasses, you can see monsters springing from the page.
In movies, red and blue filters are added to the cameras as well as the audience wearing their own red-blue glasses.
Red and blue are used as they are opposite colors: you cannot see through a red filter what you can see through a blue one, and the reverse is true.
The stereo separation is correct for each eye but the stereo image is black and white.

If you're aware about computer image formats, when reading "red-blue" you've probably the temptation to insert "green" to read "red-green-blue".
Congratulations, you've found the first step to the solution.

Chapter III

All color screens are stereo ready

Computer's images are displayed on color screens and those screens use the RGB (Red, Green, Blue) system to create the color of each pixel of the image.
That means all computer's images are made with three bands: a red one, a green one and a blue one.
Suppose now we have tools to take only one color band from an image.
If we take the red band from the left image and the blue band from the right image, we will just need a tool to glue those two bands together and we will have a computer anaglyph giving black and white stereo when wearing red-left and blue-right glasses.

Numerous software to manipulate images and to translate them between the different formats can be used to process the color bands and produce 3D (PhotoShop, PaintShopPro, The Gimp...).
You just need tools which allows the separation of bands and which allows black and white bands to be glued back as color bands; thus producing a color image.

If you rush immediately to convert your own stereo pairs into red-blue anaglyphs by playing with the RGB bands you will probably be deceived.
First, you will only have magenta and white stereo images, not really black and white ones (Red + blue = magenta).
Secondly, stereo images are definitively not flat images and special manipulations have to be applied to them for correct viewing.

Chapter IV

Playing on the green

Magenta and white stereo is not interesting, black and white should be better, but color should be much more interesting.
So, how can we produce color stereo images on the screen?
Flat color images are made with three bands.
This means that the three bands will probably also have to be used for color stereo.
From which image must we take the green band ?
Red and green filters are opposite and turn to dark if added.
In contrast, blue and green filters are not opposite and turn to cyan if added.
The green information must come from the same filter as the blue.
This means that the blue and green band will both have to come from the same image: the right one.

Why use the red filter on the left and the cyan on the right ?
It could be the reverse but the International Stereoscopic Union has chosen the red on left for standard disposal.
It is also in coordination with the red used in international marking such as: ships, planes, and politicians !

Now, if you convert a stereo pair into a color anaglyph by separating the bands and after, glue them, you will be able to see in stereo and in color directly on your monitor by just using red-cyan anaglyph glasses.
Easy !

If you do not have anaglyph glasses with a cyan filter, you can use ones with a blue or a green filter: stereo 3D effect will remain but colors will change.
With a blue filter, colors will slightly shift to blue.
Avoid green anaglyph glasses as it wipes really too much colors.

Chapter V

Color tricks

Note : For best colors, most images are in PNG format and might be a little long to display if you're using a low speed connection.

Things are a bit more complex than they should be relating to the previous explanations.

The fact is that not all images can be converted.
Images with strong contrast zones are definitively not adapted.
They produce what stereo addicts call "ghosts".
Strong contrast zones produce anaglyphs with too close and too strong red and cyan spots.
This produce a very uncomfortable sensation through the red-cyan glasses.
Images with large zones of saturated colors will produce "ghosts" too. The exact name of this trouble (that produces a visual flashy effect quite different from the "ghosts" from contrast) is : retinal rivalry.
All the left information comes from one color, red.
If your image has large red zones, there will be no information (No green nor blue) for the right eye about those zones.
No stereo effect will appear there.
The same trouble happens with green and blue zones.
There is nothing to do for images with strong contrast (excepted creating the same image without the strong contrast . . .).
For example, you can change a black background into a grey background or you can try to change the lighting), even though it is still possible to use images with saturated colors.
If those saturated zones were gray there should be no problems as all the three bands should be the same on the zones.

Thus, We have to find a solution that will shift colors to grays but, yet respect the balance of space information between the two eyes.
This solution should also respect the original colors (if possible) and the three bands.
The solution will be to modify the saturation of the images.
Modifying saturation will allow us to modify the quantity of colors in an image by keeping for the resulting image only a few percentages of colors from the original image.
Notice that fully decreasing saturation turns the image into some kind of black and white version still coded on RGB.
The correct way to produce really black and white images is to use a dedicated tool as the formula used to get one value from three is not calculating the average value but using the following formula :
black and white = 0.30 red + 0.59 green + 0.11 blue.
0.30, 0.59 and 0.11 are values related to the sensitivity of the human eye.
A tool converting into black and white will allow us directly to produce black and white stereo images.

Here is an example of a picture with high saturated spots. The small balls inside the "meteor" are of bright red, green, blue and yellow :

If you directly convert the corresponding stereo pair to a color anaglyph, there are numerous saturated zones which are uncomfortable to view. If you first convert the stereo pair to black and white images then create the anaglyph, this anaglyph have, of course, no saturated zones which are uncomfortable to view and his fully comfortable :

On the left : direct RGB anaglyph, on the right B&W conversion first.

black and white conversion is the ultimate weapon against saturated spots.
The trouble is it wipes all colors.
It would be better if it were possible to modify the colors wiping only the spots that produce "ghosts".

A way to do this is to use an image processing software to change the hue or to reduce the saturation of the spots before producing the anaglyph.
The fact is that despite using an image processing software, fighting the ghosts "by hand" spot by spot or by try and error can be long.
This does not suit our original aim to produce stereo images in a quick way.
A way to speed-up color correction is to turn the reds into yellows. This is achieved by using the red layer of the right image to fill the green layer of the anaglyph.
This is OK if you have big red spots, such as in the Red Cross example below, but as it tends to turn the browns into yellows, and also saturated greens to black, it might not be suitable for most images.

A goof exemple of red to yellow stereo conversion.
Sample picture from http://www.geocities.com/lebouttedidier/avions.html

A not so good example of red to yellow stereo conversion

The drawing below sum-up the different solutions which work by changing the whole color layers.

There is however another solution, still working by changing the whole color layers, allowing to produce "perfect" color anaglyphs in a snap, whatever the color spots.

"Perfect" anaglyphs in a snap : the Automateac method

Why "perfect" with quotes ? Because there are different levels of perfection. The Automateac method is perfect in the meaning : will automatically and easily solve ALL color troubles for ALL anaglyphs. Now, of course there will be color shifts and, in some cases, you might achieve a "better" result using another method, with "better" in the meaning : with more aesthetic contrasts and color shifts in your opinion.
Below is a sample showing you the kind of result achieved with the Automateac method. The idea is to turn the reds to the closest totally comfortable anaglyphic color, which is a dark brown. A side effect of the method is to reduce the saturation of greens and yellows, as brown is in fact a dark yellow, which itself is a mix of red and green.
Note that absolutely ALL flashing spots from the original image are removed in the "perfect" anaglyphic. Regarding brightness and constrasts, all colors are as close as possible to the original when you compare both pictures seen through anaglyphic glasses.

The original colors with color ghosting.	The "perfect automatic" anaglyph.

Note the color spot troubles : on the Red-Cross helicopter there are spots on the rear wheel, on the red-cross itself and on the door lock. Then, on the background helicopter there are three color spot troubles : on the emblem (nearby the nose of the red-cross helicopter), on the wheel and on the turbine entry.
All those color spot troubles are completely removed on the Automateac anaglyph version.

The table below compare the four anaglyphic solutions between them and to the original colors (in the middle). From left to right the four solutions are : direct RGB, Black & White, Red to Yellow and Automateac.

Chapter VI

Stereo advanced rules: windows

In anaglyphs, and more generally in all stereo images, we find that they are not images but volumes.
Specific rules, which are not in use with flat images, have to be respected to display the volumes.

Unless you are standing alone with nothing more than the horizon and the sky around you, space appears relative to some frontiers.
This is what happens when you look through a window.
In the case of a stereo image displayed on a computer screen, the four physical sides of the screen (Left, right, up and down) are absolute frontiers.
They build a window through which you can see the stereo reconstructed space.
That introduces the following specific restriction:

If any side of the images of a stereo pair cuts any part of the scene this part must stand just beside the screen sides on the stereo image.

That means that you cannot see in front of a window something that is too large to go through this window.
The spatial coherency has to be respected between the stereo scene and the screen that displays it.

Very often you will have to move your stereo image back into the screen.
If you don't do it, you will produce stereo images that viewers will not be able to fuse.
A typical reason is that points that normally should be at the infinity (Or at least far away) will lie just on the screen surface.
They will have quite no parallax.
This will make an aberrant springing stereo image, completely out from the screen.

Chapter VII

Backward and forward

On the anaglyph, two pixels that reconstruct one stereo 3D point have an horizontal offset (Parallax).
The position of a stereo-reconstructed point depends on the distance between its left and right pixels.
Stereo points lying on the physical screen surface have no offset.

Moving the stereo image relatively to the screen will simply result in changing the distances between the left and right pixels.
If you do so on an anaglyph you will notice some rather blurry stripes when looking at the sides with the red-cyan glasses.
This is because the stereoscopic window is not set.
With stereo paper prints, the window is set by cutting those stripes.

If a stripe is cut from the left of the left image and another stripe is cut from the right of the right image, the entire stereo image moves backward.

Animated stereoscopic window

Click to open the animation

If a stripe is cut from the right of the left image and another stripe is cut from the left of the right image, the entire stereo image moves forward.
A version of that operation is for a computer anaglyph to roll the red band then cut the ghosting stripes appearing on the sides.
Notice that, as a consequence, the moved stereo image will be represented by an anaglyph which will be more narrow than the original images of the stereo pair.

Chapter VIII

Conclusion

The drawing below summarizes the whole computer anaglyph process.
Despite it suffering from tremendous restrictions, anaglyphs stays the easiest way to experience color stereoscopy on a computer.

Have a try: you'll be surprised to see how a stereo 3D scene is different from what you thought while just looking at it from 3D images.

Sylvain Roques
SOURCE

technorati tags:photography, stereoscopy, 3D