“Teach how fine atoms of impinging light To ceaseless change the visual sense excite; While the bright lens collects the rays, that swerve, And bends their focus on the moving nerve. How thoughts to thoughts are link’d with viewless chains.” -Dewhurst Bilsborrow
This book was created for the Imaging Systems class in the Fall of 2013. The content was delivered and gathered under the guidance of Nitin Sampat.
Table of Contents
8 Fundamentals 14 Input 22 Processing 24 Output 32 3D Scanning
Fundamentals
Spatial Resolution Spatial resolution is defined by the pixel count of the image. A higher spatial resolution will allow for a larger image size and finer detail. When resizing images, we are making up values for resolution we didn’t have before. The image below was resized to 150% of the original using the different interpolation methods in Photoshop to compare artifact generation, speed of the process and color gradients. Nearest Neighbor This method creates values using the nearest pixel. It is a simple, fast algorithm but causes jagged edges and rough color gradients. Photoshop took 0.1 seconds to resize the image.
Bilinear Unlike Nearest Neighbor, this method samples data from the four nearest pixels (x and y or 2x2) and averages the data together to give values. This gives a more realistic transition on edges and colors but took Photoshop twice as long, taking 0.2 seconds to resize the image Bicubic Similar to bilinear, this method takes averages the 8 nearest pixels (4x4). This creates a much smoother image than the other methods but takes considerably longer (0.3 seconds) if you have very large or many images to resize. 6
Tonal Resolution Tonal resolution is measured in bit depth. Bit depth determines how many tones can be displayed. The more tones or levels pixels can display, the more continuous tones and colors appear. The number of levels in a bit depth is equal to, 2 to the power of the bit depth. (i.e. An 8 bit image is 28 or 256 levels). Continuous neutral tones for print and the human eye occurs at 7 bits and 5 bits for color. The color image shows bit depth starting from the bottom. 5
4
3
2
1
7
Spectral Resolution Spectral resolution is the color data the sensor can capture. In most cameras this is Red, Green and Blue channel (RGB), also known as additive primary colors. In printing and paint, mixing cyan, magenta and yellow (CMY), with and added channel of K to achieve black are used. These are the subtractive primaries. RGB
8
CMY
File Size File size can be calculated using the formula with the result in Megabytes (mb).
Spatial • Tonal • Spectal = X mb Multiply the pixel dimensions of your image for spatial resolution, then to convert the total number of pixels to bits divide by 1 because there are 8 bits per pixel. Next multiply by the bit depth to get file size in bits. Divide this number by 8 because they’re 8 bits per byte. This number is the size of one channel , so multiply by 3 to get your final file size in bytes. To get your file size in megabytes, divide this number twice by 1024 (1024 bytes in a kilobyte, 1024 kilobytes in a megabyte). Assume you have an 8-bit, 1024 x 1024 image (1 megapixel). (X • Y) • T • 3 = x mb (1024 x 1024) • 8 bits • 3 1,048,576 Pixels • 8 bits • 3 1,048,576 Pixels • 8 bits • 3 1 pixel 8,388,608 bits • 1 byte • 3 8 bits 1,048,576 bytes • 3 3,145,728 bytes • 1 byte 1024 kb 3,072 kb • 1 mb 1024 mb 3 mb 9
Histograms Histograms display the frequency of pixels at different tones or brightness values ranging from 0-255 in an 8-bit image. The histogram on the left displays the properly exposed dispersion of values for the image below.
Overexposed histogram below shows most of the values towards the right. Light colors, like the horns and stars start to lose information.
High Contrast shows peaks at either end of the histogram. Overuse of contrast can lead to losses of detail in dark and light pixels.
Underexposed image slides the values to the left and causes losses of detail in the trees and darker structures.
10
DPI vs PPI Resolution in prints and displays are measured in dots per inch and pixels per inch. DPI in printing puts small dots of color, dispersed over a page at different screen angles to convince your eye of continuous tones. PPI is simply the number of pixels in an inch of a display. DPI Used in printing. CMYK dots are distributed across the page at varying angles to produce color. This is called halftoning
PPI Used in displays and sensors. Pixels display varying intensities of RGB to make color. PPI and LPI (lines per inch) are equal values.
11
Input
Sensors
Sensors have replaced film in almost all areas of photography. While film is still used for aesthetic purposes, digital sensors dominate the market and are becoming better every year. Fundamentally sensors and film work the same way. Film uses silver halide crystals that are sensitive to light while sensors use pixels that record light in voltages.
14
Color in film is recorded by layers of specific color sensitive film stacked on top of each other. Each wavelength penetrate the film to specific depths. Sensors use color filters over individual pixels and use demosaicing algorithms later to make color.
CCD vs CMOS CMOS
Photon
ion n Convers to Electro
CCD
ion
onvers ltage C
n to Vo
Electro
n
ersio ctron Conv
Photon to Ele
Gain
al to Digit Analog er Convert
Gain Electron to
Complementary metal-oxide semiconductor (CMOS) devices vary from CCD’s mainly on how the signals are transferred through to the analog to digital converters. CMOS are built on a standard silicon chip that allows each pixel to have its own amplifier. This greatly reduces power consumption and gives gain control to every individual pixel. At first the main draw backs were reduced light sensitivity because the pixel components took up space on the chip, and noise because of the increased travel distance and conversion of individual pixels. Since CMOS can be produced on average silicon chips this makes them inexpensive to produce and advancements in processors and noise correction have made the CMOS comparable to the CCD in most aspects. The majority of DSLR cameras utilize CMOS technology.
Analog to Converter
Digital
nversion
Voltage Co
Charged Coupling Devices (CCD) were some of the more efficient sensors in early digital photography. After recording the image the values are sent one by one through the amplifier and analog to digital converter. Since more of the chip is covered by pixels this gives CCD’s an inherent advantage in quality, light sensitivity and noise. But every pixel has to be processed individually causing CCD’s to consume significantly more power.
15
Color Filter Arrays After capture, cameras have to use demosaicing algorithms to combine pixels of different color information to create a full color image.
r ye a B
CFA’s are placed over sensors to gather color information. There are many different kinds of CFA’s but most cameras use one called the Bayer Pattern. Using additive color, It has 2 greens for every red and blue. This is because the human eye is more sensitive to shades of green than any other wavelength. Since each pixel only records the color placed over it, overall spatial resolution is compromised for spectral resolution. This also reduces the amount of light the sensor can absorb. They’re are still many different kinds of CFA’s used. They can be any arrangement of RGB and even CMY. For example Fuji uses a X-TRANS filter array that arrange diagonals of green and alternating red and blue that increases the randomness of filters in a 6x6 square. This increases the resulting resolution because it doesn’t require a low pass filter to reduce color moire and false colors. A newer CFA to be used is called the RGBW. It has equal amounts of red , green, blue and “white” pixels, that have no color filters. This gives the sensor more dynamic range but lose spatial and color resolution.
16
X-T
NS A R
Bw G R
The Foveon
Fo ve on
Fil m
The Foveon differs from other digital sensors because it doesn’t use CFA’s to produce color. Just like color film, it relies on the fact that colors penetrate materials at different depths. Still using a CMOS chip, layers of color sensitive silicon are stacked on top of each other in relation to the penetration depth of each color. This makes it so each photosite all 3 color channels are collected This makes a demosaicing algorithm unnecessary.
It produces richer tones than sensors with bayer patterns due to increased spectral resolution and high dynamic range but lacked some of the sharper details. It also suffers overall color accuracy and significant noise at higher ISO’s. Resolution is harder to compare because since each photosite contains 3 pixels, the Foveon needs less photo sites overall to have more effective pixels than a sensor with a similar overall megapixel count. This allows the Foveon to record more spectral resolution than spatial and still have similar file sizes to a higher megapixel sensor that has to undergo demosiacing algorithms. Each pixel on the foveon is 7.8 nm, which is much larger, and can gather more light, than 2 nm pixels on average sensors. This is what accounts for the higher dynamic range of the Foveon.
17
Flatbed Scanners Flatbed scanners use strips of CCD sensors to take line scans of an object. Since CCD’s have to process the information one line at a time anyways, a scanner goes over the subject and records it only a few lines each pass as it send it through the amplifier and analog to digital converter
Scanners have very high interpolated resolution and can provide excellent detail which allows images and text to be captured at very high PPI
Light Path
Light Source
Mirrors
C
18
AD
Gain
CCD Strip
PMT’s ADC Gain
Photomultiplier tubes are used in a variety
+300V
of medical and scientific imaging such as scanning electron microscopes as well as high end drum scanners.
+200V +100V
Anode
+400V
Dinode
Incident photon (or secondary electron)
PMT’s absorb photons (or secondary electrons in the SEM’s case) and then release electrons down a dinode chain of increasing voltages. When an electron strikes a dinode, it releases secondary electrons. These multiply and accelerated down the chain with the original electron allowing large amounts of gain. The electrons hit an anode at the end of the chain to be converted to an image by the analog to digital converter. Higher end confocal microscopes and drum scanners use gallium-arsenide elements with spectral responses of 300-800nm which is slightly larger than the human response of 400-700nm. Allowing for UV and near IR detection. 19
Processing
1 Software Controlled
Neutral Balance
Red #
ADC
Green #
Raw File Format
Raw Processer
Blue #
When sensors pick up RGB data, they need a reference for how tones appeared in the original scene. This process balances colors until the designated neutral pixel values are equal amounts of red, green and blue
2 Basic Aesthetic Processing Exposure
22
Correcting for exposure shifts the entire histogram to brighten or darken the image. Correcting for exposure adjusts other values to correctly represent tones across the image instead of brightening them all
Sharpening
Convolution kernals are used to sharpen images. Sharpening an image increases differences in pixels values in high frequency areas, or the details. By increasing the difference between two adjacent pixels this creates more contrast and your eye perceives it as sharp. When sharpening details, noise also gets sharpened. To combat this, masking is used to only affect the areas of high frequency.
Pipeline Gamma Correction
CFA Interpolation
Light out
Demosaicing algorithms combine values from the RGB channels to create full color images. These algorithms aim to preserve color accuracy and create simple computational data for fast processing either in the cameras firmware or in the raw processor. These use methods such as nearest neighbor, bilinear and bicubic interpolation.
Co
c rre
d te
m Ga
m
m Ca
a
a er
m Ga
m
a
M
Color Adjustment
To make images more visually appealing, adjusting colors saturation and vibrance is sometimes necessary. This can be globally, over the entire image or selectively adjusting some colors over others. Depending on the purpose of the image, this can be completely subjective or corrected to precisely represent the colors present in the original scene. Filters are often used to affect colors and tones across an entire image.
Final Image
ito on
r
m Ga
m
a
Light in
Monitors intensity to response curve affects how images are viewed. Gamma controls the brightness of an image as well as ratios between RGB. If not corrected, images will appear darker than desired.
Once being processed, image data is sent into another file format that is more readily accepted such as a TIFF JPEG, PNG, GIF or TIFF.
23
Output
Output Once the image has been captured and processed, it needs to be presented in a media that other people can view. This usually is either monitor displays or more commonly print media.
Print media has evolved many different approaches to putting ink on paper since the days of the first Guettenberg press. The two most common types of printers are inkjet and laser. Laser printers function by scanning over a negatively charged drum. When the laser hits the drum, it removes the negative charge from that area. Once the drum has a latent image, charged particle toner attaches to the neutral portions of the drum and is then fixed by heating it onto the paper. Laser printing is used in many office all-in-one printers because of its speed. Inkjet printers function by selectively spitting dots of ink on the paper as it passes through the printer heads. Inkjet is used in most other printers because it produces higher quality prints. Other techniques are used for high end printing and mass productions such as web press, Lambda and gravure printing
26
Displays S-IPS LCD
AMOLED
Liquid Crystal Displays use the polarization of light to control output of light. This is used with RGB color filters to produce color. A light shines from behind the pixel through a sandwich of polarizing filters and glass substrate with liquid crystals in the middle. The liquid crystals diffract light, either allowing it to pass through or get blocked by the polarizer. When the pixel is turned on, electrodes apply an electric field that elongates the liquid crystal, allowing light to pass around the crystals and through the polarizer to turn the pixel white. In it’s off state, the crystal concentrate to the middle and diffracts the light away so it is blocked by the polarizer. The S-IPS LCD array shown above is a specific LCD technology that allows for fast refresh rates and wider viewing angles compared to standard rectangular pixels.
Active Matrix Organic Light Emitting Diode displays utilize a organic compound layer that emits light when an electric current is ran through it. Each pixel has a TFT layer that manages the current being put into the organic layer. Unlike LCD’s, LEDs are not continuously emitting light so much deeper black points are possible which improve color and contrast. The Super AMOLED display shown above is used in Samsung devices and incorporate a pressure sensitivity layer in the pixel instead of above it like many other arrays. 27
Analog vs Digital The fundamental difference between analog and digital printing is the presence of a physical plate. Analog printers require a plate with the text or images engraved on it to be dipped into ink and forcibly pressed against the paper. Most of the cost that comes with analoge printing is in the plates. These can be made of aluminum, rubber or in the case of gravure printing, a copper coated zinc drum. Each page has to have its own plate made. Analog printers are usually sheet fed by large rolls of paper that allows for rapid, continuous printing. This makes them much faster at mass producing similar prints, making them ideal for newspapers and magazines.
The drawbacks of analog to digital are focused around flexibility of a digital system. Software essentially takes the place of a plate and allows for faster changes to print settings and design. Individual prints cost less but in most cases are lower quality than an analog printer. Inkjet printers use either thermal or piezo elements to sputter ink on the page. The heat of a thermal element expands the ink until it falls onto the page. Piezo heads vibrate drops of ink onto the page allowing for more control on the placement of dots.
28
Pigments and Dyes Printers use either pigment or dye based inks to color prints. Pigments are insoluble in water and have to be suspended in a substrate while dyes are water soluble. Dyes have a larger color gamut than pigments being water soluble makes dyes prone to water damage. Compared to pigments, dyes also have less lightfastness and longevity. This is one of the main reason most printers use pigment based inks.
Pigments
Dyes
29
Lambda Lambda printing is different from conventional printing due to the fact that it does not have to halftone to achieve gradients and colors. Rolls of photographic paper are fed through a set of lasers that expose the paper to varying levels of light. This gives the print continuous tone. This process emulates the exposure of photographic negatives to light. The lasers vary intensity to create all the colors and tones. Comparisons between Lambda and inkjet prints tend to show better color detail and saturation where as the inkjet was favored in terms of sharpness.
30
Gravure Gravure printing is an analog printing technology that uses a copper coated zinc roller engraved with the latent image being printed engraved into the surface. The copper plates make gravure an expensive process up font so multiple prints from one plate are required to make up for the cost. It is considered the highest quality printing technology available, but due to high cost and the need for high printing quantity, it is mainly used by magazines such as National Geographic and printing labels on food and beverages. The copper cylinder is rolled through the ink tray as the doctor blade scrapes away excess ink. The ink stays in the grooves of the cylinder as a rubber impression roller presses the paper to get even coverage. The paper is then dried before going through the next color.
31
3D Scanning
Laser 3D Scanning
34
Many laser scanners work by shooting a laser at an object and measuring the distance of the reflection hitting the sensor. The system knows the position of the laser relative to the sensor and the distance from the laser to the object. This is called triangulation, its similar to the time of flight method a camera uses to auto focus which calculates the time laser pulses take to reach the sensor.
The NextEngine The NextEngine desktop scanner has two arrays of 4 lasers that scan over a subject. It collects this informations using one of two 3 MP sensors, the other has a 7 color color wheel that takes stills of each object for detailed color information. It has two modes mainly affecting its field of view and resolution. Normal mode scans an area 13.5”x 10” at 75 ppi. It also has a macro mode which sees only a 5”x 4” area at 200 ppi. Using either mode, multiple scans can be put together later in software. On average each scan takes around 2 minutes and about 30 minutes total depending on the number of facets of the object taken. The plate that comes with the system holds objects slightly larger than a piece of paper and up to 20 lbs but requires the user to rotate the object manually for multiple scans. The system also has a rotable arm available for purchase that is controlled by the software to acquire the necessary views. These scanning parameters combined with the software processing the information makes the NextEngine a low cost, high precision scanner good for many applications.
35
Software The software bundled with the system gives a user interface that can control the scanning parameters in real time. This includes speed, resolution, field of view and options for giving geometry to the surface information that allows the computer to render complex 3D structures. The more geometry or meshing, that is applied to the file then the more detailed the fine structures are and smoother the curves are. The software also puts together the color information and applies it to the rendering. It also can control an automatic arm that entirely automates every scan without having to rotate the object each time.
36
Applications The precision of this system and other 3D scanners allow for a variety of applications. Major applications include medical and orthopedic’s, industrial design, reverse engineering, CGI, and artifact preservation to name a few. Artifact preservation has changed significantly with the use of 3D scanners. Now museums are able to have detailed, full color models that can digitally be stored in case anything happens to the original. Many museums even allow the public to download and print their own models.
37