Imaging Systems Mariah Texidor
Table of Contents Fundamentals
5
Image Input
13
Image Processing
17
Image Output
23
Mirrorless Cameras
29
Fundamentals
7
Spatial Resolution There are four types of resolutions: spatial, tonal, spectral, and temporal (when motion is applicable). These combined equal to the size of an image. Spatial resolution is the number of pixels in a set of x and y dimensions. It is associated with details. When resizing an image some common methods of interpolation are bilinear, nearest neighbor, and bicubic. Any form of interpolation takes the spatial resolution, the amount of pixels the image has, and either creates pixels to make your file larger or discards pixels to make your file size smaller. Some things to take into consideration when altering spatial resolution is the affect an interpolation type will have on the details and edges of your image overall.
Nearest Neighbor: This method of interpolation is used to preserve hard edges. It typically creates the most artifacting, however.
Bilinear: This method of interpolation provides smoother results with details and hard edges than nearest neighbor because it averages two pixels in a row to create new data.
Bicubic: This method of interpolation has the most complicated algorithm, but also provides the best results in preserving details and hard edges.
8
Tonal Resolution
Tonal resolution is associated with brightness and bit depth. It deals with how many gray levels are in an image. The more gray levels present the more continuous an image looks, which can been seen on the bottom of each of the graphics. Represented in the graphics are 4, 16, 64, and 128 gray levels. With 4 levels of gray, a 2-bit image, there are no continuous tones.
With 16 gray levels, a 4-bit image, we notice the tonal blending is not continuous still and is visible in the clouds as well as spikes in the histogram. The human eye usually cannot see a difference in gray levels past 64 levels, which is a 6-bit image. It is slightly noticeable still in grayscale.
With 128 levels an image is visually continuous in all three demonstrations shown above. TIP: Some magazines usually print at 5-bits which has 32 gray levels. In color images it is seen as continuous tone, but in grayscale some posterization and banding is slightly visible.
9
Spectral Resolution
Spectral resolution speaks to the color channels in an image. Most cameras record three monochromatic signals of red, green, and blue. These signals are wavelengths across the visible spectrum to produce color in the way our eye sees. As discussed with tonal resolution a normal image is 8-bits. If we think of each recording of the channels red, green, and blue as an image we then have 8-bits/channel. RGB are additive colors, when one is subtracted the remaining create a color from the subtractive color system, cyan, magenta, and yellow, which are used in printing.
10
Temporal Resolution
Temporal resolution refers to motion and video, not still images. The unit of measurement for temporal resolution is frames per second. The standard frame rate for video is 24 fps in order for our eyes to see video seamlessly. In television broadcasting the frame rate is 30 fps.
Color Spaces There are three different color spaces that are common which are RGB, CMYK, and Lab. We will go into image processing in some of these spaces later on in the book. For now, the important thing to focus on is to remember RGB mode is the one that cameras and scanners are used with and CMYK is used in printing. Since CMY are not pure colors the black component in printing is needed so images do not become a mucky brown.
11
Histograms Histograms are a visual aid to help see where your tonal data is arranged in an image. They have a range of 0 to 255 in pixel value along the x-axis, and the frequency of pixel values along the y-axis. Histograms are helpful in visually seeing what your dynamic range is in a scene of an image.
This is the histogram for the image on the right. It has a good range of shadows, midtones, and highlights.
12
Halftoning Resolution is measured in DPI and PPI, the difference between them is important. If a device is capable of continuous tone such as scanners, cameras, and monitors then assume PPI which is pixels per inch. LPI or lines per inch is synonymous with PPI. If a device can only produce binary objects, such as inkjet and laserjet printers, which we will get in more depth with later on, then assume DPI which is dots per inch. The halftone process reduces an image to binary dots to help create what looks like a continuous image and reproducible by a digital printer. In instances where a device is marketed with using the measurement DPI and it is capable of handling multiple gray levels, you can assume PPI instead. However, there is a calculation needed to convert DPI to PPI, which is when we can use the “Rule of 16�. The number 16 is used because we assume an 8-bit image which has 256 gray levels. That is then 256 dots needed per pixel and the square root of 256 is 16. Note that this is only used when using a continuous tone image and not text or line art.
13
Image Input
15
Sensors An image sensor is what is in your digital camera. It converts light energy also known as photons, into electrical energy such as electrons. They are made of a light sensitive material called silicon, which is panchromatic, meaning it is inherently sensitive to all wavelengths of visible light.
having less chance to produce noise, which is shown on the right. Less noise is produced with an active CMOS sensor.
A sensor is what film would be in analog photography. Film was only inherently sensitive to UV light making it more sensitive to blue. Therefore, it was sensitized in order to be more sensitive to the rest of the visible spectrum.
CCD Sensors: A CCD sensor, as shown to the right, collects all the information and transfers it in rows during read-out. It proves to be an effective method, but is relatively slow. They have a preferable signal because there is no dead space in between pixels, making it susceptible to less noise. In addition, it has extreme sensitivity and consumes a lot of power. CMOS Sensors: A CMOS sensor however has two variations. It can be passive or active. A passive CMOS sensor sends information from each individual pixel to an amplifier located on one side of the sensor, which often produces noise. An active CMOS sensor also sends the information from each individual pixel; however, each pixel has its own amplifier therefore
16
Vertical Scan
CCD and CMOS sensors are the most common types of sensors found in digital cameras. They stand for charged-coupled device and complementary metal oxide semiconductor, respectively.
Horizontal Scan
Color Filter Array Patterns The pixels on sensors have filters over each of them in order to produce the colors in an image. The human eye is more sensitive to green, so twice as many green filters are put on sensors. These filters over a sensor create a pattern. Any pattern on a sensor is called a color filter array (CFA) which helps produce a color image in result.
CFA interpolation then occurs after red, green, or blue has been recorded by every pixel, the interpolation helps fill in the missing pixels from each layer. The process is also known as demosaicing, they are synonymous. Multiple CFA patterns exist and each are created with a certain strong suit.
The Bayer Pattern: It is the most common CFA patter found in most digital cameras. It has one blue, one red, and two green filters.
RGBE: Similar to the Bayer pattern, but the second green filter is replaced with an “emerald” one. This is the pattern in some Sony cameras.
RGBW: This is a pattern developed by Kodak in which 50% of the pixels are replaced with “white” filters, which are actually transparent. The transparent filter was meant to increase the amount of light the sensor could collect.
17
Image Processing
19
Image Processing Pipeline 1010101101010 1101010110101 0110101011010 1 0 1 1 0 1 0 1 0 1 01 1 0
Metad ata
20
There are multiple steps that take place in image processing even before you edit the image in any post-processing software. When you take a picture the sensor relays voltage. The voltage then needs to be readable, which is where an analog-to-digital converter converts the voltage into numerical data that your computer can read. The numbers are divided up into red, green, and blue values that make up raw data. All the data is stored in its own compartment, which creates a raw file format. Using a raw processor such as Adobe Camera Raw, Capture One, Lightroom, etc. will allow all the data to be readable and shown on the screen as an image.
CFA interpolation demosaics the image. It fills in the values that pixels are missing so that all pixels have red, green and, blue data within them. In order to see the images displayed as bright as our eye sees it, an inverse curve is applied, which is gamma correction. In addition a colormetric matrix is applied which has information about the images’ color data. This is useful for profiling needed when editing images on screen or going to any kind of output. Neutral balance then corrects color that was incorrect in original capture.
However, most will adjust exposure, contrast, and sharpness to their own liking before they save to a final format. Noise correction isn’t displayed on the pipeline because it does not have a specific placement since it is device specific and happens throughout the processing in various steps. This is also the same for sharpening. Any further edits are up to one’s artistic freedom.
If no further editing is done then your image can be converted to a file format such as .TIFF which would be the final step.
21
22
Sharpening An image is made up of both high and low frequencies, although it is mostly low frequencies. Frequency is simply the change in tones. When sharpening an image you are simply amplifying or increasing the higher frequencies. The process is similar to an equation. Convolution kernels can be applied to sharpen an image.
Using positive values surrounding a pixel will create a blurring kernel.
3 3
5
3
3
Using negative values surrounding a pixel will create a sharpening kernel.
-1 -1
5
-1
-1
An unsharp mask will take the blurred image and overlay it on the image, creating a mask that amplifies the high frequencies and creates a sharpened image.
-1
-1
-1
-1
17
-1
-1
-1
-1
23
Image Output
25
Pigments and Dyes Pigments are powder-like ink. It is good for image permanence in the sense that it isn’t water-soluble and sits on top of the paper. However, it is expensive and not ideal for high-gamut printing. Dye based inks are more often used today in everyday printing. They produce more versatility with color in images, however they are susceptible to fading from light and age as well as water damage, making them least ideal for photo preservation purposes.
Analog Printing There are analog and digital printers. Analog printers involve a plate. In digital printers the plate is replaced with software. Analog printers are used mainly in web press and are fast. In contrast, digital printers trade off speed for convenience. Rotogravure printing is an analog printing process that is very expensive. It involves a copper plated roll that presses images onto the paper. This process is expensive because it involves a change of the engraved roll every time content changes, which is why many presses have stopped using it. However, the process was very efficient in producing in mass amounts and being very fast. Also, it was the closest process to making a continuous tone image.
26
Paper Impression Roller Doctor Blade
Rotogravure Cylinder
Ink Fountain
Digital Printing Inkjet
Inkjet printing is the most popular today. There are a few ways inkjet printers apply ink onto paper. There is piezoelectric and bubble jet, which are both drop on demand printing methods, and there is continuous printing where the ink is in a constant flow from a reservoir of ink to the nozzle.
Dye Sublimation
Dye-sub printing is the closest we have to continuous tone printing in the digital world because no halftoning is needed. Using heat it transfers dyes onto a ribbon that then presses onto the paper. This method is often used in kiosks found in photo stores.
paper magenta cyan clear
yellow thermal head
carrier ribbon
Laser
Laser printers are best for text documents and not so much for photographs. They use toner that with an electrostatic charge is kept on top of the paper.
27
28
RIPs
Printers have their own language to speak, and because they don’t always speak the same language as computers or programs a translator needs to be involved. A RIP, Raster Image Processor, is a translator for printers. It translates vectors to raster images aka rasterizing, it is responsible for color management, making continuous tones into binary dots aka halftoning, and job queuing and nesting. Page Description Languages, PDLs, are within RIPs. An example of a PDL is PCL, which is Hewlett-Packards’ description language.
More familiar might be Post Script (PS) and PDFs. PDFs are by Adobe and are a simplified Post Script. Post Script is used to create vector images so it may have to be rasterized to be readable by a printer. A good way to think about RIPs for photographers is that a RIP is the equivalent to a raw file- it gives maximum control. So, you want a RIP to give that control with printing because the software driver that comes with the printer is like the equivalent of a JPEG- limited and automatic.
29
Mirrorless Cameras
31
Body Mirrorless Interchangeable Lens Cameras (MILC) come in two distinct styles: the mini DSLR and the Rangefinder.
Sensor The sensors do vary in size and we know the size of the sensor is important to image quality. Most are similar to or the same size as most popular entry-level DSLRs. The only Full Frame MILC is the Leica M9, because it is a Rangefinder (they’re all mirrorless). • Micro Four Thirds system: standard for MILC from Panasonic and Olympus. • Sony (NEX), Canon (EOS M), Fujifilm (X), Samsung (NX): use sensors closer to the size of APS-C, which are in popular cropped sensor DSLRs and is most commonly used. • The Nikon 1’s sensor is a format that is slightly larger than high-end compact cameras. The Pentax Q has the smallest one out in MILC and is close to ½ an inch.
32
Lenses and Autofocus Lenses are also important to image quality. One of the beauties of MILC is that they are closer to the sensor since there is no prism. Therefore, you can get a more accurate focus reading from the sensor. If the compactness of a MILC is most important to you then it is best to stick to the wider lenses. The longer you get in focal length the larger and bulkier the lenses get, naturally. The mounts for the lenses of every MILC is new, so existing lenses are incompatible without an adapter. That also means there are few native lenses out currently since that is a huge financial investment and cost.
MILC use contrast detection AF, so it racks ranges of distances and picks one that delivers the most contrast at the selected AF point. This is generally a slow process but newer models are becoming faster and producing great read-out times when taking single shots. This system isn’t optimal for scenes where the subject is in motion. In MILCs the autofocus speed of continuous shooting isn’t as fast as DSLRs, although they are getting there.
DSLRs use Phase Detection AF which produces an infocus image from one single measurement. This makes a good system for focus tracking aka moving subjects. The Nikon 1 is the one MILC that has successfully used a hybrid autofocus system of both contrast and phase detection systems.
33
Photographed with the Fuji X-E1
34
Viewfinder Due to the styles and compactness of most MILCs we lose the optical TTL viewfinder that is on normal DSLRs. MILC provide LCD Display, which is beneficial for arms-length shooting. Electronic Viewfinders (EVFs) are also featured on some MILC, mainly those in the style of Rangefinders. There is a slight lag from changes in the scene to EVF. However, newer ones have better response times and resolution. This definitely takes some getting used to for those who normally shoot with DSLRs.
Advantages • You can change lenses without a mirror reflex optical viewfinder making it more compact than a DSLR, but producing similar if not the same image quality. • Lightweight and compact (more so with wider lenses) • Good focus and resolution because it is closer to the sensor. • Image quality isn’t suffered much because most have APS-C sensors. • Good for those who work with interacting with people, you won’t have an intimidating camera • Good for carrying around on trips and not having to sacrifice image quality like you would for compact cameras. • Live-View so you don’t always have to keep camera to eye level. • Changeable lenses • Relatively fast autofocusing
Disadvantages • Not the greatest for motion/sports continuous shooting • The only full frame one available is the M9. However, there is rumor of Sony’s in 2014. • New mounts so there are few native lenses and need an adaptor to use current ones. • No TTL optical viewfinder. If there is one then there will be some misframing occurring
35
Note The content in this book is based on the course Imaging Systems taught by Nitin Sampat at Rochester Institute of Technology during the fall of 2013. All photographs are by Mariah Texidor. Graphics are sited below and some printing graphics are credited to Nicole Leclair. Sources: http://www.techradar.com/news/photography-video-capture/cameras/sensor-size-explained-1042035 http://en.wikipedia.org/wiki/Color_filter_array http://www.clipartsfree.net/large/3739-camera-clipart.html http://www.clipartsfree.net/large/20001-open-box-clipart.html http://www.clipartsfree.net/large/15185-computer-mouse-clipart.html http://www.iconarchive.com/show/outline-icons-by-designcontest/Eyedropper-icon.html http://images.gizmag.com/hero/mirorless-cameras.jpeg http://mimech.com/printers/electrophotography.gif http://www.xennia.com/xennia/uploads/cij-dod-diagram.gif http://www.refreshcartridges.co.uk/igloo/wp-content/uploads/2011/07/inksplat.gif http://clipartist.net/openclipart.org/2013/Mo/simple_things/Little_Robot_Black_White_Electronics_Sheet_ Page_Geometry.svg http://clipartist.net/openclipart.org/2013/Mo/simple_things/Colorful_Animal_Penguin_Geometry.svg http://colouringbook.org/social/clipartist.net/F/fotocamera_foto_camera_icon.svg http://colouringbook.org/social/black_white_line_art_colouringbook.org/P/palomaironique_bulb_black_white_ line_art_coloring_book_colouring.svg http://www.clipartsfree.net/ http://colouringbook.org/tag/scalable-vector-graphics/page/10/
37