I didn’t really understand how phone cameras work, so I thought I would investigate to find out and share my finding with you in this article.
HOW DO PHONE CAMERAS WORK?
Phone cameras are constructed from several components that all work together to enable us to take high-quality digital pictures. These components form the Image Processing Pipeline. The components consist of:
- Camera Lens
- Image Sensor
- Image Signal Processor
- Camera Control Software
- Post Processing Software
Why Do Phone Cameras Need A Lens?
Phone cameras need a lens to enable light from the subject you are photographing to be focused on the Image Sensor. The camera lens is normally made out of multiple glass elements which are needed to reduce image aberration, things like:
- Spherical Aberration – distortions caused by the curvature of lenses
- Coma – distortions caused by imperfections in lenses
- Astigmatism – distortion caused by curves in a subject hitting a flat sensor surface
- Field curvature – distortion in the sharpness of the subject
- Geometric distortion – distortion in representing three-dimension subject in a two-dimensional image
- Chromatic aberration – distortion in parts of the image due to refraction
What is Aperture?
Aperture is how much light a lens lets through to the image sensor, it’s measured in f-stops, f/1.8, f/2.0 and so on. The smaller the number the bigger the aperture and the more light gets in, so if you want good low-light performance without a flash, you want the smallest aperture possible.
Several manufacturers now produce smartphone cameras with wide apertures down to f/1.7 or f/1.8 which means they are much better in low light situations.
On smartphones that have optical image stabilization (OIS), the lens with be mounted in a rack that can move to compensate for shaking when using the camera on the move or making a video. OIS can greatly increase the chances of you getting a sharp image or stable video footage.
What Does The Image Sensor Do?
In traditional DSLR cameras, the time the sensor is exposed to light (exposure) is controlled by opening and closing a mirror over the sensor. When you press the shutter button the mirror would quickly flick open and then closed again to prevent any more light hitting the image sensor.
In digital mirrorless cameras, a digital shutter or a curtain is used to control how much light hits the image sensor. A smartphone camera has neither of these, and the exposure is controlled by activating the sensor for the time of the exposure and then deactivating it again.
What Are Megapixels?
A megapixel (MP) is 1,048,576 or just over one million pixels. In the first iPhone, the camera sensor was capable of capturing 2MP. That is over 2 million pixels, but things have changed considerably since 2007 when the first iPhone was introduced. The word pixel is made up of the words picture and element
The resolution of the image created by a smartphone camera sensor depends on the density of pixels on the sensor. Some models of smartphones have sensors with up to 40 million pixels (40 Megapixels). You can imagine that the pixels will be tiny, each pixel less than 1 micron in size. These sensors produce very high-resolution images, higher than many DSLR’s. The drawback with packing this many pixels onto the image sensor is that the pixels are tiny and not very sensitive in low light situations. Recently manufacturers have been producing 12-megapixel image sensors that have larger pixels and much better low light capabilities.
The image sensor acts as an analog to digital converter, taking light and converting it into raw bits, the raw bits are then sent to the Image Signal Processor.
What is ISO?
In the old days of cellulose camera film, you would choose the speed of the film based on its ISO, depending if you were taking pictures on a bright sunny day or a darker dingy day. ISO 100 for bright days and ISO 400 or higher for darker conditions.
We still have this concept in smartphone cameras and changing the ISO changes the sensitivity of the sensitivity ‘gain’ on the sensor. the same rules apply as before on a sunny day we could use ISO 100 and we could increase it for darker conditions. However, there is a penalty for increasing the ISO. The higher the ISO the more grainy the results. This was the same back in the days of cellulose film, as bigger light sensitive crystals were used on higher ISO film, and the bigger crystals produce grainy photographs.
Does Size Matter?
DSLR cameras usually have large image sensors sizes including; APSC and Full Frame. A full frame sensor is the equivalent size of the old 35mm film when you would get 36 shots per roll. Smartphone sensors are just a fraction of the size of a full-frame sensor, less than 6mm width diagonally. This is due to all the other technology that needs to be squeezed into the smartphone body, because of its small size the smartphone camera has limitations compared to larger camera sensors.
Smartphone manufacturers are continually improving their camera technology and it’s really only a matter of time before they start taking on the larger sensor camera manufacturers.
What is the Image Signal Processor and What Does It Do?
The image signal processor is part of the micro chip that contains the sensor, the sensor area in the outside surface of the chip while the Image Signal Processor is a dedicated chip underneath the sensor.
The Image Signal Processor is part of the Image Processing Pipeline on a smartphone camera. After the image sensor converts light into a digital signal the image signal processor then takes the digital stream of bits and processes it into a picture. During the data processing the data typically goes through several steps to ensure the image results are usable:
- Pixel Correction – removing defects due to defective pixels
- Lens Shading Correction – fixing vignetting caused by the lens
- White Balance Correction – fixing the color temperature of the image
- Denoise – remove noise (grain) from the image
- Sharpen – sharpen the image
In the latest smartphones, there are most likely several other proprietary stages, but you get the idea what the image signal processor does. It’s this pipeline that gives images from different sensor manufacturers a particular look and feel on the processed pictures.
What Is Camera Control Software?
When you are about to take a picture with your smartphone camera, you will notice on the camera screen that there are sever icons that will allow you to change setting on the camera before you take the picture. You will often see things like:
- Option to choose between photograph or video
- Change the focusing options, from automatic focusing to selective focusing.
- Select the front or Rear Camera
- Change the shutter speed
- Change the ISO
- Take a Panorama
- Shoot in RAW
All of these features are part of the camera control software, where you can take control of nearly all of the aspects of the camera settings. You can use all of the settings in auto mode or you can start using your camera for more advanced photography techniques like:
- Long exposure
- Using depth-of-field to control the focus area of the image
- Capturing fast-moving subjects
What is Post Processing Software?
You may have already noticed that when you have taken a picture with your smartphone camera, you will get options to manipulate the image, this is called post-production. Typically, you can:
- Sharpen the picture
- Change the Exposure to make it lighter or darker
- Change color saturation, add or remove color depth
- Crop the picture
- Add Text or Stickers
Some smartphone manufacturers offer more features than others, and you can also get apps that add a lot more features for post-processing. One of the main apps that I use is called Adobe Lightroom. It offers a much wider range of post-processing features than my Galaxy S8 Plus.
Post processing is required when you take your pictures in RAW format. RAW captures the image with a much wider range of detail than you will get from a JPEG image, but the image will look very flat when it comes out of the camera. It is necessary to apply post-production to a RAW image to bring it to its full glory. This is how most professional photographers take pictures, it gives them much more control over how the final image will look. The post-production of JPEG pictures is quite limited compared to RAW. I will be producing an article all about RAW so keep a look out for it.
Conclusion
This article hwe have looked at how phone cameras work. In the first section, I have identified the main components that form the image processing pipeline and I have described what each component does.
I have explained the purpose of the smartphone camera lens and described how it is constructed from multiple glass elements to try and reduce distortion in the final picture. Aperture was explained and how having a small f-stop number produces a wider aperture.
In the next section I discussed the image sensor, I described how the image sensor converted light from the lens into a digital signal that is sent to the image signal processor. I explained that image sensors can have different amounts of pixels and went to mention the advantages and drawbacks of having more and fewer pixels on the image sensor.
I then looked at how the image signal processor receives digital data from the image sensor and processes the sensor data to remove distortions and clean up the image.
Finally, we looked at software for both camera control and post-processing. I explained what post-processing was and that it is often used to improve the image and that it is often used in conjunction with the RAW image type. The last section clarified what camera controls are and how they are used to change settings before you take a picture.
I hope you enjoyed, the article on; how do phone cameras work? Like me, I hope you learned something new about how your phone camera works. I have several other articles related to smartphone photography so go give them a browse when you have time.
