How Do Image Sensors Work? Inside a Phone Camera Sensor

Where you see it: "electronic eyes" everywhere

Behind the round window on the back of your phone, there is a lens. Behind the lens sits a small chip. This chip is the image sensor. The lens gathers light and forms an image, and the image sensor turns that image into an electrical signal.

Image sensors are used in more than phones and digital cameras. They are in the cameras that watch around a car, security cameras, video doorbells and laptop webcams. In factories, inspection cameras use them to spot scratches on products moving down the line.

How it works 1: particles of light free up electrons

Light behaves as a stream of tiny packets of energy. Each packet is called a photon. To keep things simple, we will describe light here as "a stream of particles." Brighter light means more particles arrive each second.

In a silicon crystal, electrons are normally busy holding the atoms together, so they cannot move freely. When a particle of light hits one, the electron takes its energy and becomes free to move. This effect, where light frees electrons inside a material, is called the internal photoelectric effect. The electron also leaves a "hole" behind, but here we will follow only the electrons.

Each material needs a set amount of energy to free an electron. This is the "step" in "What Is a Semiconductor?" Particles of visible light carry enough energy to climb this step in silicon. That is why silicon is good at turning visible light into electrons.

The part that catches these freed electrons and stores them is the photodiode. Where two different kinds of silicon meet, an electric force forms that pulls electrons to one place. This force gathers the electrons made by light into a small "pool."

The time during which a pixel takes in light and stores electrons is called the exposure. During the exposure, electrons build up in the photodiode in step with the light that arrives. In a dark place, a longer exposure gives a brighter picture. That is because the electrons have more time to build up.

How it works 2: turning electrons into numbers

The stored electrons are not a photo yet. Here we describe the type used in phones, called CMOS (explained later). A small circuit inside each pixel turns the amount of electrons into a voltage. Voltage is the strength of the push on electricity, and it is easy to measure. The more electrons there are, the more the voltage changes.

Next, this voltage is turned into a whole number. Turning a smoothly changing voltage into a number in steps is called AD conversion (analog-to-digital conversion). For example, the result might be "some whole number from 0 to 1,023." This way, the brightness of each pixel becomes a number.

The numbers from tens of millions of pixels, laid out in the same order as the pixels, are the raw data of a photo. The phone then works on this table of numbers. It calculates colors and reduces graininess, among other steps, to produce the photo you see.

How it works 3: how does a sensor tell colors apart?

A photodiode can only tell how much light arrived. Red light and blue light free electrons in the same way. So a small colored film is placed over each pixel. It lets through only light of one set color. This film is called a color filter.

The most widely used layout of these filters is the Bayer pattern. It takes 2×2 pixels as one set: two green, one red and one blue. This set repeats across and down the sensor. The layout was devised by Bryce Bayer of Kodak and was patented in the US in 1976. Green appears twice because the human eye is most sensitive to green light. Brightness and fine detail are read mainly from the green pixels.

With this layout, each pixel measures the brightness of only one color. A red pixel, for example, has no green or blue value. So the two missing colors are estimated from the green and blue pixels around it. This calculation is called demosaicing (color interpolation).

Because it is an estimate, it sometimes goes wrong. When you photograph clothes with fine stripes, colored fringes or patterns that are not really there may appear. A camera's image processing reduces such errors, for example by checking which way the edges run.

How it works 4: catching as much light as possible

Not all of a pixel's area can catch light. A pixel also holds transistors and wiring that read out the electrons. So a tiny convex lens is placed over each pixel to focus light onto the part that senses it. This is called a microlens.

The way the chip is built also matters. A normal chip has transistors made on the surface of a silicon wafer, with many layers of metal wiring stacked on top ("How Are Chips Made?"). The older design, which lets light in from this front side, is called front-side illumination. Here, light must pass through gaps in the wiring, and the wiring blocks some of it.

So a new design appeared. The chip is flipped over, and the wafer is thinned to a few µm. (A micrometer, µm, is one-thousandth of a millimeter.) Light then enters from the back, where there is no wiring. This is called backside illumination (BSI). Sony began mass-producing CMOS image sensors of this type in 2009. They first went into its Handycam video cameras. At the time, the company said they had about twice the sensitivity of earlier products.

In a back-illuminated sensor, a support wafer is bonded to the back to hold up the thinned chip. Replace this support with a chip of signal-processing circuits, and you get a stacked sensor. Sony announced its development in January 2012 and began shipping the first product that October. Placing the pixels and the circuits one above the other makes the chip smaller and leaves room for more circuits.

A table of the differences between the three designs is in "FSI vs BSI vs Stacked Image Sensors."

CMOS and CCD: two ways to read out

There are two types of image sensor, which differ in how they read out the electrons. One is the CCD. It passes each pixel's electrons from neighbor to neighbor, like a bucket brigade. A single circuit at the exit turns them all into voltages. The CCD was invented at Bell Labs in the US in 1969. Its two inventors received the 2009 Nobel Prize in Physics.

The other is the CMOS image sensor. CMOS is the name of a way of building transistor circuits, the same kind used in CPUs ("What Is a Transistor?"). In this type, each pixel turns its electrons into a voltage itself. Circuits for each column then read the pixels out in parallel.

CMOS sensors read out quickly and use little power. Circuits for AD conversion and image processing can also sit on the same chip or on a stacked chip. In the past, though, CCDs had less noise, the random variation that makes photos look grainy. So CCDs were chosen where image quality mattered most.

Later, CMOS sensors were improved to reduce noise. Today, almost all sensors in phones and digital cameras are CMOS. CCDs are still used in some areas, such as astronomy and scientific measurement. A table of the differences is in "CMOS vs CCD Image Sensors."

Why bigger pixels do better in the dark

Take many shots of a scene with the same brightness. The number of electrons in one pixel changes a little every time. That is because particles of light arrive at random moments, like raindrops. This random variation is called shot noise. It is one of the main reasons dark photos look grainy.

The size of this variation follows a fixed rule. If a pixel collects 100 electrons on average, the variation is about ±10 electrons, or 10% of the average. If the average is 10,000, the variation is about ±100, which is only 1%. In math terms, a rough measure of the variation is the square root of the average count. (The square root is the number that gives that count when multiplied by itself.)

So the more particles of light one pixel collects, the smaller the variation as a share, and the smoother the picture. Bigger pixels, and bigger sensors overall, can collect more light. Some phone night modes take many shots and combine them. This, too, is a way to collect more light. To keep things simple, we leave out other noise, such as noise added by the circuits.

What does "50 megapixels" count?

In "50MP," MP stands for megapixels, and one megapixel is one million pixels. So 50MP means the sensor has about 50 million pixels. For a frame with a 4:3 width-to-height ratio, that works out to about 8,200 pixels across and about 6,100 down.

Fit more pixels onto a sensor of the same size, and each pixel gets smaller. Each pixel then gets less light, so the variation from the last section grows as a share. More pixels can capture finer detail. But in low light, they can be a disadvantage.

So some high-resolution phone sensors put the same color filter on four neighboring pixels. In dark scenes, the signals from these four pixels are combined and treated as one big pixel. Combining pixels like this is called binning.

For example, Apple's 2022 iPhone 14 Pro uses a 48-megapixel sensor. It combines four pixels into one, so its photos are normally 12 megapixels. With some settings, it can also shoot at the full 48 megapixels without combining.

Readout order and distortion

Many CMOS image sensors read out the pixels one row at a time, starting from the top. To match this, the start and end of the exposure also shift slightly from row to row. This method is called a rolling shutter. So a fast train or a spinning propeller can come out slanted or bent.

There is also a method where all pixels are exposed at the same moment. Each pixel briefly holds its electrons inside itself before they are read out. This is called a global shutter. Moving objects do not get distorted, but each pixel needs more circuitry for the holding part. Global shutters are used in places such as factory inspection cameras. In 2023, Sony announced the α9 III, a camera with a sensor of this type.

Companies that make image sensors

The major makers of image sensors include Sony, Samsung Electronics and OmniVision. Within the Sony group, Sony Semiconductor Solutions makes them. For phone sensors, Sony was number one in 2024, according to the research firms TechInsights and Counterpoint.

According to Sony, it was the first in the world to mass-produce both of the designs described earlier: back-illuminated and stacked sensors. In an article it published in 2022, the company said that about 90% of its image sensors are stacked.

Common misconceptions

By the numbers

Counting only shot noise, combining 4 pixels gives 4 times the light, but the variation as a share only halves. That is because the square root of 4 is 2. It also explains why tricks for collecting more light help only a little at a time.

Learn more

• See the differences in a table: CMOS vs CCD image sensors, FSI vs BSI vs stacked image sensors, Rolling vs global shutter
• Terms for the next step: quantum efficiency (the share of light particles that become electrons), dark current (electrons that appear even with no light).
• Further: full-well capacity (the most electrons one pixel can hold), correlated double sampling (a way to cancel readout noise).
• Try the numbers: with the band gap and wavelength calculator, you can check the longest wavelength of light that silicon can turn into electrons (about 1,100 nm). The image, audio and video data size calculator works out a photo's data size from its pixel count.
• Start from the basics: "What Is a Semiconductor?", "What Is a Transistor?"