Tag Archives: bayer

Technology

Pixels should not be confused for resolution.

12 Comments

Let me start with the definition of “resolution” as given by the Oxford English Dictionary:

“The smallest interval measurable by a telescope or other scientific instrument; the resolving power.
  • the degree of detail visible in a photographic or television image.”
     

OK, so that seems clear enough – measurable or visible degree of detail.

Expanding that a little further when we talk about the resolution of an image file such as a Jpeg, TIFF etc, or perhaps RGB or YCbCr* video frame, if we have a 4K image that will normally mean a 4K pixel wide image. It will have 4K wide of red, 4K wide of blue and 4K wide of green, three lots of 4K stacked on top of each other so it is capable of containing any colour or combination of colours at 4K of points or pixels across, in effect a 4K wide image will have 12K of values across the image.

Now we know what resolution means and how it is normally used when describing an image what does it mean when we say a camera has an 8K sensor? Generally this statement means that there will be 8K of pixels across the sensor. In the case of a single sensor that is used to make a colour image some of these pixels will be for Red, some for green and some for blue (or some other arrangement of a mix of colour and clear pixels).  But does this also mean that 8K sensor will be able to resolve a 8K of measurable of visible detail – no, it does not.



Typically a single sensor that uses a colour filter array (CFA) won’t be able to resolve fine details and textures anywhere close to the number of horizontal pixels. So, to say that a camera with a single 8K or 4K colour sensor is a camera that can resolve an 8K or 4K image will almost certainly be a lie. 

Would it be correct to call that 4K colour sensor a 4K resolution sensor? In my opinion no – it is not correct because if we use a bayer sensor as an example then it will only actually have 2K of green, 1K of red and 1K of blue pixels on any one row. If we compare that to a 4K image such as a Jpeg then the Jpeg image will be made up of 4K wide of green, 4K  wide of red, 4K wide of blue pixels. It has the ability to resolve any colour or combination of colours with 4K precision. Meanwhile that 4K bayer sensor can not, it simply doesn’t have sufficient pixels to sample each colour at 4K, in fact it doesn’t even get close.

Clever image processing can take the output from a 4K bayer sensor and use data from the differing pixels to calculate, estimate or guess what the brightness and colours are at each point across the whole sensor and the actual measurable luminance resolution will typically come out at around 0.7x the pixel count, the chroma resolution will be even lower.  So if we use the dictionary definition of resolution and the measured or visible details a 4K bayer sensor can resolve we can expect a camera with a 4K pixel across bayer sensor to have a resolution of around 2.8K. Your 4k camera is unlikely to actually be able to create an image that can truly be said to be 4k resolution.

But the camera manufacturers don’t care about this. They want you to believe that your 4K camera is a 4K resolution camera. While most are honest enough not to claim that the camera can resolve 4K they are also perfectly happy to let everyone assume that this is what the camera can do. It is also fair to say the most 4K bayer cameras perform similarly, so your 4K camera will resolve broadly similarly to every other 4K bayer camera and it will be much higher resolution than most HD cameras. But can it resolve 4K, no it can not.

The inconvenient truth that bayer sensor don’t resolve anywhere near the pixel count is why we see 6K or 8K sensors becoming more and more popular as these sensors can deliver visibly sharper, more detailed 4K footage than a camera with a 4K bayer sensor can.  In a 4K project the use of an 8K camera will deliver 4K luma and chroma resolution that is not far behind and as a result your 4K film will tend to have finer and more true to life textures. Of course all of this is subject to other other factors such as lens choices and how the signal from the camera is processed, but with like for like an 8K pixel camera can bring real, tangible benefits for a lot of 4K projects compared to a 4K pixel camera.  

At the same time we are seeing the emergence of alternative colour filter patterns to the tried and trusted bayer pattern. Perhaps adding white (or clear) pixels for greater sensitivity, perhaps arranging the pixels in novel and different ways. This muddies the water still further as you shouldn’t directly compare sensors with different colour filter arrays based on the specification sheet alone. When you start adding more alternately coloured pixels into the array you force the spacing between each individual colour or luma sample to increase. So, you can add more pixels but might not actually gain extra resolution, in fact the resolution might actually go down. As a result 12K of one pattern type cannot be assumed to be better than 8K of another type and vice versa. It is only through empirical testing that you can be sure of what any particular CFA layout can actually deliver. It is unsafe to simply rely on a specification sheet that simply quotes the number of pixels. And it is almost unheard of for camera manufacturers to actually publish verifiable resolution tests these days…….   ….. I wonder why that is?


* YCbCr video or component video can be recorded in a number of ways. A full 4:4:4  4K YCbCr image will have 4K of Y (luma or brightness), a full 4K of the chroma difference blue and a full 4K of chroma difference Red. The chroma difference values are a more efficient way to encode the colour data so the data takes less room but just like RGB etc there are 3 samples for each pixel within the image. Within a post production workflow if you work in YCbCr the image will normally be processed and handled as 4:4:4.

For further space savings many YCbCr systems can if desired subsample the chroma, this is when we see terms such as 4:2:2. The first digit is the luma and the 4 implies every pixel has a discrete value.  In 4:2:2 the 2:2 means that the chroma values are interleaved, every other pixel on every other line, so the chroma resolution is halved, this saves space. This is generally transparent to the viewer as our eyes have lower chroma resolution than luma.

But it is important to understand the 4:2:2 and 4:2:0 etc are normally only used for recording systems in cameras etc where saving storage space is considered paramount or in broadcasting and distribution systems and codecs where reducing the bandwidth required can be necessary. SDI and HDMI signals are typically passed as 4:2:2. The rest of the time YCbCr is normally 4:4:4. If we do compare 4K  4:2:2 YCbCr which is 4K x 2k x 2K to a 4K Bayer sensor which has 2K G, 1K R, 1K B it should be obvious that even after processing and reconstruction the image derived from a 4K bayer sensor won’t match or exceed the luma and chroma resolutions that can be passed via 4:2:2 SDI or recorded by a 4:2:2 codec. What you really want is a 6K or better still an 8K bayer sensor.

Uncategorized

When is 4:4:4 not really 4:4:4.

2 Comments

The new Sony F3 will be landing in end users hands very soon. One of the cameras upgrade options is a 4:4:4 RGB output, but is it really 4:4:4 or is it something else?

4:4:4 should mean no chroma sub-sampling, so the same amount of samples for the R, G and B channels. This would be quite easy to get with a 3 chip camera as each of the 3 chips has the same number of pixels, but what about a bayer sensor as used on the F3 and other bayer cameras too for that matter?

If the sensor is subsampling the aerial image B and R compared to G (Bayer matrix, 2x G samples for each R and B) then no matter how you interpolate those samples, the B and R are still sub sampled and data is missing. Potentially depending on the resolution of the sensor even the G may be sub sampled compared to the frame size. In my mind a true 4:4:4 system means one pixel sample for each colour at every point within the image. So for 2k that’s 2k R, 2K G and 2K B. For a Bayer sensor that would imply a sensor with twice as many horizontal and vertical pixels as the desired resolution or a 3 chip design with a pixel for each sample on each of the R,G and B sensors. It appears that the F3’s sensor has nowhere near this number of pixels, rumour has it at around 2.5k x 1.5k.

If it’s anything less than 1 pixel per colour sample, while the signal coming down the cable may have an even number of RGB data streams the data streams won’t contain even amounts of picture information for each colour, the resolution of the B and R channels will be lower than the Green, so while the signal might be 4:4:4, the system is not truly 4:4:4. Up-converting the 4:2:2 output from a camera to 4:4:4 does not make it a 4:4:4 camera. This is no different to the situation seen with some cameras with 10 bit HDSDI outputs that only contain 8 bits of data. It might be a 10 bit stream, but the data is only 8 bit. It’s like a TV station transmitting an SD TV show on an HD channel. The channel might call itself an HD channel, but the content is still SD even if it has been upscaled to fill in all the missing bits.

Now don’t get me wrong, I’m not saying that there won’t be advantages to getting the 4:4:4 output option. By reading as much information as possible from the sensor, prior to compression there should be an improvement over the 4:2:2 HDSDi output, but it won’t be the same as the 4:4:4 output from an F35 where there is a pixel for every colour sample, but then the price of the F3 isn’t the same as the F35 either!

Uncategorized

When is 4k really 4k, Bayer Sensors and resolution.

8 Comments

When is 4k really 4k, Bayer Sensors and resolution.

First lets clarify a couple of term. Resolution can be expressed two ways. It can be expressed as pixel resolution, ie how many individual pixels are there on the sensor. Or as TV lines or TVL/ph, or how many individual lines can I see. If you point a camera at a resolution chart, what you talking about is at what point can I no longer discern one black line from the next. TVL/ph is also the resolution normalised for the picture height, so aspect ratio does not confuse the equation. TVL/ph is a measure of the actual resolution of the camera system.  With video cameras TVL/ph is the normally quoted term, while  pixel resolution or pixel count is often quoted for film replacement cameras. I believe the TVL/ph term to be prefferable as it is a true measure of the visible resolution of the camera.
The term 4k started in film with the use af 4k digital intermediate files for post production and compositing. The exposed film is scanned using a single row scanner that is 4,096 pixels wide. Each line of the film is scanned 3 times, once each through a red, green and blue filter, so each line is made up of three 4K pixel scans, a total of just under 12k per line. Then the next line is scanned in the same manner all the way to the bottom of the frame. For a 35mm 1.33 aspect ratio film frame (4×3) that equates to roughly 4K x 3K. So the end result is that each 35mm film frame is sampled using 3 (RGB) x 4k x 3k, or 36 million samples. That is what 4k originally meant, a 4k x 3k x3 intermediate file.
Putting that into Red One perspective, it has a sensor with 8 Million pixels, so the highest possible sample size would be 8 million samples. Red Epic 13.8 million. But it doesn’t stop there because Red (like the F3) use a Bayer sensor where the pixels have to sample the 3 primary colours. As the human eye is most sensitive to resolution in the middle of the colour spectrum, twice as many of these pixel are used for green compared to red and blue. So you have an array made up of blocks of 4 pixels, BG above GR.
Now all video cameras (at least all correctly designed ones) include a low pass filter in the optical path, right in front of the sensor. This is there to prevent moire that would be created by the fixed pattern of the pixels or samples. To work correctly and completely eliminate moire and aliasing you have to reduce the resolution of the image falling on the sensor below that of the pixel sample rate. You don’t want fine details that the sensor cannot resolve falling on to the sensor, because the missing picture information will create strange patterns called moire and aliasing.
It is impossible to produce an Optical Low Pass Filter that has an instant cut off point and we don’t want any picture detail that cannot be resolved falling on the sensor, so the filter cut-off must start below the sensor resolution. Next we have to consider that a 4k bayer sensor is in effect a 2K horizontal pixel Green sensor combined with a 1K Red and 1K Blue sensor, so where do you put the low pass cut-off? As information from the four pixels in the bayer patter is interpolated, left/right/up/down there is some room to have the low pass cut off above the 2k pixel of the green channel but this can lead to problems when shooting objects that contain lots of primary colours.  If you set the low pass filter to satisfy the Green channel you will get strong aliasing in the R and B channels. If you put it so there would be no aliasing in the R and B channels the image would be very soft indeed. So camera manufacturers will put the low pass cut-off somewhere between the two leading to trade offs in resolution and aliasing. This is why with bayer cameras you often see those little coloured blue and red sparkles around edges in highly saturated parts of the image. It’s aliasing in the R and B channels. This problem is governed by the laws of physics and optics and there is very little that the camera manufacturers can do about it.
In the real world this means that a 4k bayer sensor cannot resolve more than about 1.5k to 1.8k TVL/ph without serious aliasing issues. Compare this with a 3 chip design with separate RGB sensors. With a three 1920×1080 pixel sensors, even with a sharp cut-off  low pass filter to eliminate any aliasing in all the channels you should still get at 1k TVL/ph. That’s one reason why bayer sensors despite being around since the 70s and being cheaper to manufacture than 3 chip designs (with their own issues created by big thick prisms) have struggled to make serious inroads into professional equipment. This is starting to change now as it becomes cheaper to make high quality, high pixel count sensors allowing you to add ever more pixels to get higher resolution, like the F35 with it’s (non bayer) 14.4 million pixels.
This is a simplified look at whats going on with these sensors, but it highlights the fact that 4k does not mean 4k, in fact it doesn’t even mean 2k TVL/ph, the laws of physics prevent that. In reality even the very best 4k pixels bayer sensor should NOT be resolving more than 1.8k TVL/ph. If it is it will have serious aliasing issues.
After all that, those that I have not lost yet are probably thinking: well hang on a minute, what about that film scan, why doesn’t that alias as there is no low pass filter there? Well two things are going on. One is that the dynamic structure of all those particles used to create a film image, which is different from frame to frame reduces the fixed pattern effects of the sampling, which causes the aliasing to be totally different from frame to frame so it is far less noticeable. The other is that those particles are of a finite size so the film itself acts as the low pass filter, because it’s resolution is typically lower than that of the 4k scanner.