Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).

workshops-275 Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).

There is a lot of confusion and misunderstanding about aliasing and moiré. So I’ve thrown together this article to try and explain whats going on and what you can (or can’t) do about it.

Sony’s FS700, F5 and F55 cameras all have 4K sensors. They also have the ability to shoot 4K raw as well as 2K raw when using Sony’s R5 raw recorder. The FS700 will also be able to shoot 2K raw to the Convergent Design Odyssey. At 4K these cameras have near zero aliasing, at 2K there is the risk of seeing noticeable amounts of aliasing.

One key concept to understand from the outset is that when you are working with raw the signal out of the camera comes more or less directly from the sensor. When shooting non raw then the output is derived from the full sensor plus a lot of extra very complex signal processing.

First of all lets look at what aliasing is and what causes it.

Aliasing shows up in images in different ways. One common effect is a rainbow of colours across a fine repeating pattern, this is called moiré. Another artefact could be lines and edges that are just a little off horizontal or vertical appearing to have stepped or jagged edges, sometimes referred to as “jaggies”.

But what causes this and why is there an issue at 2K but not at 4K with these cameras?

Lets imagine we are going to shoot a test pattern that looks like this:

F55-aliase-test-pattern Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).
Test pattern, checked shirt or other similar repeating pattern.

And lets assume we are using a bayer sensor such as the one in the FS700, F5 or F55 that has a pixel arrangement like this, although it’s worth noting that aliasing can occur with any type of sensor pattern or even a 3 chip design:

f55-bayer-samall Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).
Sensor with bayer pattern.

Now lets see what happens if we shoot our test pattern so that the stripes of the pattern line up with the pixels on the sensor. The top of the graphic below represents the pattern or image being filmed, the middle is the sensor pixels and the bottom is the output from the green pixels of the sensor:

f55-no-aliase Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).
Test pattern aligned with the sensor pixels.

As we can see each green pixel “see’s” either a white line or a black line and so the output is a series of black and white lines. Everything looks just fine… or is it? What happens if we move the test pattern or move the camera just a little bit. What if the camera is just a tiny bit wobbly on the tripod? What if this isn’t a test pattern but a striped or checked shirt and the person we are shooting is moving around? In the image below the pattern we are filming has been shifted to the left by a small amount.

f55-aliasin-all-grey Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).
Test pattern miss-aligned with pixels.

Now look at the output, it’s nothing but grey, the black and white pattern has gone. Why? Simply because the green pixels are now seeing half of a white bar and half of a black bar. Half white plus half black equals grey, so every pixel see’s grey. If we were to slowly pan the camera across this pattern then the output would alternate between black and white lines when the bars and pixels line up and grey when they don’t. This is aliasing at work. Imagine the shot is of a person with a checked shirt, as the person moves about the shirt will alternate between being patterned and being grey. As the shirt will be not be perfectly flat and even, different parts of the shirt will go in and out of sync with the pixels so some parts will be grey, some patterned, it will look blotchy. A similar thing will be happening with any colours, as the red and blue pixels will sometimes see the pattern at other times not, so the colours will flicker and produce strange patterns, this is the moiré that can look like a rainbow of colours.

So what can be done to stop this?

Well what’s done in the majority of professional level cameras is to add a special filter in front of the sensor called an optical low pass filter (OPLF). This filter works by significantly reducing the contrast of the image falling on the sensor at resolutions approaching the pixel resolution so that the scenario above cannot occur. Basically the image falling on the sensor is blurred a little so that the pixels can never see only black or only white. This way we won’t get flickering between black & white and then grey if there is any movement. The downside to this is that it does mean that some contrast and resolution will be lost, but this is better than having flickery jaggies and rainbow moiré. In effect the OLPF is a type of de-focusing filter (for the techies out there it is usually something called a birefringent filtre). The design of the OLPF is a trade off between how much aliasing is acceptable and how much resolution loss is acceptable. The OLPF cut-off isn’t instant, it’s a sharp but gradual cut-off that starts somewhere lower than the sensor resolution, so there is some undesirable but unavoidable contrast loss on fine details. The OLPF will be optimised for a specific pixel size and thus image resolution, but it’s a compromise. In a 4K camera the OLPF will start reducing the resolution/contrast before it gets to 4K.

(As an aside, this is one of the reasons why shooting with a 4K camera can result in better HD, because the OLPF in an HD camera cuts contrast as we approach HD, so the HD is never as sharp and contrasty as perhaps it could be. But shoot at 4K and down-convert and you can get sharper, higher contrast HD).

So that’s how we prevent aliasing, but what’s that got to do with 2K on the FS700, F5 and F55?

Well the problem is this, when shooting 2K raw or in the high speed raw modes Sony are reading out the sensor in a way that creates a larger “virtual” pixel. This almost certainly has to be done for the high speed modes to reduce the amount of data that needs to be transferred from the sensor and into the cameras processing and recording circuits when using high frame rates.  I don’t know exactly how Sony are doing this but it might be something like my sketch below:

F55-2K-bayer Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).
Using adjacent pixels to create larger virtual pixels.

So instead of reading individual pixels, Sony are probably reading groups of pixels together to create a 2K bayer image. This creates larger virtual pixels and in effect turns the sensor into a 2k sensor. It is probably done on the sensor during the read out process (possibly simply by addressing 4 pixels at the same time instead of just one) and this makes high speed continuous shooting possible without overheating or overload as there is far less data to read out.

But, now the standard OLPF which is designed around the small 4K isn’t really doing anything because in effect the new “virtual” pixels are now much larger than the original 4K pixels the OLPF was designed around. The standard OLPF cuts off at 4K so it isn’t having any effect at 2K, so a 2K resolution pattern can fall directly on our 2K virtual bayer pixels and you will get aliasing. (There’s a clue in the filter name: optical LOW PASS filter, so it will PASS any signals that are LOWer than the cut off. If the cut off is 4K, then 2K will be passed as this is lower than 4K, but as the sensor is now in effect a 2K sensor we now need a 2K cut off).

On the FS700 there isn’t (at the moment at least) a great deal you can do about this. But on the F5 and F55 cameras Sony have made the OLPF replaceable. By loosening one screw the 4K OLPF can be swapped out with a 2K OLPF in just a few seconds. The 2K OLPF will control aliasing at 2K and high speed and in addition it can be used if you want a softer look at 4K. The contrast/resolution reduction the filter introduces at 2K will give you a softer “creamier” look at 4K which might be nice for cosmetic, fashion, period drama or other similar shoots.

f55-olpf Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).
Replacing the OLPF on a Sony PMW-F5 or PMW-F55. Very simple.

Fs700 owners wanting to shoot 2K raw will have to look at adding a little bit of diffusion to their lenses. Perhaps a low contrast filter will help or a net or stocking over the front of the lens to add some diffusion will work to slightly soften the image and prevent aliasing. Maybe someone will bring out an OLPF that can be fitted between the lens and camera body, but for the time being to prevent aliasing on the FS700 you need to soften, defocus or blur the image a little to prevent the camera resolving detail above 2K. Maybe using a soft lens will work or just very slightly de-focussing the image.

But why don’t I get aliasing when I shoot HD?

Well all these cameras use the full 4K sensor when shooting HD. All the pixels are used and read out as individual pixels. This 4K signal is then de-bayered into a conventional 4K video signal (NOT bayer). This 4K (non raw) video will not have any significant aliasing as the OLPF is 4K and the derived video is 4K. Then this conventional video signal is electronically down-converted to HD. During the down conversion process an electronic low pass filter is then used to prevent aliasing and moiré in the newly created HD video. You can’t do this with raw sensor data as raw is BEFORE processing and derived directly from the sensor pixels, but you can do this with conventional video as the HD is derived from a fully processed 4K video signal.

I hope you understand this. It’s not an easy concept to understand and even harder to explain. I’d love your comments, especially anything that can add clarity to my explanation.

UPDATE: It has been pointed out that it should be possible to take the 4K bayer and from that use an image processor to produce a 2K anti-aliased raw signal.

The problem is that, yes in theory you can take a 4K signal from a bayer sensor into an image processor and from that create an anti-aliased 2K bayer signal. But the processing power needed to do this is incredible as we are looking at taking 16 bit linear sensor data and converting that to new 16 bit linear data. That means using DSP that has a massive bit depth with a big enough overhead to handle 16 bit in and 16 bit out. So as a minimum an extremely fast 24 bit DSP or possibly a 32 bit DSP working with 4K data in real time. This would have a big heat and power penalty and I suspect is completely impractical in a compact video camera like the F5/F55. This is rack-mount  high power work station territory at the moment. Anyone that’s edited 4K raw will know how processor intensive it is trying to manipulate 16 bit 4K data.

When shooting HD your taking 4K 16 bit linear sensor data, but your only generating 10 bit HD log or gamma encoded data, so the processing overhead is tiny in comparison and a 16 bit DSP can quite happily handle this, in fact you could use a 14 bit DSP by discarding the two LSB’s as these would have little effect on the final 10 bit HD.

For the high speed modes I doubt there is any way to read out every pixel from sensor continously without something overheating. It’s probably not the sensor that would overheat but the DSP. The FS700 can actually do 120fps at 4K for 4 seconds, so clearly the sensor can be read in it’s entirety at 4K and 120fps, but my guess is that something gets too hot for extended shooting times.

So this means that somehow the data throughput has to be reduced. The easiest way to do this is to read adjacent pixel groups. This can be done during the readout by addressing multiple pixels together (binning). If done as I suggest in my article this adds a small degree of antialising. Sony have stated many times that they do not pixel or line skip and that ever pixel is used at 2K, but either way, whether you line skip or pixel bin the effective pixel pitch increases and so you must also change the OLPF to match.

22 thoughts on “Aliasing when shooting 2K with a 4K sensor (FS700, PMW-F5, PMW-F55 and others).”

  1. Hello. At your helpful 5/55 workshop you were not sure of the pattern that 2k was going to use. You mentioned, if I understood, that there was a moire free way of making 2k from a 4k chip.

    Can you now confirm that this second method (that I did not understand or get the name of!) will not be used?

    Thanks

    1. Earlier in the year all I knew was that Sony were going to use the full sensor for 2K, no line or pixel skipping. So this suggested some form of pixel binning. This does appear to be what Sony are doing. While it does not suffer from the harsh aliasing effects that would be apparent from line or pixel skipping, the way it’s being done does not add as much anti-aliasing as I thought it might, hence the need for the 2K OLPF. I’m a little surprised at how much aliasing there is, I though there would be less than there is.
      The OLPF is not mandatory, for some scenes and occasional HFR shooting you may not need it, but having now seen examples of 2K both with and without the new OLPF, I have to say that I will definitely be getting one as I plan to do a lot of HFR filming. I don’t believe they will be too expensive. 4K with the 2K OLPF is also very pleasing when you want a smoother more filmic look, sometimes the 4K can be a little hard.

  2. Thank you for this very good explanation!
    I’ve gotten two questions:
    So if i understand you correctly, there will not be be any aliasing if recording at high speed up to 240 fps, because the camera will still derive the full HD footage from the fully processed 4K video signal? So the difference between the footage recieved from 120 fps and 240 fps is only a result of the bigger downconversion of the 4K signal?(btw: the with in “and this makes high speed continuous shooting possible with overheating or overload” should be a without, i think.)

    1. No, there will be aliasing whenever you go above 60fps as all the high speed modes are either 2K or HD and above 60fps the HD will be derived from the 2K sensor readout. The FS700 can do 120fps at 4K for 4 seconds, this might be aliase free, but I’d need to test this.

      Thanks for picking up on the “without”.

  3. Thanks for your fast answer.
    I’ve got two more questions:

    Is it possible to record RAW Full hd with the s-log upgrade?
    If not not which settings do you recommend to get a result thats good for editing in post production?

  4. I assume your referring to the FS700? I’m not sure what you mean by “raw full HD”.
    Raw is raw sensor data. The modes you get (via the external R5 recorder) are either 4K raw or 2K raw. When converting from raw sensor data to baseband video there the final resolution is typically between 70 and 80% or the raw pixel count. So 4K raw results in an image that is 4096 x 2160 pixels but the resolution is only around 3K. 2K raw results in an image that is 2048 x 1080 with a resolution of about 1.5K so sub HD.
    But all of this is only via an external raw recorder.
    Internally all you get is the same HD modes as before but now with the addition of the SLog2 gamma curve in the picture profiles. This is there to provide a way to monitor the cameras full dynamic range when shooting raw to an external recorder. The internal 8 bit recordings don’t really have enough grey shades for SLog2 to be effective. As the normal HDSDI output is also only 8 bit a conventional external recorder won’t help much with Slog2 either. I’d stick with the Cinegammas for the best results.

  5. Hi, Alister! When I read about less quality of 2K recording from FS700 I asking Sony to give as AN OPTION: to read just a central PART of the sensor, as RED camera did it. Here is “my Idea” in pictures:
    http://www.sergecinema.com/temp/FS700_Sensor.jpg
    as you see I asking an option to record in 2,35:1 Aspect Ratio with size about 17 X 7.23mm .. This will be about “3K resolution” Crop, and seems to be even better, than 2K in RED Scarlet 🙂 –> : http://www.sergecinema.com/temp/FS700_Sensor_b.jpg 🙂
    maybe this is not the BEST way to shoot Slow Motion, but I would like to TRY IT 🙂
    I was place it here a long time ago: http://community.sony.com/t5/FS700/Great-presentation-from-Art-Adams-and-Adam-Wilt/m-p/146061#U146061 ..but no one reply me .. 🙂 I think Sony should at list to look to it and thinking about this as ADDITIONAL MODE in FS-700 (.. and maybe in F5/F55 cameras.. )

    1. The F5 and F55 are getting a 2K center crop mode in December. This also allows you to use Super16 lenses or B4 lenses with the MTF 1.35x FZ to B4 adapter. Of course you can also use existing lenses too.

      Whether they will add this to the FS700 too, I do not know. It is a feature I requested when I received one of the very first pre-production FS700 for testing. It’s an obvious function.

  6. What size if Image Circle for Super16 lenses is ? I’d like to see what size of Crop will be available in this Mode… about FS-700 – I think Sony have to make it possible for FS700 as for F5, F55 cameras.. if Sony want us to be able to use FS700 as BACK UP camera for much more expensive F5, F55 🙂 Not so many people are able to buy TWO F5, and a specially TWO F55 cameras, but for security reason we still need to have second camera on shoot, don’t you think so? 🙂

  7. >>” This 4K (non raw) video will not have any significant aliasing as the OLPF is 4K and the derived video is 4K. Then this conventional video signal is electronically down-converted to HD. During the down conversion process an electronic low pass filter is then used to prevent aliasing and moiré in the newly created HD video.”<<

    Would this down conversion with the electronic low pass filter be equally true for 2k recorded internally in XAVC? The post always states that downconversion to 2k creates the aliasing, but the logic seems to only apply to 2k Raw.

    So are we only talking about using the OLPF for 2k Raw shooting, or is does it give the same benefit for 2k XAVC recording?

    Thanks!

  8. I would be very interested to know what’s the answer to: “So are we only talking about using the OLPF for 2k Raw shooting, or is does it give the same benefit for 2k XAVC recording?”

    Shooting after tomorrow XAVC 180 fps with a F5

    Thanks

  9. In layman terms – F55 options for shooting 2K RAW
    F55 comes with the OLPF for 4K sensor, so can either film 2K RAW with some aliasing, or centre crop (which will not give me aliasing?) and give my lenss 2x magnification which is grate at times.
    OR I can buy the 2K OLPF for roughly $800 and get a cleaner picture, and should I want 4K then produce a slightly softer film look (but different from the rest of my 2K)
    What I really want to ask is, would the 2K RAW picture using the 4K OLPF give obvious aliasing if delivering in HD (purely wildlife doc)
    Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *