Following on from the ongoing discussions about the way the noise reduction on the PXW-FS5 works I’d like to try to answer a few points that have been raised.
Noise reduction is incorporated in to almost every single video camera on the market today. Without it cameras would not be achieving the sensitivity levels that we are becoming accustomed to. Big sensors have helped increase sensitivity, but noise reduction also plays a major part. 4 years ago the typical sensitivity of a video camera was around the equivalent of 300 ISO. Today 1000 ISO is common. The laws of physics have not changed but image processing has.
However image processing has limitations. Noise reduction is a trade off as it typically introduces other artefacts including blurring or softening of the image, smearing of the image when there is motion, posterisation, blockiness or banding.
The more noise you introduce into the image the harder the NR has to work and as a result the worse, or more noticeable the artefacts will become.
Every time you add 6dB of gain (or double the ISO) of a sensor you will also double the underlying noise level, that is a very significant noise increase.
Different sensors will exhibit different noise characteristics. A sensor with a very high pixel count may exhibit finer grain, but may be less sensitive, have aliasing issues, skew issues or heat problems. A sensor with fewer pixels may have coarser grain but be more sensitive have fewer aliasing issues, fewer skew issues and be less prone to overheating. A sensor with an excess of pixels compared to the final image resolution may be easier to noise reduce as the excess sensor resolution can be used during the NR process without degrading the end image, but the high pixel count may introduce some of the other issues listed above.
So to get an image with an acceptable noise level, low skew, a desirable level of sensitivity without overheating is a very fine balancing act. Different cameras will perform differently and each will have strength’s and weaknesses. A Sony A7S has big pixels, so it’s very sensitive and only needs low levels of NR at the base sensitivity compared to say an FS7 with it’s smaller pixels. But the A7S suffers from a lot of image skew that is unacceptable for many types of video production plus there are heat issues when shooting for long periods. Meanwhile the FS7, while less sensitive has very, very low skew levels and no overheating.
But what about the new PXW-FS5? There is much discussion about the noise reduction in this camera. First of all it is a different camera to any other. So it will not perform the same as any other. It uses a different combination of sensor and processing to any other camera on the market.
Does it perform as well as an A7R in low light? Actually it’s not all that different in terms of base sensitivity. What about if you add gain? Well the image quality of both cameras deteriorates when you add gain. Both become noisier and both have more NR artefacts. The A7RII is possibly a better performer in terms of noise and artefacts, maybe due to the higher pixel count allowing the use of a lot of pixel binning. Does this mean that the FS5 is somehow broken or defective? No, it just means that the A7RII has the edge in low light. Do remember that the A7RII suffers from a lot of image skew, really appalling flash band issues and overheats on long shots. Does that mean the A7RII is broken, because after all the FS5 doesn’t have these issues problems. Of course not, these are just limitations of these two very different cameras.
Anyway coming back to the noise reduction. As we can’t actually turn off the NR on the FS5 it’s hard to understand exactly how beneficial it is, even though it does introduce some inevitable artefacts (NR will always have undesirable side effects), especially when you add more than 6dB of gain. But we can get some clues. The edge tearing artefact that can be see on vertical motion at high gain levels appears to simply be noise leaking through the temporal NR when there is a lot of vertical motion, and it’s pretty obvious that there’s actually a lot of noise being hidden.
Why does this only occur in 4K? Well that’s probably because in HD the camera has a surplus of sensor resolution, so different noise reduction processes that soften the image can be used instead of temporal NR as the softening won’t be noticed in HD.
Another clue as to how well the NR works is that if you quickly switch the gain switch from one gain level to another it takes a couple of frames for the temporal NR to catch up so for one or two frames the temporal NR level is reduced (not eliminated just reduced a bit). If you look at these frames it gives some insight into how amazingly effective the temporal NR is and as well as temporal NR the FS5 is also employing spatial NR, so this is only part of the story.
I have a couple of frame grabs. One is a frame where the temporal NR is stabilised and doing it’s thing reducing the noise in the picture. This is with +12 dB of gain applied and to be honest it’s a pretty clean looking image for a 4000 ISO shot, imagine adding +24dB gain to an EX1 or PMW-500 to get up to 4000 ISO and what that would look like! The other frame is a frame grabbed as the camera is switching between +6dB to +12dB, so what you are seeing is about 6dB’s worth of temporal noise reduction. Just look at all that very nasty looking noise. Notice the very blocky areas in the shadows, in motion these can be seen to be fluttering from frame to frame as the NR kicks in, it’s really nasty. The noise is having a serious impact on the image resolution. On the far left table leg in the close up you can no longer see the wood grain. Now imagine what at least double if not 3 times that would look like because if you were to shoot at 3200 ISO with the standard gammas without NR, that’s what it’s going to look like. It would almost certainly overload the internal coded resulting in compression artefacts that you won’t see while shooting, only when you play the footage back when quite possibly it’s too late to do anything about it.
So next time you look at a noise reduction artefact do think about just how dreadful the image would look without any NR. We are not talking about seeing a little bit more noise and grain with reduced NR but a blizzard of noise and grain. It’s also worth remembering that all the camera manufacturers are doing this. Also consider what a typical 1/2″ or 2/3″ broadcast camera would look like with +24dB of gain added, it really would be unusable, yet the FS5 can deliver an image at 4000ISO that is really not all that bad. It’s not perfect, it does have some artefacts but really it’s quite remarkable what a camera like this can do thanks to modern noise reduction processes and large sensors.
Temporal NR: Temporal noise reduction works by taking a single pixel and measuring it’s average output over several frames. Using this average value a correction can be applied to the current frame that helps reduce the instantaneous brightness fluctuations that are seen as noise. However if there is motion in that area this can create nasty local blurring or smearing effects. On option is to include a process that detects motion and locally reduces or eliminates the noise reduction during motion to counter the smear or blurring. Either way motion in the image introduces undesirable artefacts.