Tag Archives: raw

Noise, ISO, Gain, S-Log2 v S-Log3 and exposure.

Even though I have written about these many times before the message still just doesn’t seem to be getting through to people.

Since the dawn of photography and video the only way to really change the signal to noise ratio and ultimately how noisy the pictures are is by changing how much light you put onto the sensor.

Gain, gamma, log, raw, etc etc only have a minimal effect on the signal to noise ratio. Modern cameras do admittedly employ a lot of noise reduction processes to help combat high noise levels, but these come at a price. Typically they soften the image or introduce artefacts such as banding, smear or edge tearing. So you always want to start off with the best possible image from the sensor with the least possible noise and the only way to achieve that is through good exposure – putting the optimum amount of light onto the sensor.

ISO is so confusing:

But just to confuse things the use of ISO to rate an electronic cameras sensitivity has become normal. But the problem is that most people have no clue about what this really means. On an electronic camera ISO is NOT a sensitivity measurement, it is nothing more than a number that you can put into an external light meter to allow you to use that light meter to obtain settings for the shutter speed and aperture that will give you the camera manufacturers suggest optimum exposure. That’s it – and that is very different to sensitivity.

Lets take Sony’s FS7 as an example (most other cameras behave in a very similar way).

If you set the FS7 up at 0dB gain, rec-709, it will have an exposure rating of 800 ISO. Use a light meter to expose with the meters ISO dial set to 800. Lets say the light meter says set the aperture to f8. When you do this the image is correctly exposed, looks good (well as good as 709 gets at least) and for most people has a perfectly acceptable amount of noise.

Now switch the camera to S-Log2 or S-Log3. With the camera still set to 0dB the ISO rating changes to 2000 which give the impression that the camera may have become more sensitive. But did we change the sensor? No.  Have we added any more gain? No, we have not, the camera is still at 0dB. But if you now expose at the recommended levels, after you have done your grading and you grade to levels similar to 709 the pictures will look quite a lot noisier than pictures shot using Rec-709.

So what’s going on?

If you now go back to the light meter to expose the very same scene, you turn the ISO dial on the light meter from 800 to 2000 ISO and the light meter will tell you to now set the aperture to f13 (approx). So starting at the f8 you had for 800 ISO, you close the aperture on the camera by 1.3 stops to f13 and you will have the “correct” exposure.

BUT: now you are putting 1.3 stops less light on to the sensor so the signal coming from the sensor is reduced by 9dB and as a result the sensor noise that is always there and never really changes is much more noticeable. As a result compared to 709 the graded S-Log looks noisy and it looks noisier by the equivalent of 9dB. This is not because you have changed the cameras sensitivity or changed because you have changed the amount of camera gain but because compared to when you shoot in 709 the sensor is being under exposed and as a result it is outputting a signal 9dB lower. So in post production when you grade or add a LUT you have to add 9dB of gain to get the same brightness as the original direct rec-709 recording and as well as making the desirable image brighter it also makes the noise 9dB higher (unless you do some very fancy noise reduction work in post).

So what do you do?

It’s common simply to open the aperture back up again, typically by 1.5 stops so that after post production grading the S-log looks no more noisy than the 709 from the FS7 – Because in reality the FS7’s sensor works best for most people when rated at the equivalent of 800 ISO rather than 2000 – probably because it’s real sensitivity is 800 ISO.

When you think about it, when you shoot with Rec-709 or some other gamma that won’t be graded it’s important that it looks good right out of the camera. So the camera manufacturer will ensure that the rec-709 noise and grain v sensitivity settings are optimum – so this is probably the optimum ISO rating for the camera in terms of noise, grain and sensitivity.

So don’t be fooled into thinking that the FS7 is more sensitive when shooting with log, because it isn’t. The only reason the ISO rating goes up as it does is so that if you were using a light meter it would make you put less light onto the sensor which then allows the sensor to handle a brighter highlight range. But of course if you put less light onto the sensor the sensor won’t be able to see so far into the shadows and the picture may be noisy which limits still further the use of any shadow information. So it’s a trade-off, more highlights but less shadows and more noise. But the sensitivity is actually the same. Its’s an exposure change not a sensitivity change.

So then we get into the S-Log2 or S-Log3 debate.

First of all lets just be absolutely clear that both have exactly the same highlight and shadow ranges. Both go to +6 stops and -8 stops, there is no difference in that regard. Period.

And lets also be very clear that both have exactly the same signal to noise ratios. S-log3 is NOT noisier than S-log2. S-log 3 records some of the mid range using higher code values than S-Log2 and before you grade it that can sometimes make it appear like it’s noisier, but the reality is, it is not noisier.  Just like the differing ISO ratings for different gamma curves, this isn’t a sensitivity change, it’s just different code values being used. See this article if you want the hard proof: http://www.xdcam-user.com/2014/03/understanding-sonys-slog3-it-isnt-really-noisy/

Don’t forget when you shoot with log you will be grading the image. So you will be adjusting the brightness of the image. If you grade S-Log2 and S-Log3 to the same brightness levels the cumulative gain (the gain added in camera and the gain added in post) ends up the same. So it doesn’t matter which you use in low light the final image, assuming a like for like grade will have the same amount of noise.

For 8 bit records S-Log2 has different benefits.

S-Log2 was designed from the outset for recording 14 stops with an electronic video camera. So it makes use of the cameras full recording range. S-Log3 is based on an old film log curve (cineon) designed to transfer 16 stops or more to a digital intermediate. So when the camera only has a 14 stop sensor you waste a large part of the available recording range. On a 10 bit camera this doesn’t make much difference. But on a 8 bit camera where you are already very limited with the number of tonal values you can record it isn’t ideal and as a result S-Log2 is often a better choice.

But if I shoot raw it’s all going to be so much better – isn’t it?

Yes, no, maybe…. For a start there are lot’s of different types of raw. There is linear raw, log raw, 10 bit log raw, 12 bit linear, 16 bit linear and they are all quite different.

But they are all limited by what the sensor can see and how noisy the sensor is. So raw won’t give you less noise (it might give different looking noise). Raw won’t give you a bigger dynamic range so it won’t allow you to capture deeper or brighter highlights.

But what raw does normally is to give you more data and normally less compression than the cameras internal recordings. In the case of Sony’s FS5 the internal UHD recordings are 8 bit and highly compressed while the raw output is 12 bit, that’s a 4 fold increase in the amount of tonal values. You can record the 12bit raw using uncompressed cDNG or Apples new ProResRaw codec which doesn’t introduce any appreciable compression artefacts and as a result the footage is much more flexible in post production. Go up to the Sony Venice, F5 or F55 cameras and you have 16 bit raw and X-OCN (which behaves exactly like raw) which has an absolutely incredible range of tonal values and is a real pleasure to work with in post production. But even with the Venice camera the raw does not have more dynamic range than the log. However because there are far more tonal values in the raw and X-OCN you can do more with it and it will hold up much better to aggressive grading.

It’s all about how you expose.

At the end of the day with all of these camera and formats how you expose is the limiting factor. A badly exposed Sony Venice probably won’t end up looking anywhere near as good as a well exposed FS7. A badly exposed FS7 won’t look as good as a well exposed FS5. No camera looks good when it isn’t exposed well.

Exposure isn’t brightness. You can add gain to make a picture brighter, you can also change the gamma curve to change how bright it is.  But these are not exposure changes. Exposure is all about putting the optimum amount of light onto the sensor. Enough light to produce a signal from the sensor that will overcome the sensors noise. But also not so much light that the sensor overloads. That’s what good exposure is. Fiddling around with gamma curves and gain settings will only every make a relatively small difference to noise levels compared to good exposure. There’s just no substitute for faster lenses, reflectors or actually adding light if you want clean images.

And don’t be fooled by ISO ratings. They don’t tell you how noisy the picture is going to be, they don’t tell you what the sensitivity is or even if it’s actually changing. All it tells you is what to set a light meter to.

Advertisements

ProRes Raw Over Exposure Magic Tricks – It’s all smoke and mirrors!

There are a lot of videos circulating on the web right now showing what appears to be some kind of magic trick where someone has shot over exposed, recorded the over exposed images using ProRes Raw and then as if by magic made some adjustments to the footage and it goes from being almost nothing but a white out of over exposure to a perfectly exposed image.

This isn’t magic, this isn’t raw suddenly giving you more over exposure range than you have with log, this is nothing more than a quirk of the way FCP-X handles ProRes Raw material.

Before going any further – this isn’t a put-down of raw or ProRes raw. It’s really great to be able to take raw sensor data and record that with only minimal processing. There are a lot of benefits to shooting with raw (see my earlier post showing all the extra data that 12 bit raw can give). But a magic ability to let you over expose by seemingly crazy amounts isn’t something raw does any better than log.

Currently to work with ProRes Raw you have to go through FCP-X. FCP-X applies a default sequence of transforms to the Raw footage to get it from raw data to a viewable image. These all expect the footage to be exposed exactly as per the camera manufacturers recommendations, with no leeway. Inside FCP-X it’s either exposed exactly right, or it isn’t.

The default decode settings include a heavy highlight roll-off. Apple call it “Tone Mapping”. Fancy words used to make it sound special but it’s really no different to a LUT or the transforms and processes that take place in other raw decoders. Like a LUT it maps very specific values in the raw data  to very specific output brightness values. So if you shoot just a bit bright – as you would often do with log to improve the signal to noise ratio – The ProRes raw appears to be heavily over exposed. This is because anything bright ends up crushed into nothing but flat white by the default highlight roll off that is applied by default.

In reality the material is probably only marginally over exposed, maybe just one to 2 stops which is something we have become used to doing with log. When you view brightly exposed log, the log itself doesn’t look over exposed, but if you apply a narrow high contrast 709 LUT to it, it then the footage looks over exposed until you grade it or add an exposure compensated LUT.  This is what is happening by default inside FCP-X, a transform is being applied that makes brightly exposed footage look very bright and possibly over exposed – because thats the way it was shot!

This is why in FCP-X  it is typical to change the color library to WCG (Wide Color Gamut) as this changes the way FCP-X processes the raw, changing the Tone Mapping and most importantly getting rid of the highlight roll off. With no roll-off, highlights and any even slight over exposure will still blow out as you can’t show 14 stops on a conventional 6 stop TV or monitor. Anything beyond the first 6 stops will be lost, the image will look over exposed until you grade or adjust the material to control the brighter parts of the image and bring them back into a viewable range. When you are in WCG mode in FCP-X the there is no longer a highlight roll off crushing the highlights and now because they are not crushed they can be recovered, but there isn’t any more highlight range than you would have if you shot with log on the same camera!

None of this is some kind of Raw over exposure magic trick as is often portrayed. It’s simply not really understanding how the workflow works and appreciating that if you shoot bright – well it’s going to look bright – until you normalise it in post. We do this all the time with log via LUT’s and grading too! It can be a little more straight forward to recover highlights from Linear Raw footage as comes form an FS5 or FS7 compared to log. That’s because of the way log maintains a constant data level in each highlight stop and often normal grading and colour correction tools don’t deal with this correctly. The highlight range is there, but it can be tricky to normalise the log without log grading tools such as the log controls in DaVinci Resolve.

Another problem is the common use of LUT’s on log footage. The vast majority of LUT’s add a highlight roll off, if you try to grade the highlights after adding a LUT with a highlight roll off it’s going to be next to impossible to recover the highlights. You must do the highlight recovery before the LUT is added or use a LUT that has compensation for any over exposure. All of these things can give the impression that log has less highlight range than the raw from the same camera. This is not normally the case, both will be the same as it’s the sensor that limits the range.

The difference in the highlight behaviour is in the workflows and very often both log and raw workflows are miss-understood. This can lead to owners and users of these cameras thinking that one process has more than the other, when in reality there is no difference, it’s appears to be different because the workflow works in a different way.

Why I Choose To Shoot ProRes Raw with the FS5

This is a much discussed topic right now, so as I promised in my last article about this, I have put together a video. Unfortunately YouTube’s compression masks many of the differences between the UHD XAVC and the ProRes Raw, but you can still see them, especially on the waveform scopes.

To really appreciate the difference you should watch the video on a large screen at at high quality, preferably 4K.

Log and Raw Don’t have highlight a highlight roll off.

This just keeps coming up over and over. Almost all log gamma curves and the majority of raw recording formats don’t have a highlight roll-off. Any roll off that you might see is probably in the LUT’s that you are using.

The whole point of log and raw is to capture as much information about the scene that you are shooing as you can. Log normally achieves this by recording every stop above middle grey with a constant amount of data, so even the very brightest stop has the same amount of recording data as the ones below it – there is no roll off.

In conventional limited range recordings such as Rec-709, hypergamma, cinegamma etc, highlight roll-offs work by reducing the contrast in the highlights to make the amount of data needed to record the very brightest stops much smaller than used for the rest of the image. This allows 2 or 3 stops to be squeezed into a very small recording range, keeping most of the recording data available for a nice bright high contrast image. The reduction in contrast in the extreme highlights helps hide any highlight handling problems and makes it appear as though the sensors clipping point is reach in a more pleasing soft manner.

But you don’t want this in a log or raw recording as it makes grading much harder as the footage will contain different contrast ranges, each needing it’s own grading adjustments. Also by reducing contrast in the highlights you are reducing the data. It would be very difficult to un-pick a highlight roll off and if you did want to expand the data back out you will get issues such as banding.

S-log-levels Log and Raw Don't have highlight a highlight roll off.
Chart showing S-Log2 and S-Log3 plotted against f-stops and code values. Note how little data there is for each of the darker stops, the best data is above middle grey and there is no highlight roll-off. Note that current sensor only go to +6 stops over middle grey so S-Log2 and S-Log record to different peak levels.

S-Log2 and S-Log3 like almost all log gammas have no highlight roll-off. The only roll off is from middle grey and down. So if you underexpose you will start to roll away the data in your scenes mid range and that’s not good. Expose for the mid range, this is the most important part of any image. If your highlights are a bit clipped don’t worry about this too much. In post production you can add a roll off in the grade that will make any clipped highlights roll away gently. Adding a bit of highlight diffusion in post will also nicely mask any clipped highlights and make them look natural.

What are the benefits of ProRes Raw with the PXW-FS5?

There has been a lot of discussion recently and few videos posted that perhaps give the impression that if you shoot with S-Log2 on an FS5 and compare it to raw shot on the FS5 there is very little difference.

Many of the points raised in the videos are correct. ProRes raw won’t give you any more dynamic range. It won’t improve the cameras low light performance. There are features such as automatic lens aberration correction applied when shooting internally which isn’t applied when shooting raw.  Plus it’s true that shooting ProRes raw requires an external recorder that makes the diminutive little FS5 much more bulky.

So why in that case shoot ProRes Raw?

Frankly, if all you are doing is producing videos that will be compressed to within an inch of their life for YouTube, S-Log2 can do an excellent job when exposed well, it can be graded and can produce a nice image.

But if you are trying to produce the highest quality images possible then well shot ProRes raw will give you more data to work with in post production with fewer compression artefacts than the internal 8 bit UHD XAVC.

I was looking at some shots that I did in preparation for my recent webinar on ProRes raw earlier today and at first glance there isn’t perhaps much difference between the UHD 8 bit XAVC S-Log2 files and the ProRes raw files that were shot within seconds of each other. But look more closely and there are some important differences, especially if skin tones are important too you.

Skin tones sit half way between middle grey and white and typically span around 2 to 3 stops. So with S-Log 2 and an 8 bit recording a face would span around 24 to 34 IRE and have a somewhere between 24 and 35 code values – Yes, that’s right, maybe as few as 24 shades in each of the R, G and B channels. If you apply a basic LUT to this and then view it on a typical 8 bit monitor it will probably look OK.

But compare that to 12 bit linear raw recording and the same face with 2 to 3 stops across it will have anywhere up to 10 to 20 times as many code values ( somewhere around 250 – 500 code values depending on exactly how it’s exposed) . Apply the same LUT as for the S-Log2 and on the surface it looks pretty much the same – or does it?

Look closely and you will see more texture in the 12 bit raw. If you are lucky enough to have a 10 bit monitor the differences are even more apparent. Sure, it isn’t an in-your-face night and day difference but the 12 bit skin tones look less like plastic and much more real, they just look nicer, especially if it’s someone with a good complexion.

In addition looking at my test material I am also seeing some mpeg compression artefacts on the skin tones in the 8 bit XAVC that has a smoothing effect on the skin tones, reducing some of the subtle textures and adding to the slightly un-real, video look.

The other deal with a lack of code values and H624 compression  is banding. Take 8 bit S-Log2 and start boosting the contrast in a sky scene, perhaps to bring out some cloud details and you will start to see banding and stair stepping if you are not very careful. You will also see it across wall and other textureless surfaces. You can even see this on your grading suite waveform scopes in many cases. You won’t get this with well exposed 12 bit linear raw (for any normal grading at least).

None of these are huge deals perhaps. But what is it that makes a great picture? Take Sony’s Venice or the Arri Alexa as examples. We know these to be great cameras that produce excellent images. But what is it that makes the images so good? The short answer is that it is a combination of a wide range of factors, each done as well as is possible. Good DR, good colour, good skin tones etc. So what you want to record is whatever the sensor in your camera can deliver as well as you can. 8 bit UHD compressed to only 100Mb/s is not really that great. 12 bit raw will give you more textures in the mid range and highlights. It does have some limitations in the shadows, but that is easily overcome with a nice bright exposure and printing down in post.

And it’s not just about image quality.

Don’t forget that ProRes Raw makes shooting at 4K DCI possible. If you hope to ever release your work for cinema display, perhaps on the festival circuit, you are going to be much better off shooting in the cinema DCI 4K standard rather than the UHD TV standard. It also allows you to shoot 60fps in 4K (I’m in the middle of a very big 4K 60p project right now). Want to shoot even faster – well with ProRes Raw you can, you can shoot at up to 120fps in 4K. So there are many other benefits to the raw option on the FS5 and recording to ProRes raw on a Shogun Inferno.

There is also the acceptability of 8 bit UHD. No broadcaster that I know of will ever consider 8 bit UHD unless there is absolutely no other way to get the material. You are far more likely to be able to get them to accept 12 bit raw.

Future proofing is another consideration. I am sure that ProRes raw decoders will improve and support in other applications will eventually come. By going back to your raw sensor data with better software you may be able to gain better image quality from your footage in the future. With Log you are already somewhat limited as the bulk of the image processing has already been done and is baked into the footage.

It’s late on Friday afternoon here in the UK and I’ve promised to spend some time with the family this evening. So no videos today. But next week I’ll post some of the examples I’ve been looking at so that you can see where ProRes raw elevates the image quality possible from the FS5.

Free ProRes Raw and Sony Venice Webinars

I will be doing a couple of free interactive Webinars with Visual Impact in July. The first is on ProRes Raw looking at what it is, the equipment you will need and how to shoot with it.

http://www.visuals.co.uk/events/events.php?event=eid11028-839

The second is about my practical experiences shooting with a Sony Venice using the version 2 firmware.

http://www.visuals.co.uk/events/events.php?event=eid14021-841

Both webinars will feature a Q&A session where you will be able to ask questions online. You will find the full details about both webinars by following the links above, including how to register. The webinars are free but registration is required to obtain the login details for the events.

 

Why is the white balance limited to 3 presets when using S-Log2, S-Log3 or raw?

This seems to be a source of frustration for many people shooting  raw or using S-Log2 or S-Log3 on a Sony camera. When shooting log and raw you should also be using a matching S-Gamut colour gamut if you want to get the best from the camera and this ties you into one of 3 preset white balances.

With a PXW-FS7, PMW-F5 or F55 it is possible to use custom mode to select a different colour space to mix with S-Log2 or S-Log3 and then have a variable white balance. With the Alpha cameras, PXW cameras such as the FS5 you can choose any Gamut you want in the picture profiles, but I don’t recommend this. For a start, if you don’t use one of the S-Gamuts you will be limited to Rec-709 Gamut, so you won’t be recording  the cameras full colour range. Also in custom mode there are some other things like noise reduction that you really don’t want when shooting S-log2/3 (it can cause banding).

So why is the S-Gamut white balance fixed to the 3 presets for daylight, fluorescent and tungsten? The main reason is to ensure you get the cameras full dynamic range in each colour. White balance is a gain function, it adjusts the gain of the red, green and blue channels so that white objects appear white under differing light sources. So if the light source lacks blue light – making the pictures look excessively warm – you add extra gain to the blue channel to compensate.

But the problem with this is that gain affects dynamic range. When shooting log (or raw) the camera needs to operate the sensor at the optimum gain level to squeeze the highest possible dynamic range from the it. Changing the gain in just one colour channel to shift the white balance could result in a reduction of dynamic range in the channel. This could manifest itself as colours in one channel that clip sooner than the others. This can be really hard to deal with in post production and can show up as things like bright clouds with a colour cast that isn’t in the rest of the picture.

Another potential issue is that because of the way silicon sensors work the blue channel is almost always noisier than the red and green. So you want to keep the gain in the blue channel as low as possible to prevent the pictures getting too noisy. This is particularly important when shooting log as you won’t see your end result until after the images have been graded. So manually shifting the gain of the blue channel in camera to correct the white balance could lead to footage that ends up noisier than you would expect.

So – Sony chose to fix the white balance to 3 carefully tuned presets designed to avoid this situation and maximise the dynamic range. After all, when shooting log or raw it is expected that the footage will be graded anyway, so the white balance will normally be adjusted as part of the post production process.

There are some people that advocate adjusting the FS5’s white balance via the picture profile settings, personally I don’t recommend this or feel that it’s necessary. But yes, you can do this, but just keep a very close eye on your highlights and if you can use monitor with RGB parade to make sure you have equal recording levels for your whites without one colour channel clipping ahead of the others. Also apply a LUT in the monitor that is close to your desired output so that you can keep an eye on the noise levels.

In summary – the white balance is preset to ensure you don’t encounter problems later on. You should be able to fully adjust and fine tune your white balance in post production to a far greater degree than is possible in camera anyway, so don’t worry if the WB is a touch off when shooting.

The only exception to this is the new Sony Venice. Venice has enough dynamic range and enough internal processing power to allow you to make a wide range of white balance adjustments in camera. Hopefully we will see some of this flexibility trickle down to the next generations of lower cost Sony digital cinema cameras.

ProRes Raw and Atomos Inferno and Sumo – BIG deal for the FS5 and FS7!!

proresraw-logo ProRes Raw and Atomos Inferno and Sumo - BIG deal for the FS5 and FS7!!Over the last few days there have been various rumours and posts coming from Apple about how they intend to get back to providing decent support for professional users of their computers. Apple have openly admitted that the Trash Can Mac Pro has thermal problems and as a result has become a dead end design, which is why there haven’t been any big updates to the flagship workstation from Apple. Apple have hinted that new workstations are on the way, although it would seem that we won’t see these until next year perhaps.
Another announcement came out today, a new version of FCP-X is to be released which includes support for a new ProRes codec called ProRes Raw. This is BIG!

PRORES RAW.

Raw recordings can be made from certain cameras that have bayer sensors such as the Sony FS5 and FS7. Recording the raw data from the sensor maximises your post production flexibility and normally offers the best possible image quality from the camera. Currently if you record 4K raw with these cameras using an Atomos Shogun or similar the bit rate will be close to 3Gb/s at 24p. These are huge files and the cDNG format used to record them is difficult and clunky to work with.  As a result most users take the raw output from the camera and transform it to S-Log2 or S-Log3 and record it as 10 bit ProRes on the external recorder. This is a bit of a shame as going from 12 bit linear raw to 10 bit S-log means you are not getting the full benefit of the raw output.

Enter ProRes Raw:  ProRes Raw will allow users to record the cameras raw output at a much reduced bit rate with no significant of quality. There are two versions, ProRes Raw and ProRes Raw HQ. The HQ bit rate is around 1Gb/s at 24fps. This is not significantly bigger than the ProRes HQ (880Mb/s) that most users are using now to record the raw, yet the full benefit of 12 bit linear will be retained. A 1TB SSD will hold around an hour of ProRes Raw, compare that to uncompressed raw where you only get around 20 mins and you can see that this is a big step forwards for users of the FS5 in particular.

ProRes Raw (the non HQ version) is even smaller! The files are smaller than typical ProRes HQ files. This is possible because recording raw is inherently more efficient than recording component video.

It is claimed by Apple that ProRes Raw will play back in real time on MacBook Pro’s and iMacs without any additional rendering or external graphics cards, so it obviously isn’t terribly processor intensive. This is excellent news! Within FCP-X the playback resolution can be decreased to bring improved playback performance in less powerful systems or mutistream playback.

It looks like you will be able to record from a 4K DCI  from an FS5 or FS7 at up to 60fps continuously. This breaks through the previous limits for the Shogun of 30fps. The FS7 will be able to record 2K raw at up to 240fps and the FS5 will be able to record 4K raw at 100 and 120fps for 4 seconds. Other raw cameras are also supported by the Atomos recorders at differing frame sizes and frame rates.

At the moment the only recorders listed as supporting ProRes Raw are the Atomos Shogun Inferno and the Sumo19 and it looks like it will be a free update. In addition the DJI Inspire 2 drone and Zenmuse X7 Super 35mm camera will also support ProRes Raw.

Whether you will be able to use ProRes Raw in other applications such as Resolve or Premiere is unclear at this time. I hope that you can (or at least will be able to in the near future).

SEE: Apple Press Release.

SEE: Apple ProRes Raw White Paper

SEE: ATOMOS ProRes Raw INFO PAGE.

 

 

DaVinci Resolve, ACES and the “Sony Raw” input transform.

A quick heads up for users of Resolve with Sony Raw and X-OCN. Don’t make the same mistake I have been making. For some time I have been unhappy with the way the Sony raw looked in DaVinci Resolve and ACES prior to grading. Apparently there used to be a small problem with the raw input transform that could lead to a red/pink hue getting added to the footage. This problem was fixed some time ago. You should now not use the the “Sony Raw” input transform, if you do, it will tint your Raw or X-OCN files slightly pink/red. Instead you should select “no transform”. With no transform selected my images look so much nicer and match Sony’s own Raw Viewer so much better. Thanks to Nick Shaw of Antler Post for helping me out on this and all on the CML list.

How can 16 bit X-OCN deliver smaller files than 10 bit XAVC-I?

Sony’s X-OCN (XOriginal Camera Negative) is a new type of codec from Sony. Currently it is only available via the R7 recorder which can be attached to a Sony PMW-F5, F55 or the new Venice cinema camera.

It is a truly remarkable codec that brings the kind of flexibility normally only available with 16 bit linear raw files but with a files size that is smaller than many conventional high end video formats.

Currently there are two variations of X-OCN.

X-OCN ST is the standard version and then X-OCN LT is the “light” version. Both are 16 bit and both contain 16 bit data based directly on what comes off the cameras sensor. The LT version is barely distinguishable for a 16 bit linear raw recording and the ST version “visually lossless”. Having that sensor data in post production allows you to manipulate the footage over a far greater range than is possible with tradition video files. Traditional video files will already have some form of gamma curve as well as a colour space and white balance baked in. This limits the scope of how far the material can be adjusted and reduces the amount of picture information you have (relative to what comes directly off the sensor) .

Furthermore most traditional video files are 10 bit with a maximum of 1024 code values or levels within the recording. There are some 12 bit codecs but these are still quite rare in video cameras. X-OCN is 16 bit which means that you can have up to 65,536 code values or levels within the recording. That’s a colossal increase in tonal values over traditional recording codecs.

But the thing is that X-OCN LT files are a similar size to Sony’s own XAVC-I (class 480) codec, which is already highly efficient. X-OCN LT is around half the size of the popular 10 bit Apple ProRes HQ codec but offers comparable quality. Even the high quality ST version of X-OCN is smaller than ProRes HQ. So you can have image quality and data levels comparable to Sony’s 16 bit linear raw but in a lightweight, easy to handle 16 bit file that’s smaller than the most commonly used 10 bit version of ProRes.

But how is this even possible? Surely such an amazing 16 bit file should be bigger!

The key to all of this is that the data contained within an X-OCN file is based on the sensors output rather than traditional video.  The cameras that produce the X-OCN material all use bayer sensors. In a traditional video workflow the data from a bayer sensor is first converted from the luminance values that the sensor produces into a YCbCr or RGB signal.

So if the camera has a 4096×2160 bayer sensor in a traditional workflow this pixel level data gets converted to 4096×2160 of Green plus 4096×2160 of Red, plus 4096×2160 of Green (or the same of Y, Cb and Cr). In total you end up with 26 million data points which then need to be compressed using a video codec.

Bayer-to-RGB How can 16 bit X-OCN deliver smaller files than 10 bit XAVC-I?However if we bypass the conversion to a video signal and just store the data that comes directly from the sensor we only need to record a single set of 4096×2160 data points – 8.8 million. This means we only need to store 1/3rd as much data as in a traditional video workflow and it is this huge data saving that is the main reason why it is possible for X-OCN to be smaller than traditional video files while retaining amazing image quality. It’s simply a far more efficient way of recording the data from a bayer camera.

Of course this does mean that the edit or playback computer has to do some extra work because as well as decoding the X-OCN file it has to be converted to a video file, but Sony developed X-OCN to be easy to work with – which it is. Even a modest modern workstation will have no problem working with X-OCN. But the fact that you have that sensor data in the grading suite means you have an amazing degree of flexibility. You can even adjust the way the file is decoded to tailor whether you want more highlight or shadow information in the video file that will created after the X-OCN is decoded.

Why isn’t 16 bit much bigger than 10 bit? Normally a 16 bit file will be bigger than a 10 bit file. But with a video image there are often areas of information that are very similar. Video compression algorithms take advantage of this and instead of recording a value for every pixel will record a single value that represents all of the similar pixels. When you go from 10 bit to 16 bit, while yes, you do have more bits of data to record a greater percentage of the code values will be the same or similar and as a result the codec becomes more efficient. So the files size does increase a bit, but not as much as you might expect.

So, X-OCN, out of the gate, only needs to store 1/3rd of the data points of a similar traditional RGB or YCbCr codec. Increasing the bit depth from the typical 10 bit bit depth of a regular codec to the 16 bits of X-OCN does then increase the amount of data needed to record it. But the use of a clever algorithm to minimise the data needed for those 16 bits means that the end result is a 16 bit file only a bit bigger than XAVC-I but still smaller than ProRes HQ even at it’s highest quality level.