I have created a new and improved HLG camera LUT for the PXW-FS7, PMW-F5 and PMW-F55 cameras. This 3D LUT can be used to shoot HLG directly by baking the LUT in to the recordings in camera. This allows you to create “instant HDR” footage that just like the HLG footage from an FS5 or Z90 does not need to be graded or modified to provide an HDR image on an equipped HDR TV. Skin tones should be exposed at around 55-60% and white at around 70-75%.
The use o f the LC709 Type A LUT in Sony’s Cinealta cameras such as the PXW-FS7 or PMW-F55 is very common. This LUT is popular because it was designed to mimic the Arri cameras when in their Rec-709 mode. But before rushing out to use this LUT and any of the other LC709 series of LUT’s there are some things to consider.
The Arri cameras are rarely used in Rec-709 mode for anything other than quick turn around TV. You certainly wouldn’t normally record this for any feature or drama productions. It isn’t the “Arri Look” The Arri look normally comes as a result of shooting using Arri’s LogC and then grading that to get the look you want. The reason it exists is to provide a viewable image on set. It has more contrast than LogC and uses Rec 709 color primaries so the colors look right, but it isn’t Rec-709. It squeezes almost all of the cameras capture range into a something that can be viewed on a 709 monitor so it looks quite flat.
Because a very large dynamic range is being squeezed into a range suitable to be viewed on a regular, standard dynamic range monitor the white level is much reduced compared to regular Rec-709. In fact, white (such as a white piece of paper) should be exposed at around 70%. Skin tones should be exposed at around 55-60%.
If you are shooting S-Log on a Sony camera and using this LUT to monitor, if you were to expose using conventional levels, white at 85-90% skin tones at 65-70%, then you will be offsetting your exposure by around +1.5 stops. On it’s own this isn’t typically going to be a problem. In fact I often come across people that tell me that they always shoot at the cameras native EI using this LUT and get great, low noise pictures. When I dig a little deeper I often find that they are exposing white at 85% via the LC709 LUT. So in reality they are actually shooting with an exposure the equivalent of +1 to +1.5 stops over the base level.
Where you can really run into problems is when you have already added an exposure offset. Perhaps you are shooting on an FS7 where the native ISO is 2000 ISO and using an EI of 800. This is a little over a +1 stop exposure offset. Then if you use one of the LC709 LUT’s and expose the LUT so white is at 90% and skin tones at 70% you are adding another +1.5 stops to the exposure, so your total exposure offset is approaching 3 stops. This large an offset is rarely necessary and can be tricky to deal with in post. It’s also going to impact your highlight range.
So just be aware that different LUT’s require different white and grey levels and make sure you are exposing the LUT at it’s correct level so that you are not adding an additional offset to your desired exposure.
A quick heads up for users of Resolve with Sony Raw and X-OCN. Don’t make the same mistake I have been making. For some time I have been unhappy with the way the Sony raw looked in DaVinci Resolve and ACES prior to grading. Apparently there used to be a small problem with the raw input transform that could lead to a red/pink hue getting added to the footage. This problem was fixed some time ago. You should now not use the the “Sony Raw” input transform, if you do, it will tint your Raw or X-OCN files slightly pink/red. Instead you should select “no transform”. With no transform selected my images look so much nicer and match Sony’s own Raw Viewer so much better. Thanks to Nick Shaw of Antler Post for helping me out on this and all on the CML list.
Sony’s X-OCN (X–Original Camera Negative) is a new type of codec from Sony. Currently it is only available via the R7 recorder which can be attached to a Sony PMW-F5, F55 or the new Venice cinema camera.
It is a truly remarkable codec that brings the kind of flexibility normally only available with 16 bit linear raw files but with a files size that is smaller than many conventional high end video formats.
Currently there are two variations of X-OCN.
X-OCN ST is the standard version and then X-OCN LT is the “light” version. Both are 16 bit and both contain 16 bit data based directly on what comes off the cameras sensor. The LT version is barely distinguishable for a 16 bit linear raw recording and the ST version “visually lossless”. Having that sensor data in post production allows you to manipulate the footage over a far greater range than is possible with tradition video files. Traditional video files will already have some form of gamma curve as well as a colour space and white balance baked in. This limits the scope of how far the material can be adjusted and reduces the amount of picture information you have (relative to what comes directly off the sensor) .
Furthermore most traditional video files are 10 bit with a maximum of 1024 code values or levels within the recording. There are some 12 bit codecs but these are still quite rare in video cameras. X-OCN is 16 bit which means that you can have up to 65,536 code values or levels within the recording. That’s a colossal increase in tonal values over traditional recording codecs.
But the thing is that X-OCN LT files are a similar size to Sony’s own XAVC-I (class 480) codec, which is already highly efficient. X-OCN LT is around half the size of the popular 10 bit Apple ProRes HQ codec but offers comparable quality. Even the high quality ST version of X-OCN is smaller than ProRes HQ. So you can have image quality and data levels comparable to Sony’s 16 bit linear raw but in a lightweight, easy to handle 16 bit file that’s smaller than the most commonly used 10 bit version of ProRes.
But how is this even possible? Surely such an amazing 16 bit file should be bigger!
The key to all of this is that the data contained within an X-OCN file is based on the sensors output rather than traditional video. The cameras that produce the X-OCN material all use bayer sensors. In a traditional video workflow the data from a bayer sensor is first converted from the luminance values that the sensor produces into a YCbCr or RGB signal.
So if the camera has a 4096×2160 bayer sensor in a traditional workflow this pixel level data gets converted to 4096×2160 of Green plus 4096×2160 of Red, plus 4096×2160 of Green (or the same of Y, Cb and Cr). In total you end up with 26 million data points which then need to be compressed using a video codec.
However if we bypass the conversion to a video signal and just store the data that comes directly from the sensor we only need to record a single set of 4096×2160 data points – 8.8 million. This means we only need to store 1/3rd as much data as in a traditional video workflow and it is this huge data saving that is the main reason why it is possible for X-OCN to be smaller than traditional video files while retaining amazing image quality. It’s simply a far more efficient way of recording the data from a bayer camera.
Of course this does mean that the edit or playback computer has to do some extra work because as well as decoding the X-OCN file it has to be converted to a video file, but Sony developed X-OCN to be easy to work with – which it is. Even a modest modern workstation will have no problem working with X-OCN. But the fact that you have that sensor data in the grading suite means you have an amazing degree of flexibility. You can even adjust the way the file is decoded to tailor whether you want more highlight or shadow information in the video file that will created after the X-OCN is decoded.
Why isn’t 16 bit much bigger than 10 bit? Normally a 16 bit file will be bigger than a 10 bit file. But with a video image there are often areas of information that are very similar. Video compression algorithms take advantage of this and instead of recording a value for every pixel will record a single value that represents all of the similar pixels. When you go from 10 bit to 16 bit, while yes, you do have more bits of data to record a greater percentage of the code values will be the same or similar and as a result the codec becomes more efficient. So the files size does increase a bit, but not as much as you might expect.
So, X-OCN, out of the gate, only needs to store 1/3rd of the data points of a similar traditional RGB or YCbCr codec. Increasing the bit depth from the typical 10 bit bit depth of a regular codec to the 16 bits of X-OCN does then increase the amount of data needed to record it. But the use of a clever algorithm to minimise the data needed for those 16 bits means that the end result is a 16 bit file only a bit bigger than XAVC-I but still smaller than ProRes HQ even at it’s highest quality level.
With the first of the production Venice cameras now starting to find their way to some very lucky owners it’s time to take a look at some features that are not always well understood, or that perhaps no one has told you about yet.
Dual Native ISO’s: What does this mean?
An electronic camera uses a piece of silicon to convert photons of light into electrons of electricity. The efficiency at doing this is determined by the material used. Then the amount of light that can be captured and thus the sensitivity is determined by the size of the pixels. So, unless you physically change the sensor for one with different sized pixels (which will in the future be possible with Venice) you can’t change the true sensitivity of the camera. All you can do is adjust the electronic parameters.
With most video cameras the ISO is changed by increasing the amount of amplification applied to the signal coming off the sensor. Adding more gain or increasing the amplification will result in a brighter picture. But if you add more amplification/gain then the noise from the sensor is also amplified by the same amount. Make the picture twice as bright and normally the noise doubles.
In addition there is normally an optimum amount of gain where the full range of the signal coming from the sensor will be matched perfectly with the full recording range of the chosen gamma curve. This optimum gain level is what we normally call the “Native ISO”. If you add too much gain the brightest signal from the sensor would be amplified too much and exceed the recording range of the gamma curve. Apply too little gain and your recordings will never reach the optimum level and darker parts of the image may be too dark to be seen.
As a result the Native ISO is where you have the best match of sensor output to gain. Not too much, not too little and hopefully low noise. This is typically also referred to as 0dB gain in an electronic camera and normally there is only 1 gain level where this perfect harmony between sensor, gain and recording range is achieved, this becoming the native ISO.
Side Note: On an electronic camera ISO is an exposure rating, not a sensitivity measurement. Enter the cameras ISO rating into a light meter and you will get the correct exposure. But it doesn’t really tell you how sensitive the camera is as ISO has no allowance for increasing noise levels which will limit the darkest thing a camera can see.
Tweaking the sensor.
However, there are some things we can tweak on the sensor that effect how big the signal coming from the sensor is. The sensors pixels are analog devices. A photon of electricity hits the silicone photo receptor (pixel) and it gets converted into an electron of electricity which is then stored within the structure of the pixel as an analog signal until the pixel is read out by a circuit that converts the analog signal to a digital one, at the same time adding a degree of noise reduction. It’s possible to shift the range that the A to D converter operates over and the amount of noise reduction applied to obtain a different readout range from the sensor. By doing this (and/or other similar techniques, Venice may use some other method) it’s possible to produce a single sensor with more than one native ISO.
A camera with dual ISO’s will have two different operating ranges. One tuned for higher light levels and one tuned for lower light levels. Venice will have two exposure ratings: 500 ISO for brighter scenes and 2500 ISO for shooting when you have less light. With a conventional camera, to go from 500 ISO to 2500 ISO you would need to add just over 12dB of gain and this would increase the noise by a factor of more than 4. However with Venice and it’s dual ISO’s, as we are not adding gain but instead altering the way the sensor is operating the noise difference between 500 ISO and 2500 ISO will be very small.
You will have the same dynamic range at both ISO’s. But you can choose whether to shoot at 500 ISO for super clean images at a sensitivity not that dissimilar to traditional film stocks. This low ISO makes it easy to run lenses at wide apertures for the greatest control over the depth of field. Or you can choose to shoot at the equivalent of 2500 ISO without incurring a big noise penalty.
One of Venice’s key features is that it’s designed to work with Anamorphic lenses. Often Anamorphic lenses are typically not as fast as their spherical counterparts. Furthermore some Anamorphic lenses (particularly vintage lenses) need to be stopped down a little to prevent excessive softness at the edges. So having a second higher ISO rating will make it easier to work with slower lenses or in lower light ranges.
COMBINING DUAL ISO WITH 1 STOP ND’s.
Next it’s worth thinking about how you might want to use the cameras ND filters. Film cameras don’t have built in ND filters. An Arri Alexa does not have built in ND’s. So most cinematographers will work on the basis of a cinema camera having a single recording sensitivity.
The ND filters in Venice provide uniform, full spectrum light attenuation. Sony are incredibly fussy over the materials they use for their ND filters and you can be sure that the filters in Venice do not degrade the image. I was present for the pre-shoot tests for the European demo film and a lot of time was spent testing them. We couldn’t find any issues. If you introduce 1 stop of ND, the camera becomes 1 stop less sensitive to light. In practice this is no different to having a camera with a sensor 1 stop less sensitive. So the built in ND filters, can if you choose, be used to modify the base sensitivity of the camera in 1 stop increments, up to 8 stops lower.
So with the dual ISO’s and the ND’s combined you have a camera that you can setup to operate at the equivalent of 2 ISO all the way up to 2500 ISO in 1 stop steps (by using 2500 ISO and 500 together you can have approximately half stops steps between 10 ISO and 650 ISO). That’s an impressive range and at no stage are you adding extra gain. There is no other camera on the market that can do this.
On top of all this we do of course still have the ability to alter the Exposure Index of the cameras LUT’s to offset the exposure to move the exposure mid point up and down within the dynamic range. Talking of LUT’s I hope to have some very interesting news about the LUT’s for Venice. I’ve seen a glimpse of the future and I have to say it looks really good!
The raw and X-OCN material from a Venice camera (and from a PMW-F55 or F5 with the R7 recorder) contains a lot of dynamic metadata. This metadata tells the decoder in your grading software exactly how to handle the linear sensor data stored in the files. It tells your software where in the recorded data range the shadows start and finish, where the mid range sits and where the highlights start and finish. It also informs the software how to decode the colors you have recorded.
I recently spent some time with Sony Europe’s color grading guru Pablo Garcia at the Digital Motion Picture Center in Pinewood. He showed me how you can manipulate this metadata to alter the way the X-OCN is decoded to change the look of the images you bring into the grading suite. Using a beta version of Black Magic’s DaVinci Resolve software, Pablo was able to go into the clips metadata in real time and simply by scrubbing over the metadata settings adjust the shadows, mids and highlights BEFORE the X-OCN was decoded. It was really incredible to see the amount of data that Venice captures in the highlights and shadows. By adjusting the metadata you are tailoring the the way the file is being decoded to suit your own needs and getting the very best video information for the grade. Need more highlight data – you got it. Want to boost the shadows, you can, at the file data level before it’s converted to a traditional video signal.
It’s impressive stuff as you are manipulating the way the 16 bit linear sensor data is decoded rather than a traditional workflow which is to decode the footage to a generic intermediate file and then adjust that. This is just one of the many features that X-OCN from the Sony Venice offers. It’s even more incredible when you consider that a 16 bit linear X-OCN LT file is similar in size to 10 bit XAVC-I(class 480) and around half the size of Apples 10 bit ProRes HQ. X-OCN LT looks fantastic and in my opinion grades better than XAVC S-Log. Of course for a high end production you will probably use the regular X-OCN ST codec rather than the LT version, but ST is still smaller than ProRes HQ. What’s more X-OCN is not particularly processor intensive, it’s certainly much easier to work with X-OCN than cDNG. It’s a truly remarkable technology from Sony.
Next week I will be shooting some more test with a Venice camera as we explore the limits of what it can do. I’ll try and get some files for you to play with.
If using a LUT to judge the exposure of a camera shooting log or raw it’s really important that you fully understand how that LUT works.
When a LUT is created it will expect a specific input range and convert that input range to a very specific output range. If you change the input range then the output will range will be different and it may not be correct. As an example a LUT designed and created for use with S-Log2 should not be used with S-Log3 material as the the higher middle grey level used by S-Log3 would mean that the mid range of the LUT’s output would be much brighter than it should be.
Another consideration comes when you start offsetting your exposure levels, perhaps to achieve a brighter log exposure so that after grading the footage will have less noise.
Lets look at a version of Sony’s 709(800) LUT designed to be used with S-Log3 for a moment. This LUT expects middle grey to come in at 41% and it will output middle grey at 43%. It will expect a white card to be at 61% and it will output that same shade of white at a little over 85%. Anything on the S-Log3 side brighter than 61% (white) is considered a highlight and the LUT will compress the highlight range (almost 4 stops) into the output range between 85% and 109% resulting in flat looking highlights. This is all perfectly fine if you expose at the levels suggested by Sony. But what happens if you do expose brighter and try to use the same LUT either in camera or in post production?
Well if you expose 1.5 stops brighter on the log side middle grey becomes around 54% and white becomes around 74%. Skin tones which sit half way between middle grey and white will be around 64% on the LUT’s input. That’s going to cause a problem! The LUT considers anything brighter than 61% on it’s input to be a highlight and it will compresses anything brighter than 61%. As a result on the output of your LUT your skin tones will not only be bright, but they will be compressed and flat looking. This makes them hard to grade. This is why if you are shooting a bit brighter it is much, much easier to grade your footage if your LUT’s have offsets to allow for this over exposure.
If the camera has an EI mode (like the FS7, F5, F55 etc) the EI mode offsets the LUT’s input so you don’t see this problem in camera but there are other problems you can encounter if you are not careful like unintentional over exposure when using the Sony LC709 series of LUTs.
Sony’s 709(800) LUT closely matches the gamma of most normal monitors and viewfinders, so 709(800) will deliver the correct contrast ie. contrast that matches the scene you are shooting plus it will give conventional TV brightness levels when viewed on standard monitors or viewfinders.
If you use any of the LC709 LUT’s you will have a miss-match between the LUT’s gamma and the monitors gamma so the images will show lower contrast and the levels will be lower than conventional TV levels when exposed correctly. LC709 stands for low contrast gamma with 709 color primaries, it is not 709 gamma!
Sony’s LC709 Type A LUT is very popular as it mimics the way an Arri Alexa might look. That’s fine but you also need to be aware that the correct exposure levels for this non-standard LC gamma are middle grey at around 41% and white at 70%.
An easy trap to fall into is to set the camera to a low EI to gain a brighter log exposure and then to use one of the LC709 LUT’s and try to eyeball the exposure. Because the LC709 LUT’s are darker and flatter it’s harder to eyeball the exposure and often people will expose them as you would regular 709. This then results in a double over exposure. Bright because of the intentional use of the lower EI but even brighter because the LUT has been exposed at or close to conventional 709 brightness. If you were to mistakenly expose the LC709TypeA LUT with skin tones at 70%, white at 90% etc then that will add almost 2 stops to the log exposure on top of any EI offset.
Above middle grey with 709(800) a 1 stop exposure change results in an a 20% change in brightness, with LC709TypeA the same exposure change only gives a just over 10% change, as a result over or under exposure is much less obvious and harder to measure or judge by eye with LC709. The cameras default zebra settings for example have a 10% window. So with LC709 you could easily be a whole stop out, while with 709(800) only half a stop.
Personally when shooting I don’t really care too much about how the image looks in terms of brightness and contrast. I’m more interested in using the built in LUT’s to ensure my exposure is where I want it to be. So for exposure assessment I prefer to use the LUT that is going to show the biggest change when my exposure is not where it should be. For the “look” I will feed a separate monitor and apply any stylised looks there. To understand how my highlights and shadows, above and below the LUT’s range are being captured I use the Hi/Low Key function.
If you are someone that creates your own LUT’s an important consideration is to ensure that if you are shooting test shots, then grading these test shots to produce a LUT it’s really, really important that the test shots are very accurately exposed.
You have 2 choices here. You can either expose at the levels recommended by Sony and then use EI to add any offsets or you can offset the exposure in camera and not use EI but instead rely on the offset that will end up in the LUT. What is never a good idea is to add an EI offset to a LUT that was also offset.
With Sony’s log capable cameras (and most other manufacturers) when you switch between the standard gamma curves and log gamma there is a change in the cameras ISO rating. For example the FS7 is rated at 800 ISO in rec709 but rated at 2000 ISO in log. Why does this change occur and how does it effect the pictures you shoot?
As 709 etc has a limited DR (between around 6 and 10 stops depending on the knee settings) while the sensor itself has a 14 stop range, you only need to take a small part of the sensors full range to produce that smaller range 709 or hypergamma image. That gives the camera manufacturer some freedom to pick the sweetest part of the sensors range. his also gives some leeway as to where you place the base ISO.
I suspect Sony chose 800 ISO for the FS7 and F5 etc as that’s the sensors sweet spot, I certainly don’t think it was an accidental choice.
What is ISO on an electronic camera? ISO is the equivalent sensitivity rating. It isn’t a measure of the cameras actual sensitivity, it is the ISO rating you need to enter into a light meter if you were using an external light meter to get the correct exposure settings. It is the equivalent sensitivity. Remember we can’t change the sensor in these cameras so we can’t actually change the cameras real sensitivity, all we can do is use different amounts of gain or signal amplification to make the pictures brighter or darker.
When you go switch the camera to log you have no choice other than to take everything the sensor offers. It’s a 14 stop sensor and if you want to record 14 stops, then you have to take 100% of the sensors output. The camera manufacturer then chooses what they believe is the best exposure mid point point where they feel there is an acceptable compromise between noise, highlight and lowlight response. From that the manufacture will get an ISO equivalent exposure rating.
If you have an F5, FS7 or other Sony log camera, look at what happens when you switch from rec709 to S-Log2 but you keep your exposure constant.
Middle grey stays more or less where it is, the highlights come down. White will drop from 90% to around 73%. But the ISO rating given by the camera increases from 800ISO to 2000ISO. This increased ISO number implies that the sensor became more sensitive – This is not the case and a little missleading. If you set the camera up to display gain in dB and switch between rec709 (std gamma) and S-Log the camera stays at 0dB, this should be telling you that there is no change to the cameras gain, no change to it’s sensitivity. Yet the ISO rating changes – why?
The only reason the ISO number increases is to force us to underexpose the sensor by 1.3 stops (relative to standard gammas such as rec709 and almost every other gamma) so we can squeeze a bit more out of the highlights. If you were using an external light meter to set your exposure if you change the ISO setting on the light meter from 800 ISO to 2000 ISO the light meter will tell you to close the aperture by 1.3 stops. So that’s what we do on the camera, we close the aperture down a bit to gain some extra highlight range.
But all this comes at the expense of the shadows and mid range. Because you are putting less light on the sensor if you use 2000 ISO as your base setting the shadows and mids are now not as good as they would be in 709 or with the other standard gammas.
This is part of the reason why I recommend that you shoot with log between 1 and 2 stops brighter than the base levels given by Sony. If you shoot 1 stop brighter that is the equivalent to shooting at 1000 ISO and this is closer to the 800 ISO that Sony rate the camera at in standard gamma. Shooting that bit brighter gives you a much better mid range that grades much better.
Looks like I will have a busy day today upgrading cameras. As well as the re-release of Version 4 for the PXW-FS5, Sony have also released version 9 for the PMW-F5 and PMW-F55 cameras.
Version 9 adds some additional high speed frame rates when recording with the R7 recorder. It also adds extra parallel recording functions when using the CBK-WA100 Wireless Adapter and interestingly also adds Long GoP recording (XAVC-L) when shooting at 29.97fps and 59.94fps (sadly no 24,25 or 50fps Long GoP).
The new firmware can be downloaded from here:
FROM SONY: IMPORTANT NOTE:
Please install Version 9 ONLY if your F5 or F55 camera has been successfully updated to Firmware Version 8 or higher. Otherwise, it is very important to note that for the F5 and F55 cameras with the serial numbers range listed in the Release Notes document, the user cannot perform the Upgrade to Version 9 and should contact your local Sony Service agent.
1) Frame rates for 4K and high frame rate recordings added.
72, 75, 90, 96, and 100 FPS have been added to the available values in “Frame Rate” when the AXS-R7 is attached to the PMW-F55.
2) Compatible with the “Parallel Rec” mode with CBK-WA100 attached.
The “Parallel Rec” mode enables synchronization with the same file name between the XAVC Proxy recording by using the wireless adapter CBK-WA100.
3) XAVC HD Long added (when the system frequency is set to 29.97 or 59.94).
Sony will be releasing an update for the firmware in the Sony PXW-FS5 in the next few days. This update amongst other things will allow users of the FS5 to shoot to HDR directly using the Hybrid Log Gamma HDR gamma curve and Rec2020 color. By doing this you eliminate the need to grade your footage and could plug the camera directly in to a compatible HDR TV (the TV must support HLG) and see an HDR image directly on the screen.
But what about FS7 and F5/F55 owners? Well, for most HDR productions I still believe the best workflow is to shoot in S-Log3 and then to grade the footage to HDR. However there may be times when you need that direct HDR output. So for the FS7, F5 and F55 I have created a set of Hybrid Log Gamma LUT’s that you can use to bake in HLG and Rec2020 while you shoot. This gives you the same capabilities as the FS5 (with the exception of the ability to add HLG metadata to the HDMI).
For a video explanation of the process please follow the link to my new Patreon page where you will find the video and the downloadable LUT’s.