Deeper Understanding Of Log Gamma. Experiments with a Waveform Display.

I started writing this as an explanation of why I often choose not to use log for low light. But instead it’s ended up as an experiment you can try for yourself if you have a waveform monitor that will hopefully allow you to better understand the differences between log and standard gamma. Get a waveform display hooked up to your log camera and try this for yourself.

S-Log and other log gammas are wonderful things, but they are not the be-all and end-all of video gammas. They are designed for one specific purpose and that is to give cameras using conventional YCbCr or RGB recording methods the ability to record the greatest possible dynamic range with a limited amount of data, as a result there are some compromises made when using log. Unlike conventional gammas with a knee or gammas such as hypergammas and cinegammas, log gammas do not normally have any highlight roll off, but do have a shadow roll-off. Once you get above middle grey log gammas normally record every stop with almost exactly the same amount of data, right up to the clipping point where they hard clip. Below middle grey there is a roll off of data per stop as you go down towards the black clip point (as there is naturally less information in the shadows this is expected). So in many respects log gammas are almost the reverse of standard gammas. The highlight roll off that you may believe that you see with log is often just the natural way that real world highlights roll off anyway, after all there isn’t an infinite amount of light floating around (thank goodness). Or that apparent roll off is simply a display or LUT limitation.

An experiment for you to try.

xdcam-greyscale-300x169 Deeper Understanding Of Log Gamma. Experiments with a Waveform Display.
Click on the chart to go to larger versions that you can download. Display it full screen on your computer and use it as a test chart. You may need to de-focus the camera slightly to avoid aliasing from the screens pixels.

If you have a waveform display and a grey scale chart you can actually see this behaviour. If you don’t have a chart use the grey scale posted here full screen on your computer monitor. Start with a conventional gamma, preferably REC-709. Point the camera at the chart and gradually open up the aperture. With normal gammas as you open the aperture you will see the steps between each grey bar open up and the steps spread apart until you reach the knee point, typically at 90% (assuming the knee is ON which is the default for most cameras).  Once you hit the knee all those steps rapidly squash back together again.

What you are seeing on the waveform is conventional gamma behaviour where for each stop you go up in exposure you almost double the amount of data recorded, thus capturing the real world very accurately (although only within a limited range). Once you hit the knee everything is compressed together to increase the dynamic range using only a very small recording range, leaving the shadows and all important mid range well recorded. It’s this highlight compression that gives video the “video look”, washed out highlights with no contrast that look electronic.

If you repeat the same exercise with a hypergamma or cinegamma once again in the lower and mid range you will see the steps stretch apart on the waveform as you increase the exposure. But once you get to about 65-70% they stop stretching apart and now start to squeeze together. This is the highlight roll off of the hypergamma/cinegamma doing it’s thing. Once again compressing the highlights to get a greater dynamic range but doing this in a progressive gradual manner that tends to look much nicer than the hard knee. Even though this does look better than 709 + Knee in the vast majority of cases, we are still compressing the highlights, still throwing away a lot of data or highlight picture information that can never be recovered in post production no matter what you do.

Conventional video = Protect Your Highlights.

So in the conventional video world we are taught as cameramen to “protect the highlights”. Never overexpose because it looks bad and even grading won’t help a lot. If anything we will often err on the side of caution and expose a little low to avoid highlight issues. If you are using a Hypergamma or Cinegamma you really need to be careful with skin tones to keep them below that 65-70% beginning of the highlight roll off.

Now repeat the same experiment with Slog2 or S-log3. S-log2 is best for the experiment as it shows what is going on most clearly. Before you do it though mark middle grey on your waveform display with a piece of tape or similar. Middle grey for S-log2 is 32% (41% for S-log3).

Now open up the aperture and watch those steps between the grey scale bars. Below middle grey, as with the standard gammas you will see the gap between each bar open up. But take careful note of what happens above middle grey. Once you get above middle grey and all the way to the clip point the gap between each step remains the same.

So what’s happening now?

Well this is the S-log curve recording each stop above middle grey with the same amount of data. In addition there is NO highlight roll off. Even the very brightest step just below clipping will be same size as the one just above middle grey. In practice what this means is that it doesn’t make a great deal of difference where you expose for example skin tones, provided they are above middle grey and below clipping. After grading it will look more or less the same. In addition it means that that very brightest stop contains a lot of great, useable picture information. Compare that to Rec-709 or the Cinegammas/Hypergammas where the brightest  stops are all squashed together and contain almost no contrast or picture information.

300x250_xdcam_150dpi Deeper Understanding Of Log Gamma. Experiments with a Waveform Display.

Now add in to the equation what is going on in the shadows. Log has less data in the shadows than standard gammas because you are recording a greater overall dynamic range, so each stop is recorded with overall less data.

Standard Gammas = More shadow data per stop, much less highlight data = Need to protect highlights.

Log= Less shadow data per stop, much more highlight data = Need to protect shadows.

Hopefully now you can see that with S-log we need to flip the way we shoot from protecting highlights to protecting shadows. When you shoot with conventional gammas most people expose so the mid range is OK, then take a look at the highlights to make sure they are not too bright and largely ignore whats going on in the shadows. With Log you need to do the opposite. Expose the mid range and then check the shadows to make sure they are not too dark. You can ignore the highlights.

Yes, thats’ right, when shooting log: IGNORE the highlights!

WF-Cinegamma-3-1024x576 Deeper Understanding Of Log Gamma. Experiments with a Waveform Display.
Cinegamma highlight roll off. Note how the tree branches in the highlights look strangled and ugly due to the lack of highlight data, hence “protect your highlights”.
WF-Slog-graded-1024x576 Deeper Understanding Of Log Gamma. Experiments with a Waveform Display.
Graded S-Log2. Note how nice the same tree branches look because there is a lot of data in the highlights, but the shadows are a little crunchy. Hence: protect your shadows.

For a start you monitor or viewfinder isn’t going to be able to accurately reproduce the highlights as bright as they are . So typically they will look a lot more over exposed than they really are. In addition there is a ton of data in those highlights that you will be able to extract in the grade. But most importantly if you do underexpose your mid range will suffer, it will get noisy and your shadows will look terrible because there will be no data to work with.

When I shoot with log I always over expose by at least 1 stop above the manufacturer recommended levels. If you are using S-log2 or S-log3 that can be achieved by setting zebras to 70% and then checking that you are JUST starting to see zebras on something white in your shot such as a white shirt or piece of paper. If your camera has CineEI use an EI that is half of the cameras native ISO (I use 1000 or 800 EI for my FS7 or F5).

I hope these experiments with a grey scale and waveform help you understand what is going on with you gamma curves. One thing I will add is that while controlled over exposure is beneficial it can lead to some issues with grading. That’s because most LUT’s are designed for “correct” exposure so will typically look over exposed. Another issue is that if you simply reduce the gain level in post to compensate than the graded footage looks flat and washed out. This is because you are applying a linear correction to log footage. Fo a long tome I struggled to get pleasing results from over exposed log footage. The secret is to either use LUT’s that are offset to compensate for over exposure or to de-log the footage prior to grading using an S-Curve. I’ll cover both of these in a later article.

S-log-levels Deeper Understanding Of Log Gamma. Experiments with a Waveform Display.
Chart showing S-Log2 and S-Log3 plotted against f-stops and code values.

 

 

 

 

 

 

 

 

 

What about shooting in low light?

OK, now lets imagine we are shooting a dark or low light scene. It’s dark enough that even if we open the aperture all the way the brightest parts of the scene (ignoring things like street lights) do not reach clipping (92% with S-Log3 or 109% with S-Log2). This means two things. 1: The scene has a dynamic range less than 14 stops and 2: We are not utilising all of the recording data available to us. We are wasting data.

Log exposed so that the scene fills the entire curve puts around 100 code values (or luma shades) per stop above middle grey for S-log2 and 75 code values for S-Log3 with a 10 bit codec. If your codec is only 8 bit then that becomes 25 for S-log2 and 19 code values for S-Log3. And that’s ONLY if you are recording a signal that fills the full range from black clip to white clip.

3 stops below middle grey there is very little data, about thirty 10 bit code values for S-Log2 and about 45 for S-log3. Once again if the codec is 8 bit you have much less, about 7 for S-Log2 and about 11 for S-log2. As a result the darker parts of your recorded scene will be recorded with very little data and very few shades. This impacts how much you can grade the image in post as there is very little picture information in the darker parts of the shot and noise tends to look quite coarse as it is only recorded with a limited number of steps or levels (this is particularly true of 8 bit codecs and an area where 8 bit recordings can be problematic).

So what happens if we use a standard gamma curve?

Lets say we now shoot the same scene with a standard gamma curve, perhaps REC-709. One point to note with Sony cameras like the FS5, FS7, F5/F55 etc is that the standard gammas normally have a native ISO one to two stops lower than S-Log. That’s because the standard gammas ignore the darkest couple of stops that are recorded when in log. After all there is very little really useable picture information down there in all the noise.

Now our limited dynamic range scene will be filling much more of our recording range. So straight away we have more data per stop because we are utilising a bigger portion of the recording range. In addition because our recorded levels will be higher in our recording range there will be more data per stop, typically double the data especially in the darker parts of the recorded image. This means than any noise is recorded more accurately which results in smoother looking noise. It also means there is more data available for any post production manipulation.

But what about those dark scenes with problem highlights such as street lights?

This an area where Cinegammas or Hypergammas are very useful. The problem highlights like strret lights normally only make up a very small part of your your overall scene. So unless you are shooting for HDR display it’s a huge waste to use S-log just to bring some highlights into range as you make big compromises to the rest of the image and you’ll never be able to show them accurately in the finished image anyway as they will exceed the dynamic range of the TV display.  Instead for these situations a Hypergamma or Cinegamma works well because below about 70% exposure Hypergammas and cinegammas are very similar to Rec-709 so you will have lots of data in the shadows and mid range where you really need it. The highlights will be up in the highlight roll off area where the data levels or number of recorded shades are rolled off. So the highlights still get recorded, perhaps without clipping, but you are only giving away a small amount of data to do this. The highlights possibly won’t look quite as nice as if recorded with log, but they are typically only a small part of the scene and the rest of the scene especially the shadows and mid tones will end up looking much better as the noise will be smoother and there will be more data in that all important mid-range.

 

460x150_xdcam_150dpi Deeper Understanding Of Log Gamma. Experiments with a Waveform Display.

23 thoughts on “Deeper Understanding Of Log Gamma. Experiments with a Waveform Display.”

  1. I’m a bit confused, above you state:
    “When I shoot with log I always over expose by at least 1 stop above the manufacturer recommended levels. If you are using S-log2 or S-log3 that can be achieved by setting zebras to 70% and then checking that you are JUST starting to see zebras on something white in your shot such as a white shirt or piece of paper.”
    I would think that setting your exposure so that a white shirt is at 70% would be underexposing, not overexposing…
    Thanks in advance
    -Matt

  2. Hi Alister,
    Following your explenation here and in the excellent VOCAS talks https://youtu.be/U-HrXmRqpME I have one question about uncompressed recording using an external recorder. I understand s-log is a compression log curve. Cinegamma a different compression curve. So it is the way the uncompressed signal is compressed to save space.

    Recording uncompressed would mean: a straight compression line over the exposure range?

    How does this work with dual compressed (card) / uncompressed recording (just received my BM video assist recently which can record the uncompressed 8bit signal 4:2:2 and record as prores 10 bit 4:2:2.) How do I set exposure etc etc. When do I really benefit (I don’t see any apparent improvements in quality first hand).
    Stefan

    1. Gamma determines the cameras response to light, how it records contrast and the light range it can record. Gamma is added to the video signal to reduce the amount of data needed to record the signal and to make the signal compatible with existing video monitors and displays before it is passed to the video outputs or video codec for compression. So even recording uncompressed on and external recorder, gamma is still applied to the signal.

      The ONLY exception to this is when you record the direct signal off the sensor using RAW. There will be a firmware update in the future for the FS5 to enable an external raw recorder to do this. The FS7 can output raw for an external recorder or the FS5/F55 can have a raw recorder attached directly to them.

  3. I’m a little confused about S-LOG and the color gamuts that the A7s-II and FS5 has. Does much of this great information apply to the luma channel only? But what about chroma and RGB? Is a similar curve applied to each RGB channel? If so, or if not, what is the relationship between luma and the chroma channel curves in reference to S-LOG?

    I’m finding the new S-Gamut3 on the A7s-II is much better than any other gamut settings I have used in the past. S-Gamut3 seems to protect the blue channel from clipping in low light. Example: Police car blue lights or LED blue lights in a concert would always clip VERY…and I mean VERY harshly with other S-Gamuts before. S-Gamut3 seems to do something very different to protect that blue channel form clipping.

    I’m very interested in knowing more about S-Gamut3 and S-LOG2/3 gamma curves.

  4. Would really love to see a instructional video on Sony PP in general…many of the questions I am seeing posted are the exact same issues as the fs100 only now we have even more choices gamma, cine colors, log…. Would happily pay for it (I know it would be a great deal of work so $50-$150 depending on how in depth you would go… From what I have seen on the forums I am not alone here and really feel like you could get a pretty decent amount of people to purchase. Anyway. thank you for all the post they have been a huge help over the years.

    1. Trying to get Sony to sponsor a video on picture profiles. I think they will do it and if so the video would be ready end of Jan.

  5. Alister,

    Fantastic information on Log2/Log3. The 70% zebra white object exposure will be very helpful for correct exposure. Thanks for the excellent information you provide.

  6. Hi.
    As a 61 year old expat from the UK,I live in the lower northern region of Thailand and am in the process of setting up an editing suite to do editing work and color grading work for my own small film business and to offer our editing services as the nearest facilities are in Bangkok some 355km away.

    We use the Sony FS7 and Canon 5D MK III as our main cameras and have many accessories from Zacuto and other well known brands our filming as well as the Inspire 1 Drone. Where necessary, we use the GoPro Black Hero as well.

    For editing I have the full spec 5K Retina iMac and an excellent viewing room with high end sound system as well as a 65 inch Sony HD TV, which I’m about to upgrade to at least 85 inch 4K.

    I’m now looking for a professional calibrated monitor and want to ask for your advice.

    The color Edge CG 318 – 4K looks like it has all the features we’d need, but is over £6000 here in Thailand. Is there another monitor you would recommend? We will want to move into RAW filming with the FS7 later next year, so need a monitor for now and the future. Most video businesses here are at the lower end of the market, so we are looking to produce high end products for our customers.

    Any other advice would be gratefully received.

    My Thai film crew and drone pilot are excellent cameramen, but not so up on editing technology as the cost has always been prohibitive for them.

    I’m very happy to pay you for your time in advising me if you can let me know how much and what the best way to make payment is for you.

    Many Thanks

    Anthony

  7. For the FS7, low light, what do you suggest?
    HG3 or HG8? Does the Matrix Preset matter? Cinema okay?

    Just curious too, is the Cinema Preset Matrix color space close to the DCI space?
    Thanks!

  8. I recently shot on a Sony a7s2 with everything correctly exposed in full sunlight with an nd filter but the picture quality is very bad with little contrast and a ton of colour noise. The sun was very bright though but everything is between 35 and 90 IRE. I’ve tried grading on resolve using aces but it’s still unusable

    1. What was your exposure based on? Where did you put middle grey or 90% white? If it’s exposed correctly it won’t be noisy.

      1. I used a white card and exposed to the usual range I get on the waveform using a shogun monitor and checking the zebra and histogram in the camera. The white was around 80 IRE. I had to stop down a lot using the ND filter as it was very afternoon summer sun so that may have been the problem. I couldn’t shoot any later in the day as the subject would have add areas of heavily contrasted shadows and sunlight. I guess ideally it should have been overcast. I also had to shoot at 3200 ISO in HD cropped as it was in slow motion at 100fps (I use an camera bought in Australia set for PAL) I think If I had shot in real time 4K to the shogun I would have been fine. I’m trying to coreect in Davinci Resolve grading in ACEs and using neat noise plugin but it’s only a bandaid solution.

        1. Ah, 100fps, that’s why it’s noisy. In the high frame rates the sensor readout is done differently and it’s never as good as the regular speeds. Then throw in 6dB of gain (1600 ISO to 3200 ISO) and you make the noise twice as bad.

          1. Yeah it seems to be ok if you shoot later in the day and close up – mid shot rather than wide. I’ve done it before in those conditions using a 50mm zeiss loxia and a shallow depth of field and although there was a little noise it looked pretty good. So I guess the lesson learned here is if shooting in HFR on the A7S2 don’t shoot wide landscapes. Shoot in full 4K then cut to a close up for slow mo unless you have super high end camera like a Phantom Flex? Thanks for your advice anyway Alex.

  9. All I can go by is my own tests with the camera. When I shoot with HG3 in a low-light situation, the image is dark and the noise is usually minimal. However, when I shoot with SLOG 3 EL CINE, the image is much brighter and the noise is significant. However, when I grade the SLOG 3 image to eliminate the noise, the image (and colors) is still superior to HG3. In short, I can grade SLOG 3 to look identical to HG3, if not better. I see a distinct different in color between HG3 and SLOG 3 EL CINE and so despite HG3 having more data, I don’t think the image looks better than a properly graded SLOG 3 image. Does this make sense? How is having more data in a more limited color space a better thing?

    1. Did you grade the HG3 footage? Did you try adding the same amount of gain (6dB) when shooting the Hypergamma, grading that and comparing that with the log? I’m not sure which camera you have but unless you have an F55 the color range difference is actually very small between 709 and SGamut because the sensors color filters cannot see that much beyond 709. The benefit from recording with a smaller recording color space is you have more tonal gradations. You can have SGamut.cine and only part fill it because of the sensor limitations, wasting some a lot of data or you can use 709 and fill it 100% and make use of all the data to record with greater accuracy. There are very few natural colors beyond 709.

      It isn’t just about noise, it’s also about tonal values. But if you feel that SLog works better for you, then that’s what you should use. You’ve done the right thing, performed a test and drawn your own conclusions. I wish more people would take the time to do this.

      1. Hi,

        How did you come to the conclusion that the bayer filter limits color aquisition? If you have some tests please share them! Are the bayer filters on the A7 line much different than the FS line? I deal with mostly AlexalogC footage at work ( VFX ) and those plates certainly capture colors well past rec709 gamut. In fact, I’ve encountered several plates that clip the AlexalogC gamut. Obviously, they are not Sony cameras but I would be surprised if they were terribly different.

        1. Because I’ve measured the gamut of the FS series cameras using a Leader LV5600. It’s also stated by Sony that the colour filters used on the FS sensor cannot capture colours significantly beyond Rec-709.

          You have to remember that a bayer sensor is a luma only sensor that relies on colour filters to isolate individual parts of the colour spectrum, before the light reaches the luma only pixels. So the colour filters are critical to colour range and response as their ability to isolate one colour from another has a huge impact on the gamut. Too much cross colour contamination results in a high luma level at the pixel but poor chroma separation, so you get a big luma range but small colour range.

          I’m curious as to what colours are so far out of Rec-709 that they have resulted in colour space clipping. Pointers Gamut suggests that in the real world there are very few colours significantly beyond Rec-709 anyway.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.