Advertisements

Brewing up a Scene File: Gamma and Knee

Before anyone complains that I have missed stuff out or that some technical detail is not quite right, one of the things I’m trying to do here is simplify the hows and why’s to try and make it easier for the less technical people out there. Lets face it this is an art form, not a science (well actually a bit of both really).

So what is a gamma curve anyway? Well the good old fashioned cathode ray tube television was a very non-linear device. You put 1 unit of power in and get one unit of light out. You put 2 units in and get 1.5 units out, put 3 in and get 2 out… and so on. So in order to get a natural picture the output of the camera also has to be modified to compensate for this. This compensation is the gamma curve, an artificial modification of the output signal from the camera to make it match TV’s and monitors around the world. See Wikipedia for a fuller explaination:   http://en.wikipedia.org/wiki/Gamma_correction

So, all video cameras will have a gamma curve, whether you can adjust it or not is another matter. Certainly most pro level cameras allow you some form of gamma adjustment.

The PMW-350 has 6 standard gamma curves, these are all pretty similar, they have to be otherwise the pictures wouldn’t look right, but small changes in the curve effect the relationship between dark and bright parts of the pictures. Todays modern cameras have a far greater dynamic range (range of dark to bright) than older cameras. This means that the full dynamic range of the sensor no longer fits within the gamma curves used for TV’s and monitors. In broadcast television any signal that goes over 100% gets clipped off and is discarded, so the cameras entire brightness range has to be squeezed into 0 to 100%. The PMW-350 sensors are capable of far more than this (at least 600%) so what can you do?

The older and simpler solution is called the “Knee”. The knee works because in most cases the brightest parts of a scene contains little detail and is generally ignored by our brains. We humans tend to focus on mid-tone faces, animals and plants rather than the bright sky. Because of this you can compress the highlights (bright) parts of the picture quite heavily without it looking hugely un-natural (most of the time at least). What the knee does is takes a standard gamma curve and up near it’s top, bends it over. This has the effect of compressing the brighter parts of the image, squashing a broad range of highlights (clouds for example) into a narrow range of brightness. While this works fairly well, it does tend to look rather “electronic” as the picture is either natural (below the knee) or compressed (above the knee).

The answer to this electronic video look is to replace the hard knee with gentle bend to the gamma curve. This bend starts some way down the gamma curve, very gentle at first but getting harder and harder as you go up the gamma curve. This has the effect of compressing the image gently at first with the compression getting stronger and stronger as you go up the curve. This looks a lot more natural than a hard knee and is far closer to the way film handles highlights. The downside is that because the compression starts earlier a wider tonal range is compressed. This makes the pictures look flat and uninteresting. You have to watch exposure on faces as these can creep into the compressed part of the curve. The plus point is that it’s possible to squeeze large amounts of latitude into the 100% video range. This video can then be worked on in post production by the editor or colorist who can pull out the tonal range that best suits the production.

These compressed gamma curves are given different names on different products. Panasonic call them “Film Rec”, on the EX1 they are “Cinegammas” on the PMW-350 they are “Hypergammas”. The 350 has four Hypergammas. The first is 3250. this takes a brightness range the equivalent to 325% and compresses it down to 100%. HG 4600 takes 460% and squeezes that down to 100%. Both of these Hypergammas are “broadcast safe” and the recordings made with them can be broadcast straight from the camera without any issues. The next Hypergamma is 3259. This takes a 325% range and squeezes this down to a 109% range, likewise 4609 takes 460% down to 109%. But why 109%? well the extra 9% gives you almost 10% more data to work with in post production compared to broadcast safe 100%. It also gives you the peak white level you need for display on the internet. Of course if you are doing a broadcast show you will need to ensure that the video levels in the finished programme don’t exceed 100%.

My preferred gamma is Hypergamma 4 (4609) as this gives the maximum dynamic range and gives a natural look, however the pictures can look a little flat so if I’m going direct from the camera to finished video without grading I use either a standard gamma or use the Black Gamma function to modify the curve. I’ll explain the Black Gamma in my next post.

There are 6 standard gammas to choose from. I like to stick with gamma 5 which is the ITU-709 HD standard gamma. To increase the dynamic range I use the Knee. The default knee point setting is 90, this is a reasonable setting, but if your shooting with clipping set to 100% you are not getting all the cameras latitude (the Knee at 90 works very well with clipping at 108%). Lowering the knee down to 83 gives you almost another stop of latitude, but you have to be careful as skin tones and faces can creep up towards 83%. It’s very noticeable if skin becomes compressed so you need to watch your exposure. This is also true of the Hypergammas and with them you may need to underexpose faces very slightly. The other option is to set the knee point to 88 and then also adjust the knee slope. The slope is the compression amount. A positive value is more compressed, negative less compressed. With the knee at 88 and slope set to +20 you get good latitude, albeit with quite highly compressed highlights.

If you want to play with the gammas and knee and see how they work one method you can use is to use a paint package on your PC (such as photoshop) to create a full screen left to right graduated image going from Black to white. Then shoot this with the camera (slightly out of focus) while making adjustments to the curves or knee and record the results along with a vocal description of each setting. Import the clips into your favorite editing package and use the waveform monitor or scopes you should be able to see a reasonable representation of the shape of the gamma curve and knee.

So my Gamma Choices are:

For material that will be post produced: Hypergamma 4609 (HG4)

For material that will be used straight from the camera: Standard Gamma 5 Knee at 90 with clip at 108% for non broadcast or Knee at 88 with slope +20 with white clip at 100% for direct to broadcast.

Advertisements

PMW-350 Detail Settings


I have finally managed to get my hands on a production PMW-350. I am going to start dialing it in. The first thing to address for me is the over sharpened pictures, so I have been playing with the Paint settings aiming towards a natural, yet sharp look. I have come up with these detail setings. Everything is default except:

Detail Level -16?H/V Ratio +20?Detail Frequency +35?White limit +39?Black Limit +20?Aperture -30

This is still a work in progress.

Next I’m going to start looking at the Gamma curves and Knee. I have a nice Hamlet MicroFlex scope to help with this. Previously I have had to rely on my eye and then check the footage against the scopes in the edit suite, now I can see the waveforms on location. I’ll be writting up both the gamma settings and a microflex review in due course.

PMW-350 Scene Files. Two to get you started.

With the PMW-350 now starting to ship I thought I would dig out the settings I used when I did my 350 review. I created two scene files and the details are as follows:

File One: Aimed at giving high latitude with deep almost crushed blacks and the image well saturated and slightly warmed up.

Detail: ON    Aperture: ON    Detail Level: -15     Aperture Level: -10     Detail Frequency: +24

Matrix: ON      Matrix (User): ON      Matrix (Preset): ON      Matrix (Prst) Sel: 6

R-G 1, R-B 12, G-R 2, G-B 11, B-R 0, B-G 0

Gamma: ON         Gamma Table: STD           Gam Table (STD): 5

Black Gamma: ON       Black Gamma Range: HIGH        Master BLK Gamma -24

Knee: ON     Knee Point: 85.2      Knee Slope: -14

White Clip: ON     White Clip Level: 109.0 (If will be graded) 104 (if not graded or for broadcast)

Master Black: -2

File Two: Aimed at giving maximum latitude with deep but not crushed blacks and vivid slightly warmed up colours.

Matrix: ON      Matrix (User): ON      Matrix (Preset): OFF      Matrix (Prst) Sel: 1

R-G 8, R-B 10, G-R 0, G-B 15, B-R 5, B-G 6

Gamma: ON         Gamma Table: HG           Gam Table (HG): 4

Black Gamma: ON       Black Gamma Range: H.MID        Master BLK Gamma -28

White Clip: ON     White Clip Level: 109.0 (If will be graded) 104 (if not graded or for broadcast)

Master Black: -3

Detail: ON    Aperture: ON    Detail Level: -15     Aperture Level: -10     Detail Frequency: +24

These settings were created on a pre-production PMW-350 so it would be wise to try them and look at them before using them in anger. All other settings are default or 0.

HD, SD and Depth of Field.


I was reminded of this by Perrone Ford on DVINFO.net. With HD cameras compared to SD cameras the depth of field appears shallower. Why is this and why is it important?

Visually depth of field is the loss of focus as you move away from the object that you have focussed on. If you have two cameras, one HD and one SD and they both have the same lens at the same aperture along with sensors of the same size then the change in focus with distance for both cameras will be exactly the same. However with the HD camera, because the image is sharper to start with, any small changes in focus will be more apparent than with the softer picture from the SD camera. So visually the HD camera will have a shallower depth of field. Now if you take that HD image and convert it to SD then the depth of field appears to increase again. This can be calculated and measured and is defined by the “circle of confusion”

So why is this important? Well lets look at what happens when you shoot an interview or face. The human brain is very good at looking at faces, we “read” faces day in and day out, taking in expressions, skin tone and subtle changes. We use these tiny visual cues to gauge emotion and see how someone is responding to the things that we do. Because of this any imperfection in the look of a face in a video tends to stand out (thats also why you normally expose for faces). With HD it’s quite possible to have a shot of a face where the tip of the persons nose or their ears are in sharp focus while the eyes are slightly soft. With an SD image we would be unlikely to notice this because of the greater depth of field, but HD with it’s visually shallower DoF can show up this small difference in focus and our brain flags it up. Very often you see the HD face and it looks OK, but something in your brain tells you it’s not quite right as the eyes are not quite as sharp as the nose or ears. So this apparently shallower DoF means that you can’t just focus on a face with HD but you must focus on the eyes, as that’s where we normally look when engaged in a conversation with someone.

PXU-MS240 SxS Backup device. First Impressions.


I have been playing with a Sony PXU-MS240 SxS backup device. It’s quite different to my NextoDi NVS2500 even though it essentially does the same job. I will be reviewing it in some detail very soon, but here are my first thoughts.

The key feature is that unit has a removable 240Gb hard drive module. Extra drives are readily available and the removable drives can be used as stand-alone USB hard drives without the main unit. Each hard drive cartridge comes in a sturdy box that is much like a Betacam cassette box. There is space on the drives for labels and the box has an insert sleeve that can be used to write on, just like a tape. Clearly this has been done so that as you fill up drives you can pop them on a shelf for longer term storage as you would with a tape. The beauty of the MS240 is that you never need to off load footage, you just add cartridges as you fill them up.

The main unit is 12 volt powered or can run off a standard EX battery. There is a slot at the front for a SxS card and a big Copy button on the top panel along with the power button and menu controls. There is also a small and very clear LCD display that tells you what the unit is doing. In the setup menu you can choose whether to simply copy the SxS cards contents or to do a copy with full verification in one pass.

Another way to verify your clips is to plug it in to an EX camera. The MS240 is supplied with a USB to Express card adapter. You plug the adapter into an EX’s SxS slot and the USB end into the MS240 and then you can use the EX to  playback any clips on the  MS240 in full HD. This is something the Nexto cannot do. It also means that you could use the MS240 to store finished edits for playback via an EX over HDSDi.

The build quality is good and the range of connectivity is also good with eSATA and USB on the main unit and USB on the cartridges. A 16Gb card can be copied to the drive in around 5 mins.

Getting SD from HD and the problems of oversampling.


Ever since the release of the XDCAM EX cameras users have been having problems getting good looking SD pictures out of downconverted HD.?Why is this and what can be done about it? This is an issue that effects all high resolution HD cameras and is not unique to the EX’s. There are two key issues. The first is the way basic software converters handle fields in interlace material and the second is the amount of information in an HD image that must in effect be discarded to get a SD image.?At first glance you would think that starting off with lots of picture detail would be a good thing, but in this case it’s not. Let’s see if I can explain.?Imagine that you have something in you HD picture that over 4 pixels goes from light to dark, in Hd you get a gradual transition from light to dark and all looks good. Now what happens when you take those 4 pixels and convert them to SD. The 4 pixels become just 2 and instead of a stepped change from light to dark the picture now goes instantly from a light pixel to a dark pixel. If these pixels were the edge of a moving object, as it moved the pixels would be switching instantly from on to off and unless the object moved at exactly one pixel per frame you will get a flickering effect. Clearly our nice gradual transition from light to dark has been lost and if there is any motion we may now be seeing flickering edges. Niether of these look good.

Take a look at these images:

hd-sd-full-frame Getting SD from HD and the problems of oversampling.
Original Frame showing box with area of interest
hd-sd-original Getting SD from HD and the problems of oversampling.
Original HD Image
hd-sd-sd-no-blur Getting SD from HD and the problems of oversampling.
Same image, downconverted to SD

As you can see the down converted SD is very blocky and there is some strange patterning (aliasing) going on amongst the bricks of the houses in the background. This does not look good and if there was motion the brickwork would shimmer and flicker.

So what can be done?

Well the best way to improve the SD down conversion is to soften the HD image before it is down converted to prevent this single pixel light to dark switch from happening. You need to end up with an SD image where you go from full light to full dark over at least 3 pixels to prevent flicker (Twitter).

How much you will need to soften you HD by will depend on how sharp it is to start with. Simply turning down the cameras detail settings can be a big help, but even then the best results are often obtained by applying some kind of blur filter in post production. In FCP I find the flicker filter works quite well. As you can see from the frame grab below the difference in the quality of the downconvert is quite striking.

hd-sd-SD-blur Getting SD from HD and the problems of oversampling.
SD Image created by adding blur to HD before conversion.

I have also found that another problem is that the detail settings on an HD camera are not optimised for SD. The detail correction edges created in HD are very thin and when these are down converted to SD they all but disappear and can cause further aliasing. The solution is to make the detail correction edges thicker (on an EX turn detail frequency down to -60 to -99) but this then looks ugly in HD. The bottom line is that a camera optimised for HD works best in HD and SD will be a compromise.

PMW-350 – Good for Indie Movies?

I have been giving a lot of thought to the PMW-350 and whether it fits in with what I do. I’m really, really torn. While I love my PDW-700 and the optical disc workflow I also really liked the PMW-350 package. Now to bring my PDW-700 up to a similar package would mean purchasing a colour viewfinder (C35W £6,500) and a new ALAC compatible HD lens (£8k +). The cost of these options is around the same price as a PMW-350 kit with lens. The 350 has a wonderful colour VF as standard and is very nice to use. It has much lower power consumption than the 700 and weighs a lot less, which would be very nice for me with all the traveling I do.

350-nano-right-side-300x200 PMW-350 – Good for Indie Movies?
PMW-350 – With Convergent Designs NanoFlash

The downside to the PMW-350 is the 35 Mb/s data rate and the use of CMOS sensors. The 35 Mb/s issue is easy enough to get around as I have a couple of NanoFlashes which can record at higher bit rates, making the 350 suitable for HD broadcast without any issues. The CMOS sensors, for most people would not really be an issue, especially now Sony have incorporated flash band removal into the clip browser software, but for me it’s important because of the amount of lightning I shoot. Certainly I can get good results with my EX1 and EX3 but I really don’t know yet how the 350 will perform.

While thinking about all this, it occurred to me that the 350 would probably make a really good camera for Indie Films. The use of 2/3? sensors means that it’s easier to get a reasonably shallow depth of field compared to 1/3? or even 1/2? cameras, without having to resort to 35mm lens adapters. The very high resolution images with very, very low noise would certainly look good projected on to a big screen. If you took the HDSDi output and record it on a NanoFlash at 100Mb/s or higher then it will grade very well. The lack of camera noise (59db) means that you can really push and pull the picture very hard during grading before it will degrade. Even if you only record at 35 Mb/s this low noise floor is going to help with grading as less noise means less stress on the codec in the first place. On top of all that the supurb sensitivity means that you will be able to use very minimal lighting rigs, perhaps just using practicals to light a scene, which should really give you more scope in your composition.

Further PMW-350 advantages include the use of Sony’s Hypergammas, step gamma and multi-matrix. These settings are normally only found right at the high end of the product range and give you excellent control of the look of your images. Add to that 24P (23.98P) and the ability to overcrank and undercrank and it all adds up to an extremely capable camera.

So, will I be getting a PMW-350? Well if I didn’t have my PDW-700 I would certainly be getting a 350. At the moment I’m still undecided….. if only I could work out a way to afford both.

Working with XDCAM EX (or other file based) material.

The one big question of everyones lips is how do you archive your material? Well here are my thoughts and some ideas.

The most important thing is to think ahead and plan your end to end workflow. You also need to consider the fact that hard drives will almost certainly fail at some point (maybe not now but in the future) and the vast majority of problems are due to human error. Possibly simply forgetting to copy something or not fully understanding the workflow.

Some Golden Rules, no matter what format or workflow you are using are:

Copy EVERYTHING off the card, keeping the original file structure.

Ensure all copies incorporate some kind of error correction or error checking.

Don’t skimp on the quality of your backup system.

Check, check and double check your workflow before you start shooting.

So you go on a shoot and start filling up you expensive memory cards, at some point you will have to start off loading your material onto something else. In the field this is likely to be hard drives of some sort. Backing up to a single hard drive should only be done as a last resort or for media that you don’t mind loosing. You have several options here, you could use Shotput Pro to backup to single or multiple drives. I really like shotput as you can use it to eliminate a lot of user errors. For a start shotput can be set to backup to multiple locations simultaneously from the source media. Then once it has made the copies and verified the copies it can, if you wish, format the card, ready for re-use. Allowing Shotput to format the media helps prevent human error. How? Well if I ever put a card in my camera and find it has footage on it, it means that card has not been backed up and verified by Shotput. This is better than backing up yourself as there is always the risk of a mix up between backed up and not backed up cards. The other way to backup with a computer is to use the Sony XDCAM EX Clip Browser. You should never use the windows explorer or Mac finder to backup your valuable media as there is no form of error checking. Clip Browser has built in error checking which is enabled under the preferences tab.

A further option is to use a dedicated backup device such as the NextoDi products or soon to be released Sony PXU-MS240 backup device. These are easier to use than taking a laptop into the field. The NextoDi devices can backup to 2 drives at once (full review of the NVS2500 comming soon) and the Sony device backs up to removable esata drive cartridges.

So what sort of hard drives should you use? Well I am currently using pairs of USB Western Digital “Elements” hard drives. Where possible I use 3.5? drives as opposed to the smaller 2.5? laptop type drives. These are low cost yet so far have proven to be reliable and of good quality. The larger 3.3? drives should be more reliable, but they are bigger and bulkier and require mains power, so in the field I use the 2.5? drives. By storing these drives at separate locations, one at home and one in the office, I have a very safe system. If my office were to burn down or get flooded, I would have a spare copy at home. Over time however these drive will fail so every couple of years I move my footage on to new, larger hard drives. Another hard drive option is to use G-Tech G-Raid drives. These units contain two separate hard drives and can be used in raid 1 mode so should one of the drives fail your data should be safe. The cost is similar to using a pair of drives and it’s certainly less fiddly than using pairs of drives but it doesn’t give the security of separate storage locations. If you are doing corporate videos then you could consider selling drives to your clients. The client then keeps the drive and as a result you are no longer responsible for it’s storage or safety, just like if the client kept your rushes tapes.

For longer term storage, again there are many options. I backup a lot of my material to BluRay discs. This is not a fast process, use high quality discs and you should be good for 20+ years. Another option is to backup to Sony Professional discs using a Sony PMW-U1 drive. This is a lot faster than most current BluRay burners and the discs are protected in a rugged caddy. Sony claim a life of 50 years for the discs so it is a very good long term storage solution. The new Sony PMW-350 and EX1R as well as the Convergent Design NanoFlash (next firmware release)  have shooting modes that allow footage to be saved on XDCAM discs (Sony Professional Discs) as video clips and not just data files. Using these modes you can put the discs in a player and play back the material directly.

A further long term storage solution is LTO tape. It seems strange to be going back to tape, but LTO4 tape is very reliable and widely supported. It’s not suited to applications where you need quick access to your footage, but is very good for long term security. A good compromise may be one copy on a hard drive as a working copy along with a backup on LTO for archive.

Raid Arrays can be used for long term storage, but even Raid arrays can fail. If the lookup table becomes corrupted it can be next to impossible to recover the data off the discs, so do be careful. Do remember however you store your footage try and be organised. Store your material in a sensible folder structure that will help you find your rushes quickly and easily. If you are out shooting for a day you may be generating a hundred or more files, do that day in, day out and you will generate thousands and thousands of files. Make sure you work out you clip naming and clip prefixes in such a way that you won’t get duplicate names and can find your footage quickly and simply.

And just one more reminder, always save the full file structure. In the case of XDCAM EX keep the full BPAV folder and all it’s contents, also don’t rename the BPAV folder. Even if you edit on a Mac and use the Sony Transfer Tool to make .mov files you should keep the BPAV folders as trying to edit the  .movs on a PC or AVID is a nightmare. If you have the original material you can easily work with it on any platform.

Camera setup, reviews, tutorials and information for pro camcorder users from Alister Chapman.