Recovering footage from formatted cards.

Following a series of recent discussions about whether or not it was possible to recover files from XQD cards that have been formatted by mistake I have obtained some clarification from Sony of what can or can’t be done.

This information is specifically for XQD cards and the PXW-FS7 but probably applies to most Sony cameras and also SxS media. I’m not sure about SD cards.

Formatted In-Camera:

The bottom line is that if you format the card in the camera you will not be able to recover any previously shot material. An in-camera format completely erases everything on the card. This is done to ensure that material shot on the cards cannot be recovered by another production company in the case of card or camera rentals. So there is no point in attempting any form of data recovery on a card formatted in the camera as there is nothing recoverable left on the card.

Formatted by a computer:

When you format a card with a computer it is possible that the material will still be on the card. However different operating systems handle the formatting of the cards differently, so there is no guarantee that the data will be recoverable and often it won’t be recoverable. For very important material it may be worth attempting to recover the card. Sony may be able to assist with this in some cases.

Deleted Clips:

Clips deleted from a card can typically be recovered provided they have not be recorded over by a later recording. Again Sony may be able to assist with this.

Delete or Format?

Based on this new information from Sony I may be adjusting my workflow. My own workflow has always been to off-load material from a card. Then to do a parity check to compare the original files on the card and what is now on the hard drives. This checks not just the file size but also the general structure of the files so should pick up most problems with any copies. My last check is then to skim through the files with Catalyst Browse or my edit application to make sure the clips are there and playable. Only then do I format a card. In light of this new information I may use my computer to delete the clips from a card rather than format it. Of course this will only ever offer some benefit if the card is not recorded on again causing the previous files to be over written, but it might add an extra chance of data recovery should the backups get lost or some other disaster occur. From time to time I would format the cards in camera as this helps keep the cards in the best possible condition.

Do the images from my Sony camera have to look the way they do?

— And why do Sony cameras look the way they do?

It all about the color science.

“Color Science” is one of those currently in fashion phrases that gets thrown around all over the place today. First of all – what the heck is color science anyway? Simply put it’s how the camera sees the colors in a scene, mixes them together, records them – and then how your editing or grading software interprets what is in the recording and finally how the TV or other display device turns the digital values it receives back into a color image. It’s a combination of optical filters such as the low pass filter, color filters, sensor properties, how the sensor is read out and how the signals are electronically processed both in the camera, by your edit/grading system and by the display device. It is no one single thing, and it’s important to understand that your edit process also contributes to the overall color science.

Color Science is something we have been doing since the very first color cameras, it’s not anything new. However us end users now have a much greater ability to modify that color science thanks to better post production tools and in camera adjustments such as picture profiles or scene files.

Recently, Sony cameras have sometimes been seen by some as having less advanced or poor color science compared to cameras from some other manufacturers. Is this really the case? For Sony part of the color science issue is that historically Sony have deliberately designed their newest cameras to match previous generations of cameras so that a large organisation with multiple cameras can use new cameras without having them look radically different to their old ones. It has always been like this and all the manufacturers do this, Panasonic cameras have a certain look as do Canon etc. New and old Panasonics tend to look the same as do old and new Canon’s, but the Canon’s look different to the Panasonics which look different to the Sony’s.

Sony have a very long heritage in broadcast TV and that’s how their cameras look out of the box, like Rec-709 TV cameras with colors that are similar to the tube cameras they were producing 20 years ago. Sony’s broadcast color science is really very accurate – point one at a test chart such as a Chroma DuMonde and you’ll see highly repeatable, consistent and accurate color reproduction with all the vectors on a vector scope falling exactly where they should, including the skin tone line.

On the one hand this is great if you are that big multi-camera business wanting to add new cameras to old ones without problems, where you want your latest ENG or self-shooters cameras to have the same colors as your perhaps older studio cameras so that any video inserts into a studio show cut in and out smoothly with a consistent look.

But on the other hand it’s not so good if you are a one man band shooter that wants something that looks different. Plus accurate is not always “pretty” and you can’t get away from the fact that the pictures look like Rec-709 television pictures in a new world of digital cinematography where TV is perhaps seen as bad and the holy grail is now a very different kind of look that is more stylised and much less true to life.

So Sony have been a bit stuck. The standard look you get when you apply any of the standard off-the shelf S-Log3 or S-Log2 LUT’s will by design be based on the Sony color science of old, so you get the Sony look. Most edit and grading applications are using transforms for S-Log2/3 based on Sony’s old standard Rec-709 look to maintain this consistency of look. This isn’t a mistake. It’s by design, it’s a Sony camera so it’s supposed to look like other Sony cameras, not different.

But for many this isn’t what they want. They want a camera that looks different, perhaps the “film look” – whatever that is?

Recently we have seen two new cameras from Sony that out of the box look very different from all the others. Sony’s high end Venice camera and the lower cost FS5 MKII. The FS5 MKII in particular proves that it’s possible to have a very different look with Sony’s existing colour filters and sensors. The FS5 MK II has exactly the same sensor with exactly the same electronics as the MK I. The only difference is in the way the RGB data from the sensor is being processed and mixed together (determined by the different firmware in the Mk1 and mk2) to create the final output.

The sensors Sony manufacture and use are very good at capturing color. Sony sensors are found in cameras from many different manufacturers. The recording systems in the Sony cameras do a fine job of recording those colors as data within the files the camera records as data with different code values representing what the sensor saw. Take that data into almost any half decent grading software and you can change the way it looks by modifying the data values. In post production I can turn almost any color I want into any other color. It’s really up to us as to how we translate the code values in the files into the colors we see on the screen, especially when recording using Log or raw. A 3D LUT can change tones and hues very easily by shifting and modifying the code values. So really there is no reason why you have to have the Sony 709 look.

My Venice emulation LUT’s will make S-Log3 from an FS5 or FS7 look quite different to the old Sony Broadcast look. I also have LUT’s for Sony cameras that emulate different Fuji and Kodak film stocks, apply one of these and it really looks nothing like a Sony broadcast camera. Another alternative is to use a color managed workflow such as ACES which will attempt to make just about every camera on the market look the same applying the ACES film style look and highlight roll-off.

We have seen it time and time again where Sony footage has been graded well and it then becomes all but impossible to identify what camera shot it. If you have Netflix take a look at “The Crown” shot on Sony’s F55 (which has the same default Sony look as the FS5 MK1, FS7 etc). Most people find it hard to believe the Crown was shot with a Sony because it has not even the slightest hint of the old Sony broadcast look.

If you use default settings, standard LUT’s etc it will look like a Sony, it’s supposed to! But you have the freedom to choose from a vast range of alternative looks or better still create your own looks and styles with your own grading choices.

But for many this can prove tricky as often they will start with a standard Sony LUT or standard Sony transform. So the image they start with has the old Sony look. When you start to grade or adjust this it can sometimes look wrong because you have perhaps become used to the original Sony image and then anything else just doesn’t seem right, because it’s not what you are used to. In addition if you add a LUT and then grade, elements of the LUT’s look may be hard to remove, things like the highlight roll off will be hard baked into the material, so you need to do need to think carefully about how you use LUT’s. So try to break away from standard LUT’s. Try ACES or try some other starting point for your grade.

Going forward I think it is likely that we will see the new Venice look become standard across all of the Cinema style cameras from Sony, but it will take time for this to trickle down into all the grading and editing software that currently uses transforms for s-Log2/3 that are based on the old Sony Rec-709 broadcast look. But if you grade your footage for yourself you can create just about any look you want.

Black and White LUT set.

This was asked for on one of the online groups I follow. It’s a simple Black and White LUT that gives a wide dynamic range black and white image. The LUT has been designed to give a pleasing and gentle highlight roll-off with a reasonably contrasty mid range. I decided not to go too contrasty via the LUT so that more contrast can be added in post if needed. The output is legal range (broadcast safe). There is a single camera LUT for use in camera as well as a set of exposure compensated LUT’s going from -2 to plus 2 stops in half stop steps.

As always (to date at least) I offer these as a free download available by clicking at the bottom of the page. It takes time to create them and money to host them. I feel that this LUT set is worth $5.00 and would really appreciate that being paid if you find the LUT’s useful. But I will let you pay what you feel is fair, all contributions are greatly appreciated and it really does help keep this website up and running. If you can’t afford to pay, then just download the LUT’s and enjoy using them. If in the future you should choose to use them on a paying project, please remember where you got them and come back and make a contribution. More contributions means more LUT offerings in the future.

Please feel free to share a link to this page if you wish to share these LUT’s with anyone else or anywhere else.

To make a contribution please use the drop down menu here, there are several contribution levels to choose from.


Your choice:



Click here to download AC Black and White LUT’s V1

SEE ALSO:

Film Emulation LUT’s set 1.

Venice Look LUT’s V3 (lower contrast and minus green)

Venice Look LUT’s V1 (high contrast)

HLG HDR LUT for FS7, F5 and F55.

WYSIWYG LUT’s (Designed to be baked in)

Film Emulation LUT’s for S-Log3 and S-Log2.

I’ve uploaded these LUT’s before, but they are tucked away under a slightly obscure heading, so here they are again!

There are 4 different LUTs in this set. A basic 709 LUT which is really good for checking exposure etc. It won’t give the best image, but it’s really good for getting your exposure just right. Diffuse white should be 85%, middle grey 43% and skin tones 65-70%.

Then there are 3 film emulation LUT’s that mimic 3 different types of film stock form different manufacturers. These are primarily designed for post production or for use on a client monitor on set. My recommendation is to use the 709 LUT for your viewfinder and exposure and then add the film emulation LUT later in post.

As always (to date at least) I offer these as a free download available by clicking on the links below. However a lot of work goes into creating and hosting these. I feel that this LUT set is worth $25.00 and would really appreciate that being paid if you find the LUT’s useful. Try them before you decide then pay what you feel is fair. All contributions are greatly appreciated and it really does help keep this website up and running. If you can’t afford to pay, then just download the LUT’s and enjoy using them, tell your friends and send them here. If in the future you should choose to use them on a paying project, please remember where you got them and come back and make a contribution. More contributions means more LUT offerings in the future.

Click here for the Slog3_cine-film-look-luts

Click here for the S-Log2  filmic look luts – Alisters A7S Filmic LUTS set1

Download the S-Log3/SGamut3.cine to 709(800) and S709 Legal Input Range LUTS for external ProRes recorders.

Please feel free to share a link to this page if you wish to share these LUT’s with anyone else or anywhere else.

To make a contribution please use the drop down menu here, there are several contribution levels to choose from.


Your choice:



Out and about with the PXW-Z280.

Sony’s 4K PXW-Z280 handycam.

I have recently returned from a trip around Canada. While I was there I spent some more time shooting with Sony’s new PXW-Z280 handycam camcorder. This neat little camera continues to surprise me. I used a pre-production sample to shoot parts of an airshow in the summer and it worked really well. It was so easy to use, I had forgotten how much quicker it is to work with a camera with a 17x zoom lens compared to a large sensor camera with a very limited zoom range or prime lenses.

The Z280 uses 3x state of the art EMOR Stacked multi layer sensors. Each is full 4K, so you have full RGB 4K, unlike a single chip camera where the chroma resolution is much reduced by the bayer layout of the pixels. The 3 chip, full resolution design also means no aliasing in the color channels as is often typical of single chip designs.

The color splitting prism is more efficient than the absorption color filters on a single chip design, so more light gets to the pixels. The multi layer sensors have very good on-sensor processing so even though the pixels are rather small you get good sensitivity, low noise and good DR. The Z280 is approx  650-700 ISO with the base gammas so very close to an FS7 with it’s standard gammas and the colors match an FS7  extremely well. The picture look really nice.

3x 4K sensors, 17x zoom and variable ND filter is a great combination on the PXW-Z280

From the testing I have done in the cameras dedicated HDR mode, where you can choose between HLG and S-Log3, with S-Log3 the DR of the Z280 appears to be around 13 stops, which is really quite remarkable for this type of camcorder. The sensor readout is very fast so rolling shutter is minimal.

When you factor in the Z280’s f1.9 lens, compared to an FS7 with the Sony F4 zoom or many other zooms that are typically around F4 the Z280 with it’s f1.9 lens does better in low light and offers similar DoF when both are wide open. Of course you can change the lens on an FS7 and use a faster lens, but then you won’t have anywhere near the zoom range of the Z280.
 

Like any small compact camera, it isn’t 100% perfect. Overall the lens is pretty good for a low cost 4K zoom, but like many 17x zooms it does have a touch of barrel distortion when fully wide. As well as the LCD It has an excellent OLED viewfinder that is much, much better than those typically found on Sony’s smaller cameras. It has Timecode in/out and genlock, all the XAVC-I and L codecs as well as MpegHD. There is a full suite of wifi, LAN and network functions for streaming, ftp and remote control as well as the ability to offload files from the cards to a USB drive or memory stick without a computer. It’s a modern camera designed for the modern news or documentary shooter and a big step up in terms of image quality from the PXW-X200 IMHO.

A full review and sample video will be coming in the very near future with lot’s more information.

Adobe still can’t get XAVC levels right!

I’m often asked at the various workshops I run why I don’t grade in Adobe Premiere. Here’s why – they can’t even get basic import levels right.

Below are two screen grabs. The first is from Adobe Premiere CC 2019 and shows an ungraded, as shot, HLG clip. Shot with a Sony Z280 (love that little camera). Note how the clip appears over grossly exposed with a nuclear looking sky and clipped snow, it doesn’t look nice. Also note that the waveform suggest the clips peak levels exceed 110%. Now I know for a fact that if you shoot HLG with any Sony camera white will never exceed 100%.

Incorrect levels with an XAVC clip in Adobe Premier. Click on the image to view a larger version.

The second screen grab is from DaVinci Resolve and it shows the same clip. Note how in Resolve that although bright the clip certainly doesn’t look over exposed as it does in Premiere. Note also how the levels show by the waveform now no longer exceeds code value 869 (100% white is 940).  These are the correct and expected levels, this is how the clip is supposed to look. Not the utter nonsense that Adobe creates.

Same XAVC clip in Resolve and now the levels are correct. Click on the image to view a larger version.

Why can’t Adobe get this right. This problem has existed for ages and it really screws up your footage. If you are using S-Log and you try to add a LUT then things get even worse as the LUT expects the correct levels, not these totally incorrect levels.

Take the SDI or raw out from the camera and record a ProRes file on something like a Shogun while recording XAVC internally and the two files look totally different in Premiere but they look the same in Resolve. Come on Adobe – you should be doing better than this.

If they can’t even bring clips in at the correct levels, what hope is there of being able to get a decent grading output? I can make the XAVC clips look OK in Premiere but I have to bring the levels down to do this. I shouldn’t have to. I exposed it right when I shot it so I expect it to look right in my edit software.

ISO and EI – using the right terms makes what you are doing easier to understand.

ISO and EI are different things and have different meanings. I find that it really helps understand what you are doing if you use the terms correctly and understand exactly what each really means.

ISO is the measured sensitivity of film stock. There is no actual direct equivalent for electronic cameras as the camera manufacturer is free to determine what they believe is an acceptable noise level. So one camera with an ISO of 1000 may be a lot more or less sensitive than another camera rated at 1000 ISO, it all depends on how much noise the manufacturer things is acceptable for that particular camera.

Broadly speaking on an electronic camera ISO is the number you would enter in to a light meter to achieve the a normally exposed image. It is the nearest equivalent to a sensitivity rating, it isn’t an actual sensitivity rating, but it’s what you need to enter into a light meter if you want to set the exposure that way.

EI is the Exposure Index. For film this is the manufacturers recommended best setting for your light meter to get the best results following the standard developing process for the chosen film stock. It is often different from the films true sensitivity rating. For example Kodak 500T is a 500 ISO film stock that has an EI of 350 when shooting under tungsten light. In almost all situations you would use the EI and not the ISO.

On an electronic camera EI normally refers to an exposure rating that you have chosen to give the camera to get the optimum results for the type of scene you are shooting. ISO may give the median/average/typical exposure for the camera but often rating the camera at a different ISO can give better results depending on your preferences for noise or highlight/shadow range etc. If you find exposing a bit brighter helps your images then you are rating the camera slower (treating it as though it’s less sensitive) and you would enter your new lower sensitivity rating into your light meter and this would be the EI.

Keeping EI and ISO as two different things (because they are) helps you to understand what your camera is doing. ISO is the base or manufacturer sensitivity rating and in most (but not all) log or raw cameras you cannot change this.

EI is the equivalent sensitivity number that you may choose to use to offset the exposure away from the manufacturers rating.

If you freely interchange ISO and EI it’s very confusing for people as they don’t know whether you are referring to the base sensitivity rating or a sensitivity rating that is not the base sensitivity but actually some kind of offset.

If you have a camera with an ISO rating of 2000 and you say “I’m shooting at 800 EI” then it’s clear that you are using a 1.3 stop exposure offset. But if you just say “I’m shooting at 800 ISO” it is less clear as to exactly what you are doing. Have you somehow changed the cameras base sensitivity or are you using an offset? While the numbers used by EI and ISO are the same, the meaning of the terms ISO and EI are importantly different.

Learning to be a film maker? Don’t shoot short pretty videos – shoot documentaries.

Over the years I’ve met many well known high end cinematographers. Most of them are really no different to you or I. Most of them are passionate about their craft and almost always willing to chat about ideas or techniques. But one thing that at first surprised me  was how many of these high end movie makers cut their teeth shooting documentaries. I guess I had imagined them to have all been film school graduates that shot nothing but drama, but no, a very large percentage started out in the world of documentary production.
Documentary production is full of challenges. Location shooting, interviews, beauty shots, challenging lighting etc. But one of the big things with documentary production is that you very often don’t have a choice about what, where or when you shoot. You are faced with a scene or location needed for the story and you have to make it look good. So you have to think about how best to approach it, how to frame it, light it or how to use the available light in the best way.
A lot of todays aspiring film makers like to hone their skills by shooting beautiful people in beautiful locations when the light is at it’s best. These short beauty films very often do come out looking very nice. But, if you have a pretty girl, a pretty scene, and good light then really there is no excuse for it not to look good and the problem is you don’t learn much through doing this.
When you are shooting a typical TV documentary you will be faced with all kinds of locations, all kinds of people and the challenge is to make it all look good, no matter what your presented with. Having to take sometimes ugly scenes and make them look good you learn how to be creative. You have to think about camera angles, perhaps to hide something or to emphasise something important to the story the programme tells. And, like a feature film it is normally also story telling, most documentaries have a story with a beginning middle and end.
If you can master the art of documentary production it will give you a great set of skills that will serve you well elsewhere. If you move on from documentaries to drama, then when the director asks you to shoot a scene that takes place in a specific location you may well have already done something similar in a documentary so you may already have some ideas about how to light it, what focal lengths to use. Plus now you will probably have a more time to light and a much more control over the scene, so it should actually be easier.
 
In an ideal world I guess the best way to learn how to shoot movies is…. to shoot movies. But very often it’s hard to get the people, locations and good script that you need. So very often aspiring film makers will fall back on shooting 90 second vignettes of pretty people in pretty places as it’s easy and simple to do.
But instead I would suggest that you would be much better off shooting a documentary. Perhaps about something that happened near where you live or an issue you are interested in.
Go out and shoot  documentaries in less than perfect locations with all the challenges they present. I bet the resulting videos won’t look nearly as perfectly pretty as all those slow-mo pretty girl on a beach/forest/field of flowers videos. But it will teach you how to deal with different types locations, people, lighting, weather and many of the other things that can be challenges when shooting features. It will make you a better camera operator. Making something that’s already pretty look pretty – that’s easy. Making what would otherwise be an un-interesting living room, factory or city street look interesting, that’s a much tougher challenge that will help bring out the creativity in you.