This keeps cropping up more and more as users of Adobe Premiere Pro upgrade to the 2022 version.
What people are finding is that when they place S-Log3 (or almost any other log format such as Panasonic V-Log or Canon C-Log) into a project, instead of looking flat and washed as it would have done in previus versions of Premiere, the log footage looks more like Rec-709 with normal looking contrast and normal looking color. Then when they apply their favorite LUT to the S-Log3 it looks completely wrong, or at least very different to the way it looked in previous versions.
So, what’s going on?
This isn’t a bug, this is a deliberate change. Rec-709 is no longer the only colourspace that people need to work inand more and more new computers and monitors support other colourspaces such as P3 or Rec2020. The Macbook Pro I am writing this on has a wonderful HDR screen that supports Rec2020 or DCI P3 and it looks wonderful when working with HDR content!
Color Management and Colorspace Transforms.
Premiere 2022 isn’t adding a LUT to the log footage, it is doing a colorspace transform so that the footage you shot in one colorspace (S-Log3/SGamut3.cine/V-Log/C-Log/Log-C etc) gets displayed correctly in the colorspace you are working in.
S-Log3 is NOT flat.
A common misconception is that S-log3 is flat or washed out. This is not true. S-log3 has normal contrast and normal colour.
The only reason it appears flat is because more often than not people use it in a miss matched color space and the miss match that you get when you display material shot using the S-log3/SGamut3 colorspace using the Rec-709 colorspace causes it to be displayed incorrectly and the result is images that appear to be flat, lack contrast and colour when in fact your S-Log3 footage isn’t flat, it has lots of contrast and lots of colour. You are just viewing it incorrectly in the incorrect colorspace.
So, what is Premiere 2022 doing to my log footage?
What is now happening in Premiere is that Premiere 2022 reads the clips metadata to determine its native colorspace and it then adds a colorspace transform to convert it to the display colourspace determined by your project settings.
The footage is still S-Log3, but now you are seeing it as it is actually supposed to look, albeit within the limitations of the display gamma. S-log3 isn’t flat, it just that previously you were viewing it incorrectly, but now with the correct colorspace being added to match the project settings and the type of monitor you are using the S-Log3 is being displayed correctly having been transformed fro S-Log3/SGamut3 to Rec-709 or whatever your project is set to.
If your project is an HDR project, perhaps HDR10 to be compatible with most HDR TV’s or for a Netflix commission then the S-log3 would be transformed to HDR10 and would be seen as HDR on an HDR screen without any grading being necessary. If you then changed you project settings to DCI-P3 then everything in your project will be transformed to P3 and will look correct without grading on a P3 screen. Then change to Rec709 and again it all looks correct without grading – the S-Log3 doesn’t look flat, because in fact it isn’t.
Color Managed Workflows will be the new “normal”.
Colour managed workflows such as this are now normal in most high end edit and grading applications and it is something we need to get used to because Rec709 is no longer the only colorspace that people need to deliver in. It won’t be long before delivery in HDR (which may mean one of several different gamma and gamut combinations) becomes normal. This isn’t a bug, this is Premiere catching up and getting ready for a future that won’t be stuck in SDR Rec-709.
A color managed workflow means that you no longer need to use LUT’s to convert your Log footage to Rec-709, you simply grade you clips within the colorspace you will be delivering in. A big benefit of this comes when working with multiple sources. For example S-Log3 and Rec-709 material in the same project will now look very similar. If you mix log footage from different cameras they will all look quite similar and you won’t need separate LUT’s for each type of footage or for each final output colorspace.
The workaround if you don’t want to change.
If you don’t want to adapt to this new more flexible way of working then you can force Premiere to ignore the clips metadata by right clicking on your clips and going to “Modify” and “Interpret Footage” and then selecting “Colorspace Override” and setting this to Rec-709. When you use the interpret footage function on an S-Log3 clip to set the colorspace to Rec709 what you are doing is forcing Premiere to ignore the clips metadata and to treat the S-Log3 as though it is a standard dynamic range Rec-709 clip. In a Rec-709 project his re-introduces the gamut miss-match that most are used to and results in the S-Log3 appearing flat and washed out. You can then apply your favourite LUTs to the S-Log3 and the LUT then transforms the S-Log3 to the projects Rec-709 colorspace and you are back to where you were previously.
This is fine, but you do need to consider that it is likely that at some point you will need to learn how to work across multiple colorspaces and using LUTs as colorspace transforms is very inefficient as you will need separate LUTs and separate grades for every colorspace and every different type of source material that you wish to work in. Colour managed workflows such as this new one in Premiere or ACES etc are the way forwards as LUTs are no longer needed for colorspace transforms, the edit and grading software looks after this for you. Arri Log-C will look like S-Log3 which will look like V-Log and then the same grade can be applied no matter what camera or colorspace was used. It will greatly simplify workflows once you understand what is happening under the hood and allows you to output both SDR and HDR versions without having to completely re-grade everything.
Unfortunately I don’t think the way Adobe are implementing their version of a colour managed workflow is very clear. There are too many automatic assumptions about what you want to do and how you want to handle your footage. On top of this there are insufficient controls for the user to force everything into a known set of settings. Instead different things are in different places and it’s not always obvious exactly what is going on under the hood. The color management tools are all small addons here and there and there is no single place where you can go for an overview of the start to finish pipeline and settings as there is in DaVinci Resolve for example. This makes it quite confusing at times and it’s easy to make mistakes or get an unexpected result. There is more information about what Premiere 2022 is doing here: https://community.adobe.com/t5/premiere-pro-discussions/faq-premiere-pro-2022-color-management-for-log-raw-media/
If you are running the latest Mac Sierra OS the recent Pro Video Formats update, version 2.0.5 adds the ability to play back MXF OP1a files in Quick Time Player without the need to transcode.
You can also preview MXF files in the finder window directly! This is a big deal and very welcome, finally you don’t need special software to play back files wrapped in one of the most commonly used professional media wrappers. Of course you must have the codec installed on your computer, it won’t play a file you don’t have the codec for, but XAVC, ProRes and many other pro codecs are include in the update.
At the moment I am able to play back most MXF files including most XAVC and ProRes MXF’s. However some of my XAVC MXF’s are showing up as audio only files. I can still play back these files with 3rd party software, there is no change there. But for some reason I can’t play back every XAVC MXF file directly in Quicktime Player, so play as audio only. I’m not sure why some files are fine and others are not, but this is certainly a step in the right direction. Why it’s taken so long to make this possible I don’t really know, although I suspect it is now possible due to changes in the core Quicktime components of OS Sierra. You can apply this same Video Formats update to earlier OS’s but don’t gain the MXF playback.
I had heard about the QIO some time ago, so I approached Sonnet to see if I could borrow a unit to review. I was given the loan of a Sonnet QIO at NAB. I have been playing with it since then and you know what, it’s a great device. So what exactly is it? Well it is an extension box that allows you to connect a range of peripherals and flash memory cards to your computer via the PCI bus. The reason I wanted to borrow one was because the QIO is one of the few devices (the only device?) that allows you to connect SxS, Compact Flash and P2 cards to a computer using the high speed PCI bus with hot-swappable functionality. Hot Swap means you can eject and remove cards without having to re-boot the computer or do anything else, something that some of the other adapters on the market force you to do.
Installation was very straight forward. On my Mac Pro workstation I had to plug in a small PCI-X card into one of the vacant slots inside the rear of the machine. This is easy to do and should not put anyone off buying the device, it took me all of 5 minutes to plug the card in and install the drivers. Then a short cable runs from the back of the Mac Pro to the QIO and a separate power supply is plugged into the QIO for power.
On my Mac Book Pro I simply slotted the Sonnet express card PCI bus expansion adapter into the express card slot and then connected this to the main QIO unit via the extension cable and installed the drivers, again a 5 minute job, very simple.
If you do want to use it with a Mac Book Pro, you will need a model that has the express card slot. At the time of writing the device only works with Mac’s, but Windows support should be coming very soon. When buying a QIO there are two versions. The desktop version supplied with the desktop adapter or the laptop version with the express card slot adapter. The functionality is the same for both, it’s just a case of which adapter you need. You can buy the alternate adapter should you want both as an accessory.
So, I have it installed, how is it to use?
It’s really extremely straight forward. You simply pop your media into the slot and away you go. When your done with that card you eject it as you would with any other removable media and stick in the next card. On the workstation this was so much better than plugging in my XDCAM camcorder via USB.
Of course convenience is one thing, but how about performance? The QIO is fast, very fast. I was able to offload a full 16Gb SxS card in about 150 seconds, less than 3 minutes to the internal drive on the Mac Pro. That equates to an hours worth of XDCAM EX material in around 3 minutes or 20x real time. The performance for compact flash cards doesn’t disappoint either at around 15 seconds per Gb so clearly the transfer speed is limited by the speed of the CF card and not the connection as would be the case with USB or firewire. If you want to use the QIO for SD cards then you can use the supplied adapter. Again the performance is very good, but not as good as SxS and CF due mainly to the lower speeds of the SD cards.
Laptop Performance and Expansion.
One of the issues with Laptops is how do you expand them? It’s all very well being able to put an SxS card into the express card slot for fast off load, but where do you then put the material? On a Mac Book Pro you do have firewire 800 but this is still nowhere near as fast as the SxS card. As the SxS card is in the express card slot you can’t use it to add an eSATA drive, so your a little stuck. But not with the QIO. You see the QIO has a built in eSATA controller and 4 eSATA connectors on it’s rear. This means that you can plug in one or more eSATA drives to the QIO and transfer directly from the SxS card to an eSATA drive or drives. So now even on my Mac Book I can make multiple eSATA copies of my media at speeds of up to 200MB/s (total). So once again the speed is usually limited by the card and not the interface.
For a real torture test I put two full 16Gb SxS cards into the QIO and offloaded both cards at the same time to the Mac Pro’s raid drive. Where one card had taken a little under 3 minutes, two cards took abut 190 seconds, just a little over 3 minutes. Transferred this way, two cards at a time you could offload 2 hours of XDCAM EX material in around 4 mins, that’s an incredible 30x real time. I tried the same test with CF cards and again there was little difference in transfer speed between one card and two cards.
This is one fast device. If you have lots of media to off-load and backup it’s going to save you a lot of time. If you are a production company that works with large volumes of solid state media it will pay for itself very quickly in saved man-hours. If your working in the field with a Mac Book Pro the ability to connect both the media and eSATA devices at the same time makes the QIO a very interesting proposition. It is well constructed, simple to install and use, what more could you ask for.
Value for money?
That’s a little harder to answer. It depends on how much material you work with. It’s a fairly pricey device at around $800US or £700GBP for a card reader, but the time savings are substantial, especially if you are asking people to back up material at the end of a days shoot. The faster it can be done, the more likely it is that it will be done straight away, rather than put off until later. It’s also a lot more than just a card reader, the eSATA ports make it so much more useful for connecting drives or even a raid array to a laptop. Overall I think it is actually well worth the investment for the time savings alone. 8/10 (it would have been 9/10 if it didn’t require the power adapter). Great product.
I approached Sonnet and requested a loan QIO for this review, which Sonnet provided. I was not paid to write this and the views expressed are entirely my own. Speed tests were conducted using my own SxS (blue) cards with the QIO attached to a 1.1 first generation Mac Pro with an internal 4 drive raid array, or with a 15″ Mac Book Pro.
Those clever Korean’s at Nexto DI have been at it again. At first glance this might appear to be one of the original NVS2500 backup devices, which in itself is a very clever device that in my mind no self respecting solid state shooter should be without. However this is actually the new NVS2525 and Larry (hello Larry) from Nexto gave me a quick demo of it’s new features.
The main external change is the removal of the CF and SD cards slots from the top. These have been replaced with a dedicated slot for P2 cards. As before SxS cards slot in to an express card slot in the side of the unit. For SD, CF, MS and all those other sticks that you might one day use, an adapter is supplied that slides in to the express card slot. This has improved the speed of offload for these cards and now CF cards can be backed up almost as fast as SxS cards. Once again the backup speed is impressive, 80MB/s, yes Mega Bytes! In Larry’s demo he offloaded 2Gb of SxS data in 27 seconds, very impressive. But it doesn’t end there. The Nexto uses a hard drive internally for storage. I’ve had my 2500 for about a year now and I have dropped it, banged it around, taken it to the arctic and storm chasing. Despite this it has been 100% reliable. However despite the built in crash protection, gravity sensor, sector checking and all the other safety features, it is at the end of the day a single hard drive. So what Nexto have added is the ability to write to an external drive. The NVS2500 can also write to an external drive, but the difference is that the NVS2525 can write to it’s internal drive as well as an external drive…… in parallel. So backing up your valuable data takes exactly the same amount of time as before.. 27 seconds for 2Gb, but now you have two copies on two drives.
This is what a lot of production companies have been waiting for. Safe, secure, backups with full verification on to two separate drives without the need for a laptop or any other bulky gear. You can use most bus powered 2.5? esata drives for the external backup drive. Nexto supply a nice looking wallet style case that holds both the NVS2525 and the external drive.
While I was talking to Larry I also spotted a Nexto DI box with red rubber buffers. This one it turns out has been developed specifically for backing up SxS cards from the Arri Alexa. The other little box in my pictures is an external battery pack for the Nexto DI devices. It has a little LED battery meter and looks really nice. Perhaps Larry could give us some more info?
I first saw this at NAB earlier in the year, but it wasn’t available then, it is now. Basically it allows you to import and work with native MXF, MP4, IMX and even Proxy files files in FCP. When you install the Cinemon plugin you no longer have to re-warp your MXF’s to .MOV’s so import is greatly simplified and it makes sharing projects across platforms much simpler. In the US it will be known as Cinemon and in the rest of the world XDCAM Workflow Accelerator. Sadly it’s not free, but if you do a lot of work with XDCAM material and FCP it’s probably worth having. USA link
At the recent BVE show in London, NextoDI were showing a demo of what will be available in the next firmware release, due around NAB time. The most significant feature is the ability to archive your material directly from the NVS2500 to BluRay!
You will be able to plug the NVS2500 into a USB BluRay burner and burn your files from the NextoDI straight on to BD-R discs!
So, no need for a computer or BluRay software, just the NextoDI and a BD burner. This makes the NVS2500 a truly pivotal device. You take your rushes in the feild, back them up on to the 2500, then plug in a BD writer and produce your archive discs. Remember that BD-R should have a shelf life of around 25 years. It doesn’t tie up a computer and there’s no reason why you couldn’t do this on location, the recommended LG drive even runs off 12v so you could do it in a car or running off another 12v battery.
As well as writing to BluRay the new firmware will also have the ability to recover some types of corrupt clips. To be honest, if you use SxS this should not be a problem, but it’s a useful tool to have. It will also reject bad sectors when writing to a hard drive thus reducing the risk of errors in the first place.
I know the NVS2500 is not the cheapest piece of kit on the planet, but it really, really, really is worth every penny. If you took mine away from me now it would be like cutting off an arm. It is one piece of the whole file based jigsaw that has really made my life so much simpler, faster, less stressful.
I have been playing with a Sony PXU-MS240 SxS backup device. It’s quite different to my NextoDi NVS2500 even though it essentially does the same job. I will be reviewing it in some detail very soon, but here are my first thoughts.
The key feature is that unit has a removable 240Gb hard drive module. Extra drives are readily available and the removable drives can be used as stand-alone USB hard drives without the main unit. Each hard drive cartridge comes in a sturdy box that is much like a Betacam cassette box. There is space on the drives for labels and the box has an insert sleeve that can be used to write on, just like a tape. Clearly this has been done so that as you fill up drives you can pop them on a shelf for longer term storage as you would with a tape. The beauty of the MS240 is that you never need to off load footage, you just add cartridges as you fill them up.
The main unit is 12 volt powered or can run off a standard EX battery. There is a slot at the front for a SxS card and a big Copy button on the top panel along with the power button and menu controls. There is also a small and very clear LCD display that tells you what the unit is doing. In the setup menu you can choose whether to simply copy the SxS cards contents or to do a copy with full verification in one pass.
Another way to verify your clips is to plug it in to an EX camera. The MS240 is supplied with a USB to Express card adapter. You plug the adapter into an EX’s SxS slot and the USB end into the MS240 and then you can use the EX to playback any clips on the MS240 in full HD. This is something the Nexto cannot do. It also means that you could use the MS240 to store finished edits for playback via an EX over HDSDi.
The build quality is good and the range of connectivity is also good with eSATA and USB on the main unit and USB on the cartridges. A 16Gb card can be copied to the drive in around 5 mins.
Ever since the release of the XDCAM EX cameras users have been having problems getting good looking SD pictures out of downconverted HD.?Why is this and what can be done about it? This is an issue that effects all high resolution HD cameras and is not unique to the EX’s. There are two key issues. The first is the way basic software converters handle fields in interlace material and the second is the amount of information in an HD image that must in effect be discarded to get a SD image.?At first glance you would think that starting off with lots of picture detail would be a good thing, but in this case it’s not. Let’s see if I can explain.?Imagine that you have something in you HD picture that over 4 pixels goes from light to dark, in Hd you get a gradual transition from light to dark and all looks good. Now what happens when you take those 4 pixels and convert them to SD. The 4 pixels become just 2 and instead of a stepped change from light to dark the picture now goes instantly from a light pixel to a dark pixel. If these pixels were the edge of a moving object, as it moved the pixels would be switching instantly from on to off and unless the object moved at exactly one pixel per frame you will get a flickering effect. Clearly our nice gradual transition from light to dark has been lost and if there is any motion we may now be seeing flickering edges. Niether of these look good.
Take a look at these images:
As you can see the down converted SD is very blocky and there is some strange patterning (aliasing) going on amongst the bricks of the houses in the background. This does not look good and if there was motion the brickwork would shimmer and flicker.
So what can be done?
Well the best way to improve the SD down conversion is to soften the HD image before it is down converted to prevent this single pixel light to dark switch from happening. You need to end up with an SD image where you go from full light to full dark over at least 3 pixels to prevent flicker (Twitter).
How much you will need to soften you HD by will depend on how sharp it is to start with. Simply turning down the cameras detail settings can be a big help, but even then the best results are often obtained by applying some kind of blur filter in post production. In FCP I find the flicker filter works quite well. As you can see from the frame grab below the difference in the quality of the downconvert is quite striking.
I have also found that another problem is that the detail settings on an HD camera are not optimised for SD. The detail correction edges created in HD are very thin and when these are down converted to SD they all but disappear and can cause further aliasing. The solution is to make the detail correction edges thicker (on an EX turn detail frequency down to -60 to -99) but this then looks ugly in HD. The bottom line is that a camera optimised for HD works best in HD and SD will be a compromise.
The one big question of everyones lips is how do you archive your material? Well here are my thoughts and some ideas.
The most important thing is to think ahead and plan your end to end workflow. You also need to consider the fact that hard drives will almost certainly fail at some point (maybe not now but in the future) and the vast majority of problems are due to human error. Possibly simply forgetting to copy something or not fully understanding the workflow.
Some Golden Rules, no matter what format or workflow you are using are:
Copy EVERYTHING off the card, keeping the original file structure.
Ensure all copies incorporate some kind of error correction or error checking.
Don’t skimp on the quality of your backup system.
Check, check and double check your workflow before you start shooting.
So you go on a shoot and start filling up you expensive memory cards, at some point you will have to start off loading your material onto something else. In the field this is likely to be hard drives of some sort. Backing up to a single hard drive should only be done as a last resort or for media that you don’t mind loosing. You have several options here, you could use Shotput Pro to backup to single or multiple drives. I really like shotput as you can use it to eliminate a lot of user errors. For a start shotput can be set to backup to multiple locations simultaneously from the source media. Then once it has made the copies and verified the copies it can, if you wish, format the card, ready for re-use. Allowing Shotput to format the media helps prevent human error. How? Well if I ever put a card in my camera and find it has footage on it, it means that card has not been backed up and verified by Shotput. This is better than backing up yourself as there is always the risk of a mix up between backed up and not backed up cards. The other way to backup with a computer is to use the Sony XDCAM EX Clip Browser. You should never use the windows explorer or Mac finder to backup your valuable media as there is no form of error checking. Clip Browser has built in error checking which is enabled under the preferences tab.
A further option is to use a dedicated backup device such as the NextoDi products or soon to be released Sony PXU-MS240 backup device. These are easier to use than taking a laptop into the field. The NextoDi devices can backup to 2 drives at once (full review of the NVS2500 comming soon) and the Sony device backs up to removable esata drive cartridges.
So what sort of hard drives should you use? Well I am currently using pairs of USB Western Digital “Elements” hard drives. Where possible I use 3.5? drives as opposed to the smaller 2.5? laptop type drives. These are low cost yet so far have proven to be reliable and of good quality. The larger 3.3? drives should be more reliable, but they are bigger and bulkier and require mains power, so in the field I use the 2.5? drives. By storing these drives at separate locations, one at home and one in the office, I have a very safe system. If my office were to burn down or get flooded, I would have a spare copy at home. Over time however these drive will fail so every couple of years I move my footage on to new, larger hard drives. Another hard drive option is to use G-Tech G-Raid drives. These units contain two separate hard drives and can be used in raid 1 mode so should one of the drives fail your data should be safe. The cost is similar to using a pair of drives and it’s certainly less fiddly than using pairs of drives but it doesn’t give the security of separate storage locations. If you are doing corporate videos then you could consider selling drives to your clients. The client then keeps the drive and as a result you are no longer responsible for it’s storage or safety, just like if the client kept your rushes tapes.
For longer term storage, again there are many options. I backup a lot of my material to BluRay discs. This is not a fast process, use high quality discs and you should be good for 20+ years. Another option is to backup to Sony Professional discs using a Sony PMW-U1 drive. This is a lot faster than most current BluRay burners and the discs are protected in a rugged caddy. Sony claim a life of 50 years for the discs so it is a very good long term storage solution. The new Sony PMW-350 and EX1R as well as the Convergent Design NanoFlash (next firmware release) have shooting modes that allow footage to be saved on XDCAM discs (Sony Professional Discs) as video clips and not just data files. Using these modes you can put the discs in a player and play back the material directly.
A further long term storage solution is LTO tape. It seems strange to be going back to tape, but LTO4 tape is very reliable and widely supported. It’s not suited to applications where you need quick access to your footage, but is very good for long term security. A good compromise may be one copy on a hard drive as a working copy along with a backup on LTO for archive.
Raid Arrays can be used for long term storage, but even Raid arrays can fail. If the lookup table becomes corrupted it can be next to impossible to recover the data off the discs, so do be careful. Do remember however you store your footage try and be organised. Store your material in a sensible folder structure that will help you find your rushes quickly and easily. If you are out shooting for a day you may be generating a hundred or more files, do that day in, day out and you will generate thousands and thousands of files. Make sure you work out you clip naming and clip prefixes in such a way that you won’t get duplicate names and can find your footage quickly and simply.
And just one more reminder, always save the full file structure. In the case of XDCAM EX keep the full BPAV folder and all it’s contents, also don’t rename the BPAV folder. Even if you edit on a Mac and use the Sony Transfer Tool to make .mov files you should keep the BPAV folders as trying to edit the .movs on a PC or AVID is a nightmare. If you have the original material you can easily work with it on any platform.
With Convergent expecting to start shipping Nano Flash HDSDi recorders at the end of the month I thought I would start looking at workflows. I already have Calibrated Q’s MXF importer for FCP and this works just fine, recognising the MXF files and allowing you to see thumbnails in Finder and then import the clips directly in to FCP. In addition Imagine Products ShotPut pro will automatically back up Compact Flash cards with NanoFlash files. As the device uses Sony’s XDCAM HD codec (but at 25mbps, 50mbps, 100mbs and higher) FCP is able to read the files. If you shoot at 100Mbps with the NanoFlash or Flash XDR then converting the files from 100Mbps to 50Mbps to export back to a Sony Professional Disc is very fast indeed taking only seconds to convert a 30 second clip.