Colorspace of DPX files

Jake Richards

September 01, 2015 14:24

None

Hello:

I've got some dpx files that I've written from Nuke 6.0v1 that are showing up in the wrong colorspace/conversion in rv 3.10.11. (linux)

If I write a dpx from Nuke using a linear colorspace, rv opens the file with Color->Cineon/DPX Log and View->sRGB resulting in my image appearing wrong. Here is an rvls -x of the file:

                       DPX/Version   V1.0
                       DPX/FileName   /var/tmp/test.dpx
                     DPX/CreateTime   2011:03:01:16:30:41:EST
                        DPX/Creator   Nuke
                    DPX/Orientation   Top Left
   DPX-Source/HorizontalPixelAspect   1200
     DPX-Source/VerticalPixelAspect   1200
                   DPX-0/Descriptor   RGBA
                 DPX-0/Colorimetric   Other (0)
                     DPX-0/Transfer   Other (0)

Now the odd part is if I choose to output the dpx file from Nuke using the Cineon colorspace, rv opens the file with Color->No Conversion and no linear to display conversion, resulting in the image looking wrong. Here is an rvls -x of that file:

                        DPX/Version   V1.0
                       DPX/FileName   /var/tmp/testCineon.dpx
                     DPX/CreateTime   2011:03:01:17:16:54:EST
                        DPX/Creator   Nuke
                    DPX/Orientation   Top Left
   DPX-Source/HorizontalPixelAspect   1200
     DPX-Source/VerticalPixelAspect   1200
                   DPX-0/Descriptor   RGBA
                 DPX-0/Colorimetric   Linear
                     DPX-0/Transfer   Linear

Does rv use the DPX-0/Colorimetric field to determine how to display the image? Is nuke setting this field incorrectly? What's odd is that we were recently using version 3.8.5 and it displayed these dpx files correctly. I apologize if I'm doing something wrong or haven't set a preference correctly, I've only recently started using RV. Thank you!

Jake

Date Votes

14 comments

0

Alan Trombla September 03, 2015 07:51

Hi Jake,

I'm going to do some more investigation, then get back to you about this, but first one follow-up question for you: I'm not actually able to replicate the second case you site above. IE when I output a dpx file from Nuke with a Write node whose colorspace knob is set to "Cineon", I get a DPX with Transfer set to "Other(0)", which btw means "User Defined" according to the spec.

Could you please confirm that you get the "Linear" result listed above ? And if so, could you maybe send a small nuke script to support@tweaksoftware.zendesk.com ? Also, what version of Nuke are you using ? (I've tried 6.1v2 and 6.2v1.)

Thanks,

Alan

Comment actions Permalink
0

Jake Richards September 03, 2015 07:51

Certainly! Here is a Nuke script. It was from version 6.0v1 and after writing out a frame, here is the full output of a rvls -x on the file:

/u/truant $ rvls -x /var/tmp/testCin.dpx
RV_HOME = /usr/net/app/rv/rv-3.10.11
INFO: dpx requires plugin io_dpx.so
/var/tmp/testCin.dpx:

                         Resolution   1920 x 1080, 4ch, 8 bits/ch
                           Channels
                     ChannelsInFile   4
                        DPX/Version   V1.0
                       DPX/FileName   /var/tmp/testCin.dpx
                     DPX/CreateTime   2011:03:04:16:54:55:EST
                        DPX/Creator   Nuke
                    DPX/Orientation   Top Left
   DPX-Source/HorizontalPixelAspect   1200
     DPX-Source/VerticalPixelAspect   1200
                   DPX-0/Descriptor   RGBA
                 DPX-0/Colorimetric   Linear
                     DPX-0/Transfer   Linear
                        DPX/BitSize   10
                      DPX-0/Packing   LSB Padded
                   DPX-MP/FrameRate   0
                DPX-MP/ShutterAngle   0
               DPX-MP/FramePosition   32895
                    DPX-TV/UserBits   00:00:00:00
                    DPX-TV/TimeCode   00:00:00:00
                 DPX-TV/FieldNumber   0
                 DPX-TV/VideoSignal   0
        DPX-TV/HorizontalSampleRate   0
          DPX-TV/VeritcalSampleRate   0
                   DPX-TV/FrameRate   0
                  DPX-TV/TimeOffset   0
                       DPX-TV/Gamma   0
                  DPX-TV/BlackLevel   0
                   DPX-TV/BlackGain   0
                  DPX-TV/BreakPoint   0
                  DPX-TV/WhiteLevel   0
            DPX-TV/IntegrationTimes   0
                   DPX-TV/Interlace   Noninterlaced
                   PixelAspectRatio   1

I will also email this to the support address you gave me (but it may be monday as I have to leave work now)

Thank you!

Jake

Comment actions Permalink
0

Alan Trombla September 03, 2015 07:51

Hi Jake,

Thanks very much for the sample. I still don't really understand what's going on with Nuke here (I don't really know that much about it), but at least I understand why our two tests differ. At least according to my tests, if you set the "log files" LUT in Nuke's project settings to "Cineon", then there will be no change in the output pixel values, but the Transfer Characterstic DPX field will now be set to 0 (User Defined). That's obviously not what we want (it should be Cineon/Printing Density), but at least now it doesn't say "Linear" (which will cause RV to assume that it's linear), and in the lack of other information, RV will assume that it's Cineon, and it should display with the inverse-Cineon transform automatically.

The upshot: If your Write node says "linear" be sure the log files LUT setting is also "linear", and then the DPX Transfer field will be "Linear" and RV will display it accordingly. On the other hand, if your Write node says "Cineon", set the log files LUT setting to "Cineon", and then the DPX transfer field will be "User Defined" which will cause RV to guess "Printing Density" (Cineon) and display it correctly.

(And if you're writing sRGB data into a DPX file, may the video gods have mercy on your soul.)

Hope that helps,

Alan

Comment actions Permalink
0

Jake Richards September 03, 2015 07:51

Alan:

Hi, I apologize for not replying sooner, I've been under the weather lately. But, I downloaded Nuke 6.2v2 just to make sure I've got the most recent version and attempted to write out a few dpx files for testing in RV. I did as you said and set the log files lut to linear and the write node to linear as well but in RV it still uses the Color->Cineon/Dpx log conversion and View->sRGB settings. I then tried setting the log files lut to Cineon and the write node to Cineon and it sets the Color->Cineon/Dpx log correctly but still has the View->sRGB set. Here are full rvls -x for the two files:

../nuke6.22Linear.dpx:

                         Resolution   2048 x 1556, 3ch, 8 bits/ch
                           Channels
                     ChannelsInFile   3
                        DPX/Version   V1.0
                       DPX/FileName   /var/tmp/nuke6.22Linear.dpx
                     DPX/CreateTime   2011:03:09:11:24:33:EST
                        DPX/Creator   Nuke
                    DPX/Orientation   Top Left
   DPX-Source/HorizontalPixelAspect   1200
     DPX-Source/VerticalPixelAspect   1200
                   DPX-0/Descriptor   RGB
                 DPX-0/Colorimetric   Other (0)
                     DPX-0/Transfer   Other (0)
                        DPX/BitSize   10
                      DPX-0/Packing   LSB Padded
                   DPX-MP/FrameRate   0
                DPX-MP/ShutterAngle   0
               DPX-MP/FramePosition   32895
                    DPX-TV/UserBits   00:00:00:00
                    DPX-TV/TimeCode   00:00:00:00
                 DPX-TV/FieldNumber   0
                 DPX-TV/VideoSignal   0
        DPX-TV/HorizontalSampleRate   0
          DPX-TV/VeritcalSampleRate   0
                   DPX-TV/FrameRate   0
                  DPX-TV/TimeOffset   0
                       DPX-TV/Gamma   0
                  DPX-TV/BlackLevel   0
                   DPX-TV/BlackGain   0
                  DPX-TV/BreakPoint   0
                  DPX-TV/WhiteLevel   0
            DPX-TV/IntegrationTimes   0
                   DPX-TV/Interlace   Noninterlaced
                   PixelAspectRatio   1

../nuke6.22Cineon.dpx:

                         Resolution   2048 x 1556, 3ch, 8 bits/ch
                           Channels
                     ChannelsInFile   3
                        DPX/Version   V1.0
                       DPX/FileName   /var/tmp/nuke6.22Cineon.dpx
                     DPX/CreateTime   2011:03:09:11:25:55:EST
                        DPX/Creator   Nuke
                    DPX/Orientation   Top Left
   DPX-Source/HorizontalPixelAspect   1200
     DPX-Source/VerticalPixelAspect   1200
                   DPX-0/Descriptor   RGB
                 DPX-0/Colorimetric   Other (0)
                     DPX-0/Transfer   Other (0)
                        DPX/BitSize   10
                      DPX-0/Packing   LSB Padded
                   DPX-MP/FrameRate   0
                DPX-MP/ShutterAngle   0
               DPX-MP/FramePosition   32895
                    DPX-TV/UserBits   00:00:00:00
                    DPX-TV/TimeCode   00:00:00:00
                 DPX-TV/FieldNumber   0
                 DPX-TV/VideoSignal   0
        DPX-TV/HorizontalSampleRate   0
          DPX-TV/VeritcalSampleRate   0
                   DPX-TV/FrameRate   0
                  DPX-TV/TimeOffset   0
                       DPX-TV/Gamma   0
                  DPX-TV/BlackLevel   0
                   DPX-TV/BlackGain   0
                  DPX-TV/BreakPoint   0
                  DPX-TV/WhiteLevel   0
            DPX-TV/IntegrationTimes   0
                   DPX-TV/Interlace   Noninterlaced
                   PixelAspectRatio   1

I've also attached the small nuke script I used to generate the cineon image. I don't know what is going on inside of nuke's dpx writer either, so I guess it's possible they are writing incorrect meta data? Thank you for your time!

Jake

Comment actions Permalink
0
Alan Trombla September 03, 2015 07:51
Hi Jake,

Very sorry for the delay, this thread kind of fell off my radar.

(Note to all forums users: If you see us seemingly ignoring a forum post, please feel free to prod us with email to support@tweaksoftware.zendesk.com !)

A few points:
- Different versions of Nuke produces slightly different results (for the record I'v been testing 6.1v2, 6.2v1, 6.2v2, all on the mac), but none of them are different enough to help with this problem.
- I was mistaken above. You can, as you note above, produce a DPX file with a Linear transfer in the header, but as far as I can tell, it will still have Cineon-encoded pixels.
- If you set the lut pref and the output color space to "linear", you get linear pixels, but the header field will be set to "User Defined".
- I have found no way to get Nuke to produce a linear-encode DPX file that says "Linear" in the header.
- I have found no way to get Nuke to produce a Cineon-encoded DPX file that says "Printing Density" (which is what it should be set to according to the spec), or even "Logarithmic."
Findamentally, I think the aswer to your question is yes, Nuke is not writing the DPX header correctly. After all, your test above shows that you can produce two DPX files, in different color spaces, with identical header values.

Unfortunately, given that, there's not much RV can do to automatically handle these files. I'm going to look into something like a preference, or maybe an environment variable, to let you specify what RV should do with files from Nuke with undefined color space. That seems like it would work well at least in cases where pretty much all the DPX files in the pipeline (that come from Nuke) are in the same color space. Is that the case for you ?

Thanks for your patience,

Alan
Comment actions Permalink
0

Jake Richards September 03, 2015 07:51

Alan:

Hey, no problem, I know you guys are probably super busy! A preference or a environment variable or even just a parameter to pass to rv would be nice. I know there is a lot of power in the command line options, could I set the colorspace settings from the command line? (ie rv [-colorspace linear 101.dpx] ? ) But, I should log a bug report with The Foundry about their DPX writer producing an incorrect header.

You are correct, we output one kind of DPX here that is linear but the header makes it seem user defined, so when I try to view our DPX's, I have to change the colorspaces in RV each time. Thank you for your time!

Jake

Comment actions Permalink
0

Nathalie Girard September 03, 2015 07:51

Good morning,

I am coming across the same issue. I have a very quick question to start with, for Jake.

You mentioned that you have to change the colorspaces in RV to look at your DPX properly, ***what are the settings that you are applying?***

In this forum, the main issue is the output Nuke produces on a DPX, but what is the source ( Read node image ), is it a DPX as well? I am asking since

I will have to write out imges as DPX in linear space with a Rec709 on. But I am suspecting ( with all the info above ) that Nuke is maybe not the best way

to create DPX files, or is there some changes that have been made to Nuke?

Thank you for your time,

Nathalie

Comment actions Permalink
0
Jake Richards September 03, 2015 07:51
Natalie:

To view the dpx's written by nuke on my monitor correctly, I had to do two things after opening my dpx:
1. Click on the Color menu then set the "File Nonlinear to Linear Conversion" to "No Conversion"
2. Click on the View menu then uncheck the sRGB checkbox in the "Linear to Display Correction" field
This became tedious after a while so I wrote a package that sets these fields for me when it detects the DPX/Creator field in the header is Nuke. As far as I can tell, this is still a problem as I just tested Nuke6.3v5 and rv-3.12.12 this morning. Our source imagery is a read node of our proprietary image format which is in linear.
Comment actions Permalink
0

Nathalie Girard September 03, 2015 07:51

Hello,

I wanted to thank you all for your help. Everything is working smoothly now.
Thanks for taking the time to reply, very helpful.

Nathalie

Comment actions Permalink
0

Colin Davies September 03, 2015 07:51

Hey,

I am trying to find a command line flag (or some such thing) that will allow me to set the View > Linear to Display Correction to "No Conversion"

Thanks,

Colin

Comment actions Permalink
0

Jon Morley September 03, 2015 07:51

Hi Colin,

There is no such command line argument for that. The way that value is set is controlled through RV's source_setup scripts which are user modifiable. If you let me know more about when you want to disable display conversion I can help guide you through creating a custom setup. However if you do not want to deal with that we do have a custom LUT package that among many other features allows you to turn off display conversion.

I have attached it here (linked below).

Please let me know what you would like to do.

Thanks,

Jon

Comment actions Permalink
0

Frank Rueter September 03, 2015 07:51

Hi Jon,

I just tried the the cdlimport package and ran into a few of problems:

in line 95 (importCDL method), you use "file" as a variable name, but "file" is actually a python object, so you effectively overwrite it here. I just changed the variable to "file_" instead.

Also, the openFileDialog() method is attached to the commands module, but it's only called from the current name space which raises an error, preventing the file browser from silently failing because of the try/except blocks. I changed the line to this to make it work:

file_ = commands.openFileDialog(True, False, False,"ccc|cdl",None)[0]

However, after using "Force Color->To Linear to No Conversion on load" option in a previous session and calling rv from the command line with a dpx file like this:

rv alexaFile.dpx

I get the below:

ERROR: Exception thrown while calling commands.setIntProperty -- exception: "invalid property name sourceGroup000000_colorPipeline_0.color.logtype", program exception
Traceback (most recent call last):
File "/ohufx/pipeline/tools/rv/Python/cdlimport.py", line 62, in customColorSetup
self.setNoLinConvert(event.contents().split(';')[0])
File "/ohufx/pipeline/tools/rv/Python/cdlimport.py", line 50, in setNoLinConvert
commands.setIntProperty(linNode + ".color.logtype", [0], True)

Cheers,

frank

Comment actions Permalink
0

Frank Rueter September 03, 2015 07:51

sorry, found another type in line 26:

self._dl = not seld._dl

should read:

self._dl = not self._dl

Comment actions Permalink
0

Jon Morley September 03, 2015 07:51

Hi Frank,

Sorry about that. I should update to the latest version with fixes likes these as well as updating for RV-4 compatibility (moving to RVLinearize node from RVColor). Attached (linked below) is the latest. But keep in mind this still a proof of concept example. It is a starting off point. It sounds like you might want to customize for you workflow even further.

Please let me know if there is anything else I can help with.

Thanks,
Jon

The cdlimport package is no longer supported and its feature set has been rolled into the now included "Custom LUT Menu Mode" package found in each release.

Comment actions Permalink

Please sign in to leave a comment.