New post
0

Annotated frames dark when using OCIO Source Setup

Avatar
Tobias Pfeiffer
None
Follow

Hi,

we are using a slightly adjusted OCIO package for RV 7.2.0 (behaviour happens in RV6 as well) and it seems that annotated frames (uploaded through SG Review) are just running through the the RVLinearizeGroup (or no group at all - still need to check) and are not affected by the Look or Display Group. 

Our OCIO source setup is slightly adjusted to put the OCIO Display Transforms into the RVLookPipelineGroup. The RVDisplayPipelineGroup is just a pass-through where nothing happens, as you can see here:

We would prefer that annotations are taken after running through all groups (Linearize, Look, Display) to simply save the exact same picture as seen on screen. Thats the way when actually using the new Snapshot feature of SG Review where you can draw a rectangle around the desired area. Slight differences that might occur due to different displays (P3 vs. rec.709) are negligible compared to the offset we are seeing right now. Many paint strokes simply coudn't be seen as linear clipping JPGs.

Is there an fix that we can do on our side?

Any hint would be highly appreciated.

Regards,

Tobias

Date Votes

10 comments

  • 0
    Avatar
    (Michael) Kessler

    Hi Tobias,

     

    That is indeed odd.  You say you have slightly modified your setup, do you use the $OCIO variable for your config, or do you seek this out programmatically?

    The export for annotated frames utilizes RVIO, which means it must save your session for export, then re-read the session in RVIO while exporting frames.  The RV session does not capture the state of which OCIO config is loaded, so my leading theory is that you might not have your OCIO nodes properly initialized in the RVIO session.

    You said that you have moved most of your nodes away from the display group; the export will actually uses the defaultOutputGroup, so it would be useful to see what the exported rv file looks like with your defaultOutputGroup.

    Something that we often use when debugging exports is to use the following flag when launching rv:

    -flags debug_export

    This should leave the RVIO .rv file on disk; you can find it by looking at your terminal for a message that looks similar to this:

    DEBUG: Leaving '/var/folders/ww/j1y70bb52lvcq5v_pj9cmp500000gn/T/temp_12360_3.rv' for inspection

     

    So in short, if you are doing something fancy to load your OCIO config dynamically, you may need to set the OCIO variable for the subprocess to pick up the config correctly.  If the OCIO config is purely generated at run-time, you will need to use PyOCIO to write the config to a tempfile and then set the OICO variable for the subprocess.  Additionally, check to see if any display conditioning you need to do pre-file writing is applied to the exported .rv session file using the -flags debug_export.

     

    Thanks,

    -Kessler

  • 0
    Avatar
    Tobias Pfeiffer

    Hi Michael,

    yes - we are not using $OCIO, but getting the path to the config programatically. 

    Makes total sense that this causes the behaviour.

    Do you think its sufficient to set the environemnt variable inside of our custom ocio_config_from_media function, or does it need to be set prior to the initial start of RV?

    As I am now on vacation I might not have a chance to check this soon, but will immediately when I'm back at the office.

    Thank you very much!

    Tobias

  • 0
    Avatar
    (Michael) Kessler

    You should be able to set the environment variable from within the ocio_config_from_media, since the environment will be passed on to the subprocess.

    I'm glad we are on a promising lead, let me know what you find once you are back in the office.

    Thanks
    -Kessler

  • 0
    Avatar
    Tobias Pfeiffer

    Do I need to take care about other OCIO realted env vars?

    $SEQ and $SHOT are used on our side and need to be set correctly in order to put shot based de-neutralization, CDLs and such on top.

    Thanks,

    Tobias

  • 0
    Avatar
    (Michael) Kessler

    Tobias,

    If you expand those variables into each node's context variables, then those should be stored on the node and not needed in the environment. That would cause problems if you have different SEQ/SHOT for each source and they aren't fully baked.  You can validate you are set properly if those context variables show up as fully defined under the associated nodes in the exported session.

    If you rely on environment variables for your context variables, then yes, you would need to define those, but in that case, they should already be in your environment.

    Thanks,
    -Kessler

  • 0
    Avatar
    Tobias Pfeiffer

    Hi Michael,

    I tested your recommendations today and unfortunately it didn't even export a frame. Even after re-enabling all plugins and testing on different machines I cant repdroduce the too dark (linear) output I mentioned initially. RV is hangig while saying that it is rendering annotated frames:

    But after using the debug flag I am at least able to give you some more insight.

     

    This grab from the RV Console shows which contexts and colorspaces are fed into the Linearize / Look / Display Pipeline Groups of RV. 

    At the end it shows the error why its failing.

    INFO: sgtk_rv_bootstrap Looking for tk-core here: //tools/tools_vfx/tools_config/system/software/rv/7.2.0/windows_nt.x64\src\python\sgtk\bundle_cache\manual\tk-core\v1.0.42\python 
    INFO: Toolkit initialization: ready to import sgtk at 0.000301361 sec. 
    INFO: Toolkit initialization: sgtk import complete at 3.35665 sec. 
    INFO: Toolkit initialization: ToolkitManager complete at 3.67083 sec. 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    WARNING: libpng warning: iCCP: known incorrect sRGB profile 
    INFO: Toolkit initialization took 9.56644 sec. 
    INFO: Queued events waited 10.43 seconds. 
    INFO: Processing event 'external-gma-play-entity' 21.2409 seconds after startup. 
    OCIO config from media for: //data2/8001000921/0001/0020/output/comp/main/v002/3424x2202_exr/hpt_0001_0020_comp_main_v002_jko.%04d.exr 
    OCIO using config: \\data2\8001000921\_gen\settings\ocio\8001000921.ocio 
    OCIO node from media for: //data2/8001000921/0001/0020/output/comp/main/v002/3424x2202_exr/hpt_0001_0020_comp_main_v002_jko.%04d.exr 
    OCIO pipeline group: RVLookPipelineGroup 
    OCIO using context: {'SHOT': '0020', 'SEQ': '0001'} 
    OCIO assuming source cs: linear 
    OCIO assuming working cs: linear 
    OCIO using view (display): Grade (default) 
    INFO: using OCIOLook node for sourceGroup000000_source RVLookPipelineGroup 
    OCIO node from media for: //data2/8001000921/0001/0020/output/comp/main/v002/3424x2202_exr/hpt_0001_0020_comp_main_v002_jko.%04d.exr 
    OCIO pipeline group: RVLinearizePipelineGroup 
    OCIO using context: {'SHOT': '0020', 'SEQ': '0001'} 
    OCIO assuming source cs: linear 
    OCIO assuming working cs: linear 
    OCIO using view (display): Grade (default) 
    INFO: using OCIOFile node for sourceGroup000000_source RVLinearizePipelineGroup 
    OCIO node from media for: None 
    OCIO pipeline group: RVDisplayPipelineGroup 
    OCIO using context: {'SHOT': '0000', 'SEQ': '0000'} 
    OCIO assuming source cs: linear 
    OCIO assuming working cs: linear 
    OCIO using view (display): Raw (default) 
    INFO: using OCIODisplay for display: 0 
    OCIO node from media for: None 
    OCIO pipeline group: RVDisplayPipelineGroup 
    OCIO using context: {'SHOT': '0000', 'SEQ': '0000'} 
    OCIO assuming source cs: linear 
    OCIO assuming working cs: linear 
    OCIO using view (display): Raw (default) 
    INFO: using OCIODisplay for display: 1 
    OCIO node from media for: None 
    OCIO pipeline group: RVDisplayPipelineGroup 
    OCIO using context: {'SHOT': '0000', 'SEQ': '0000'} 
    OCIO assuming source cs: linear 
    OCIO assuming working cs: linear 
    OCIO using view (display): Raw (default) 
    INFO: using OCIODisplay for display: 2 
    DEBUG: Leaving 'D:/TEMP/temp_0_1.profile' for inspection 
    DEBUG: Leaving 'D:/TEMP/temp_0_2.profile' for inspection 
    Traceback (most recent call last): 
    File "\\tools\tools_vfx\tools_config\system\software\rv\7.2.0\windows_nt.x64\src\python\sgtk\bundle_cache\manual\tk-framework-qtwidgets\v1.0.42\python\note_input_widget\widget.py", line 260, in _submit 
    self.pre_submit_callback(self) 
    File "\\tools\tools_vfx\tools_config\system\software\rv\7.2.0\windows_nt.x64\src\python\sgtk\bundle_cache\manual\tk-rv-shotgunreview\v1.0.42\python\tk_rv_shotgunreview\rv_activity_mode.py", line 749, in make_note_attachments 
    out = subprocess.check_output(args) 
    File "\\tools\tools_vfx\tools_config\system\software\rv\7.2.0\windows_nt.x64\lib\python2.7\subprocess.py", line 566, in check_output 
    process = Popen(stdout=PIPE, *popenargs, **kwargs) 
    File "\\tools\tools_vfx\tools_config\system\software\rv\7.2.0\windows_nt.x64\lib\python2.7\subprocess.py", line 702, in __init__ 
    errread, errwrite), to_close = self._get_handles(stdin, stdout, stderr) 
    File "\\tools\tools_vfx\tools_config\system\software\rv\7.2.0\windows_nt.x64\lib\python2.7\subprocess.py", line 833, in _get_handles 
    p2cread = self._make_inheritable(p2cread) 
    File "\\tools\tools_vfx\tools_config\system\software\rv\7.2.0\windows_nt.x64\lib\python2.7\subprocess.py", line 884, in _make_inheritable 
    _subprocess.DUPLICATE_SAME_ACCESS) 
    WindowsError: [Error 6] The handle is invalid

     

    This is the actual environment of that RV session

    It shows that the OCIO env var is now properly set

    ************ Environment Variables ****************** 
    RV_SUPPORT_PATH=//tools/tools_vfx/vfx/production/mv_vfx_rv/RV_SUPPORT_PATH 
    IOJPEG_ARGS= --ioSize 61440 --ioMethod 1 --ioMaxAsync 16 
    TWK_FB_PLUGIN_PATH=//tools/tools_vfx/vfx/production/mv_vfx_rv/RV_SUPPORT_PATH/ImageFormats;//tools/tools_vfx/tools_config/system/software/rv/7.2.0/windows_nt.x64/plugins/ImageFormats 
    OCIO=\\data2\8001000921\_gen\settings\ocio\8001000921.ocio 
    ALLUSERSPROFILE=C:\ProgramData 
    ...

     

    These are the session files from RVIO: temp_0_1.profile & temp_0_2.profile that were availabe due
    the export debug flag you mentioned.

    temp_0_1.profile

    GTOa (4)

    profile_c5028688 : Profile (1)
    {
     root
     {
     string name = "c5028688"
     string comment = ""
     }
    }

    c5028688 : RVDisplayPipelineGroup (1)
    {
     pipeline
     {
     string nodes = "OCIODisplay"
     }

     membership
     {
     string contains = [ "c5028688_a_displayGroup0_stereo" "c5028688_0" ]
     }

     evaluation
     {
     string[2] connections = [ [ "c5028688_a_displayGroup0_stereo" "c5028688_0" ] ]
     string root = "c5028688_0"
     }
    }

    c5028688_0 : OCIODisplay (1)
    {
     ocio
     {
     string function = "display"
     int active = 1
     int lut3DSize = 32
     string inColorSpace = "linear"
     }

     ocio_color
     {
     string outColorSpace = ""
     }

     ocio_look
     {
     string look = ""
     int direction = 0
     string outColorSpace = ""
     }

     ocio_display
     {
     string display = "default"
     string view = "Raw"
     }

     config
     {
     string description = "Mackevision OCIO config based on ACES OCIO config 1.0.3"
     string workingDir = "\\\\data2\\8001000921\\_gen\\settings\\ocio"
     }

     ocio_context
     {
     string SHOT = "0000"
     string SEQ = "0000"
     }
    }

    c5028688_a_displayGroup0_stereo : Adaptor (1)
    {
     input
     {
     int index = 0
     }
    }

    Obvious "mistakes" in here are for me the missing OCIO Look and Linearization Nodes. Only the display node is in here, which uses the wrong context, as I am alway passing SEQ: 0000 and SHOT: 0000 as context when the ocio_node_from_media gets None as media

    temp_0_2.profile

    GTOa (4)

    profile_4fa107bb : Profile (1)
    {
     root
     {
     string name = "4fa107bb"
     string comment = ""
     }
    }

    4fa107bb : RVDisplayStereo (1)
    {
     stereo
     {
     int swap = 0
     float relativeOffset = 0
     float rightOffset = 0
     string type = "off"
     }

     rightTransform
     {
     int flip = 0
     int flop = 0
     float rotate = 0
     float[2] translate = [ [ 0 0 ] ]
     }
    }

    Last but not least, here are our modified ocio_node_from_media and ocio_config_from_media functions:

    import os
    from rv import commands, runtime
    import PyOpenColorIO as OCIO
    import mv_vfx_ocio
    import mv_vfx_ocio.utilities
    from mv_vfx_rv_ocio_source_setup import ocio_source_setup
    import mv_vfx_rv_lib.ui



    def _get_fp_from_rv_fp(fp):
     result = fp
     if not '##' in fp and '#' in fp:
     split = fp.split('.')
     result = '%s.####.%s' % (split[-3], split[-1])
     return result



    def ocio_config_from_media(fileName, attributes):
     """
     Overrides the original 'ocio_config_from_media' from
     the 'ocio_source_setup' plugin (plugin-interface).
     """

     if fileName:
     print '\n\nOCIO config from media for:', fileName
     ocio_utils = mv_vfx_ocio.utilities.ocio_manager(fileName)
     config_file = ocio_utils.ocio_config()
     if config_file is not None:
     config = OCIO.Config.CreateFromFile(config_file)
     import os
     os.environ['OCIO'] = config_file
     print 'OCIO using config:', config_file, '\n\n'
     return config



    def ocio_node_from_media(config, node, default, media=None, attributes=None):
     """
     Overrides the original 'ocio_node_from_media' from
     the 'ocio_source_setup' plugin (plugin-interface).
     """
     node_type = commands.nodeType(node)
     ocio_defaults = mv_vfx_ocio.get_ocio_defaults_from_config(config)

     if media:
     fp = _get_fp_from_rv_fp(media)
     ocio_utils = mv_vfx_ocio.utilities.ocio_manager(fp)

     source_cs = ocio_utils.color_space()
     if source_cs not in ocio_defaults['colorspaces']:
     source_cs_name = ocio_utils.color_space_name()
     source_cs = ocio_defaults[source_cs_name]

     linear_cs = ocio_utils.ocio_settings()['linear_cs']
     if linear_cs not in ocio_defaults['colorspaces']:
     linear_cs_name = ocio_utils.color_space_name()
     linear_cs = ocio_defaults[linear_cs_name]

     display = ocio_utils.display()
     if display not in ocio_defaults['displays']:
     display = ocio_defaults['display_default']

     view = ocio_utils.view()
     if view not in ocio_defaults['views']:
     view = ocio_defaults['view_default']

     context = ocio_utils.oiio_shot_environment()

     else:
     fp = None
     context = {}
     source_cs = ocio_defaults['linear_cs']
     linear_cs = ocio_defaults['linear_cs']
     display = ocio_defaults['display_default']
     view = 'Raw'

     if context == {}:
     context = {'SEQ':'0000', 'SHOT':'0000'}

     print '\n\nOCIO node from media for: %s\nOCIO pipeline group: %s\nOCIO using context: %s\nOCIO assuming source cs: %s\nOCIO assuming working cs: %s\nOCIO using view (display): %s (%s)\n\n' % (fp, node_type, str(context), source_cs, linear_cs, view, display)

     result = [{"nodeType" : d, "context" : {}, "properties" : {}} for d in default]

     if node_type == "RVDisplayPipelineGroup":
     result = [{"nodeType": "OCIODisplay",
     "context": context,
     "properties": {
     "ocio.inColorSpace": linear_cs,
     "ocio_display.view": view,
     "ocio_display.display": display}}]

     elif node_type == "RVLinearizePipelineGroup":
     result = [
     {"nodeType": "OCIOFile",
     "context" : context,
     "properties" : {
     "ocio.function" : "color",
     "ocio.inColorSpace" : source_cs,
     "ocio_color.outColorSpace" : linear_cs}},
     {"nodeType" : "RVLensWarp", "context" : {}, "properties" : {}}]

     elif node_type == "RVLookPipelineGroup":
     result = [{"nodeType" : "OCIOLook",
     "context" : context,
     "properties" : {
     "ocio.function" : "display",
     "ocio.inColorSpace" : linear_cs,
     "ocio_display.view" : view,
     "ocio_display.display" : display}}]

     return result

     

    Hopy this makes some more sense to you send it does to me.

    Thank you in advance,

    Tobias

  • 0
    Avatar
    Tobias Pfeiffer

    Just wanted to let you know that it works perfectly in RV6 using Screening Room when setting the OCIO ENV VAR in ocio_config_from_media.

    In RV7 it is not working even when OCIO is disabled and only the standard source_setup is used. 

    It seems like its not caused by our modified ocio source setup.

    Best,

    Tobias

     

  • 0
    Avatar
    (Michael) Kessler

    Very strange!!!  Does the problem also exhibit itself when not running from the commandline (obviously that would make running the debug flag harder)?  I ask because when looking up those errors, I found this python bug on Windows:

    http://bugs.python.org/issue3905

    If you were launching RV6 from either a play in RV or from the start menu/desktop it wouldn't hit this case, but from cmd.exe it would :(.

     

  • 0
    Avatar
    Tobias Pfeiffer

    Hi Michael,

    it seems like the bug should not be the problem here as it happens when running from a batch file as well when using "Play in RV".

    Weird thing is that it works with Screening Room in RV7, but not with SG Review in RV7 - no matter where it has been launched from. 

    It seems that we will go with RV7 + Screening Room for now.

    I will come back to you when I will make the next attempt with the cut support feature, but this will take some time for sure.

    Thank you!

    Tobias

  • 0
    Avatar
    Tobias Pfeiffer

     

    The problem is still present.

    Because the topic is mixed up with the dark output mentioned above I created a support ticket #94658

    Regards,

    Tobias

Please sign in to leave a comment.