New post
0

Nuke render farm integration : Old bootstrap script is not working for Core API v0.14.67 ?

Avatar
Donat Van Bellinghen
None
Follow

Hi,

We've installed Shotgun Desktop and I've recently set a few new toolkit projects with it

Those new projects use the core api v0.14.67. I'm not sure if this is related but I have observed that in those projects the core (and the apps) are localised inside the project structure. In our previous toolkit configs we were using a centralised core.

For our nuke render farm we slightly adapted your script :

https://toolkit.shotgunsoftware.com/entries/23481723#Bootstrap%20the%20Shotgun%20Pipeline%20Toolkit%20engine%20using%20init.py

Importing the sgtk is fairly straightforward because we could hardcode the path to our centralised location. This is not as easy anymore as we now have to determine the correct core path, add it to the sys.path, before being able to import sgtk.

Could you propose some code/ideas/suggestions to solve this problem ? In other words could you adapt the aforementioned script ?

We also have other scripts related to our render farm that use the centralised location which would need to be adapted as well. 

 

This is a fairly urgent request, because in a few weeks we will start projects that have been setup using the new core, and we have to decide if we adapt to this core of if we downgrade to the earlier core

Regards

 

Donat Van Bellinghen

Date Votes

7 comments

  • 0
    Avatar
    Permanently deleted user

    Hello!

    Excellent question! The ability to have the Core API in a single location hasn't gone away with the desktop engine - it is still possible, and in many cases a great way to set up toolkit. What we did do in the Shotgun Desktop is to change the defaults around so that projects by default contain their own independent API instead of sharing a single API. This was a way to make the setup process and initial configuration a little bit more straight forward for people.

    With this new behaviour, we also released two new tank commands, share_core and attach_to_core, which allows you to take such a core API that new projects come integrated with and either discard it in a favour of a shared studio API (run the tank attach_to_core command to do this) or to take the shared API and move it out into an external location. (This is useful if you don't have any studio core API set up yet, and want to start a brand new one). You can read more about it here: https://support.shotgunsoftware.com/entries/95443887

    If you like the old behaviour, where toolkit tired to use a shared core as its default, you can always use the tank setup_project command. This hasn't changed.

    Hope this helps! Hopefully it should be a quick process for you guys to run tank attach_to_core your new projects and have the connect to your studio wide setup.

    Thanks!
    Manne

  • 0
    Avatar
    Donat Van Bellinghen

    Thanks for the info. I can indeed 'de-localise' those new toolkit projects and mzkr them use our centralised core api. But what if we prefer to use a localised core api ? My question remains, for scripts outside shotgun (on our render farm) how would I know where the localised core is ? Is there a yml file in the config somewhere with the path to the core  ?

     

    Regards

  • 0
    Avatar
    Permanently deleted user

    Ahh, sorry I think I misunderstood your question! :)

    If you need a list all the pipeline configurations for a project and where they are located on disk, check out the Pipeline Configuration entity in Shotgun. Based on the information in this entity, you could effectively code up a "dispatch" where a script connects to Shotgun, figures out the location for a particular configuration (and in the case when it is localized also its Core API) and then start it up.

    This is effectively what the tank command is doing behind the scenes when you for example run tank Shot ABC123 from the studio level tank command.

    Does this answer the question?

    Thanks!
    Manne

  • 0
    Avatar
    Donat Van Bellinghen

    Ok, I'm checking the webpage with the pipeline configs and I don't find the path to the core being used. I find these infos : 

     

    Config Name:
    Primary
    Created by:
    Date Created:
    02/26/14 11:53am
    Date Updated:
    02/26/14 11:53am
    Description:
     
    Id:
    56
    Linux Path:
    /mnt/shotgun/xxxxx
    Mac Path:
     
    Project:
    Tags:
     
    Thumbnail:
    Upload Thumbnail
    Updated by:
    User Restrictions:
     
    Windows Path:
    \\VSERVER01\shotgun\xxxx

     

    Could you please tell me where the path to the core is located ?

  • 0
    Avatar
    Permanently deleted user

    Each pipeline configuration, whether it is localized or not, has got a tank command and a python API. This is to allow you to access both the toolkit python API and the tank command in a generic fashion, regardless of whether it has got a local API or a shared API.

    You can find the tank command at the root of each pipeline configuration. Just run it by opening a terminal or command shell and navigate to the toolkit configuration folder (as given by your shotgun info above). Then run the tank command.

    If you need to start an API session for a particular configuration, you can add the path root/install/core/python to your pythonpath and then go import sgtk, with root being the root path of your pipeline configuration, as given in Shotgun.

    The general idea is that you shouldn't have to worry about whether toolkit is localized or not, it should work consistently in both modes. The benefit with a shared API approach is that you can add a fixed location (the studio API) to your pythonpath and use that to reach all your projects. If you instead are using localized cores, you need to first look up the configuration location in Shotgun, then add that specific configuration's python folder to your pythonpath.

    Hope this helps!
    manne

  • 0
    Avatar
    Donat Van Bellinghen

    Ok, I think I understand now. I looked into the __init__.py of the core and found the method used to find the real sgtk. 

    Thanks a lot for the explanations.

     

    Regards

     

    Donat

  • 0
    Avatar
    Permanently deleted user

    Cool! Keep in mind that the logic used in https://github.com/shotgunsoftware/tk-core/blob/master/setup/tank_api_proxy/tank/__init__.py is internal to toolkit and may change - this is why we advise people to just import the proxy API and leave that to do the heavy lifting. This way, we can change things later on without breaking anyone's code!

    We need to update our render farm examples to show how to set things up when you are using a series of localized projects, as created by the Shotgun Desktop. I have added this to our list of team discussion topics this week and hopefully our product team will prioritize this work - it definitely feels to me like this is an area where we should provide some clarification now that we have a lot of localized projects being created by default in Toolkit!

    Thanks for your patience and all your great replies!
    Manne

Please sign in to leave a comment.