New post
0

What's the easiest way to publish without a UI?

Avatar
Benoit Leveau
None
Follow

Hi,

I'd like to call the publish process (scan => pre_publish => primary_publish => secondary_publish => post_publish) but outside of a user session. There are a few use cases:

* a job has been rendered on the farm and the user wants to publish the scene+renders without having to open the scene again (which might take some time on heavy scenes)

* a user is launching a render on the farm with a "Publish when job is done" option so that the renders will be ready to be picked up as soon as the job is finished

I don't think duplicating the logic in each of the publish hooks is a good idea, so ideally this lightweight "publish process" would run a batch maya/nuke and would call tk-multi-publish. What's the best way to do that without a GUI?

Thanks for your help,

Benoit

Date Votes

4 comments

  • 0
    Avatar
    Benoit Leveau

    The following code works and allows to automate the publish process, but sadly it still requires a Gui because the PublishForm class derives from a QWidget. It fails when used in batch: "QWidget: Cannot create a QWidget when no GUI is being used. Abort (core dumped)"

    # retrieve the toolkit engine
    curr_engine = sgtk.platform.current_engine()
    # retrieve the publish app
    publish_app = curr_engine.apps["tk-multi-publish"]
    # get access to the underlying module to import the publish_form.py module
    tk_multi_publish = publish_app.import_module("tk_multi_publish")
    publish_form = tk_multi_publish.publish_form
    # create the publish form that holds all the tasks, comment, etc.
    pub_form = publish_form.PublishForm(publish_app, publish_app._publish_handler)
    # override comment
    pub_form._ui.publish_details.comment = "auto publish test"# call the publish procedure
    # publish
    publish_app._publish_handler._on_publish(pub_form)

     

  • 0
    Avatar
    Benoit Leveau

    I managed to get it to work by:

    1. duplicating the logic of the tk-multi-workfile's version up feature in a workfile.py module. this is used when submitting the job to the farm, so that the current work scene is set to be published later, and it is immediately versionned up so the user works on the next version.
    2. the publish will run from the farm so I had to edit the core hook get_current_login so that when the renderfarm user is detected, we use the environment variable USER (which points to the original human user)
    3. when running maya in batch mode we don't usually initialize Shotgun as we don't need it. I added some functions that start the shotgun engine, taking a Task id as a parameter.
    4. Shotgun is adding some callbacks to Maya, which would be triggered during publish and will do lots of nasty Qt things, so I added a function to remove the tank callbacks, which is calling engine.watcher.stop_watching(). Because watcher is private, it is actually calling engine._MayaEngine__watcher.stop_watching().
    5. The tk-multi-publish app relies on a lot of GUI during publish. I created a publish_batch module which duplicates the tk-multi-publish's PublishForm class into a Mockup class that basically just constructs the list of tasks and doesn't rely on QWidget. This class can be passed to the original publish handler's _on_publish function. If there are errors during publish, QMessageBox instances are created and the publish fails without a possibility to get the error messages.
    6. Some logic in the hooks need to be bypassed: our primary_pre_publish hook is erroring out if there are any work file with a higher version than the one we're trying to publish. The post_publish hook is versionning up the scene after publish, whereas in our case the next version already exists and shouldn't be overwritten (the user might have made some work on it). For these two cases, I relied on a new batch boolean variable that I added to the publish app.

    1, 4 and 5 are quite ugly hacks. They wouldn't be needed by adding a few functions to the app/engine API.

    2 and 3 are necessary customizations, no matter what the app do.

    6 are also necessary customizations for batch but it would be cool to have this batch parameter part of the API (maybe as an extra parameter dictionary passed to the publish function, that each hook would be able to access, so each site can setup their own customizations).

    None of this is in production or will be used like this. Instead of duplicating the PublishForm class I should just fork the tk-multi-publish and make it more batch-friendly. But I think it illustrates well that the current version of the apps are not designed to work as batch processes but at the same time they're not really far off.

    Hopefully the next version of the apps will have a cleaner API!

    Benoit

  • 0
    Avatar
    Alan Dann

    Hi Benoit,

    Sorry this proved to be a bit of  a headache!

    Going through the steps you had to take:

    1. It seems like exposing the 'save-as' and 'set-version' operations on the app interface would help with this?

    2. Yeah, this is a known problem that we don't have a great solution for atm - what you've done is a good workaround though.

    3. This is something we hope to look at in the near future (Toolkit on the farm) - it's been on the list for a while!

    4. Another area we want to improve is making sure that all engines behave themselves in batch mode (this would be part of the Toolkit on the farm work) - I'll make sure that this is ticketed up though, specifically for the Maya engine with the issues you found.

    5. This is annoying as the functionality exists in the app, it just never got exposed through the app interface!  I'll take a look and see how much would be involved in sorting this out...

    6. This is interesting and not something we've considered.  Rather than passing a boolean from the app though, could you instead just query the engines 'has_ui' property?  So from the hook do: self.parent.engine.has_ui

    Thanks for persevering with this and reporting your findings - I'll take a look to see if there are any quick wins we could do and ticket everything else up so that we don't lose track of it.

    Alan

     

  • 0
    Avatar
    Benoit Leveau

    Hi Alan,

    1, 4 & 5. Exposing "Save as" and "version up", "publish" would be nice indeed. The more functions we have in the app, the easier it is to integrate toolkit with other tools without hacks that can stop working when we update the app.

    2. I don't think it's really a toolkit problem. We're running the jobs as a different user so we can reasonnably expect to have to write a bit of code to set the original user, and the core hook does the job perfectly.

    3. Actually, the engines can be initialised fine on the farm, I added code to force the context to a specific task (the one the user was in when submitting the job) to make sure we publish in the proper environment. In the end, I don't think there's much more to do on your side.

    6.  That worked great! It would be cleaner though if this kind of choice (versioning up or not, etc.) was left to the caller of the publish function. For instance, we could have a 'extra' parameter, a dictionary that would be then given to each hook, to customise them. In my case I would use extra["version_up"] and set it to False when calling from the farm, and my hook would check the presence and value of this entry.

    I think the biggest thing is really to open the apps so we can call directly the functions. It would be great to be able to do that for the breakdown as well, to check whether a scene is up to date or not when the user wants to publish it. Lots of possibilities!

    Thanks,

    Benoit

Please sign in to leave a comment.