We have an episodic workflow at our studio that uses Episodes and Sequences:
The Hiero docs mention that it can be configured to work with Episodes but I don't understand what I need to modify to get it working. We have an Episode field (sg_episode) on both Shot and Sequence to make it easier for querying in Shotgun.
The method for getting the Hiero Export configured for your Episode > Sequence > Shot hierarchy will vary slightly depending on your exact workflow and Shotgun setup. But we'll make a couple of assumptions and provide a complete working example which should be enough to help apply this to how your studio works. This is a rather lengthy explanation for a process that isn't too involved. But we're hoping the detail will help clarify a bit better how you can customize the Hiero export workflow to match your needs.
Here are the assumptions by example:
CustomEntity01 entitysg_episode)sg_episode) as well which is redundant but helps for reporting in Shotgun.Note that this walkthrough is for implementing 3 layers of hierarchy (Episode > Sequence > Shot). If you are simply trying to replace the Sequence entity type with one you have repurposed for Episodes (Episode > Shot), that process is easier to implement and hopefully this document will give you the knowledge you need to modify your configuration.
First, you'll want to setup your schema. Assuming you want to have episodes built-in to your filesystem structure, you'll need to modify the schema folder in your pipeline configuration. For this example, we'll use the schema defined in the default config but modify it slightly so that episodes live above sequences. The schema folder will look something like this
We'll need to define the query for the episode folder in episode.yml so that the folder names on disk match the episode name in Shotgun. We can do that with by using this code:
# the type of dynamic content
type: "shotgun_entity"
# the shotgun field to use for the folder name
name: "code"
# the shotgun entity type to connect to
entity_type: "CustomEntity01"
# shotgun filters to apply when getting the list of items
# this should be a list of dicts, each dict containing
# three fields: path, relation and values
# (this is std shotgun API syntax)
# any values starting with $ are resolved into path objects
filters: [ { "path": "project", "relation": "is", "values": [ "$project" ] } ]
We also need to update the query for the sequence folder in sequence.yml so that the query is more accurate. We're no longer querying for Sequences in the Project, we're querying for Sequences linked to the parent Episode. So we can modify the file to use this code:
# the type of dynamic content
type: "shotgun_entity" # the shotgun field to use for the folder name
name: "code" # the shotgun entity type to connect to
entity_type: "Sequence" # shotgun filters to apply when getting the list of items
# this should be a list of dicts, each dict containing
# three fields: path, relation and values
# (this is std shotgun API syntax)
# any values starting with $ are resolved into path objects
filters: [ { "path": "sg_episode", "relation": "is", "values": [ "$episode" ] } ]
For more information on how to modify your schema and organize your filesystem, see the Filesystem Configuration Reference docs. There's also discussion about this very topic (adding Episodes to your schema in Toolkit) in the FXPhD videos in Chapter 7: "Toolkit: Administering & Customizing" at 29:16.
The other part of configuring your schema and filesystem is your "templates file" which lives in your pipeline configuration at <pipeline configuration>/config/core/templates.yml. We want to update the Hiero templates so they take advantage of this new episodic schema we've defined.
The Hiero Export app includes settings for:
template_nuke_script_pathtemplate_plate_pathtemplate_render_pathThese settings point to templates defined in your templates.yml file so you'll want to be sure they match the schema you've defined above for the episode hierarchy.
We use the CustomEntity01 entity type in Shotgun to represent our Episodes. So first we need to add the episode key which we can define as the code field on the entity type CustomEntity01.
Add this to the keys section of your templates.yml file like so:
keys:
...
episode:
type: str
shotgun_entity_type: CustomEntity01
shotgun_field_name: code
...
Then further down in your templates.yml file you'll want to update the templates that define where plates and renders from Hiero will go. The defaults are hiero_plate_path and hiero_render_path. This should match up with the structure you've defined in your schema above:
...
# export of shot asset data from hiero
hiero_plate_path: 'episodes/{episode}/{Sequence}/{Shot}/editorial/{YYYY}_{MM}_{DD}/plates/{project}_{Shot}.mov'
hiero_render_path: 'episodes/{episode}/{Sequence}/{Shot}/editorial/{YYYY}_{MM}_{DD}/renders/{project}_{Shot}.{SEQ}.dpx'
...
We'll assume you've updated the rest of your templates.yml file to match your schema definition using episodes. The default setting for template_nuke_script_path is nuke_shot_work so you'll need to be sure that is correctly defined as well.
Let's look at the hiero_translate_template.py hook which translates a Toolkit template to a string that Hiero can understand as an export path. So any keys used in the Toolkit template, either have to be translated to a sting value or to an export token that Hiero can understand. Since we've added an {episode} key and Hiero doesn't have a built-in export token for {episode}, we need to translate it into something that Hiero understands.
...
...
# first convert basic fields
mapping = {
"{Sequence}": "{sequence}",
"{Shot}": "{shot}",
"{name}": "{clip}",
"{version}": "{tk_version}",
}
...
...
If you look at the mapping dict at the top of the file, you'll see this maps a key name from the Toolkit template to an export token that Hiero knows about. So if we wanted to map {episode} to some other export token in Hiero, we could just add it here. Otherwise you could just add your own logic to replace it with a string value in this hook.
But we're actually going to make {episode} a valid export token in Hiero. So we won't need to map it to anything since it will be valid in both Toolkit and Hiero. Therefore, we can leave this file alone and just modify the setting custom_template_fields in the environment file for tk-hiero-export.
In your <pipeline_configuration>/config/env/project.yml file, you can add the following:
...
...
tk-hiero:
apps:
tk-hiero-export:
custom_template_fields: [{keyword: episode, description: The episode number}]
...
...
This adds a valid export token named {episode} to the Hiero exporter.
So why didn't we just tell you to do that from the start? Because it's important to know what's happening behind the scenes here. If we decided we wanted the export token in Hiero to be named {episode_tag}, but still use {episode} in your Toolkit templates, then we would have to use that mapping dict in hiero_translate_template.py in order to map the key name in Toolkit to the export token name in Hiero:
...
...
# first convert basic fields
mapping = {
"{Sequence}": "{sequence}",
"{Shot}": "{shot}",
"{name}": "{clip}",
"{version}": "{tk_version}",
"{episode}": "{episode_tag}",
}
...
...
And your custom_template_fields setting would have to add episode_tag instead of episode.
We're almost done. But now we need to tell the hiero_get_shot.py hook how to get the correct Shot from Shotgun to match up with the Shot when doing an export.
The default version of the hook will return the Shot from Shotgun with the same name as the TrackItem. The Shot must also be linked to a Sequence with the same name as the Hiero Sequence. If the Sequence doesn't exist in Shotgun, the hook will create it. And if the Shot doesn't exist in Shotgun, it will create that as well and then return the result.
We're adding another level of hierarchy so we need to tell the Hook to also create the Episode if it doesn't exist. And since the Sequence is linked to the Episode, we should tie this in to the code that looks up the Sequence, _get_sequence(). Below is the complete code for the hiero_get_shot.py hook. Let's walk through it a bit.
from tank import Hook
class HieroGetShot(Hook):
"""
Return a Shotgun Shot dictionary for the given Hiero items
"""
def execute(self, item, data, **kwargs):
"""
Takes a hiero.core.TrackItem as input and returns a data dictionary for
the shot to update the cut info for.
"""
# get the parent sequence for the Shot
# (which in-turn has the parent episode)
sequence = self._get_sequence(item, data)
# grab shot from Shotgun
sg = self.parent.shotgun
filt = [
["project", "is", self.parent.context.project],
["sg_sequence", "is", sequence],
["code", "is", item.name()],
]
fields = kwargs.get("fields", [])
fields.append("sg_episode")
shots = sg.find("Shot", filt, fields=fields)
if len(shots) > 1:
# can not handle multiple shots with the same name
raise StandardError("Multiple shots named '%s' found", item.name())
if len(shots) == 0:
# create shot in shotgun
shot_data = {
"code": item.name(),
"sg_sequence": sequence,
"sg_episode": sequence["sg_episode"],
"project": self.parent.context.project,
}
shot = sg.create("Shot", shot_data)
self.parent.log_info("Created Shot in Shotgun: %s" % shot_data)
else:
shot = shots[0]
# update the thumbnail for the shot
self.parent.execute_hook(
"hook_upload_thumbnail",
entity=shot,
source=item.source(),
item=item,
task=kwargs.get("task")
)
return shot
def _get_sequence(self, item, data):
"""Return the shotgun sequence for the given Hiero items"""
# stick a lookup cache on the data object.
if "seq_cache" not in data:
data["seq_cache"] = {}
hiero_sequence = item.parentSequence()
if hiero_sequence.guid() in data["seq_cache"]:
return data["seq_cache"][hiero_sequence.guid()]
# sequence not found in cache, grab it from Shotgun
sg = self.parent.shotgun
filt = [
["project", "is", self.parent.context.project],
["code", "is", hiero_sequence.name()],
]
fields = ['sg_episode']
sequences = sg.find("Sequence", filt, fields)
if len(sequences) > 1:
# can not handle multiple sequences with the same name
raise StandardError("Multiple sequences named '%s' found" % hiero_sequence.name())
if len(sequences) == 0:
episode = self._get_episode(item, data)
# create the sequence in shotgun
seq_data = {
"code": hiero_sequence.name(),
"sg_episode": episode,
"project": self.parent.context.project,
}
sequence = sg.create("Sequence", seq_data)
self.parent.log_info("Created Sequence in Shotgun: %s" % seq_data)
else:
sequence = sequences[0]
# update the thumbnail for the sequence
self.parent.execute_hook("hook_upload_thumbnail", entity=sequence, source=hiero_sequence, item=None)
# cache the results
data["seq_cache"][hiero_sequence.guid()] = sequence
return sequence
def _get_episode(self, item, data):
"""Return the shotgun episode for the given Hiero items.
We define this as any tag linked to the sequence that starts
with 'Ep'."""
# stick a lookup cache on the data object.
if "epi_cache" not in data:
data["epi_cache"] = {}
hiero_episode = None
for t in item.parentSequence().tags():
if t.name().startswith('Ep'):
hiero_episode = t
break
if not hiero_episode:
raise StandardError("No episode has been assigned to the sequence: " % item.parentSequence().name())
if hiero_episode.guid() in data["epi_cache"]:
return data["epi_cache"][hiero_episode.guid()]
# episode not found in cache, grab it from Shotgun
sg = self.parent.shotgun
filt = [
["project", "is", self.parent.context.project],
["code", "is", hiero_episode.name()],
]
episodes = sg.find("CustomEntity01", filt)
if len(episodes) > 1:
# can not handle multiple sequences with the same name
raise StandardError("Multiple episodes named '%s' found" % hiero_episode.name())
if len(episodes) == 0:
# create the sequence in shotgun
epi_data = {
"code": hiero_episode.name(),
"project": self.parent.context.project,
}
episode = sg.create("CustomEntity01", epi_data)
self.parent.log_info("Created Episode in Shotgun: %s" % epi_data)
else:
episode = episodes[0]
# cache the results
data["epi_cache"][hiero_episode.guid()] = episode
return episode
We've modified the _get_sequence() method to also fetch the Episode field when querying Shotgun. And when creating a new Sequence, we provide the Episode to link the Sequence to.
So how do we get the Episode? The code is very similar to _get_sequence() but we modify it slightly for our needs.
We are assigning the episode in Hiero by using Tags. So, for example, we created a Tag in Hiero named "Ep01". Then we applied that Tag to the Sequence in Hiero. This is just an example of how to assign an episode to your sequence in Hiero. If you have a different way of doing this, that's great! You should modify this part of the hook to get the Episode from Hiero however you're doing it.
So walking through _get_episode() at a high level... we look at all of the Tags applied to the Sequence in Hiero and if we encounter one that starts with the string "Ep", we assume that is the Episode name. We then return the matching Episode from Shotgun (and create it if it doesn't exist yet). Then we cache this info so we don't have to roundtrip to Shotgun again.
The main purpose of this hook is to return the Shot data from Shotgun. In order to do that, we've had to lookup both the Sequence and the Episode, but now we're ready to lookup the Shot. We have an Episode field on Shot so we are sure to include sg_episode no matter what. If the Shot needs to be created in Shotgun, we know now that when looking up the Sequence, it will be returned with the Episode it's linked to, so we can get the Episode from the Sequence entity sequence["sg_episode"].
That should be it. It's quite a verbose walkthrough when you're really only doing some minor modification to the defaults. But this should give you a better understanding of what's going on under the hood so that you can further customize things as needed.
And, as always, if you have questions please post them in the comments below or email us at support@shotgunsoftware.com.
Hi KP,
this is is great - thank you for the detailed description. The one thing I can't work out is how to create the tag so that my exports go to the right episode and sequence in our project. I've created a tag that has key and date fields but I'm not sure what to put in these.
Im curious to know where people save there NukeStudio/Hiero scripts using this approach. Is it good practise to save them in project>editorial or episodes>sequences>editorial?
it would be good the see how the setup is used once all the hooks and entities are all in place?
many thanks, Michael
HI KP,
Thanks for the walkthrough. It is really great!!!
i'm just curious as an example how you solved the tweaking on the "hiero_resolve_custom_string.py" to get hiero to fully understand your {episode} token?
I admit first time a read through it I skipped that and everything in shotgun worked fine, but when it came down to the filesystem, hiero was building folders ignoring the folder {episode} just creating a {sequence} folder inside the static folder "episodes".
could you post how you "told" hiero how to interpret that {episode} tag using your tag logic?
cheers