2

Adding meta data to publish files

Stephane Deverly

July 28, 2015 16:14

None

Follow

Hi there,

very often it is useful to add meta data to publish files, e.g. colour space and frame range for images sequences, animation range for animation caches, etc ...

What would be the best way to do that with Tank ?

One solution would be to use a Shotgun custom entity to store meta data as fields, but fields might change a lot from one asset type to the other.

Another solution would be to store meta data as json code in a custom "blob" field on Tank published files.

A last one would be to store the json code in a file, and reference it from a field on Tank published files.

Wonder what would be the best solution to get it done ?

Thanks.

• Facebook
• Twitter
• LinkedIn

10 comments

• 0
  
  Stephane Deverly July 28, 2015 19:16

  We are using a meta_data text field, where we store some arbitrary json code. It seems to do the trick. It would be nice to have a json field type in Shotgun, so data would be displayed nicely, though.

• 0
  
  Stephane Deverly July 28, 2015 19:16

  Hi there, 
  Here is the kind of meta data we store in our field :

  For a published file ( images in that case ) : 
  {"dversion": 1.1, "color_space": "log"}

  For a sub asset ( a prop in a set ) : 
  {"lods": {"maya": {"HighRes": {"keys": {"TankType": "maya_scene", "Step": "modeling", "sg_asset_type": "PROP", "name": "HighRes", "Asset": "chair_A"}, "template": "maya_asset_publish"}, "LowRes": {"keys": {"TankType": "maya_scene", "Step": "MODEL", "sg_asset_type": "PROPS", "name": "LowRes", "Asset": "chair_A"}, "template": "maya_asset_publish"}, "RigLow": {"keys": {"TankType": "maya_scene", "Step": "rigging", "sg_asset_type": "PROP", "name": "RigLow", "Asset": "chair_A"}, "template": "maya_asset_publish"}}}, "world_matrix": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, -9.3330028004020313, 0.0, -8.1994737270316129, 1.0]}

  It defines the prop's position in world space, and how to retrieve the asset data for various LOD, with some ( keys, template ) couples. 
  I guess I could manage to add some formatting when turning my json dict into a string, although it could be done when displaying the field content with some kind of pretty print. It would be better in my point of view to avoid to add some formatting characters, making the field contents bigger, where the formatting could be done at display / editing time. 
  Cheers

• 0
  
  Ryan Mayeda July 28, 2015 19:16

  Hey!

  What kind of formatting would you expect to see in special JSON field type?  Is it more than just indentation?  If so, you might already be covered by the regular text field.  If you feed Shotgun text that has the proper spaces/tabs as indentation, it will display that way if you set the column to display with wrapped text in Shotgun:

  

  Is there other formatting besides this that you'd want to see, or for Shotgun to help with?

  Ryan.

• 0
  
  Ryan Mayeda July 28, 2015 19:16

  Ok, thanks for the info - I'll take this and discuss it with some folks on the Shotgun side and report back...

  Ryan.

• 0
  
  Ryan Mayeda July 28, 2015 19:16

  Hey!

  I checked with the Shotgun devs, and we can accommodate this request for only JSON (i.e. not any other formats like XML or YML, for now at least) by applying a custom renderer to this particular field, since the browser will pretty-print JSON natively.  It's possible a native JSON field type could be implemented in Shotgun down the road, but for now, the Shotgun Pipe/Support team can hook a field like this up for you as a one-off.

  Can you create a ticket on the Shotgun side through support@shotgunsoftware.com and indicate the field and entity you want to apply this treatment to, as well as the server it should be done on?  Someone on the Shotgun Pipe/Support team will then be in touch to schedule its deploy.  For anyone else monitoring this thread that wants a field like this, please do the same!

  Ryan.

• 0
  
  Ryan Mayeda July 28, 2015 19:16

  Just to close the loop on this, here's what the JSON formatting is going to look like in Shotgun on a field with the custom renderer applied:

  

• 0
  
  Dave Lajoie July 28, 2015 19:16

  Is there anything special to add or we just need to create a simple text field, and put json data in there, voila you have nice syntax highlighting?

• 0
  
  Permanently deleted user July 28, 2015 19:16

  Unfortunately, it is not that simple! It's possible a native JSON field type could be implemented in Shotgun down the road, but for now, the Shotgun Pipe/Support team can hook a field like this up for you as a one-off.

• 0
  
  Dave Lajoie July 28, 2015 19:16

  What is the length of the text field? I could convert a python dict to a string, and convert it back using eval(). I have used that trick to store complex metadata in an attribute. Now we have to determine whether there is a limitation on the text field.

  Anyway, the initial implementation will be converted to a real json published file which will be linked to the entity/version. This way we won't have size limitation.

  for me it is not urgent to have json field, but it can be handy.

• 0
  
  Permanently deleted user July 28, 2015 19:16

  Hello Dave!

  I am following up on old tickets, and I just realised that I owe you a response here! The string fields in Shotgun are currently text fields in postgres, theoretically allowing them to store large chunks of data. Having very large amounts of data stored in a text field is probably nothing we would recommend though as it may posssibly affect overall system performance as the overall data set size goes up.

  Thanks!
  manne