0

Shotgun "Automator"

Joe Frayne

July 10, 2009 20:37

None

Follow

I think a Shotgun version of the "Automator" app in Mac OS X would be an incredible addition. It would mean I could create custom templates for creating or modifying or deleting any number of any kind of entities with complex rules for how to fill out their fields or determine relationships, etc. A user could then run it as I created it, or modify it slightly to fit their immediate needs and then use it as a one-off.

I think this is a better overall solution than attempting to solve what is essentially the same problem over and over in specific ways, like a feature for creating multiple shots and using "task templates" to add tasks to them. And it's better than having to always write a special script outside of Shotgun to handle it.

This is something I always wanted to add to our system, but instead we have been fielding regular requests for what we call "wizards", which have been getting more and more complex and powerful in themselves, but are still not a unified solution, even though there is a lot of overlap in the logic and functionality. I really wish we had put the time in the beginning to write something like Automator that users themselves could create and customize and manage on their own.

Thanks!

Joe

• Facebook
• Twitter
• LinkedIn

19 comments

• 0
  
  KP July 14, 2009 07:30

  Joe,

  This is something we talk about very frequently. The EventLog observer method of setting up automation and triggers is very useful in some ways and less so in others. We have brainstormed about some sort of UI-based trigger or automator system for the future of Shotgun but it's clear we can't get there in one leap and I think there's still some fundamental framework within Shotgun that still need to be solidified before we can make the jump.

  That said, let's start talking about it more. What sort of requests do you guys field over and over that you would want to see Shotgun do? Do you see the same sort of UI as Apple's solution or something different? How would you want to interact with them through the API? and how would you want to manage permissions and feedback if updates failed, or had output to return?

  If we were to start with one small piece of this, what do you think would be most useful?

  It's a very meaty topic... so sorry for all the questions, but feel free to dive in.

• 0
  
  Don Parker July 14, 2009 08:41

  Oh good, glad we're talking about this.  As a non programmer, this is the tool I'd like to help build out all the business logic and automation I want in the system.  In particular, I think it should be easy to do the following:

  - When a task status is changed to a value, automatically flip other task(s) statues to another value.

  - When some query is met, automatically create a note, addressed to specified folks, linked to specific things, with specified subject and body text.

  - Automatically switch assignments on tickets when a status is set.

  - Automatically assign a new ticket to a group based on a query.

  - Automatically flip a shot status when all task statues are set to a set of values.

  etc.

  I've been thinking about the UI like Apple Mail Rules.  You define the query first, then you define the action.  Like this:

  

• 0
  
  Stu Aitken July 14, 2009 13:09

  I like that  a lot - you pretty much cover most of what we would want also

  this would remove about 90% of the work we would be looking at in terms of building our own trigger framework

  IMHO I think shotgun should supply some kind of framework for building triggers within the UI for things that deal with shotgun only functionality - I am totally happy with end users writing triggers via the API for things that hook into studio pipeline stuff

• 0
  
  Pliny (John Eremic) July 15, 2009 12:54

  Stu, I think that is a nice way to encapsulate it.  Shotgun-only triggers inside the UI; pipeline stuff in the API.

  Don - another example - time- and date-based triggers.  When the current date reached X, flip the switch on a project from "Waiting" to "Active". Etc.

• 0
  
  Joe Frayne July 15, 2009 20:49

  These are all great ideas! I agree with all that has been said.

  When I wrote this I actually wasn't thinking about events and "automatically" running anything. I was thinking about creating potentially complicated "workflows" (as Apple calls them) which are a set of actions that can have "inputs", and "results" that can be carried into subsequent actions as inputs. I don't think the Mail Rules take it that far. But like mail rules, these workflows could be run manually, or triggered to run automatically based on some set of event criteria, as Don described.

  For example, a producer might create a workflow for easily creating a whole bunch of Shots using a particular naming convention, adding thumbnails and tasks to them, and assigning those tasks to the correct artists. This workflow might have actions that do the following:

  1. Prompt user for a set of shot storyboard thumbnails (present a browse dialog and allow user to pick a set of thumbs).
  2. Get the thumbnail file names, strip the extension and use the names to create one shot per image.
  3. Prompt user for two artists: the animator, and the lighter/compositor.
  4. For each Shot, add one task called "Animate" and another called "Light" and a third "Comp", and assign the given artists appropriately.

  Once the producer has created the workflow, he can run it to create the shots he had in mind when he made it. But later, he might have another couple shots he wants to create that don't require any animation, and have a different person for lighting and compositing. He could just modify his workflow appropriately and run it as a one-off to create those shots, or save it off as a different workflow for later re-use. Or perhaps make the original workflow a little more complicated and give the user, at runtime, the choice of which tasks to create and how to assign them.

  I think this goes beyond "Task Templates" and "multiple shot creation" and other kinds of targetted, "hard-coded" solutions for common situations. It opens it up for creating that kind of time-saving functionality with any entity, including custom entities.

  I think Apple's UI for Automator looks great and is intuitive, though I'd like to have the ability to put results into variables that can be used in later actions, instead of just the very next action. But I haven't really used it much, so maybe there's a lot I still don't know about it. Like I don't see any way to perform a "loop", for cases where you want the action to be performed a certain number of times, either explicitly given, or based on the number of values in an input list provided to the action. But maybe there's a way...

  Someone here suggested that you could also expose Shotgun actions directly to Automator, so you can use Automator to build workflows for Shotgun -- but then of course you can only use the feature on Mac OS X. But if it's a lot easier to do, and perhaps in some ways more powerful because it could interact with other automated apps, then maybe it would be worth trying out!

  What do you think? Is there enough complicated but predictable "work" to do regularly in Shotgun that would make a powerful feature like this worth your time and effort developing?

  Thanks!

  Joe

• 0
  
  Don Parker July 16, 2009 07:30

  Holy crap that's cool.  And what if you could save your action, and register it to show up in context menus or on the toolbars?  Or maybe have a set of saved actions (global and personal)?

• 0
  
  Pliny (John Eremic) July 16, 2009 14:23

  Joe - thanks for the clarification. This makes a ton of sense.  SG guys - this is definitely something my company could use.

  Example: we are servicing a RED feature.  Every time a new batch of footage drops, I need to create 3 work orders: one for Avid dailies; one for DVD dailies; one for tape archival.  Then I need to create tasks associated with those work orders and push them to the appropriate talent (colorists, lab techs).  A lot of repetitive work on a daily basis (or twice daily if production breaks footage).  So - an "automator workflow" here would be really nice.

  And yep, it would be even cooler if these actions could be automatically triggered every time I checked in a new batch of footage.  (E.g. I have a custome entity for "batch".)  But even if I had to manually run that workflow in the absence of a trigger -- that would handle 90% of the repetitive work.

  This plays into some other stuff, including templating (which is near the top of my "feature request release list"  (Yes, I have one and am only doling them out slowly.)  It would be great to save a work order template for (a) my Avid specs (b) my DVD specs, etc.  The "automator workflow" would then pull from those templates to generate the new work orders, etc.

  I will post more about templating separately.  I'm coming at this from a different angle than a lot of your clients, I imagine.  Feature dailies is a much more repetitive thing than managing unique FX shots, but a platform like SG is ideal for managing all this.

• 0
  
  Pliny (John Eremic) July 16, 2009 14:26

  Question to customers using the API -- any reason you can't already do what Joe mentioned, and run Python API calls from an Automator workflow?

• 0
  
  Stu Aitken July 16, 2009 15:47

  not that we have done so, but I think this would involve having to build a trigger framework for everything that could be automated then have automater use that framework to interact with shotgun - ie doing pretty much what is suggested here anyway :)

   

  one of the things we are trying to evaluate at the moment is how much work at our end would be required just to get the shotgun internal triggers we would need to do stuff like automated notification and status updating as things cascade through various production stages, etc

  templating new entity creation would be good as well

• 0
  
  Don Parker July 16, 2009 17:25

  Here is a link to our API docs on how to create event driven triggers:  https://shotgunsoftware.zendesk.com/forums/48807/entries/44575  Stu, have you seen this page yet?

• 0
  
  Don Parker July 16, 2009 17:27

  It would be helpful to see more use cases here of what you all would want to do with this automator.  I'm thinking about the UI...

• 0
  
  Stu Aitken July 16, 2009 18:36

  yep saw that page :)

  thanks - the event model looks pretty easy, but I was more pointing out that building a significant internal trigger framework around it is a fairly big undertaking for someone our size and if shotgun had the type of bult in automater stuff we are talking about here, we could then restrict internal dev to pipeline hooks (eg asset file management and the like) rather than have to also look at internal shotgun triggering

  the main kind of things we would want to do would concern triggering asset status's in downstream tasks based on upstream task status changes, notifying people when stuff is late or status hasn't changed when expected, logged hours have exceeded bid hours, etc - all of that kind of thing that really doesn't affect anything outside of shotgun

  another obvious example might be automating creation of sub assets from a main asset (ie for a new character also build new sub assets for geo caches, etc)

  ie a script that basically does:

  create new entity of type X, using template A, also create some new child/linked-entities of type Y (name based on X somehow), using template B, and assign these tasks to person C, and those tasks to person D, then notify person E

  to do this we would probably need some kind of access to entity properties (preferably just as we would in python code -eg: entity.name)...

  mm the more I think about this the more I think "just use python" :)

  I guess what I reaslly want is a simple way to trigger a script from within the UI (or via an event) without having to write my own demaon and server socket code as well :)

• 0
  
  Stu Aitken July 16, 2009 18:45

  actually having read again that this is EXACTLY what I want - not an automator at all, more an internal means to launch small python scripts and an exposed object model to interact with entities inside shotgun that can automatically listen to the internal event log - maybe this need some sugar on top to make it  a bit easier than the external client API but no much...and a method (new entity type?) to store these scripts within shotgun itself.

  I guess that does throw up some issues re error handling and the like...not sure how that would work

• 0
  
  Pliny (John Eremic) July 16, 2009 19:47

  Stu - re: the latter - That, or if your Python script decides to write to the boot sector of your server's OS drive.

• 0
  
  Joe Frayne July 16, 2009 19:53

  What you say makes sense, Stu. The Automator is a kind of UI-based scripting, which will never be as powerful as "just using python"... :)

  But our producers and production managers don't know python, so whenever they want to do something a just little more complicated and very repetitive, they have to put in a tools request and wait for it. Even some very limited functionality that can be created and customized by anyone would be helpful, imo. And even for those that do know python, a good UI might be simpler, faster, and safer, and still often meet their immediate needs.

  I really like your idea of being able to run custom scripts through the UI -- you could incorporate it into a Shotgun Automator like this:

  1. Prompt user for a list of values.

  2. Run script using the given list of values as the arguments.

  I believe when Apple made it's Automator it initially could do only some very simple actions, and they've been adding onto it ever since, making it more and more powerful. I would imagine that's how it would go with Shotgun, too. Start with just a few available actions, like: prompt user for a some primitive values, multiple entity creation of any type, and custom script execution.

  Joe

• 0
  
  Stu Aitken July 16, 2009 20:46

  yeah I appreciate where you are coming from re usuability by a wider group of people

   

  I guess what I was trying to describe would be a like a very tiny specific 'version' of python that ran inside shotgun, which would only actually execute a limited set of functions that directly related to shotgun features or entity objects 

  essentially  tacking on a very small internal scripting language that would be VERY limited in terms of what it would allow you to do outside the system (ie writing to boot sectors etc) but would be always available inside shotgun and be able to handle the more 'difficult' things involved in writing external client API scripts for you (ie picking up events,  creating objects from entities returned from queries etc)

  that way maybe even programming agnostics could get a handle on it?

  people tend to pick up simple scripting fairly quickly in my experience - especialy if the synax you have to learn is quiet contained and obviously relates wuite closely to stuff you would have done in the UI anyway...

  maybe this could be an interim step?

  for a full blown UI way of doing this I keep thinking of houdini - ie provide some shotgun 'SOPS'  (low level operator nodes that do something useful) , some nodes to represent entitty objects or just simple variables (parameters in houdini parlance) and allow people to chain them together in a nodal workspace...

• 0
  
  Joe Frayne July 16, 2009 21:27

  Ah, I see now! That sounds cool! A potentially easy way to get complex functionality in without having to wait for a good UI widget...

  Joe

• 0
  
  Marten Coombe July 17, 2009 04:34

  I agree with Joes comments re: who will be using the structures.

   

  It is important from our perspective here as we have minimal programming support for our task management CMS systems and the users who are accessing the system are largely artists and producers/coordinators. For the coordinators-producers, most are used to the paradigm of filemaker and/or excel and can handle building things with widgets and formulas but that is about as far as it goes.

  I would really be keen to see tools for us non programmers to use to create macros for repetitive tasks, for alerts or for more complex behaviour driven events based activities such as warnings if certain tasks are falling behind or for analysing performance across a project. I can see many of us admin/artist types taking to that sort of set up. Even formulas integrated into the system could be useful.

   

  It all bares more thinking about.

   

  Great idea btw.

   

   

  Marten

• 0
  
  Tom Stratton December 19, 2012 16:45

  In addition to all these great ideas I would love to see the ability for an admin to enter scripts into a field which could access other info in the shot/version/etc. In much the same way that a URL can reference data in another field using the {field_name} tag, we could use tags in pure python code. This would allow some of the "if this, then that" actions - like automatically changing a value or automatically performing a calculation.

   

  To me, the difficulty here is in creating a mechanism where shotgun does the calculation ONCE and not every time the data is requested - otherwise list views would really slow down. I guess what I am looking for is a way to inject script INTO the UI in addition to having it run on a separate server or on the user's desktop.

   

  Tom