1

Per-Project Disk Storage

As we have an increasing number of large projects going into a single Shotgun instance, we're starting to think about disk space issues, especially as they start uploading (potentially voluminous) reference material as attachments. At present, the Shotgun filesystem stuff is all in a single place regardless of which project an entity is related to - how hard would it be to add the ability to do some kind of mapping? Each VFX show here has its own disk (e.g. /show/gun_slinger) and it would be great it we could configure Shotgun so that each Project used a filesystem location specific to that show so that they could manage their own diskspace.

9 comments

  • 0
    Avatar
    Marten Coombe

    Does that mean we create links that play off our own storage rather than having to be uploaded to SG? Do we create thumbs or proxies to hold on the SG servers?

  • 0
    Avatar
    Andy Geers

    I was imagining that UI-wise it would behave exactly the same as it does now - it's just that once you've uploaded stuff, Shotgun saves it to the designated disk rather than its own Rails storage space

  • 0
    Avatar
    Pliny (John Eremic)

    Andy, are you running internal SG servers or using the hosted service?

  • 0
    Avatar
    Andy Geers

    Ah, we're using internal SG servers. Guess this issue doesn't mean a lot to the hosted version!

  • 0
    Avatar
    KP

    We're hoping to add more per-project options in general as more clients are working on multiple projects at the same time. We're also looking at ways to help save disk space for studios using the installed version of Shotgun too. We have another ticket for archiving projects and I just created one for this specific request as well (ticket 8015).

    There's lots of different opinions on how people want to store and link files depending on speciifc needs, hosted/installed, etc. We definitely wanna hear any other suggestions people may have.

    cheers
    kp

     

  • 0
    Avatar
    Pliny (John Eremic)

    KP, I actually have some questions about storage and backups for the hosted version, which I will post separately.

  • 0
    Avatar
    Don Parker

    Hosted folks, we're also thinking about ways to allow you to manage your media locally, so thumbnails and other linked files can stay on your own storage where you can control access.

  • 1
    Avatar
    Isaac Reuben

    Hey Andy,

    What if we put the project id (more stable, as can rename projects) as the first part of that file path (so file id 100 with project id 12 would be /files/12/000/000/0100/file_name.jpg).  Then if you want to store the files for a project somewhere else, you could symlink that project id directory to the new location.  This would also allow an easy filesystem level way to check how much disk space a project is using or to offline all their files (can do this now by first getting a list of all the file ids for a project, and then looping through them, but extra work).

  • 0
    Avatar
    Andy Geers

    Hi Isaac, I think that would suit us just fine

Please sign in to leave a comment.