Organization and backups of our work

spotlight

Ah, storage space. One never seems to have enough storage space.

In addition to the storage on our computer, peripheral drives plugged into our USB ports and (perhaps) file servers we have on our local network… there are also various “cloud” services (DropBox, OneDrive, Google Drive, Amazon S3, iCloud to name a few) clamoring for our attention.

How do we organize our work? We have raw audio files, the project files that go with our favorite digital-audio-workstation (DAW) software, the finished audio files that we publish as podcasts… do we trust the online services (Descript, Zencastr, Spotify for podcasters, to name a few) to save our work? …do they backup our work? What about transcripts text files? What about our notes for each episode?

And what about backups? Should we (can we? how do we?) separately backup our local files to a cloud service? Or do we trust our preferred cloud service to just mirror everything in one, local spot where we just keep everything?

Yeah. Let’s talk about that in this spotlight.

This is a high-level map of what you should be doing. I’ll write a separate reply in a bit, with the specific details of exactly what I do every step of the way.

:thinking: This topic is one of the Podcaster Community’s occasional Spotlights.

Each spotlight is an opportunity for the entire Podcaster Community: To provide deep, generous feedback on someone’s work; To brainstorm and give first-impressions on someone’s idea; To investigate or problem-solve someone’s issue.

Uniform naming for everything

The first 90% of anything is vastly easier than the second 90%.

I want to suggest that you come up with a system that you use to name your work.

The best way to organize your work is into “sessions.” You will invariably assemble raw recordings, some artwork, some notes, then maybe later more audio files, etc. You may also have a folder full of stuff created by your favorite audio-editing software. All of that goes inside one folder (or “directory”) for each session.

I suggest that you decide on a system, and that you go back to your first work and organize it all. Yes, from the beginning…

I suggest using a letter and then the date, to invent a unique name for each session.

For example, use “R” if you think of your sessions as “recording”, or “S” if you want to imagine “sessions”. But, any letter will do. Then use 6 digits, in this yoda-backwards ordering: year, month, date.

S240528 is the session you began working with on May 28, 2024.

:thinking: If you need more than one session, on one day, add a letter— S240528a is the first, S240528b is the second on that day, etc. Don’t bother with “a” on every session, just append the letters if you need multiple sessions on one day.

You now have a unique label, that you never need to change. They will quickly feel familiar.

If you want, you can add some info (I do this) for more of a hint. “S240528 - John Doe”

With one name, now I can even write that on any notes I make by hand. It’s easy to note “s240528”.

:thinking: As I mentioned at the top, I’ll write a separate reply detailing what I do at every step of the way.

Two locations for storage

Active work

First, for sessions you’re actively working on, you need those files to be on your computer (on your local storage.) This is one of the locations you have to manage.

This can be in your Documents folder, your desktop, wherever. Each session you’re working on gets a uniquely labeled folder, in your local storage.

Archived work

Second, you need a separate place where you put your work when you are done.

Since you have everything for a session in one folder, when you’re done with that session, you just copy that one folder to your archive place.

…and then you remove it from your local storage.

That takes some courage, and trust in your archive space. It’s critical to finish each session up, by freeing up your local (precious, limited) storage space.

Backups

Then your local storage, and your archive storage, get backed up.

The adage is that if you don’t regularly test restoring something from your backups… then you do not actually have backups.

Yes, regular testing—not just once on the day you set it up, but repeatedly test restoring a file, say, every few months. Recall my comment about the second 90%?

Let’s discuss

That’s a high level plan for organizing (unique labels), storing (active and archived) and backing up, your work.

Let’s discuss the specifics of what you’re doing—

Are you already doing any of what I’m suggesting?
What aren’t you doing? Why not?
Which services are you using?
What else are you doing that I’ve not thought of?

What questions do you have about the tools you’re using now?
What questions do you have about tools or services you’re thinking of using?

ɕ

1 Like

Time to circle back and drop some details on how I do things…

Local storage — I’m in the Mac universe, so Apple’s iCloud service maintains a copy of everything in the “iCloud/” folder on my mac. Inside there I have a folder full of zillions of folders and text files and pdf files…

Backup of local storage — iCloud synchronizes all my files across all my Apple devices. I pay $1/month to have 50GB of space in my iCloud. I don’t want to “go back in time”… this isn’t a snapshots of backups service. I only want to be sure my files aren’t lost if something happens to the computer itself. iCloud does that perfectly.

Unique session names — Because I use a slipbox addressing system, my session identifiers start with the number “3”. So “3.240528a” would be the first (the “a” on the end) May 28, 2024 session.

Non-local storage — I have a Synology, 2-drive, NAS. I’m not sure it’s that exact model, but you get the idea. It doesn’t come with the hard drives—you buy those separately. This thing is an entire computer system with a web-based management console. This of course means more technology maintenance—visiting it’s console now and then to do updates, etc.

File server — That Synology can do many (many!) things, but I’m using it to be a file server (using the “SMB” protocol if you care). So any computer in my local network (w/ user/pass security of course) can mount the storage on the fileserver. On there I have Terabytes of storage space.

Archiving work — On the file share, I have a folder for each project (Movers Mindset, Podtalk, and many other things). Inside each project are first folders by year, “2024” etc. In each year, folders with the month, “202405” etc. And inside there are folders for each session. Here’s part of Podtalk’s archive:

File server aside — The advantage of a file server is two-fold: Larger capacity, and drive redundancy. The Synology I’m using is the smallest one. It’s configured with large, 2TB drives (2,048 Gigabytes, each). The file servers uses RAID technology, to protect against loss if any one drive fails. And I can’t put a 2TB drive in my local computer. File servers with more bays can create even larger storage areas… or I could have bought 4TB drives, etc…

Backing up the archives — This, gets tricky, because you have this other computer system that you don’t normally do anything with. The Synology has lots of options—it’s an entire computer with myriad applications you can install. Instead, I use a tool called Arq Backup. This is installed on a desktop iMac. It runs every day and looks at the file server. It’s configured to use Amazon’s S3 storage as a “remote” place to backup. Arq is handy because it’s on the system I use when I’m at my desk… if I’m away, it just keeps doing it’s job. Amazon charges me a few dollars (like $2.31) each month to keep terabytes of my data backed up.

…that’s how I do it. But I’m guessing no one else wants that level of complexity :wink:

Any questions on any part of that? Desires to see more details, etc. ?

ɕ

This month’s idea club zoom doubles as the chance to talk about this Spotlight topic. Zoom URL details are in…

1 Like