Moment's Notice

Everything i do is about the moment. The moment i capture in photographs, the moment i act and react in a jazz improvisation, the moments i live through on a run. The moments i experience in my life every day. It is all connected and can and should be written and talked about.

The photographer's file organization and data backup

About a year ago, I posted a video on YouTube about how to fix a corrupted Lightroom catalog and ever since then I get mails form people asking me for help with their corrupted files. I do help them of course, but this is after the fact, but to help prevent this from happening, i thought I just jot down how i organize and backup my files. I hope this is helpful to you and you don't even have to recover corrupted files anymore.

LIGHTROOM CATALOG BACKUP

The first thing you do, if you haven't done so already is adjust the catalog backup settings in Lightroom. To do that, go to Edit > Catalog Settings, then go to the General tab (Figure 1) . It will show you where your catalog file is located, it's name, creation date and last backup and optimization date as well as the file size. Now you can choose when it will backup. I have it set to Every time Lightroom exits.

 Figure 1: Catalog settings

Figure 1: Catalog settings

This way when Lightroom exits, it will show you the Back Up Catalog dialog (Figure 2). There you can again change when it backs up and you'll also see the location of the backup folder. You can slso determine whether the integrity will be tested and the catalog file will be optimized. You can also skip it for this one time if you're in a hurry or so. The dialog will pop up again next time you exit Lightroom.

 Figure 2: On Lightroom exit

Figure 2: On Lightroom exit

You should check the backup folder every now and then and delete all backup files but the last five so they don't take up too much space. Those five files should then be copied to external drives. I'll get to that external drives in a minute.

 

ORGANIZING FILES IN LIGHTROOM

I am describing a system here i originally learned from a Tim Grey presentation, a Scott Kelby book, a Harold Davis bookletand a PHLEARN video, i just took the parts i liked from each and made it work for me. (Figure 3)

My main folder is called "My Lightroom Photos". In there are folders for every year. In these years i have folders for every shoot.  "Zoutelande 2017-04" means i was in the city of Zoutelande, The Netherlands in April 2017. I have the name of the place first since i probably will remember the place better than the date when i am looking for a picture.

Under these folders i always create five folders whenever i import from a new shoot.

1 - Capture: Holds all captures form the shoot

2 - Selects: Holds all pictures I selected as keepers and i will work on

3 - External WIP: Holds all files i have worked on in other programs than Lightroom - PSDs from Photoshop or TIFF files from Color Efex Pro etc.

4 - Master: Holds the master files i will make prints from or versions for the web and such.

5 - Output: Holds the files i send to customers, post on the internet or send to labs.

 Figure 3: Lightroom file structure

Figure 3: Lightroom file structure

All the folders sit on an external hard drive. I currently have two Western Digital My Passport Ultra external hard drives. The original location to which I import is the first of these drives I call "MP Photo 1". It is where I also copy the latest five of the Lightroom catalog backups as well as some other data i need backed up. (Figure 4).

 Figure 4: The folder structure on my first hard drive

Figure 4: The folder structure on my first hard drive

DATA BACKUP

Once you have a folder structure in place in Lightroom as well as on the hard drive, it is time to back this data up. The principle is that you want the original files and two copies of which one has to be in a different location in case something happens to your home or office.

The original catalog file is on the main drive of my computer so this one I will even have on more copy of. As I said, I copy the five most recent catalog backups from the location as shown in the "Backup Catalog" dialog (Figure 2) to the first external hard drive.

The images will always be imported to the "My Lightroom Photos" folder on the first external hard drive, they never go to my computer's hard drive. I usually import to the "000_Lightroom Import folder" as i will have to review them, create the folders for the shoot and you might want to keyword them as well (I never did and with almost 30,000 images I am not starting now. I can find my images the way i have set it all up, but for some keywording is a better concept to find images.

 Once everything is on the first external hard drive, it gets copied to the second external hard drive i called "MP Photo 2". I don't do this manually though, I use backup software. In my case Goodync (www.goodsync.com). (Figure 5)

 Figure 5: Goodsync user interface

Figure 5: Goodsync user interface

Whenever you imported pictures or changed anything on the first drive, you run GoodSync. You can also have it do it automatically, but I'd rather see it being done and it doesn't take too much time anyway. So what you see on the left is the first external hard drive and on the right is the second one. Whatever you change on the left, will be mirrored on the left, so the second drive will always be an exact copy of the first. You can also do other things like having the second drive keep everything which will mean you would always be able to retrieve deleted files, but that would use more space on the second drive.

So now you have two versions of your files at home or at your office. As i said you need one more off site to be safe. You could burn DVDs and take them someplace else, but you can also save all your stuff to the cloud. There are several services you can choose and the pricing is very affordable. You could of course use services like Dropbox or Google Drive, but they were not meant for that. Backblaze (www.backblaze.com) and Crashplan (www.crashplan.com) were designed for that and offer unlimited storage space. I use Crashplan and have currently more than 610 GB of data backed up there. (Figure 6).

 Figure 6: Crashplan user interface

Figure 6: Crashplan user interface

Once the Crashplan software is installed on your computer and you have selected which folders to back up in the cloud, it will run in the background. The first upload will of course take quite some time, depending on your connection. But once everything is uploaded, it will only transfer changes you made to the original folder. Just as Goodsync does in my setup. You can even adjust how much of your computer's resources are being used for the backup when you are at your machine or when it is idle, but I didn't notice changes in the performance since I am running this. If your machine goes down, you just reinstall Crashplan after you are back in business and restore the files.

So that's how I set up my folder structure and backup and I can say it never failed me. I hope you find this useful and it helps you to just concentrate on the fun part of photography more rather than having to worry about the technical bit too much. If you have any further questions, feel free to mail me.