Can two PCs, running Darktable with photos and library/styles ie configuration files synced over the internet work? YES
I work on my photos in two main locations and it was important for me to establish a system that allowed me to do this seamlessly. I could use a cloud but call me old fashioned or cheap or simple, whatever floats your boat. For a number of reasons I chose to do it this way along time ago and its been working ever since even after a few software and hardware upgrades.
What is Darktable
For those uninitiated Darktable is a magic bit of free software written for Linux (Ubuntu etc) which allows editing of RAW photos. It includes cataloguing, application of styles etc Go check it out. The main point I want to draw your attention to here is that its non destructive editing/manipulation of the original file. It keeps those changes in a separate .XMP which is extremely small. It is that file, the RAW image and the configuration files which must be seen as the same form both PCs. I didn’t want one location. RAW are large files speed is important. Also two locations gives me that security of an off site backup.
I’m going to get to the trickier stuff further down but for now the simpler moves in our game of photo chess. As mentioned we have large RAW files, the .XMP definition file and our output printable, web or JPG files. As the RAW files are large I generally tend to move them either on the SD card or USB sticks from location to location. Especially after a large photo session incorporating 100s of images. Once the RAW files are in both locations tho they dont change so never add to the bandwidth or internet burden. Only the XMP or output files change and these are small so zip along the net highway.
Via an SSH tunnel linking both machines across the net I use Rsync or the gui version Grsync to keep both HDs looking the same. Keeps all the photos up to date. No matter what machine I am on it all looks the same. Any style I create, any edit can be done on either machine. This also means no matter where I am in the world I could via the SSH access the photos.
Syncing Library, styles etc
The one thing I was lacking was the configuration setup the library database used by Darktable to keep track of where things are, tags, styles, the last thing I was editing , history and that sort of thing. Luckily Rsyncing the /home/user/.config/darktable folder works perfectly. All your styles, presets, watermarks and the current configuration of the Darktable view is seamless across machines. Again a rsync command takes care of this.
Using external HDs
My Photos are stored on removable drives. In one location I have extended the SATA cables outside the box and the drive just sits on the top. Prior to that the drive was kept inside but I do lots of different things and was often disk swapping. In location 2 I have a SATA to USB dock where I plug the HD in. Ubuntu changed the way it mounted things a while back and also due to HD updates (you always need more or bigger HDs when playing with images) so naming started to become an issue. That is the darktable (DT) library stores the location of each image so it can cross reference tags etc. This is great when searching thousands of images for a faint memory. Of course this relies on you diligently tagging your photos on the initial import..but that’s for another post.
Linux makes this easy we can make any location “look” the same to DT. Ubuntu now mounts drives in /media/user/ so what I did was create a symbolic link in /media/ to point to whatever the name of the drive was .
Here is the PHOTOS HD as mounted by Ubuntu. /media/user/
How it looks when all is working
How it looks when the HD is not physically there
Links can break if the HD is not physically there but thats not a problem It will re link as soon as the HD becomes available. Just don’t use DT when its not there.
To create the symbolic link I found it best to set it up in sudo or super user mode. you can either do it manually or run your file manager sudo nautilus or sudo thunar etc from. I wont go into the nitty grit here because doing stuff sudo can create havoc. Not a beginners thing. This post is more conceptual than an exact how to.
Ctrl shift Click drag to create link in /media/
Broken but not busted
To prove all this works well, I broke it. It was that time again to upgrade to a bigger HD on both machines. I noticed in the copy process that one of the drives was running under NTFS. Not sure how that happened but it was time to set it all to FAT32, making it all a little bit more compatible.
Everbody loves FAT32, or so the internet says. Its the most compatible across platforms. As we are using Linux but may want to jaunt over to a WinPC or even a MAC, logic would suggest we format our drives in FAT32. Problem is by default FAT32 labels its drives in uppercase only. There are ways around this but why make things tricky. That sort of thing bites you in the bum later on, normally when you are under pressure.
So I formatted the two new drives as FAT32, copied the terabytes of data across to both of them(yes this took a while) and installed the drives in their PCs. All booted , all photos were viewable…then I invoked DT and it was all broken.
Darktable had the wrong address
If you look at the library.DB in .config/darktable/ with say SQlite manager you will see in table film_rolls a field called folder. This is where the image address records are stored. Straight away I noticed the problem. My library was looking for /media/Photos due to the NTFS hangover. I could have fixed this with the symbolic link, I could have forced the label to be lowercase but lets try and clean it up so it always works and it matches the default scenario. I had also in the past moved drives around and new that other locations in the library were kind of broken. Time to fix this all up once and for all.
SQliteman-modifying the Library.db
I am not an SQL guy, in fact its the first time I have used it. I find the syntax and online help messy and confusing. Creating a “query” to search or find the problem was easy enough. replacing the address too was easy.
But saving the db, with the changes took me a while to work out. I bobbled my way thru examples online and finally came up with this.
UPDATE film_rolls set folder =replace(folder,’/media/Photos/’,’/media/PHOTOS/’);
Using this new found info you could “touchup” or modify any weird things in the DT library.db. WARNING applies of course. backup first.