#rrsync — Public Fediverse posts

Using Photoprism and Immich in Tandem Across Three Machines

1. Propagating from Photoprism to Immich
2. Woupsie Daisies
3. Drive not Mounted
4. A Sync Shell Script
5. The Standalone Immich Backup
6. The Tertiary Backup
7. And Finally

I have an HP laptop running Photoprism, and two Raspberry Pi 5 running Immich. One of the Pi devices is running Immich, solo, and is made accessible via Tailscale when I am on the move. Photoprism runs on the HP laptop and is not available at all times. When the laptop sleeps, Photoprism is not available. This means that I have to consciously decide to ingest the photos to photoprism.

Propagating from Photoprism to Immich

I setup Photosync on my phone to upload the most recent photos and organise them by year and month. This is easy to forget, so I consciously set that up. The reason for this decision is two-fold. The first reason is that by organising photos in this manner it is easy to mirror them to multiple drives, as well as to make them accessible to Immich, and Photoprism on the second Pi I mentioned earlier.

At the time when I set this up I was using the Ubuntu machine daily so every morning I could upload the photos with photosync.

Woupsie Daisies

Recently I allowed the HP laptop to run out of battery, and when it rebooted everything seemed fine, so I went to sync between devices and that’s when the “woopsie daisisies” incident occured. The “Woopsie Daisies Moment” was both catastrophic, but luckily recoverable. The SD card that I use within the laptop didn’t mount properly but I didn’t notice. When I told rsync to sync from the HP SD card to the external hard drive locally, and to Immich on the Pi it deleted all the files on the Mac spare drive. I suddenly noticed that Nextcloud had no more photos so I killed the sync within seconds. When a delete command is sent it goes for a while before the kill command was received.

The impact was that my secondary backup was wiped, and that my cloud backup, as a result was also in danger. When I realised I unplugged drives, safely, to isolate them, until I understood what happened.

Drive not Mounted

When I looked at the mounted drives on the HP machine I checked to see if the external hard drive was mounted. It was, but the photos were gone. It occured a few weeks ago so I don’t remember if the Pi5 was also wiped, but I don’t think it was. I then realised that the primary drive that I was using had not mounted after the dead battery incident. I mounted the drive, and the photos were intact.

A Sync Shell Script

To avoid this mistake from occuring again I made two key changes. The first is that I removed –delete from the rsync commands. If a file is not present rsync will not delete anything. Accidental deletion is almost instanteneous. The second change was to make sure that the shell script checked that the drives are mounted before doing anything. If the drive is not mounted it aborts the transfer.

Now I sync from the local SD card to the local external hard drive. When that’s done I sync the external hard drive to the Pi5 currently running Immich before then syncing from Linux to MacOS and my kDrive cloud instance. Within two or three minutes I have things backed up across the HP machine, the MacOS machine, a Pi5, and Infomaniak.

The Standalone Immich Backup

I have an entirely seperate workflow for my second Pi5. I synchronise to this Pi5 via the Immich app directly. In theory, within seconds of taking a photo it can be synced and indexed by this Pi5. If the Photoprism and Immich suffers an issue such as a describe above, this one is entirely seperate and isolated. A mistake with one worfkflow only affects one workflow.

The Tertiary Backup

The primary worflow goes from Photosync to Photoprism, and then Immich across multple machines and drives. The secondary worflow is from the Immich iPhone app to the Immich instance on a seperate Pi. The tertiary backup is kDrive. All of the photos are automatically backed up by kDrive in the year/month folder structure, so if the primary and secondary workflow fail, I can still revert to the tertiary solution.

Each library uses the same structure therefore migration data from one library to the other should be seamless. This gives me the opportunity to continue learning, and if I make a mistake I have several recovery solutions.

And Finally

The key mistake was not to check that the volume was available. If I had done that, then I would not have run rsync. I have removed that risk by replacing manual operation with a shell script. The second mistake was to have a delete flat in an rsync workflow by default. I removed it. The third mistake was to allow the battery on the laptop to run flat. That’s why when the machine was rebooted it did not mount correctly.

If you see that files are missing, the first thing to do is check that the volume is mounted. If I had done that, then I would have seen it wasn’t and I could have mounted it, and the issue would have been resolved within seconds.

The current workflow is now more resilient, and automated.

Using Photoprism and Immich in Tandem Across Three Machines

1. Propagating from Photoprism to Immich
2. Woupsie Daisies
3. Drive not Mounted
4. A Sync Shell Script
5. The Standalone Immich Backup
6. The Tertiary Backup
7. And Finally

I have an HP laptop running Photoprism, and two Raspberry Pi 5 running Immich. One of the Pi devices is running Immich, solo, and is made accessible via Tailscale when I am on the move. Photoprism runs on the HP laptop and is not available at all times. When the laptop sleeps, Photoprism is not available. This means that I have to consciously decide to ingest the photos to photoprism.

Propagating from Photoprism to Immich

I setup Photosync on my phone to upload the most recent photos and organise them by year and month. This is easy to forget, so I consciously set that up. The reason for this decision is two-fold. The first reason is that by organising photos in this manner it is easy to mirror them to multiple drives, as well as to make them accessible to Immich, and Photoprism on the second Pi I mentioned earlier.

At the time when I set this up I was using the Ubuntu machine daily so every morning I could upload the photos with photosync.

Woupsie Daisies

Recently I allowed the HP laptop to run out of battery, and when it rebooted everything seemed fine, so I went to sync between devices and that’s when the “woopsie daisisies” incident occured. The “Woopsie Daisies Moment” was both catastrophic, but luckily recoverable. The SD card that I use within the laptop didn’t mount properly but I didn’t notice. When I told rsync to sync from the HP SD card to the external hard drive locally, and to Immich on the Pi it deleted all the files on the Mac spare drive. I suddenly noticed that Nextcloud had no more photos so I killed the sync within seconds. When a delete command is sent it goes for a while before the kill command was received.

The impact was that my secondary backup was wiped, and that my cloud backup, as a result was also in danger. When I realised I unplugged drives, safely, to isolate them, until I understood what happened.

Drive not Mounted

When I looked at the mounted drives on the HP machine I checked to see if the external hard drive was mounted. It was, but the photos were gone. It occured a few weeks ago so I don’t remember if the Pi5 was also wiped, but I don’t think it was. I then realised that the primary drive that I was using had not mounted after the dead battery incident. I mounted the drive, and the photos were intact.

A Sync Shell Script

To avoid this mistake from occuring again I made two key changes. The first is that I removed –delete from the rsync commands. If a file is not present rsync will not delete anything. Accidental deletion is almost instanteneous. The second change was to make sure that the shell script checked that the drives are mounted before doing anything. If the drive is not mounted it aborts the transfer.

Now I sync from the local SD card to the local external hard drive. When that’s done I sync the external hard drive to the Pi5 currently running Immich before then syncing from Linux to MacOS and my kDrive cloud instance. Within two or three minutes I have things backed up across the HP machine, the MacOS machine, a Pi5, and Infomaniak.

The Standalone Immich Backup

I have an entirely seperate workflow for my second Pi5. I synchronise to this Pi5 via the Immich app directly. In theory, within seconds of taking a photo it can be synced and indexed by this Pi5. If the Photoprism and Immich suffers an issue such as a describe above, this one is entirely seperate and isolated. A mistake with one worfkflow only affects one workflow.

The Tertiary Backup

The primary worflow goes from Photosync to Photoprism, and then Immich across multple machines and drives. The secondary worflow is from the Immich iPhone app to the Immich instance on a seperate Pi. The tertiary backup is kDrive. All of the photos are automatically backed up by kDrive in the year/month folder structure, so if the primary and secondary workflow fail, I can still revert to the tertiary solution.

Each library uses the same structure therefore migration data from one library to the other should be seamless. This gives me the opportunity to continue learning, and if I make a mistake I have several recovery solutions.

And Finally

The key mistake was not to check that the volume was available. If I had done that, then I would not have run rsync. I have removed that risk by replacing manual operation with a shell script. The second mistake was to have a delete flat in an rsync workflow by default. I removed it. The third mistake was to allow the battery on the laptop to run flat. That’s why when the machine was rebooted it did not mount correctly.

If you see that files are missing, the first thing to do is check that the volume is mounted. If I had done that, then I would have seen it wasn’t and I could have mounted it, and the issue would have been resolved within seconds.

The current workflow is now more resilient, and automated.

Using Photoprism and Immich in Tandem Across Three Machines

I have an HP laptop running Photoprism, and two Raspberry Pi 5 running Immich. One of the Pi devices is running Immich, solo, and is made accessible via Tailscale when I am on the move. Photoprism runs on the HP laptop and is not available at all times. When the laptop sleeps, Photoprism is not available. This means that I have to consciously decide to ingest the photos to photoprism.

Propagating from Photoprism to Immich

I setup Photosync on my phone to upload the most recent photos and organise them by year and month. This is easy to forget, so I consciously set that up. The reason for this decision is two-fold. The first reason is that by organising photos in this manner it is easy to mirror them to multiple drives, as well as to make them accessible to Immich, and Photoprism on the second Pi I mentioned earlier.

At the time when I set this up I was using the Ubuntu machine daily so every morning I could upload the photos with photosync.

Woupsie Daisies

Recently I allowed the HP laptop to run out of battery, and when it rebooted everything seemed fine, so I went to sync between devices and that’s when the “woopsie daisisies” incident occured. The “Woopsie Daisies Moment” was both catastrophic, but luckily recoverable. The SD card that I use within the laptop didn’t mount properly but I didn’t notice. When I told rsync to sync from the HP SD card to the external hard drive locally, and to Immich on the Pi it deleted all the files on the Mac spare drive. I suddenly noticed that Nextcloud had no more photos so I killed the sync within seconds. When a delete command is sent it goes for a while before the kill command was received.

The impact was that my secondary backup was wiped, and that my cloud backup, as a result was also in danger. When I realised I unplugged drives, safely, to isolate them, until I understood what happened.

Drive not Mounted

When I looked at the mounted drives on the HP machine I checked to see if the external hard drive was mounted. It was, but the photos were gone. It occured a few weeks ago so I don’t remember if the Pi5 was also wiped, but I don’t think it was. I then realised that the primary drive that I was using had not mounted after the dead battery incident. I mounted the drive, and the photos were intact.

A Sync Shell Script

To avoid this mistake from occuring again I made two key changes. The first is that I removed –delete from the rsync commands. If a file is not present rsync will not delete anything. Accidental deletion is almost instanteneous. The second change was to make sure that the shell script checked that the drives are mounted before doing anything. If the drive is not mounted it aborts the transfer.

Now I sync from the local SD card to the local external hard drive. When that’s done I sync the external hard drive to the Pi5 currently running Immich before then syncing from Linux to MacOS and my kDrive cloud instance. Within two or three minutes I have things backed up across the HP machine, the MacOS machine, a Pi5, and Infomaniak.

The Standalone Immich Backup

I have an entirely seperate workflow for my second Pi5. I synchronise to this Pi5 via the Immich app directly. In theory, within seconds of taking a photo it can be synced and indexed by this Pi5. If the Photoprism and Immich suffers an issue such as a describe above, this one is entirely seperate and isolated. A mistake with one worfkflow only affects one workflow.

The Tertiary Backup

The primary worflow goes from Photosync to Photoprism, and then Immich across multple machines and drives. The secondary worflow is from the Immich iPhone app to the Immich instance on a seperate Pi. The tertiary backup is kDrive. All of the photos are automatically backed up by kDrive in the year/month folder structure, so if the primary and secondary workflow fail, I can still revert to the tertiary solution.

Each library uses the same structure therefore migration data from one library to the other should be seamless. This gives me the opportunity to continue learning, and if I make a mistake I have several recovery solutions.

And Finally

The key mistake was not to check that the volume was available. If I had done that, then I would not have run rsync. I have removed that risk by replacing manual operation with a shell script. The second mistake was to have a delete flat in an rsync workflow by default. I removed it. The third mistake was to allow the battery on the laptop to run flat. That’s why when the machine was rebooted it did not mount correctly.

If you see that files are missing, the first thing to do is check that the volume is mounted. If I had done that, then I would have seen it wasn’t and I could have mounted it, and the issue would have been resolved within seconds.

The current workflow is now more resilient, and automated.