#rsync — Public Fediverse posts

How do I configure Syncthing paths between two Ubuntu VMs in Proxmox? #virtualization #sync #rsync

https://askubuntu.com/q/1566769/612

Camilo Aguilar on the Rsync Algorithm [PWL NYC] (2018)

https://www.youtube.com/watch?v=X3Stha8pxXc

#rsync #paper #algorithm

Arch Tip týdne: Zálohování – Borg vs rsync

💾 Dva osvědčené nástroje pro zálohu dat na Archu.
Každý řeší něco jiného – tady je rozdíl a kdy co použít.

═══════════════════════════════════════════
📦 BORG – chytré zálohy s historií
═══════════════════════════════════════════

Co umí:
→ deduplikace (stejný soubor se uloží jen jednou)
→ šifrování (AES-256)
→ komprese (lz4, zstd, zlib)
→ verze v čase – můžeš se vrátit k libovolnému snapshotu

Instalace:
sudo pacman -S borg

Základní workflow:

inicializace repozitáře (jen jednou)

borg init --encryption=repokey /mnt/backup/repo

vytvoření zálohy

borg create --stats --progress
/mnt/backup/repo::archos-{now}
~/Dokumenty ~/Obrázky ~/.config

výpis archivů

borg list /mnt/backup/repo

obnova souboru

borg extract /mnt/backup/repo::archos-2025-01-15 home/archos/Dokumenty

retence – nech 7 denních, 4 týdenní, 6 měsíčních

borg prune -v --list
--keep-daily=7 --keep-weekly=4 --keep-monthly=6
/mnt/backup/repo

Výhody: úspora místa, šifrování, historie
Nevýhody: data nejsou přímo čitelná – potřebuješ borg

═══════════════════════════════════════════
🔄 RSYNC – rychlý mirror 1:1
═══════════════════════════════════════════

Co umí:
→ synchronizace souborů (lokálně i přes SSH)
→ přenáší jen změny (delta)
→ zachová oprávnění, ACL, xattrs, hardlinky
→ data zůstávají čitelná jako normální soubory

Instalace:
sudo pacman -S rsync

Základní workflow:

mirror /home na externí disk

rsync -aAXHv --delete --progress
~/ /mnt/backup/home/

suchý běh (nic nezapíše, jen ukáže)

rsync -aAXHvn --delete ~/ /mnt/backup/home/

přes SSH na vzdálený server

rsync -aAXHv --delete -e ssh
~/Dokumenty/ user@server:/backup/dokumenty/

vynechat složky

rsync -aAXHv --delete
--exclude='.cache' --exclude='node_modules'
~/ /mnt/backup/home/

Výhody: rychlý, čitelný výstup, univerzální
Nevýhody: žádná historie (přepisuje), bez šifrování

═══════════════════════════════════════════
💡 KDY CO POUŽÍT
═══════════════════════════════════════════

Borg:
→ pravidelné zálohy s historií verzí
→ šifrované zálohy na cizí server (Hetzner, Borgbase)
→ když potřebuješ šetřit místo (deduplikace)

rsync:
→ mirror na externí disk
→ rychlá synchronizace mezi stroji
→ když chceš data přímo procházet
→ záloha webů, konfiguráků, deploy

Kombinace:
→ rsync pro denní mirror na NAS
→ borg pro týdenní šifrovanou zálohu na vzdálený server

#archlinux #backup #borg #rsync #linux #sysadmi

Arch Tip týdne: Zálohování – Borg vs rsync

💾 Dva osvědčené nástroje pro zálohu dat na Archu.
Každý řeší něco jiného – tady je rozdíl a kdy co použít.

═══════════════════════════════════════════
📦 BORG – chytré zálohy s historií
═══════════════════════════════════════════

Co umí:
→ deduplikace (stejný soubor se uloží jen jednou)
→ šifrování (AES-256)
→ komprese (lz4, zstd, zlib)
→ verze v čase – můžeš se vrátit k libovolnému snapshotu

Instalace:
sudo pacman -S borg

Základní workflow:

inicializace repozitáře (jen jednou)

borg init --encryption=repokey /mnt/backup/repo

vytvoření zálohy

borg create --stats --progress
/mnt/backup/repo::archos-{now}
~/Dokumenty ~/Obrázky ~/.config

výpis archivů

borg list /mnt/backup/repo

obnova souboru

borg extract /mnt/backup/repo::archos-2025-01-15 home/archos/Dokumenty

retence – nech 7 denních, 4 týdenní, 6 měsíčních

borg prune -v --list
--keep-daily=7 --keep-weekly=4 --keep-monthly=6
/mnt/backup/repo

Výhody: úspora místa, šifrování, historie
Nevýhody: data nejsou přímo čitelná – potřebuješ borg

═══════════════════════════════════════════
🔄 RSYNC – rychlý mirror 1:1
═══════════════════════════════════════════

Co umí:
→ synchronizace souborů (lokálně i přes SSH)
→ přenáší jen změny (delta)
→ zachová oprávnění, ACL, xattrs, hardlinky
→ data zůstávají čitelná jako normální soubory

Instalace:
sudo pacman -S rsync

Základní workflow:

mirror /home na externí disk

rsync -aAXHv --delete --progress
~/ /mnt/backup/home/

suchý běh (nic nezapíše, jen ukáže)

rsync -aAXHvn --delete ~/ /mnt/backup/home/

přes SSH na vzdálený server

rsync -aAXHv --delete -e ssh
~/Dokumenty/ user@server:/backup/dokumenty/

vynechat složky

rsync -aAXHv --delete
--exclude='.cache' --exclude='node_modules'
~/ /mnt/backup/home/

Výhody: rychlý, čitelný výstup, univerzální
Nevýhody: žádná historie (přepisuje), bez šifrování

═══════════════════════════════════════════
💡 KDY CO POUŽÍT
═══════════════════════════════════════════

Borg:
→ pravidelné zálohy s historií verzí
→ šifrované zálohy na cizí server (Hetzner, Borgbase)
→ když potřebuješ šetřit místo (deduplikace)

rsync:
→ mirror na externí disk
→ rychlá synchronizace mezi stroji
→ když chceš data přímo procházet
→ záloha webů, konfiguráků, deploy

Kombinace:
→ rsync pro denní mirror na NAS
→ borg pro týdenní šifrovanou zálohu na vzdálený server

#archlinux #backup #borg #rsync #linux #sysadmi

Arch Tip týdne: Zálohování – Borg vs rsync

💾 Dva osvědčené nástroje pro zálohu dat na Archu.
Každý řeší něco jiného – tady je rozdíl a kdy co použít.

═══════════════════════════════════════════
📦 BORG – chytré zálohy s historií
═══════════════════════════════════════════

Co umí:
→ deduplikace (stejný soubor se uloží jen jednou)
→ šifrování (AES-256)
→ komprese (lz4, zstd, zlib)
→ verze v čase – můžeš se vrátit k libovolnému snapshotu

Instalace:
sudo pacman -S borg

Základní workflow:

inicializace repozitáře (jen jednou)

borg init --encryption=repokey /mnt/backup/repo

vytvoření zálohy

borg create --stats --progress
/mnt/backup/repo::archos-{now}
~/Dokumenty ~/Obrázky ~/.config

výpis archivů

borg list /mnt/backup/repo

obnova souboru

borg extract /mnt/backup/repo::archos-2025-01-15 home/archos/Dokumenty

retence – nech 7 denních, 4 týdenní, 6 měsíčních

borg prune -v --list
--keep-daily=7 --keep-weekly=4 --keep-monthly=6
/mnt/backup/repo

Výhody: úspora místa, šifrování, historie
Nevýhody: data nejsou přímo čitelná – potřebuješ borg

═══════════════════════════════════════════
🔄 RSYNC – rychlý mirror 1:1
═══════════════════════════════════════════

Co umí:
→ synchronizace souborů (lokálně i přes SSH)
→ přenáší jen změny (delta)
→ zachová oprávnění, ACL, xattrs, hardlinky
→ data zůstávají čitelná jako normální soubory

Instalace:
sudo pacman -S rsync

Základní workflow:

mirror /home na externí disk

rsync -aAXHv --delete --progress
~/ /mnt/backup/home/

suchý běh (nic nezapíše, jen ukáže)

rsync -aAXHvn --delete ~/ /mnt/backup/home/

přes SSH na vzdálený server

rsync -aAXHv --delete -e ssh
~/Dokumenty/ user@server:/backup/dokumenty/

vynechat složky

rsync -aAXHv --delete
--exclude='.cache' --exclude='node_modules'
~/ /mnt/backup/home/

Výhody: rychlý, čitelný výstup, univerzální
Nevýhody: žádná historie (přepisuje), bez šifrování

═══════════════════════════════════════════
💡 KDY CO POUŽÍT
═══════════════════════════════════════════

Borg:
→ pravidelné zálohy s historií verzí
→ šifrované zálohy na cizí server (Hetzner, Borgbase)
→ když potřebuješ šetřit místo (deduplikace)

rsync:
→ mirror na externí disk
→ rychlá synchronizace mezi stroji
→ když chceš data přímo procházet
→ záloha webů, konfiguráků, deploy

Kombinace:
→ rsync pro denní mirror na NAS
→ borg pro týdenní šifrovanou zálohu na vzdálený server

#archlinux #backup #borg #rsync #linux #sysadmi

Arch Tip týdne: Zálohování – Borg vs rsync

💾 Dva osvědčené nástroje pro zálohu dat na Archu.
Každý řeší něco jiného – tady je rozdíl a kdy co použít.

═══════════════════════════════════════════
📦 BORG – chytré zálohy s historií
═══════════════════════════════════════════

Co umí:
→ deduplikace (stejný soubor se uloží jen jednou)
→ šifrování (AES-256)
→ komprese (lz4, zstd, zlib)
→ verze v čase – můžeš se vrátit k libovolnému snapshotu

Instalace:
sudo pacman -S borg

Základní workflow:

inicializace repozitáře (jen jednou)

borg init --encryption=repokey /mnt/backup/repo

vytvoření zálohy

borg create --stats --progress
/mnt/backup/repo::archos-{now}
~/Dokumenty ~/Obrázky ~/.config

výpis archivů

borg list /mnt/backup/repo

obnova souboru

borg extract /mnt/backup/repo::archos-2025-01-15 home/archos/Dokumenty

retence – nech 7 denních, 4 týdenní, 6 měsíčních

borg prune -v --list
--keep-daily=7 --keep-weekly=4 --keep-monthly=6
/mnt/backup/repo

Výhody: úspora místa, šifrování, historie
Nevýhody: data nejsou přímo čitelná – potřebuješ borg

═══════════════════════════════════════════
🔄 RSYNC – rychlý mirror 1:1
═══════════════════════════════════════════

Co umí:
→ synchronizace souborů (lokálně i přes SSH)
→ přenáší jen změny (delta)
→ zachová oprávnění, ACL, xattrs, hardlinky
→ data zůstávají čitelná jako normální soubory

Instalace:
sudo pacman -S rsync

Základní workflow:

mirror /home na externí disk

rsync -aAXHv --delete --progress
~/ /mnt/backup/home/

suchý běh (nic nezapíše, jen ukáže)

rsync -aAXHvn --delete ~/ /mnt/backup/home/

přes SSH na vzdálený server

rsync -aAXHv --delete -e ssh
~/Dokumenty/ user@server:/backup/dokumenty/

vynechat složky

rsync -aAXHv --delete
--exclude='.cache' --exclude='node_modules'
~/ /mnt/backup/home/

Výhody: rychlý, čitelný výstup, univerzální
Nevýhody: žádná historie (přepisuje), bez šifrování

═══════════════════════════════════════════
💡 KDY CO POUŽÍT
═══════════════════════════════════════════

Borg:
→ pravidelné zálohy s historií verzí
→ šifrované zálohy na cizí server (Hetzner, Borgbase)
→ když potřebuješ šetřit místo (deduplikace)

rsync:
→ mirror na externí disk
→ rychlá synchronizace mezi stroji
→ když chceš data přímo procházet
→ záloha webů, konfiguráků, deploy

Kombinace:
→ rsync pro denní mirror na NAS
→ borg pro týdenní šifrovanou zálohu na vzdálený server

#archlinux #backup #borg #rsync #linux #sysadmi

Arch Tip týdne: Zálohování – Borg vs rsync

💾 Dva osvědčené nástroje pro zálohu dat na Archu.
Každý řeší něco jiného – tady je rozdíl a kdy co použít.

═══════════════════════════════════════════
📦 BORG – chytré zálohy s historií
═══════════════════════════════════════════

Co umí:
→ deduplikace (stejný soubor se uloží jen jednou)
→ šifrování (AES-256)
→ komprese (lz4, zstd, zlib)
→ verze v čase – můžeš se vrátit k libovolnému snapshotu

Instalace:
sudo pacman -S borg

Základní workflow:

inicializace repozitáře (jen jednou)

borg init --encryption=repokey /mnt/backup/repo

vytvoření zálohy

borg create --stats --progress
/mnt/backup/repo::archos-{now}
~/Dokumenty ~/Obrázky ~/.config

výpis archivů

borg list /mnt/backup/repo

obnova souboru

borg extract /mnt/backup/repo::archos-2025-01-15 home/archos/Dokumenty

retence – nech 7 denních, 4 týdenní, 6 měsíčních

borg prune -v --list
--keep-daily=7 --keep-weekly=4 --keep-monthly=6
/mnt/backup/repo

Výhody: úspora místa, šifrování, historie
Nevýhody: data nejsou přímo čitelná – potřebuješ borg

═══════════════════════════════════════════
🔄 RSYNC – rychlý mirror 1:1
═══════════════════════════════════════════

Co umí:
→ synchronizace souborů (lokálně i přes SSH)
→ přenáší jen změny (delta)
→ zachová oprávnění, ACL, xattrs, hardlinky
→ data zůstávají čitelná jako normální soubory

Instalace:
sudo pacman -S rsync

Základní workflow:

mirror /home na externí disk

rsync -aAXHv --delete --progress
~/ /mnt/backup/home/

suchý běh (nic nezapíše, jen ukáže)

rsync -aAXHvn --delete ~/ /mnt/backup/home/

přes SSH na vzdálený server

rsync -aAXHv --delete -e ssh
~/Dokumenty/ user@server:/backup/dokumenty/

vynechat složky

rsync -aAXHv --delete
--exclude='.cache' --exclude='node_modules'
~/ /mnt/backup/home/

Výhody: rychlý, čitelný výstup, univerzální
Nevýhody: žádná historie (přepisuje), bez šifrování

═══════════════════════════════════════════
💡 KDY CO POUŽÍT
═══════════════════════════════════════════

Borg:
→ pravidelné zálohy s historií verzí
→ šifrované zálohy na cizí server (Hetzner, Borgbase)
→ když potřebuješ šetřit místo (deduplikace)

rsync:
→ mirror na externí disk
→ rychlá synchronizace mezi stroji
→ když chceš data přímo procházet
→ záloha webů, konfiguráků, deploy

Kombinace:
→ rsync pro denní mirror na NAS
→ borg pro týdenní šifrovanou zálohu na vzdálený server

#archlinux #backup #borg #rsync #linux #sysadmi

⚔️ Il giorno in cui ho abbandonato SFTP

Dopo lunga meditazione tecnica...
dopo prove, trasferimenti, osservazioni mistiche davanti al terminale...

ho scelto RSYNC.

Non perché fa scena.
Non perché “lo usano i guru”.
Non perché ha un nome aggressivo.

Ma perché funziona dannatamente bene.

🛡️ SFTP, grazie di tutto

Hai servito con onore.

Finché i backup erano piccoli o tranquilli...
sei andato benissimo.

Ma poi i backup sono cresciuti.
Hanno iniziato a pesare come meteoriti compressi.

E lì ogni interruzione diventava una tragedia greca.

Scaricavi file.
Li portavi a casa.
Ogni tanto ti fermavi nel momento peggiore possibile.

E lì iniziava il rito antico:

“Ricomincia tutto da capo.”

🚀 RSYNC entra in scena

Con lui il dialogo è diverso:

Connessione cade?

Riprendo da dove ero arrivato.

File già presente?

Controllo.

Backup grosso come un mammut?

Andiamo avanti con calma, ma andiamo.

📦 Ma attenzione...

Io NON voglio backup che si aggiornano da soli.
NON voglio magie silenziose.
NON voglio script che decidono al posto mio.

Io voglio:

• backup nuovo
• data nuova
• file nuovo
• storico locale sul mio HDD da 4 TB
• 4 o 5 vecchie glorie conservate con rispetto

Come un archivista medievale del sysadmin.

🧠 E soprattutto... manuale

Sì, manuale.

Perché facendo a mano:

• vedo cosa succede
• controllo tutto
• imparo davvero
• se qualcosa esplode, so dove guardare

L’automazione cieca è comoda...
finché non prende fuoco alle 03:17.

🏆 Morale finale

Non sto scaricando file.

Sto costruendo esperienza.

Non sto facendo backup.

Sto addestrando me stesso.

E mentre qualcuno clicca “Download” su interfacce dubbie...

io avanzo nel terminale,
tra date, tarball e gloria.

RSYNC sia con me.

#RSYNC

#SFTP

#BACKUP

⚔️ Il giorno in cui ho abbandonato SFTP

Dopo lunga meditazione tecnica...
dopo prove, trasferimenti, osservazioni mistiche davanti al terminale...

ho scelto RSYNC.

Non perché fa scena.
Non perché “lo usano i guru”.
Non perché ha un nome aggressivo.

Ma perché funziona dannatamente bene.

🛡️ SFTP, grazie di tutto

Hai servito con onore.

Finché i backup erano piccoli o tranquilli...
sei andato benissimo.

Ma poi i backup sono cresciuti.
Hanno iniziato a pesare come meteoriti compressi.

E lì ogni interruzione diventava una tragedia greca.

Scaricavi file.
Li portavi a casa.
Ogni tanto ti fermavi nel momento peggiore possibile.

E lì iniziava il rito antico:

“Ricomincia tutto da capo.”

🚀 RSYNC entra in scena

Con lui il dialogo è diverso:

Connessione cade?

Riprendo da dove ero arrivato.

File già presente?

Controllo.

Backup grosso come un mammut?

Andiamo avanti con calma, ma andiamo.

📦 Ma attenzione...

Io NON voglio backup che si aggiornano da soli.
NON voglio magie silenziose.
NON voglio script che decidono al posto mio.

Io voglio:

• backup nuovo
• data nuova
• file nuovo
• storico locale sul mio HDD da 4 TB
• 4 o 5 vecchie glorie conservate con rispetto

Come un archivista medievale del sysadmin.

🧠 E soprattutto... manuale

Sì, manuale.

Perché facendo a mano:

• vedo cosa succede
• controllo tutto
• imparo davvero
• se qualcosa esplode, so dove guardare

L’automazione cieca è comoda...
finché non prende fuoco alle 03:17.

🏆 Morale finale

Non sto scaricando file.

Sto costruendo esperienza.

Non sto facendo backup.

Sto addestrando me stesso.

E mentre qualcuno clicca “Download” su interfacce dubbie...

io avanzo nel terminale,
tra date, tarball e gloria.

RSYNC sia con me.

#RSYNC

#SFTP

#BACKUP

I really don't get this interaction between "sync" and "rsync". Tried it a few more times and it doesn't happen always but sometimes the "sync" gets stuck until rsync is finished completely. Even if it has yet to touch e.g. 3TB of data.

WTF?!?

Anyone seeing similar things or is there something else at play here too?

#gnu #Linux #coreutils #rsync #NixOS

Почему rollback на ext4 — боль, и как я решил это через rsync

Я не собирался делать этот проект. Просто однажды сломал систему — и понял, что rollback на ext4 не такая простая вещь, как кажется. В итоге собрал своё решение на rsync: быстрый и предсказуемый откат без смены файловой системы. Кода тут минимум — он на GitHub. В статье — про сам подход: проблему, попытки решения и итог. Вернуть систему

https://habr.com/ru/articles/1018056/

#linux #devops #rollback #rsync #ext4 #backup #snapshot #файловая_система #sysadmin #infrastructure

RE: https://fosstodon.org/@borgbackup/116325322352636962

Faites des sauvegardes !

Perso je n'ai (toujours) pas pris le temps de me pencher sur #Borg, j'ai un système qui tourne avec #rsync depuis quelques années.

Mes irremplaçables photos sont stockées en triple et dans 2 lieux différents.

Quant à Borg, un jour j'irai voir ça pour peut-être remplacer rsync.

#sauvegarde #backup

RE: https://fosstodon.org/@borgbackup/116325322352636962

Faites des sauvegardes !

Perso je n'ai (toujours) pas pris le temps de me pencher sur #Borg, j'ai un système qui tourne avec #rsync depuis quelques années.

Mes irremplaçables photos sont stockées en triple et dans 2 lieux différents.

Quant à Borg, un jour j'irai voir ça pour peut-être remplacer rsync.

#sauvegarde #backup

RE: https://fosstodon.org/@borgbackup/116325322352636962

Faites des sauvegardes !

Perso je n'ai (toujours) pas pris le temps de me pencher sur #Borg, j'ai un système qui tourne avec #rsync depuis quelques années.

Mes irremplaçables photos sont stockées en triple et dans 2 lieux différents.

Quant à Borg, un jour j'irai voir ça pour peut-être remplacer rsync.

#sauvegarde #backup

RE: https://fosstodon.org/@borgbackup/116325322352636962

Faites des sauvegardes !

Perso je n'ai (toujours) pas pris le temps de me pencher sur #Borg, j'ai un système qui tourne avec #rsync depuis quelques années.

Mes irremplaçables photos sont stockées en triple et dans 2 lieux différents.

Quant à Borg, un jour j'irai voir ça pour peut-être remplacer rsync.

#sauvegarde #backup

RE: https://fosstodon.org/@borgbackup/116325322352636962

Faites des sauvegardes !

Perso je n'ai (toujours) pas pris le temps de me pencher sur #Borg, j'ai un système qui tourne avec #rsync depuis quelques années.

Mes irremplaçables photos sont stockées en triple et dans 2 lieux différents.

Quant à Borg, un jour j'irai voir ça pour peut-être remplacer rsync.

#sauvegarde #backup

Ripping Out Three Backup Systems and Replacing Them with One (Part 1: The Mess)

https://rant.mvh.dev/ripping-out-three-backup-systems-and-replacing-them-with-one-part-1-the-mess/

#selfhosted #bacula #backups #homelab #rsnapshot #rsync

Ripping Out Three Backup Systems and Replacing Them with One (Part 1: The Mess)

https://rant.mvh.dev/ripping-out-three-backup-systems-and-replacing-them-with-one-part-1-the-mess/

#selfhosted #bacula #backups #homelab #rsnapshot #rsync

Ripping Out Three Backup Systems and Replacing Them with One (Part 1: The Mess)

https://rant.mvh.dev/ripping-out-three-backup-systems-and-replacing-them-with-one-part-1-the-mess/

#selfhosted #bacula #backups #homelab #rsnapshot #rsync

Ripping Out Three Backup Systems and Replacing Them with One (Part 1: The Mess)

https://rant.mvh.dev/ripping-out-three-backup-systems-and-replacing-them-with-one-part-1-the-mess/

#selfhosted #bacula #backups #homelab #rsnapshot #rsync

Ripping Out Three Backup Systems and Replacing Them with One (Part 1: The Mess)

https://rant.mvh.dev/ripping-out-three-backup-systems-and-replacing-them-with-one-part-1-the-mess/

#selfhosted #bacula #backups #homelab #rsnapshot #rsync

RE: https://mas.to/@tezoatlipoca/116263075699722140

Well, I had to give up on the #rsync approach - rsync over #MTP seemed to have issues with copying some file metadata that rsync would need to handle incremental updates. Subsequent rsyncs would re-copy _everything_. Boo.

Likewise using #android usb-debug mode and `adb push` (android debug tool on linux) also only re-copies everyting.
I could use #termux on the phone to provide an actual rsync or ssh host on the phone (which would also let me use wifi). Or..

1/

Checked all little utilities, which I'm using in my daily computing, are they still good old programs or slopware?

I prefer to use little programs, which were created to please the needs of it's creator. And/or some folks which has the same needs. The process of such little program creation usually, if author in sane state of mind, doesn't mimick process of commercial software creation, where developers need to rush to "deliver features" to please management and investors. For now this leds to forcing developers to use LLMs on the workplace — all to "deliver features" faster. So one developer for the same price (salary) able to make more features. Profit!

So, when I see how opensource programmer uses LLM to create some opensource program — it is a red flag for me and I'll try to avoid using such program. Because it means to me that programmer doesn't like the process of creation. Like an artist who don't like to draw or photographer, who don't like to make photos. Also, (s)he possibly has a "corporate mindset" (deliver value and features faster, no fun allowed). So, looks like his/her creature is not a pet, but a cattle. When I prefer to use "pet"-programs — usually they are nicer, simpler and doesn't bring me a lot of problems.

Results are pretty good — only three programs are slopware now. These three programs, installed from repositories of my OS, have versions, when these programs were coded by humans.

Here they are:

1) rsync — version 3.4.1 is good, but the next versions will be slopware, since programming happens with Claude LLM.

2) ImageMagick7 — installed good version 7.1.2-15. But since 7.1.2-16 it become a slopware. LLM the same — Claude, was used in one commit.

3) python3 — installed version 3.11.15. Since 3.13.6, according to commits and release dates, it become a slopware too — there are some commits, where the same Claude LLM was used.

#slopware #rsync #ImageMagick #Python3

Replacing FTP with Rsync For my Blog

Recently I have been playing with rsync a lot. In the process of synching source A to B, as well as synching between machines I have grown familiar with how it works. It is for this reason that the move from using ftp for rsync to update the static part of the website began to make sense.

When I write a blog post I update wordpress with the markdown from the static blog post and then I run hugo to prepare the static site. I then used Filezilla to upload the changed files.

With a blog that is updated daily, it's not that I update two or three files. The blog post page is created of course, but the navigation from plenty of pages needs to be updated at the same time. The result is that filezilla needs to compare, and transfer hundreds of files on a daily basis.

As I use one computer for blogging, and another for other tasks the time it takes Filezilla via FTP to work through the list is time that I'm stuck waiting.

With rsync, with rsync -av --dry-run /local/path/ user@remote_host:/remote/path/ I can update the blog as soon as Hugo has run, within seconds, and from the command line rather than a dedicated app. I'm suggesting the "--dry-run" flag so that you can double check that it is doing what you expect before running it without the flag.

Getting a Push from AI

In my eyes vibe coding apps, and getting AI to write blog posts or create photographic kitsch and videos is deeply immoral. Asking AI to help you use understand tools such as rsync is worthwhile.

It's not that we can't read the manual. It's not that the manual is hard to understand. It's that sometimes we learn and think differently than those that wrote the man pages. We might not have the right context to understand the nuance of what was written.

Many times I have wanted to use a tool, read the man page, failed to understand it, tried two or three things and got nowhere. I spent half a day or more trying to get Ghost to work on an Infomaniak Node.js server without success despite asking for AI for help.

The Grsync Stepping Stone

More than once I used grsync to back up a linux machine and it works well. I can look at the interface and see the options, but it would take reading the fabulous manual to understand what everything does. I was happy with Grsync for a while.

The Gemini Advantage

With Gemini, I will say, "I want to sync the Hugo publish output from my local machine to my web server. Which flags are optimal for this task. I would also like to run it without using the password. What does that involve" and it will generate the prompt as well as explain what each prompt does and why it's used.

Beware Hallucinations

When you are given a prompt make sure that you understand it before running it, and if you do run it, try a dry run. If the output is not too long you can feed it to Gemini and ask if you can proceed. You can also say "I noticed that the output seems wrong in this manner" and it will help you debug. More than once it caught that I was missing a "/" at the end of a source. In that case the folder and it's contents would be moved, rather than just the contents.

Long Conversations with Gemini

If you're curious why I favour Gemini over Euria, MyAI, Le Chat and other solutions, it's because I rarely if ever get it telling me that I am out of tokens. Instead it hallucinates more and more. If you're playing with rsync (By playing I mean learning) you can often get long outputs and these long outputs can quickly get Gemini to hallucinate.

Who cares?

When you're learning to use rsync, you can ask it to be verbose to see what it's doing. Since that output can cover hundreds, if not thousands of lines, you can ask gemini to help you with grep and other tools to check that what you expect is happening, for quality control and quality assurance.

If you did this by eye, and by skimming you might miss something that AI, due to its optimisation for dealing with big data, might help you with.

AI as Patient Tutor

Reading a man page will tell you about the diversity of flags and how to use them but you might have reservations about trusting that you have understood what prompts do. That's where AI as a patient tutor comes in. I might run command A once, twice, three times, and with each run I become more confident, in part because Gemini or another "tutor" confirms that what I'm doing is right. It doesn't mind repeating a lesson until it sinks in.

Move From Host to Host

Imagine, you are with a web host and you have files on Hosting Solution A and Hosting Solution B. My natural instinct was to FTP the files from the web host locally, and then to ftp them back up to Hosting Solution B. Gemini said "Use rsync" and because I had experimented with ssh and transferring files via rsync locally and remotely the idea grabbed me, so I experimented, and that's why I changed how I update my blog.

And Finally

I was using rsync a lot, for moving around and synching photos between drives. In the process my confidence with this tool grew. I also grew more familiar with using rsync between machines within my "home lab" so it became a small leap to go a step further, to sync my blog.

Thanks to Gemini being my "tutor/mentor" I broke my 29 year habit of using FTP.

I have a large local #musiclibrary—mostly #flac with some #m4a. I sync it between my #nas and four computers using #rsync.
Unfortunately, I recently discovered that some files appear to be corrupted, even though I've always checked the rsync logs. 🫢

I'll need to scan the entire library now to see how widespread the damage is. I first noticed something was wrong when a few tracks started skipping mid-song while playing via #mpd / #mpc.

I guess nothing lasts forever—not even music files. 😞

I have a large local #musiclibrary—mostly #flac with some #m4a. I sync it between my #nas and four computers using #rsync.
Unfortunately, I recently discovered that some files appear to be corrupted, even though I've always checked the rsync logs. 🫢

I'll need to scan the entire library now to see how widespread the damage is. I first noticed something was wrong when a few tracks started skipping mid-song while playing via #mpd / #mpc.

I guess nothing lasts forever—not even music files. 😞

I have a large local #musiclibrary—mostly #flac with some #m4a. I sync it between my #nas and four computers using #rsync.
Unfortunately, I recently discovered that some files appear to be corrupted, even though I've always checked the rsync logs. 🫢

I'll need to scan the entire library now to see how widespread the damage is. I first noticed something was wrong when a few tracks started skipping mid-song while playing via #mpd / #mpc.

I guess nothing lasts forever—not even music files. 😞

I have a large local #musiclibrary—mostly #flac with some #m4a. I sync it between my #nas and four computers using #rsync.
Unfortunately, I recently discovered that some files appear to be corrupted, even though I've always checked the rsync logs. 🫢

I'll need to scan the entire library now to see how widespread the damage is. I first noticed something was wrong when a few tracks started skipping mid-song while playing via #mpd / #mpc.

I guess nothing lasts forever—not even music files. 😞

I have a large local #musiclibrary—mostly #flac with some #m4a. I sync it between my #nas and four computers using #rsync.
Unfortunately, I recently discovered that some files appear to be corrupted, even though I've always checked the rsync logs. 🫢

I'll need to scan the entire library now to see how widespread the damage is. I first noticed something was wrong when a few tracks started skipping mid-song while playing via #mpd / #mpc.

I guess nothing lasts forever—not even music files. 😞

NEW BLOG:
RSYNC Made Easy: Copy and Sync Files Without Losing Data

https://sajidmahmood.com/rsync-made-easy/
#blog #linux #rsync #datatransfer #tools

@[email protected] @rl_dane

This is a screen cap of Standard Notes on Android
Of course it also has on Open Source Operating System clients

#Web #links #sync #rsync #Vim #VimMasterRace #Standard #Notes #programming #technology

@rl_dane

If I need to synchronize between machines I use this program, Standard Notes, it's Open Source. For the rest I just open up vim.athena or vim.motif and put them in a .nfo file

https://f-droid.org/packages/com.standardnotes

#Web #links #sync #rsync #Vim #VimMasterRace #Standard #Notes #programming #technology

Migrating to kDrive from Flickr, Apple and Google Photo Clouds

As I write this my consolidated photo album is being uploaded to kDrive, to serve as an offsite backup but the journey to this point took about two weeks, due in part to experimentation and learning to use various tools.

Tools I used

• rsync
• Google Takeout
• Flickr Export
• jdupe
• Gemini
• Euria
• Le Chat, by Mistral

Work Flow

The first step is to request your data from Google Photos via the Google Takeout Tool, the Flickr Export tool for flickr, and to download all your photos locally from Apple Photos before disconnecting the local library from iCloud. Disconnecting Photos from iCloud gives you 30 days to realise you made a terrible mistake and fix it.

Export and organise

The next step is to unzip the Google Taekout files in one place, and the flickr export in another place. You want to keep the tree structure created by the zips for the next part.

Exiftool

Exiftools is a command line tool. Google Takeout and Flickr Export may detach metadata from your photos and add them to json files. Exiftools writes the exif data back into your photo files. If you ask Gemini or other AI solution for help it will provide you with the command you need to use. Request a dry run, and get the dry run to write to a text file to double check that it does what you expect.

Keep the zip files as they are. If you make a mistake it's good to have them on hand. Downloading 50 GB files from Google Takeout takes time.

With Flickr it's even more critical because Flickr generates 2gb files. I created a script to automatically download my 168 files.

Once you are happy that exiftool is behaving as expected you can run the command for real. Both of these steps take time so let them run in the background.

Google Takeout

Google takeout generates albums in three key ways, by individual names if you used face recognition, event name if you created an album, and by year, automatically. You will have two to three copies of some photos. In some directories you will only find json files.

When exiftool has run you can backup or delete the json files. If you have the zip files, then you're safe.

Flickr

When I expanded the Flickr zips it created a monoolithic directory with all the photos. I ran exiftools to marry json data with the photos.

Apple Photos

If you want to extract photos from Apple Photos quickly the quickest solution is to right click, show package contents, navigate to originals, and copy photos to another directory. You will need to use exiftool to create a directory where they are sorted by year, month day, and then you can run jdupe and add them to your main library.

Looking for Duplicates and Creating Chronological Libraries

With the data added by Exiftool we can now organise the photos chronologically. The issue is that we have event photos in albums, and the same event photos in the year folder. That's where jdupe comes in. It allows us to automatically compare photos within a directory before removing the duplicate copies.

Once this is done we can organise all the photos chronologically. This makes comparing photos much easier. It also adds a human accessible way of organising photos by year, month and day.

We repeat this step for Google Takeout and Flickr so that we end up with two clean chronological libraries.

The next step is to run jdupe again. This time we're comparing Flickr to Google Photos. The reason for this is that in an ideal world we have a perfect mirror, with both libraries being complete. In reality we might have interrupted payment to flickr, or Google photos so we have gaps. That's why we look for duplicates, before merging unique photos into our main photo library.

Tools such as rsync will help you merge the two libraries into the main library, as well as backup the clean library to a second hard drive on an external hard drive or on another device.

The kDrive migration

If you have not already done so, install the kDrive app and log in. Open the app and navigate to your library's folder and tell kdrive to sync the folder. It will then start copying the data to your cloud. Now you wait for it to be done.

Cleanup and Looking Forward

Once the main library is synced to kDrive I can delete two photos folders from kDrive and my local machine. I can tell kDrive on my phone to sync to the new library folder on kDrive.

That Synching Feeling

For now:

• Photosync adds photos to photoprism
• immich app adds photos to Immich
• kDrive app uploads to kDrive storage

Photoprism and Immich Watching

Both Photoprism and Immich allow you to watch an import folder(photoprism) or external library (immich). If you set the main library as a watch folder then new photos uploaded to kdrive will be added to the main library, and photoprism and Immich will add them to their own libraries. Unselect the "move" option to keep the chronological library intact.

And Finally

With jdupe, exiftool and rsync you can go from having three photo libraries wittled down to just one. You can then tell kdrive desktop to watch and sync that folder. You can use rsync to mirror the library to two or three other drives and filesystems. I have APFS, APFS (case sensitive) and ext4. I also have an offsite backup via kDrive.

Aaaw yies. Perfect integration of pv with rsync.

#rsync #pv #pipes

Back-In-Time (rsync) - heute ist Version 1.6 erschienen.
Größere Neuerung: Umstieg der Verschlüsselungsart. Lange Jahre EnfCS es - jetzt gocryptfs (alte verschlüsselte Archive wohl weiterhin lesbar).

Oberfläche unterstützt nun Qt6.

Stabilität soll verbessert sein.

#backup #rsync #backintime #verschlüsselung #crypted #linux

Haack's Networking - Informal Hacking Session #03

1) Today's agenda is rsnapshot script/stack migration from backup 8700 machine to 7920, i.e., to backup the backup.

2) Write the article for the bind9 authoritative DNS server.

3) If time and/or stamina permits, we will work on this month's timbre post where I cover my month's listening history.

Live Steam w/ Chat: https://content.haacksnetworking.org/w/byjvKm4LXLLn7q4ngYaG8f

#livestream #live #gnulinux #sysadmin #floss #freesoftware #rsync #rsnapshot #shellscripts

Hat jemand da draussen eine „Wunder-Lösung“ (ohne mühsame Bastelei) für Backups eines Debian 13 auf ein Synology NAS?

rsync und Timeshift konnten mich bisher nicht wirklich überzeugen.

Edit: Danke allen für die Tipps 😇💪🙏. Werde mich mal durcharbeiten 😅

#Timeshift #rsync #Synology #Debian13 #Debian #SSH #Backup #followerpower #pleaseboost #fedihelp #AskFedi

I generated a testset of roughly 2 million patterns and respective matches to test the glob matcher of #sydbox. The glob matching code was inherited from #rsync and was first written in 1986(!). In the process I have noticed 5 divergences between the fnmatch(3) implementations of glibc vs. musl and reported it to musl devs. Here are the divergences: https://dpaste.com/44XJS89XM.txt curious to hear inputs from #regex wizards as to whether #glibc or #musl is correct! TYVMIA! #linux #libc

My #RaspberryPi 4B is acting up and I think it's data corruption(?) from not powering off properly when it stopped responding. I really need to buy a switch to shutdown safely...

I formatted the SSD (SMART status seems ok), flashed the latest #DietPi and slowly setting back my services.

This is a great chance for me to actually sort out the messy initial setup of experimenting with #SelfHosting I'm in the process of setting up a #cron job with #rsync, #syncthing & dietpi-backup now. 🤞#Rpi4

Every day I backup my home folder to my external usb drive using #rsync. It only takes a couple minutes. Once a week or so I make a system snapshot using #Timeshift.

#Linux #LinuxMint #LMDE

🏠 Homelab Backup Evolution! 🏠

Following the "3-2-1 is the minimum" rule, I've expanded my VPS container backup strategy:

✅ Hetzner Cloud (Borg) → Offsite long-term storage
✅ Synology NAS (rsync) → Local fast recovery

The new setup does nightly automated syncs of all /opt/containers/ data to my Synology - with deduplication and all the bells and whistles! 📦

Particularly clever: hardlinks for space-efficient snapshots and morning email reports. Now I know right with my coffee ☕ whether all backups ran cleanly.

Lesson learned: Cloud-only is good, but having a local NAS mirror for quick restores is pure gold! 💪

How do you solve this in your setups? Also multi-tier or everything to cloud?

#Homelab #Backup #SelfHosting #Synology #VPS #DataSafety #321Rule #TechLife #NAS #CloudBackup #rsync #BorgBackup #InfrastructureAsCode

Как я делаю бекапы домашней системы Linux: простой пример инкрементального rsync + btrfs с zstd сжатием

Статья покажет простой rsync скрипт для инкрементального бекапа (с использованием хардлинков из предыдущего бекапа) и про использование btrfs сжатия в zstd.

https://habr.com/ru/articles/929182/

#rsync #btrfs #zstd #backup

Ich brauche nochmal #Brainpower der #EDV #Bubble 🤣

Da mein bisheriges, betagtes, leistungsschwaches 1-Bay #QNAP #NAS sehr langsam ist, z.B. beim #rsync, habe ich inzwischen das #Ugreen #NAS #DPX2800 gekauft, mit 32 GB #RAM aufgerüstet und 2 #NVNE mit je 2 TB eingebaut.

Azf den beiden M.2 ist #Proxmox im #ZFS #RAID1 installiert. #Homeassistant und #TrueNAS laufen je in einer #VM.

TrueNAS soll nun einen #Datenpool in Form eines verschlüsselten ZFS RAID 1 auf 2 16 TB Festplatten bekommen. So weit so gut.

Nur: Wie bekomme ich die #Daten vom alten NAS, die in einer verschlüsselten #LUKS #Partition liegen, in den Datenpool?

Über #Netzwerk kopieren ist keine wirkliche Option. Das ist für ~ 10 TB Nutzdaten viel zu langsam.

Bitte #Boost für mehr Reichweite. 🙏

Visto en el canal de Jonatan Castro https://www.youtube.com/watch?v=9TCOslIxh2E una web que ofrece 100 GB gratis de almacenamiento para @proxmox y compatible con #rsync, #borg y #sftp.

En principio es un proyecto personal de https://bennetgallein.de/ y, según leo en su blog, es un nuevo producto que acaba de poner en producción.

En mi @proxmox lo tengo todo lo más minimizado posible y con 100 GB puedo guardar lo más crítico de forma completa, comprimida y encriptada.

Como prueba de concepto de cómo funciona todo el proceso, está muy bien.

Lógicamente, al ser un proyecto tan personal, no se sabe si durará mucho, así que conviene tener más respaldos. Pero está bien tener esta referencia.

Y como siempre, recuerdo que todo se debe subir a cualquier sitio encriptado.

https://remote-backups.com/

#proxmox #backups #rbs #rsync #borg #sftp

@knasman
#foot - terminal emulator
#sfeed - rss feeds
#tofi - aplications menu
#libarchive - compression formats
#rsync - incremental backups
#mpv - video player
#toot - mastodon client
#fnf fuzzy finder
#rtorrent - torrent client
#aria2c - downloader
#qemu -- virtualization

Immerhin .. Backup Server neu aufgesetzt. #Bareos läuft wieder und auch noch das kaputte #Synology #HyperBackup, was per #rsync lief - und nun nicht mehr - auf S3 und #Minio auf dem neuen Backup Server umgestellt / eingerichtet.

Wie geht das mit dem Exclude bei Rsync?

Wer Rsync für die Datensicherung verwendet, möchte bestimmte Verzeichnisse ausschliessen. Doch wie macht man das?

#Rsync #Grsync #Backup #Exclude #Linux

https://gnulinux.ch/wie-geht-das-mit-dem-exclude-bei-rsync

Шифрование для облака: разные подходы

Облачные сервисы по-разному подходят к шифрованию данных на своём хостинге. В некоторых случаях это шифрование не удовлетворяет требованиям безопасности, поэтому приходится брать задачу в свои руки и шифровать файлы самостоятельно.

https://habr.com/ru/companies/ruvds/articles/846478/

#бэкапы #шифрование #Encryption_SDK #Database_Encryption_SDK #Secrets_Manager #Clean_Rooms #C3R #CloudHSM #KMS #HSM #DEK #KEK #multikeyring #restic #rsync #gocryptfs #cppcryptfs #DroidFS #gocryptfsinspect #Cryptomator #ruvds_статьи

Шифрование для облака: разные подходы

Облачные сервисы по-разному подходят к шифрованию данных на своём хостинге. В некоторых случаях это шифрование не удовлетворяет требованиям безопасности, поэтому приходится брать задачу в свои руки и шифровать файлы самостоятельно.

https://habr.com/ru/companies/ruvds/articles/846478/

#бэкапы #шифрование #Encryption_SDK #Database_Encryption_SDK #Secrets_Manager #Clean_Rooms #C3R #CloudHSM #KMS #HSM #DEK #KEK #multikeyring #restic #rsync #gocryptfs #cppcryptfs #DroidFS #gocryptfsinspect #Cryptomator #ruvds_статьи

Шифрование для облака: разные подходы

Облачные сервисы по-разному подходят к шифрованию данных на своём хостинге. В некоторых случаях это шифрование не удовлетворяет требованиям безопасности, поэтому приходится брать задачу в свои руки и шифровать файлы самостоятельно.

https://habr.com/ru/companies/ruvds/articles/846478/

#бэкапы #шифрование #Encryption_SDK #Database_Encryption_SDK #Secrets_Manager #Clean_Rooms #C3R #CloudHSM #KMS #HSM #DEK #KEK #multikeyring #restic #rsync #gocryptfs #cppcryptfs #DroidFS #gocryptfsinspect #Cryptomator #ruvds_статьи