#zfsbootmenu — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #zfsbootmenu, aggregated by home.social.
-
@dwarmstrong I love #zfsbootmenu but haven't quite got the hang of using boot environments with it. I find parts of the documentation a little opaque in places and I've been using #ZFS since it was #Solaris-only...
-
@dwarmstrong I love #zfsbootmenu but haven't quite got the hang of using boot environments with it. I find parts of the documentation a little opaque in places and I've been using #ZFS since it was #Solaris-only...
-
@dwarmstrong I love #zfsbootmenu but haven't quite got the hang of using boot environments with it. I find parts of the documentation a little opaque in places and I've been using #ZFS since it was #Solaris-only...
-
Today I used ZFSBootMenu + zfs snapshot/rollback functions for the first time to restore a machine back to its original state at first boot of a Chimera Linux system (when I had immediately taken a snapshot).
Worked like a charm. Its probably old hat to the pros but I was very pleased!
Back to experimenting!
-
Today I used ZFSBootMenu + zfs snapshot/rollback functions for the first time to restore a machine back to its original state at first boot of a Chimera Linux system (when I had immediately taken a snapshot).
Worked like a charm. Its probably old hat to the pros but I was very pleased!
Back to experimenting!
-
Today I used ZFSBootMenu + zfs snapshot/rollback functions for the first time to restore a machine back to its original state at first boot of a Chimera Linux system (when I had immediately taken a snapshot).
Worked like a charm. Its probably old hat to the pros but I was very pleased!
Back to experimenting!
-
Today I used ZFSBootMenu + zfs snapshot/rollback functions for the first time to restore a machine back to its original state at first boot of a Chimera Linux system (when I had immediately taken a snapshot).
Worked like a charm. Its probably old hat to the pros but I was very pleased!
Back to experimenting!
-
Today I used ZFSBootMenu + zfs snapshot/rollback functions for the first time to restore a machine back to its original state at first boot of a Chimera Linux system (when I had immediately taken a snapshot).
Worked like a charm. Its probably old hat to the pros but I was very pleased!
Back to experimenting!
-
Introducing USC <CR> editions 🤘
#zfs #incus #zfsbootmenu #debian -
Thanks everyone for voting and the super helpful advice!
This is my current game plan for Saturday:
- Debian root on ZFS with xfce
- She'll be a regular user with no privileges, but can install Flatpaks via the Warehouse app (https://flathub.org/en/apps/io.github.flattool.Warehouse )
Some open points:
- I'm not sure whether I'll use grub (using the openzfs guide https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Trixie%20Root%20on%20ZFS.html ) or zfsbootmenu (https://docs.zfsbootmenu.org/en/v3.1.x/guides/debian/uefi.html )
- Related to ZFS DKMS: to unattended upgrades or not? I don't know how high the risk is of breaking the system via a botched kernel upgrade.
- On another SSD, she'll keep Windows. I'm a bit hesitant if I should synchronise files two ways somehow between windows<->zfs, or just provide the windows share in Linux (so she can just copy files she'd need incidentally in Windows)
As much as I'd like to experiment with NixOS or e.g. Chimera Linux, it's a bit too risky for this project, since I have no experience with those yet.
Anyway, just for the fun of it, here's a follow up poll re: my open questions. Voting/boosting/commenting with feedback is highly appreciated!
#Debian #Xfce #ZFS #ZFSbootMenu #poll -
Updated post!
Chimera Linux is a delightful community-driven distribution built from scratch that does things differently: `musl` instead of the typical `glibc` for C library, `dinit` over `systemd` for system init, and a userland derived from FreeBSD core tools.
I show the choices I make to create an encrypted, minimal system with "just enough" to provide a solid foundation to build upon further:
-
Chimera Linux does things differently: the distro is built from scratch, musl instead of glibc, init is dinit, FreeBSD userland, ...
Fresh install on my Thinkpad using OpenZFS and ZFSBootMenu. Curious to explore further. I'm thinking of a setup where Chimera is my desktop and FreeBSD on a home server.
-
Chimera Linux does things differently: the distro is built from scratch, musl instead of glibc, init is dinit, FreeBSD userland, ...
Fresh install on my Thinkpad using OpenZFS and ZFSBootMenu. Curious to explore further. I'm thinking of a setup where Chimera is my desktop and FreeBSD on a home server.
-
Chimera Linux does things differently: the distro is built from scratch, musl instead of glibc, init is dinit, FreeBSD userland, ...
Fresh install on my Thinkpad using OpenZFS and ZFSBootMenu. Curious to explore further. I'm thinking of a setup where Chimera is my desktop and FreeBSD on a home server.
-
Chimera Linux does things differently: the distro is built from scratch, musl instead of glibc, init is dinit, FreeBSD userland, ...
Fresh install on my Thinkpad using OpenZFS and ZFSBootMenu. Curious to explore further. I'm thinking of a setup where Chimera is my desktop and FreeBSD on a home server.
-
Chimera Linux does things differently: the distro is built from scratch, musl instead of glibc, init is dinit, FreeBSD userland, ...
Fresh install on my Thinkpad using OpenZFS and ZFSBootMenu. Curious to explore further. I'm thinking of a setup where Chimera is my desktop and FreeBSD on a home server.
-
So after a week with #zfs and #zfsbootmenu on my #alpinelinux install I must say both are real gems.
Example: I made a backup of my datasets on USB SSD. You just change few zfs properties and you have not only a backup but also full, working copy of your entire system in the drawer. In case something goes wrong you can boot from it and have a great rescue system with all tools you can imagine. If a tool you need is not there you just do "apk add ...".
-
Yesterday I migrated my #alpinelinux install to #zfs and #zfsbootmenu. It was not too difficult even though I've done it from partition to partition on one drive. Although I wouldn't recommend it to a non techy person.
I even managed to compile zfs modules for my custom LLVM compiled kernels.So not a big deal actually.
Looking at you @vermaden :ablobrollingeyes: -
@joel I've not used it with Slackware but I can highly recommend #ZFSBootMenu - https://zfsbootmenu.org/ They don't have an installation guide but I'm sure it can be done...
-
I think Chimera Linux is an intriguing mix of components: Linux kernel, FreeBSD userland, apk package manager, (non-systemd) dinit. I checked it out briefly last year. Now I want to take a second, closer look.
Installed Chimera Linux with Root-on-ZFS with ZFS native encryption and ZFSBootMenu as bootloader on a VM. Time to explore!
-
@mwl oh I'm the worst person to ask but here you go
personally: Debian LTS, with #ZFSBootMenu. I build my own #OpenZFS (dogfooding) but I'd have no issues running what Debian ships in the contrib repo. Setting up ZFSBootMenu is a little fiddly, but the docs are good and you only do it once
now all the "it depends", as all Linux-related answers must be. mostly, because I don't really think of OpenZFS as being a reason to choose Linux; if OpenZFS is the most important thing, then go for FreeBSD. so if you have to have Linux+OpenZFS, the choice depends on what you're doing
at minimum, something running an LTS kernel. anything that picks up a new kernel on day of release will have a bad time, because we usually don't have support for it in a stable OpenZFS release at that moment
if you don't need OpenZFS as the root/boot filesystem, then either a Redhat-ish or Debian-ish LTS. The RHEL-derivatives (Alma, Rocky) have probably the best support from upstream OpenZFS, since the big users are in education and research and those are almost exclusively RHEL. they don't ship it in their repos though; we provide package repositories for those. on the other side, Debian do ship it in their contrib repo, and their packagers work with us on the finer details, so its pretty solid
(in previous life as systems guy for a cloud service, it was Debian and locally-built OpenZFS, and it was just great)
if you do want OpenZFS as the root/boot, then I highly recommend ZFSBootMenu. it is basically an entire Linux+OpenZFS mini-distro in a UEFI binary as a bootloader, and brings things like FreeBSD-style boot environments to Linux. its is very very well thought out and understands the complications involved
I definitely _don't_ recommend GRUB for OpenZFS on root/boot, because it mostly doesn't keep up with new OpenZFS features, so you end up needing to keep a separate pool with a bunch of features disables, and you then need to take care to not `zpool upgrade` it (which, tbf, is a bit of a footgun in OpenZFS' tooling). it also doesn't really bring much to the table; iirc it doesn't understand snapshots/clones, and if you're not doing things like snapshot-before-upgrade, I'm not sure that OpenZFS root/boot is even worth the trouble
if you want OpenZFS in the installer, then options are far more limited. for general purpose Linux, I know Void Linux and CachyOS have installation options for OpenZFS. maybe Ubuntu too, though I'm never quite sure what the truth is on any given day. Void is by the same people that do ZBM, so that's very nicely integrated.
for "applicance" type things, Proxmox is popular, though I'd be more inclined to look towards TrueNAS CE because I know the OpenZFS devs there :) HexOS is an interesting-looking up-and-comer too. though really for an appliance-style usage, I'm far more likely to look towards a FreeBSD-based option (Sylve is looking _very_ interesting).
(disclosure on last para: I have in the past done paid dev work on OpenZFS for both TrueNAS and HexOS).
-
And yes, I dogfood #OpenZFS pre-releases on my daily driver. Though I won't upgrade on-disk formats until #ZFSBootMenu is updated to understand those features, because I still like to boot my computer sometimes!
("lucy" is the local host and pool name; I build a custom OpenZFS on each machine, with a couple of tiny uninteresting patches to smooth a couple of rough edges around build and system integration).
-
My ZFS snapshot and replication setup on Ubuntu ft. sanoid and syncoid
I have known about ZFS since 2009, when I was working for Sun Microsystems as a campus ambassador at my college. But it wasn’t until I started hearing Jim Salter (on the TechSNAP and 2.5 Admins podcasts) and Allan Jude (on the 2.5 Admins podcast) evangelize ZFS that I became interested in using it on my computers and servers. With Ubuntu shipping ZFS in the kernel for many years now, I had access to native ZFS!,
Here is an overview of my setup running Ubuntu + ZFS before I explain and document some of the details.
cube– A headless server running Ubuntu 24.04 LTS (at the time of writing) with ZFS on root and a lot of ZFS storage powered by mirrorvdevs. Hassanoidfor automatic snapshots.- Desktops and laptops in my home run (K)Ubuntu (24.04 or later; versions vary) with encrypted (ZFS native encryption) ZFS on root and ZFSBootMenu. These computers also use
sanoidfor automatic snapshots.
Sanoid configuration
On my personal computers, I use a minimal
sanoidconfiguration that looks like############# datasets #############[zroot] use_template = production recursive = zfs############## templates ##############[template_production] frequently = 0 hourly = 26 daily = 30 monthly = 3 yearly = 0 autosnap = yes autoprune = yes[template_ignore] autoprune = no autosnap = no monitor = noOn servers, the
sanoidconfiguration has some additional tweaks, like the following template to not snapshot replicated datasets.[template_backup] frequently = 0 hourly = 36 daily = 30 monthly = 3 yearly = 0 # don't take new snapshots - snapshots # on backup datasets are replicated in # from source, not generated locally autosnap = noPre-apt snapshots
While
sanoidprovides periodic ZFS snapshots, I also wanted to wrapapttransactions in ZFS snapshots for the ability to roll back any bad updates/upgrades. For this, I used the following shell script,#!/usr/bin/env bashDATE="$(/bin/date +%F-%T)"zfs snapshot -r zroot@snap_pre_apt_"$DATE"with the following
apthook in/etc/apt/apt.conf.d/90zfs-pre-apt-snapshot.// Takes a snapshot of the system before package changes.DPkg::Pre-Invoke {"[ -x /usr/local/sbin/zfs-pre-apt-snapshot ] && /usr/local/sbin/zfs-pre-apt-snapshot || true";};This handles taking snapshots before
apttransactions but doesn’t prune the snapshots at all. For that, I used thezfs-prune-snapshotsscript (from https://github.com/bahamas10/zfs-prune-snapshots) in a wrapper cron shell (schedule varies per computer) script that looks like#!/bin/sh/usr/local/sbin/zfs-prune-snapshots \ -p 'snap_pre_apt_' \ 1w 2>&1 | logger \ -t cleanup-zfs-pre-apt-snapshotsSnapshot replication
The
cubeserver has sufficient disk space to provide a replication target for all my other personal computers using ZFS. It has a pool nameddpool, which will be referenced in the details to follow.For automating snapshot replication, I chose to use
syncoidfrom the samesanoidpackage. To avoid giving privileged access to the sending and the receiving user accounts, my setup closely follows the path in https://klarasystems.com/articles/improving-replication-security-with-openzfs-delegation/.On my personal computer, I granted my unprivileged (but has
sudo🤷♂️) local user account theholdandsendpermissions on the root dataset,zroot.sudo zfs allow send-user hold,send zrootzfs allow zroot---- Permissions on zroot --------------------------------------------Local+Descendent permissions: user send-user hold,sendOn the
cubeserver, I created an unprivileged user (nosudopermissions here 😌) and granted it thecreate,mount,receivepermissions temporarily on the parent of the target dataset,dpool.Then I performed an initial full replication of a local snapshot by running the following commands as the unprivileged user.
zfs send \ zroot@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostnamezfs send \ zroot/ROOT@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOTzfs send \ zroot/ROOT/os-name@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOT/os-namezfs send \ zroot/home@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/homeThe
-uflag in thezfs receivecommands above will prevent it from trying to mount the remote dataset. The target remote dataset must not exist when running this initial full replication.As it is not a good practice to allow unprivileged users to mount filesystems, I disabled automatic mounting by running
zfs set mountpoint=none dpool/local-hostnameas the
sudouser on the target server.Then I narrowed down the permissions of the receiving user to only its own dataset by running
zfs unallow remote-user \ create,mount,receive dpoolzfs allow remote-user \ create,mount,receive dpool/local-hostnameon the target server.
Next, I tried to test the snapshot replication by running
syncoidmanually likesyncoid -r \ --no-privilege-elevation \ --no-sync-snap \ zroot \ remote-user@cube:dpool/local-hostnameand it replicated all the other snapshots all on the local datasets (we had only replicated one snapshot previously).
The
sanoidpackage in Debian and Ubuntu does not ship with a systemd timer forsyncoid. So I created a user service and a timer that look like the following examples.# ~/.config/systemd/user/syncoid.service[Unit]Description=Replicate sanoid snapshots[Service]Type=oneshotExecStart=/usr/sbin/syncoid -r --no-privilege-elevation --no-sync-snap zroot remote-user@cube:dpool/local-hostname# ~/.config/systemd/user/syncoid.timer[Unit]Description=Run Syncoid to replicate ZFS snapshots to cube[Timer]OnCalendar=*:0/15Persistent=true[Install]WantedBy=timers.targetThen I reloaded systemd, enabled and started the above timer to have everything working smoothly.
-
My ZFS snapshot and replication setup on Ubuntu ft. sanoid and syncoid
I have known about ZFS since 2009, when I was working for Sun Microsystems as a campus ambassador at my college. But it wasn’t until I started hearing Jim Salter (on the TechSNAP and 2.5 Admins podcasts) and Allan Jude (on the 2.5 Admins podcast) evangelize ZFS that I became interested in using it on my computers and servers. With Ubuntu shipping ZFS in the kernel for many years now, I had access to native ZFS!,
Here is an overview of my setup running Ubuntu + ZFS before I explain and document some of the details.
cube– A headless server running Ubuntu 24.04 LTS (at the time of writing) with ZFS on root and a lot of ZFS storage powered by mirrorvdevs. Hassanoidfor automatic snapshots.- Desktops and laptops in my home run (K)Ubuntu (24.04 or later; versions vary) with encrypted (ZFS native encryption) ZFS on root and ZFSBootMenu. These computers also use
sanoidfor automatic snapshots.
Sanoid configuration
On my personal computers, I use a minimal
sanoidconfiguration that looks like############# datasets #############[zroot] use_template = production recursive = zfs############## templates ##############[template_production] frequently = 0 hourly = 26 daily = 30 monthly = 3 yearly = 0 autosnap = yes autoprune = yes[template_ignore] autoprune = no autosnap = no monitor = noOn servers, the
sanoidconfiguration has some additional tweaks, like the following template to not snapshot replicated datasets.[template_backup] frequently = 0 hourly = 36 daily = 30 monthly = 3 yearly = 0 # don't take new snapshots - snapshots # on backup datasets are replicated in # from source, not generated locally autosnap = noPre-apt snapshots
While
sanoidprovides periodic ZFS snapshots, I also wanted to wrapapttransactions in ZFS snapshots for the ability to roll back any bad updates/upgrades. For this, I used the following shell script,#!/usr/bin/env bashDATE="$(/bin/date +%F-%T)"zfs snapshot -r zroot@snap_pre_apt_"$DATE"with the following
apthook in/etc/apt/apt.conf.d/90zfs-pre-apt-snapshot.// Takes a snapshot of the system before package changes.DPkg::Pre-Invoke {"[ -x /usr/local/sbin/zfs-pre-apt-snapshot ] && /usr/local/sbin/zfs-pre-apt-snapshot || true";};This handles taking snapshots before
apttransactions but doesn’t prune the snapshots at all. For that, I used thezfs-prune-snapshotsscript (from https://github.com/bahamas10/zfs-prune-snapshots) in a wrapper cron shell (schedule varies per computer) script that looks like#!/bin/sh/usr/local/sbin/zfs-prune-snapshots \ -p 'snap_pre_apt_' \ 1w 2>&1 | logger \ -t cleanup-zfs-pre-apt-snapshotsSnapshot replication
The
cubeserver has sufficient disk space to provide a replication target for all my other personal computers using ZFS. It has a pool nameddpool, which will be referenced in the details to follow.For automating snapshot replication, I chose to use
syncoidfrom the samesanoidpackage. To avoid giving privileged access to the sending and the receiving user accounts, my setup closely follows the path in https://klarasystems.com/articles/improving-replication-security-with-openzfs-delegation/.On my personal computer, I granted my unprivileged (but has
sudo🤷♂️) local user account theholdandsendpermissions on the root dataset,zroot.sudo zfs allow send-user hold,send zrootzfs allow zroot---- Permissions on zroot --------------------------------------------Local+Descendent permissions: user send-user hold,sendOn the
cubeserver, I created an unprivileged user (nosudopermissions here 😌) and granted it thecreate,mount,receivepermissions temporarily on the parent of the target dataset,dpool.Then I performed an initial full replication of a local snapshot by running the following commands as the unprivileged user.
zfs send \ zroot@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostnamezfs send \ zroot/ROOT@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOTzfs send \ zroot/ROOT/os-name@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOT/os-namezfs send \ zroot/home@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/homeThe
-uflag in thezfs receivecommands above will prevent it from trying to mount the remote dataset. The target remote dataset must not exist when running this initial full replication.As it is not a good practice to allow unprivileged users to mount filesystems, I disabled automatic mounting by running
zfs set mountpoint=none dpool/local-hostnameas the
sudouser on the target server.Then I narrowed down the permissions of the receiving user to only its own dataset by running
zfs unallow remote-user \ create,mount,receive dpoolzfs allow remote-user \ create,mount,receive dpool/local-hostnameon the target server.
Next, I tried to test the snapshot replication by running
syncoidmanually likesyncoid -r \ --no-privilege-elevation \ --no-sync-snap \ zroot \ remote-user@cube:dpool/local-hostnameand it replicated all the other snapshots all on the local datasets (we had only replicated one snapshot previously).
The
sanoidpackage in Debian and Ubuntu does not ship with a systemd timer forsyncoid. So I created a user service and a timer that look like the following examples.# ~/.config/systemd/user/syncoid.service[Unit]Description=Replicate sanoid snapshots[Service]Type=oneshotExecStart=/usr/sbin/syncoid -r --no-privilege-elevation --no-sync-snap zroot remote-user@cube:dpool/local-hostname# ~/.config/systemd/user/syncoid.timer[Unit]Description=Run Syncoid to replicate ZFS snapshots to cube[Timer]OnCalendar=*:0/15Persistent=true[Install]WantedBy=timers.targetThen I reloaded systemd, enabled and started the above timer to have everything working smoothly.
-
My ZFS snapshot and replication setup on Ubuntu ft. sanoid and syncoid
I have known about ZFS since 2009, when I was working for Sun Microsystems as a campus ambassador at my college. But it wasn’t until I started hearing Jim Salter (on the TechSNAP and 2.5 Admins podcasts) and Allan Jude (on the 2.5 Admins podcast) evangelize ZFS that I became interested in using it on my computers and servers. With Ubuntu shipping ZFS in the kernel for many years now, I had access to native ZFS!,
Here is an overview of my setup running Ubuntu + ZFS before I explain and document some of the details.
cube– A headless server running Ubuntu 24.04 LTS (at the time of writing) with ZFS on root and a lot of ZFS storage powered by mirrorvdevs. Hassanoidfor automatic snapshots.- Desktops and laptops in my home run (K)Ubuntu (24.04 or later; versions vary) with encrypted (ZFS native encryption) ZFS on root and ZFSBootMenu. These computers also use
sanoidfor automatic snapshots.
Sanoid configuration
On my personal computers, I use a minimal
sanoidconfiguration that looks like############# datasets #############[zroot] use_template = production recursive = zfs############## templates ##############[template_production] frequently = 0 hourly = 26 daily = 30 monthly = 3 yearly = 0 autosnap = yes autoprune = yes[template_ignore] autoprune = no autosnap = no monitor = noOn servers, the
sanoidconfiguration has some additional tweaks, like the following template to not snapshot replicated datasets.[template_backup] frequently = 0 hourly = 36 daily = 30 monthly = 3 yearly = 0 # don't take new snapshots - snapshots # on backup datasets are replicated in # from source, not generated locally autosnap = noPre-apt snapshots
While
sanoidprovides periodic ZFS snapshots, I also wanted to wrapapttransactions in ZFS snapshots for the ability to roll back any bad updates/upgrades. For this, I used the following shell script,#!/usr/bin/env bashDATE="$(/bin/date +%F-%T)"zfs snapshot -r zroot@snap_pre_apt_"$DATE"with the following
apthook in/etc/apt/apt.conf.d/90zfs-pre-apt-snapshot.// Takes a snapshot of the system before package changes.DPkg::Pre-Invoke {"[ -x /usr/local/sbin/zfs-pre-apt-snapshot ] && /usr/local/sbin/zfs-pre-apt-snapshot || true";};This handles taking snapshots before
apttransactions but doesn’t prune the snapshots at all. For that, I used thezfs-prune-snapshotsscript (from https://github.com/bahamas10/zfs-prune-snapshots) in a wrapper cron shell (schedule varies per computer) script that looks like#!/bin/sh/usr/local/sbin/zfs-prune-snapshots \ -p 'snap_pre_apt_' \ 1w 2>&1 | logger \ -t cleanup-zfs-pre-apt-snapshotsSnapshot replication
The
cubeserver has sufficient disk space to provide a replication target for all my other personal computers using ZFS. It has a pool nameddpool, which will be referenced in the details to follow.For automating snapshot replication, I chose to use
syncoidfrom the samesanoidpackage. To avoid giving privileged access to the sending and the receiving user accounts, my setup closely follows the path in https://klarasystems.com/articles/improving-replication-security-with-openzfs-delegation/.On my personal computer, I granted my unprivileged (but has
sudo🤷♂️) local user account theholdandsendpermissions on the root dataset,zroot.sudo zfs allow send-user hold,send zrootzfs allow zroot---- Permissions on zroot --------------------------------------------Local+Descendent permissions: user send-user hold,sendOn the
cubeserver, I created an unprivileged user (nosudopermissions here 😌) and granted it thecreate,mount,receivepermissions temporarily on the parent of the target dataset,dpool.Then I performed an initial full replication of a local snapshot by running the following commands as the unprivileged user.
zfs send \ zroot@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostnamezfs send \ zroot/ROOT@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOTzfs send \ zroot/ROOT/os-name@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOT/os-namezfs send \ zroot/home@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/homeThe
-uflag in thezfs receivecommands above will prevent it from trying to mount the remote dataset. The target remote dataset must not exist when running this initial full replication.As it is not a good practice to allow unprivileged users to mount filesystems, I disabled automatic mounting by running
zfs set mountpoint=none dpool/local-hostnameas the
sudouser on the target server.Then I narrowed down the permissions of the receiving user to only its own dataset by running
zfs unallow remote-user \ create,mount,receive dpoolzfs allow remote-user \ create,mount,receive dpool/local-hostnameon the target server.
Next, I tried to test the snapshot replication by running
syncoidmanually likesyncoid -r \ --no-privilege-elevation \ --no-sync-snap \ zroot \ remote-user@cube:dpool/local-hostnameand it replicated all the other snapshots all on the local datasets (we had only replicated one snapshot previously).
The
sanoidpackage in Debian and Ubuntu does not ship with a systemd timer forsyncoid. So I created a user service and a timer that look like the following examples.# ~/.config/systemd/user/syncoid.service[Unit]Description=Replicate sanoid snapshots[Service]Type=oneshotExecStart=/usr/sbin/syncoid -r --no-privilege-elevation --no-sync-snap zroot remote-user@cube:dpool/local-hostname# ~/.config/systemd/user/syncoid.timer[Unit]Description=Run Syncoid to replicate ZFS snapshots to cube[Timer]OnCalendar=*:0/15Persistent=true[Install]WantedBy=timers.targetThen I reloaded systemd, enabled and started the above timer to have everything working smoothly.
-
My ZFS snapshot and replication setup on Ubuntu ft. sanoid and syncoid
I have known about ZFS since 2009, when I was working for Sun Microsystems as a campus ambassador at my college. But it wasn’t until I started hearing Jim Salter (on the TechSNAP and 2.5 Admins podcasts) and Allan Jude (on the 2.5 Admins podcast) evangelize ZFS that I became interested in using it on my computers and servers. With Ubuntu shipping ZFS in the kernel for many years now, I had access to native ZFS!,
Here is an overview of my setup running Ubuntu + ZFS before I explain and document some of the details.
cube– A headless server running Ubuntu 24.04 LTS (at the time of writing) with ZFS on root and a lot of ZFS storage powered by mirrorvdevs. Hassanoidfor automatic snapshots.- Desktops and laptops in my home run (K)Ubuntu (24.04 or later; versions vary) with encrypted (ZFS native encryption) ZFS on root and ZFSBootMenu. These computers also use
sanoidfor automatic snapshots.
Sanoid configuration
On my personal computers, I use a minimal
sanoidconfiguration that looks like############# datasets #############[zroot] use_template = production recursive = zfs############## templates ##############[template_production] frequently = 0 hourly = 26 daily = 30 monthly = 3 yearly = 0 autosnap = yes autoprune = yes[template_ignore] autoprune = no autosnap = no monitor = noOn servers, the
sanoidconfiguration has some additional tweaks, like the following template to not snapshot replicated datasets.[template_backup] frequently = 0 hourly = 36 daily = 30 monthly = 3 yearly = 0 # don't take new snapshots - snapshots # on backup datasets are replicated in # from source, not generated locally autosnap = noPre-apt snapshots
While
sanoidprovides periodic ZFS snapshots, I also wanted to wrapapttransactions in ZFS snapshots for the ability to roll back any bad updates/upgrades. For this, I used the following shell script,#!/usr/bin/env bashDATE="$(/bin/date +%F-%T)"zfs snapshot -r zroot@snap_pre_apt_"$DATE"with the following
apthook in/etc/apt/apt.conf.d/90zfs-pre-apt-snapshot.// Takes a snapshot of the system before package changes.DPkg::Pre-Invoke {"[ -x /usr/local/sbin/zfs-pre-apt-snapshot ] && /usr/local/sbin/zfs-pre-apt-snapshot || true";};This handles taking snapshots before
apttransactions but doesn’t prune the snapshots at all. For that, I used thezfs-prune-snapshotsscript (from https://github.com/bahamas10/zfs-prune-snapshots) in a wrapper cron shell (schedule varies per computer) script that looks like#!/bin/sh/usr/local/sbin/zfs-prune-snapshots \ -p 'snap_pre_apt_' \ 1w 2>&1 | logger \ -t cleanup-zfs-pre-apt-snapshotsSnapshot replication
The
cubeserver has sufficient disk space to provide a replication target for all my other personal computers using ZFS. It has a pool nameddpool, which will be referenced in the details to follow.For automating snapshot replication, I chose to use
syncoidfrom the samesanoidpackage. To avoid giving privileged access to the sending and the receiving user accounts, my setup closely follows the path in https://klarasystems.com/articles/improving-replication-security-with-openzfs-delegation/.On my personal computer, I granted my unprivileged (but has
sudo🤷♂️) local user account theholdandsendpermissions on the root dataset,zroot.sudo zfs allow send-user hold,send zrootzfs allow zroot---- Permissions on zroot --------------------------------------------Local+Descendent permissions: user send-user hold,sendOn the
cubeserver, I created an unprivileged user (nosudopermissions here 😌) and granted it thecreate,mount,receivepermissions temporarily on the parent of the target dataset,dpool.Then I performed an initial full replication of a local snapshot by running the following commands as the unprivileged user.
zfs send \ zroot@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostnamezfs send \ zroot/ROOT@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOTzfs send \ zroot/ROOT/os-name@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOT/os-namezfs send \ zroot/home@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/homeThe
-uflag in thezfs receivecommands above will prevent it from trying to mount the remote dataset. The target remote dataset must not exist when running this initial full replication.As it is not a good practice to allow unprivileged users to mount filesystems, I disabled automatic mounting by running
zfs set mountpoint=none dpool/local-hostnameas the
sudouser on the target server.Then I narrowed down the permissions of the receiving user to only its own dataset by running
zfs unallow remote-user \ create,mount,receive dpoolzfs allow remote-user \ create,mount,receive dpool/local-hostnameon the target server.
Next, I tried to test the snapshot replication by running
syncoidmanually likesyncoid -r \ --no-privilege-elevation \ --no-sync-snap \ zroot \ remote-user@cube:dpool/local-hostnameand it replicated all the other snapshots all on the local datasets (we had only replicated one snapshot previously).
The
sanoidpackage in Debian and Ubuntu does not ship with a systemd timer forsyncoid. So I created a user service and a timer that look like the following examples.# ~/.config/systemd/user/syncoid.service[Unit]Description=Replicate sanoid snapshots[Service]Type=oneshotExecStart=/usr/sbin/syncoid -r --no-privilege-elevation --no-sync-snap zroot remote-user@cube:dpool/local-hostname# ~/.config/systemd/user/syncoid.timer[Unit]Description=Run Syncoid to replicate ZFS snapshots to cube[Timer]OnCalendar=*:0/15Persistent=true[Install]WantedBy=timers.targetThen I reloaded systemd, enabled and started the above timer to have everything working smoothly.
-
My ZFS snapshot and replication setup on Ubuntu ft. sanoid and syncoid
I have known about ZFS since 2009, when I was working for Sun Microsystems as a campus ambassador at my college. But it wasn’t until I started hearing Jim Salter (on the TechSNAP and 2.5 Admins podcasts) and Allan Jude (on the 2.5 Admins podcast) evangelize ZFS that I became interested in using it on my computers and servers. With Ubuntu shipping ZFS in the kernel for many years now, I had access to native ZFS!,
Here is an overview of my setup running Ubuntu + ZFS before I explain and document some of the details.
cube– A headless server running Ubuntu 24.04 LTS (at the time of writing) with ZFS on root and a lot of ZFS storage powered by mirrorvdevs. Hassanoidfor automatic snapshots.- Desktops and laptops in my home run (K)Ubuntu (24.04 or later; versions vary) with encrypted (ZFS native encryption) ZFS on root and ZFSBootMenu. These computers also use
sanoidfor automatic snapshots.
Sanoid configuration
On my personal computers, I use a minimal
sanoidconfiguration that looks like############# datasets #############[zroot] use_template = production recursive = zfs############## templates ##############[template_production] frequently = 0 hourly = 26 daily = 30 monthly = 3 yearly = 0 autosnap = yes autoprune = yes[template_ignore] autoprune = no autosnap = no monitor = noOn servers, the
sanoidconfiguration has some additional tweaks, like the following template to not snapshot replicated datasets.[template_backup] frequently = 0 hourly = 36 daily = 30 monthly = 3 yearly = 0 # don't take new snapshots - snapshots # on backup datasets are replicated in # from source, not generated locally autosnap = noPre-apt snapshots
While
sanoidprovides periodic ZFS snapshots, I also wanted to wrapapttransactions in ZFS snapshots for the ability to roll back any bad updates/upgrades. For this, I used the following shell script,#!/usr/bin/env bashDATE="$(/bin/date +%F-%T)"zfs snapshot -r zroot@snap_pre_apt_"$DATE"with the following
apthook in/etc/apt/apt.conf.d/90zfs-pre-apt-snapshot.// Takes a snapshot of the system before package changes.DPkg::Pre-Invoke {"[ -x /usr/local/sbin/zfs-pre-apt-snapshot ] && /usr/local/sbin/zfs-pre-apt-snapshot || true";};This handles taking snapshots before
apttransactions but doesn’t prune the snapshots at all. For that, I used thezfs-prune-snapshotsscript (from https://github.com/bahamas10/zfs-prune-snapshots) in a wrapper cron shell (schedule varies per computer) script that looks like#!/bin/sh/usr/local/sbin/zfs-prune-snapshots \ -p 'snap_pre_apt_' \ 1w 2>&1 | logger \ -t cleanup-zfs-pre-apt-snapshotsSnapshot replication
The
cubeserver has sufficient disk space to provide a replication target for all my other personal computers using ZFS. It has a pool nameddpool, which will be referenced in the details to follow.For automating snapshot replication, I chose to use
syncoidfrom the samesanoidpackage. To avoid giving privileged access to the sending and the receiving user accounts, my setup closely follows the path in https://klarasystems.com/articles/improving-replication-security-with-openzfs-delegation/.On my personal computer, I granted my unprivileged (but has
sudo🤷♂️) local user account theholdandsendpermissions on the root dataset,zroot.sudo zfs allow send-user hold,send zrootzfs allow zroot---- Permissions on zroot --------------------------------------------Local+Descendent permissions: user send-user hold,sendOn the
cubeserver, I created an unprivileged user (nosudopermissions here 😌) and granted it thecreate,mount,receivepermissions temporarily on the parent of the target dataset,dpool.Then I performed an initial full replication of a local snapshot by running the following commands as the unprivileged user.
zfs send \ zroot@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostnamezfs send \ zroot/ROOT@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOTzfs send \ zroot/ROOT/os-name@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/ROOT/os-namezfs send \ zroot/home@snapshot-name | ssh \ remote-user@cube \ zfs receive -u \ dpool/local-hostname/homeThe
-uflag in thezfs receivecommands above will prevent it from trying to mount the remote dataset. The target remote dataset must not exist when running this initial full replication.As it is not a good practice to allow unprivileged users to mount filesystems, I disabled automatic mounting by running
zfs set mountpoint=none dpool/local-hostnameas the
sudouser on the target server.Then I narrowed down the permissions of the receiving user to only its own dataset by running
zfs unallow remote-user \ create,mount,receive dpoolzfs allow remote-user \ create,mount,receive dpool/local-hostnameon the target server.
Next, I tried to test the snapshot replication by running
syncoidmanually likesyncoid -r \ --no-privilege-elevation \ --no-sync-snap \ zroot \ remote-user@cube:dpool/local-hostnameand it replicated all the other snapshots all on the local datasets (we had only replicated one snapshot previously).
The
sanoidpackage in Debian and Ubuntu does not ship with a systemd timer forsyncoid. So I created a user service and a timer that look like the following examples.# ~/.config/systemd/user/syncoid.service[Unit]Description=Replicate sanoid snapshots[Service]Type=oneshotExecStart=/usr/sbin/syncoid -r --no-privilege-elevation --no-sync-snap zroot remote-user@cube:dpool/local-hostname# ~/.config/systemd/user/syncoid.timer[Unit]Description=Run Syncoid to replicate ZFS snapshots to cube[Timer]OnCalendar=*:0/15Persistent=true[Install]WantedBy=timers.targetThen I reloaded systemd, enabled and started the above timer to have everything working smoothly.
-
I switched to Kubuntu with root-on-ZFS:
― https://www.reddit.com/r/freebsd/comments/1lr4ayi/switched/
FreeBSD is not entirely abandoned:
― https://www.reddit.com/r/freebsd/comments/1lr4ayi/comment/n1cnfy2/?context=1
Pinned (seeking guidance):
― https://mastodon.bsd.cafe/@grahamperrin/114640816892663471
#FreeBSD #Linux #KDE #Plasma #OpenZFS #ZFS #bemgr #zectl #ZFSBootMenu
-
@paul @linus @stefano @whynothugo I chose Ubuntu (for KDE Plasma) because the installer provides root-on-ZFS.
With that base, I have not yet figured out which of these will be the simplest way forward:
― bemgr
― zectl
― ZFSBootMenu.I see verbose guides, the verbosity creates a sense of complication.
I'd like the simplest possible guide to getting started, with any of the three options, where the boot environment layout/structure is predetermined by the installer for Ubuntu.
TIA
-
@ahoyboyhoy @andreasgoebel in addition to ZFSBootMenu …
I'm looking at zectl, <https://ramsdenj.com/posts/2020-03-18-zectl-zfs-boot-environment-manager-for-linux/>
― zectl ZFS Boot Environment Manager for Linux · John Ramsden
Back to Manjaro. Reading <https://github.com/calamares/calamares/issues/533#issuecomment-971746992> (2021) alongside <https://en.wikipedia.org/wiki/Calamares_(software)>, I wonder why ZFS on root is not an option.
<https://www.theregister.com/2024/08/01/linux_rollback_options/> @lproven mentions licencing …
― Linux updates with an undo function? Some distros have that • The Register
Maybe I'll never need to undo :-)
-
For a switch away from everyday use of FreeBSD 15.0-CURRENT with KDE Plasma and ZFS boot environments:
― which of these two Linux distros might be best for me?
Emphases: easy installation, non-complicated upgrades, GUI.
Root on ZFS, and ZFS boot environments, will be nice, but not essential. If not easily achievable now, I can switch more than once:
– first to Linux, then from one distro to another.
Whatever Linux distros I use: I'll continue with Plasma.
I began toying with SparkyLinux in February 2024. Prior experience with Manjaro was excellent.
#zectl #ZFSBootMenu #OpenZFS #Linux #Manjaro #SparkyLinux #KDE #Plasma #FreeBSD
-
Edit:
While I still haven’t the slightest idea exactly was causing the issue, I did find a resolution to itThank you Arch Wiki
========
Okay my fellow #OpenZFS on #Linux peoples, I need helpI've attempted installs of #Debian Bookworm and Trixie in GNOME Boxes, and unlike my previous Boxes installations, this time around I'm getting this weird error (image attached) that is preventing it from booting the OS
#ZFS is the root file system, and #ZFSBootMenu is being used
I've done this exact installation method multiple times before successfully, both in VMs and real hardware, however for reasons unbeknownst to me they are now failing
Have not tried this on real hardware to see if it's just a VM thing (don't wanna grab a spare machine out at this time of night)
My "Google Fu" must be rusty, because I'm not really finding anything helpful in troubleshooting this
-
Even though I've managed to run #Ubuntu 24.04 desktop quite stable, it still doesn't inflict too much confidence. Hence my #Linux attempt no. 2: Fedora desktop 41. Which, amazingly, runs smooth and stable as it should. And since I find ZFS a nice fit, I'm trying to install it using #zfsbootmenu. Almost done except EFI boot. Tomorrow is an EFI day I guess. @lapor
-
Starting to get the itch to re-do my server, as this is the year Ubuntu 20.04 LTS loses support. Was thinking TrueNAS but wanted a little more freedom with lower ports. Now thinking about using #ZFSBootMenu on Alpine as a host for converting most everything to #docker :
docs.zfsbootmenu.org/en/v3.0.x…
Then make a few zpools for different benefits for my nextcloud docks and the other services I use...
Now not to get tempted with getting a refurb DL580 G9. 😅
-
@opensuse To hell with GRUB. It is time to move on from that mess.
#zfsbootmenu is a great start.
-
I use the Chimera Linux live install image to create an encrypted, console-only base configuration using the OpenZFS file system that can be customized further for various tasks.
This is how I do it...
-
Today I'm exploring Chimera Linux. It took a couple of attempts to setup with encrypted Root-on-ZFS... but I have it running now in a VM. Good stuff!
Next up: install to hardware (Thinkpad)
-
If you use #zfsbootmenu remember to update it or you won't be able to boot when you update your #zfs pool.
If you end up there put the binary efi recovery image from the projects website on a stick and boot from there - at least for me that worked like a charm ✨
Best experience I ever had with broken boot. Wow.
-
@thelastpsion #ZFS || GTFO. Arch works great with #zfsbootmenu as well.
https://arstechnica.com/gadgets/2021/09/examining-btrfs-linuxs-perpetually-half-finished-filesystem/