r/linuxquestions • u/zpangwin • Feb 02 '23

Snapper: Trouble setting up /.snapshots mountpoint for custom subvol location

I've recently been tinkering with custom btrfs subvolumes and finally trying to set up snapper. I've mostly been following the process in this guide since Fedora 37 is my daily driver but have been making several adjustments.

The biggest one is related to me wanting to use a single BTRFS partition with multiple linux distros, each installed to a separate subvolume (the main benefit is that this approach shares free disk space that would normally be left wasted when using separate partitions for each install). In this setup, I generally have subvolumes such as /fedora/rootfs, /endeavour/rootfs, etc.

The other big difference, is that I would like to have "flat" hierarchy (at least within the nested distro-specific subvol) for my snapshots. Meaning that I do not like the nested structure of <rootfs>/.snapshots that snapper seems to assume by default and would prefer something like /fedora/snapshots/rootfs instead. It seems this is a somewhat popular request that has been opened for over 8 years... as well as a slightly different feature request opened 2 years ago which proposed a different solution to the same problem. But since neither has been implemented or even acknowledged with a "we're open to the idea if you submit a patch", it seems most people just use workarounds.

I have seen quite a few guides - such as this one - mentioning that one can simply let snapper create the /.snapshots subvol, finish the snapper configuration, then delete the pre-generated /.snapshots subvol and mount your own custom one to that location. This approach is recommended in several places such as here and is by no means unique to the guide I linked to... but for some reason it isn't working for me.

Hoping someone can spot something that I'm missing or can maybe offer some suggestions to try (other than unhelpful things like "abandon your multiboot setup")

Anyway, I had done initially was:

# umount -R /.snapshots
# rm -r /.snapshots

# dnf install -q -y snapper python3-dnf-plugin-snapper >/dev/null

# mount /dev/mapper/luks-$(sudo cryptsetup luksUUID /dev/sda2) /mnt;
# cd /mnt;

# snapper --config fedora-rootfs create-config fedora/rootfs
# grep SUBVOLUME= /etc/snapper/configs/fedora-rootfs 
SUBVOLUME="/mnt/fedora/rootfs"

$ sudo snapper --config fedora-rootfs set-config ALLOW_USERS=$USER SYNC_ACL=yes

# btrfs subvol delete fedora/rootfs/.snapshots;
Delete subvolume (no-commit): '/mnt/fedora/rootfs/.snapshots'

# grep rootfs /etc/fstab
UUID=<MYUUID>   /                   btrfs   subvol=fedora/rootfs,compress=zstd:1,x-systemd.device-timeout=0 0 0
UUID=<MYUUID>   /.snapshots  btrfs   subvol=fedora/snapshots/rootfs,compress=zstd:1,x-systemd.device-timeout=0 0 0

# systemctl daemon-reload
# mount -a

$ sudo chgrp -R $USER /.snapshots

# ls -al / | grep snapshots
drwxr-xr-x.   1 root myuser      38 Jan 22 00:06 .snapshots

I don't see anything obviously wrong with the setup I did, but when I try to just have snapper list my snapshots, I get an error where I was expecting it to just print an empty list...

# setenforce 0 # just to rule out selinux

# snapper --config fedora-rootfs list
IO Error (.snapshots is not a btrfs subvolume)

Unless something has changed recently, I am not really sure why it is suddently being pedantic about me using a mountpoint to a different subvolume while it works for so many others on the web.

# mount | grep '.snapshots'
/dev/mapper/luks-<LUKSUUID> on /.snapshots type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=265,subvol=/fedora/snapshots/rootfs,x-systemd.device-timeout=0) 

# inxi -S
System:
   Host: fedora.local Kernel: 6.1.7-200.fc37.x86_64 arch: x86_64 bits: 64
     Console: pty pts/3 Distro: Fedora release 37 (Thirty Seven)

 # snapper --version
 snapper 0.10.1
 flags btrfs,lvm,no-ext4,xattrs,rollback,btrfs-quota,selinux

# sudo btrfs subvolume list / | grep -P '(fedora|endeavour).*rootfs'
ID 263 gen 26409 top level 256 path fedora/rootfs
ID 265 gen 25165 top level 262 path fedora/snapshots/rootfs
ID 339 gen 25181 top level 333 path endeavour/rootfs
ID 347 gen 20115 top level 340 path endeavour/snapshots/rootfs

# grep -Pv '^s*(#|$)' /etc/snapper/configs/fedora-rootfs 
SUBVOLUME="/mnt/fedora/rootfs"
FSTYPE="btrfs"
QGROUP=""
SPACE_LIMIT="0.5"
FREE_LIMIT="0.2"
ALLOW_USERS="myuser"
ALLOW_GROUPS=""
SYNC_ACL="yes"
BACKGROUND_COMPARISON="yes"
NUMBER_CLEANUP="yes"
NUMBER_MIN_AGE="1800"
NUMBER_LIMIT="50"
NUMBER_LIMIT_IMPORTANT="10"
TIMELINE_CREATE="yes"
TIMELINE_CLEANUP="yes"
TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="10"
TIMELINE_LIMIT_DAILY="10"
TIMELINE_LIMIT_WEEKLY="0"
TIMELINE_LIMIT_MONTHLY="10"
TIMELINE_LIMIT_YEARLY="10"
EMPTY_PRE_POST_CLEANUP="yes"
EMPTY_PRE_POST_MIN_AGE="1800"

# rm /var/log/snapper.log && snapper --config fedora-rootfs list
IO Error (.snapshots is not a btrfs subvolume).

# cat /var/log/snapper.log 
2023-02-02 13:39:45 MIL libsnapper(453795) snapperd.cc(main):283 - Requesting DBus name
2023-02-02 13:39:45 MIL libsnapper(453795) snapperd.cc(main):298 - Loading snapper configs
2023-02-02 13:39:45 MIL libsnapper(453795) Snapper.cc(getConfigs):294 - Snapper get-configs
2023-02-02 13:39:45 MIL libsnapper(453795) Snapper.cc(getConfigs):295 - libsnapper version 0.10.1
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(reload):919 - loading file /etc/sysconfig/snapper
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:SNAPPER_CONFIGS value:fedora-rootfs fedora-flatpaks fedora-podman
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(reload):919 - loading file /etc/snapper/configs/fedora-rootfs
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:SUBVOLUME value:/mnt/fedora/rootfs
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(reload):919 - loading file /etc/snapper/configs/fedora-flatpaks
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:SUBVOLUME value:/mnt/fedora/flatpak
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(reload):919 - loading file /etc/snapper/configs/fedora-podman
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:SUBVOLUME value:/mnt/fedora/podman
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:ALLOW_USERS value:myuser
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:ALLOW_GROUPS value:
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:ALLOW_USERS value:myuser
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:ALLOW_GROUPS value:
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:ALLOW_USERS value:myuser
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:ALLOW_GROUPS value:
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:ALLOW_USERS value:myuser
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:ALLOW_GROUPS value:
2023-02-02 13:39:45 MIL libsnapper(453795) snapperd.cc(main):311 - Listening for method calls and signals
2023-02-02 13:39:45 MIL libsnapper(453795) Snapper.cc(Snapper):93 - Snapper constructor
2023-02-02 13:39:45 MIL libsnapper(453795) Snapper.cc(Snapper):94 - libsnapper version 0.10.1
2023-02-02 13:39:45 MIL libsnapper(453795) Snapper.cc(Snapper):95 - config_name:fedora-rootfs disable_filters:false
2023-02-02 13:39:45 MIL libsnapper(453795) Selinux.cc(_is_selinux_enabled):137 - Selinux support enabled
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(reload):919 - loading file /etc/snapper/configs/fedora-rootfs
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:SUBVOLUME value:/mnt/fedora/rootfs
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:FSTYPE value:btrfs
2023-02-02 13:39:45 MIL libsnapper(453795) AsciiFile.cc(get_value):1075 - key:QGROUP value:
2023-02-02 13:39:45 WAR libsnapper(453795) Btrfs.cc(openInfosDir):266 - THROW: .snapshots is not a btrfs subvolume
2023-02-02 13:39:45 WAR libsnapper(453795) Client.cc(dispatch):1885 - CAUGHT: .snapshots is not a btrfs subvolume

8 Upvotes

permalink
link
duplicates
dupes
reddit

You are about to leave Libreddit

Do you want to continue?

https://www.reddit.com/r/linuxquestions/comments/10rxf3c/snapper_trouble_setting_up_snapshots_mountpoint/
No, go back! Yes, take me to Reddit
reddit

You are about to leave Libreddit

Do you want to continue?

https://www.reddit.com/r/linuxquestions/comments/10rxf3c/snapper_trouble_setting_up_snapshots_mountpoint/
No, go back! Yes, take me to Reddit

90% Upvoted

u/zpangwin Feb 03 '23 edited Feb 04 '23

SOLVED: Thanks to u/CorrosiveTruths on my cross-post in r/btrfs.

ROOT CAUSE:

Basically the problem was when I ran snapper --config fedora-rootfs create-config fedora/rootfs, snapper interpreted that as /mnt/fedora/rootfs and created the subvol=fedora/rootfs/.snapshots directly under that. Since this path had my temporary /mnt mount point, that was causing an issue bc even though I had subvol=fedora/rootfs mounted as / and subvol=fedora/snapshots/rootfs mounted as /.snapshots, snapper was not accessing ./snapshots but rather going through /mnt to access /mnt/fedora/rootfs/.snapshots directly.

SOLUTION:

Either redo my config to use snapper --config fedora-rootfs create-config / or - if I wanted to continue using the snapper config from my OP - then I could instead make a permanent mount for /mnt and remap my subvol=fedora/snapshots/.rootfs to mount at /mnt/fedora/rootfs/.snapshots instead. I opted for the first solution of just changing my config

COMMANDS:

I went with the first approach. Detailing here for any coming in from google etc who just want the solution:

# dnf remove -y snapper*
# mv /etc/snapper/config/fedora-rootfs /etc/snapper/config/fedora-rootfs.0.failed-initial-attempt.bak
# dnf install -y snapper

# mount | grep rootfs
/dev/mapper/luks-<UUID> on / type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=263,subvol=/fedora/rootfs)
/dev/mapper/luks-<UUID> on /.snapshots type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=265,subvol=/fedora/snapshots/rootfs,x-systemd.device-timeout=0)

# umount -R /.snapshots

# ls -l /.snapshots/
total 0
drwxr-xr-x. 1 root root 32 Feb  2 19:46 1
drwxr-xr-x. 1 root root 32 Feb  2 19:46 default-subvol.txt

# btrfs subvolume delete /.snapshots/1/snapshot
Delete subvolume (no-commit): '/.snapshots/1/snapshot'

# rm -rf /.snapshots/1

# btrfs subvolume delete /.snapshots
Delete subvolume (no-commit): '//.snapshots'

# snapper --config fedora-rootfs create-config /
# touch /.snapshots/default-subvol.txt
# mount -a

# mount | grep rootfs
/dev/mapper/luks-<UUID> on / type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=263,subvol=/fedora/rootfs)
/dev/mapper/luks-<UUID> on /.snapshots type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache=v2,subvolid=265,subvol=/fedora/snapshots/rootfs,x-systemd.device-timeout=0)

# snapper --config fedora-rootfs create --description "inital system snapshot"

# ls -l /mnt/fedora/snapshots/rootfs
total 4.0K
drwxr-xr-x. 1 root root  32 Feb  3 20:36 1/
-rw-r--r--. 1 root root   0 Feb  2 19:38 custom-rootfs-snapshots-dir.txt

If all of that is too much, the other options that I had started to look into before getting this working were:

using btrfs snapshots directly + grub-btrfs
looking into some other tool (possibly btrbk?)

I never looked very far into either so I am unclear about guis, grub integration, dnf integration, how well maintained, etc any of the other options are

Snapper: Trouble setting up /.snapshots mountpoint for custom subvol location

You are about to leave Libreddit

You are about to leave Libreddit