[ Tutorial ] Parrot on ZFS

I used the Parrot Project for years, so now i plan to give something back:

Guide: Install Parrot OS on ZFS

Requirements:

  • 1 USB-Stick (8GB)
  • Internet Connection
  • Burner (Etcher)

Steps:
1: Startup
2: Flash the USB
3: Install Parrot
4: Compile ZFS
5: Setup System
6: Finalize Setup

1: Startup:

BACKUP YOU PC!
If you plan to overwrite your OS, backup first.
“We’re off to a good start…”

Prepare the live USB:

Download your preferred parrot flavor. I chose XFCE for this guide.
Open your favorite browser and go to the Parrot OS download page: https://parrotsec.org/download/
If you can’t find a current xfce image, search here instead: https://download.parrot.sh/parrot/iso/

2: Flash the USB:

Download the Balena Etcher Burner, the best burner i know of: https://www.balena.io/etcher/
Follow its intuitive GUI and flash your USB.
“I’m Sec Parrot, and this is my favorite Burner on the Internet!”

Expert Option:
You can also compile Etcher from source, using the following script.
Instruction as to how to install it can be found later, during the zfs compilation.
The preferred folder of choice is “/usr/local/src/burner/etcher”

It will ask for root password, so READ IT FIRST!!!
Note: fails at the time of writing, likely due to upstream bug. should be resolved in a few days.

#!/bin/dash

sudo apt-get install -y git nodejs python3 jq curl npm libudev-dev || return;

sudo rm -r etcher;

git clone --recursive --recurse-submodules --single-branch --branch master https://github.com/balena-io/etcher.git || return;

cd etcher || return;

make electron-develop -j8 CFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" CPPFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" C++FLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" LDFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" || return;

npm audit fix || return;

make electron-build -j8 CFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" CPPFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" C++FLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" LDFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" || return;

if firejail --help > /dev/null; then sudo chown root:root /usr/local/src/application/etcher/etcher/node_modules/electron/dist/chrome-sandbox || return; fi || return;

if firejail --help > /dev/null; then sudo chmod 4755 /usr/local/src/application/etcher/etcher/node_modules/electron/dist/chrome-sandbox || return; fi || return;

touch etcher || return;

echo "#!""/bin/bash" > etcher || return;

echo "cd $(pwd)/" >> etcher || return;

echo "npm start" >> etcher || return;

sudo chmod uo+x etcher || return;

sudo cp etcher /usr/local/bin/etcher || return;

npm start || return;

3: Install Parrot

Once your USB has been flashed, plug it to the PC you want to change and restart the machine.
You will likely have to change the boot order, though startpage (google) has enough topics to cover this part.

Once your new USB-Menu appears, choose “Installer”.
Be aware that it boots into live mode if you are not quick enough, which can cost you a minute.
Proceed to setup your system as you like but pause at the partition menu. Enter the manual mode.

Here you have to do something unusual: Create a btrfs partition in the free space of your USB. Only create a root partition (no swap, home, or other).
If the installer mounted other partitions (preexisting efi partitions etc.) unselect them to avoid changes to your PC.
The installer will try its best to stop you, as the installation will be unusable. Ignore and proceed to install.
“Don’t you dare continue like this!”

The installer will fail at the grub setup (because we did not specify an efi partition). It will offer to go back or continue the installation,
which we have to choose (twice). this will open a hidden installation stage menu, including the option to complete the installation without boot-loader. Choose it.
The rest of the installation will proceed as normal. That is, it will appear to get stuck at the remove-live-packages step.
That’s normal, so do not worry (and take a nap).

Once your PC reboots, do not unplug your USB but enter the ram-mode to continue to step 4.

4: Compile ZFS

First a quick note: You have the option to either install zfs from packages or compile it from source.
Compiling from source has several benefits and should always be preferred (like zstd support).
“One does not simply choose zfs packages”

There are some bugs with the current image, so first apply the following fixes:
Go to Settings Manager -> Screensaver and disable the lock-screen (cannot use input field - forces restart)

Try to connect to the internet using LAN or WLAN.
Then open a terminal and execute

sudo parrot-upgrade; (a version of full-upgrade). This can take a while.

As it just happened to me: If you see the output ORIGIN DOWN… Try another day? The servers are down.
Run this command to see when they are reachable again (when it stops):

until sudo apt update; do sleep 1; done;

Then run the original command sudo parrot-upgrade;

Now create a directory to compile zfs:

sudo chown user:user /usr/local/src;
mkdir /usr/local/src/filesystem;
mkdir /usr/local/src/filesystem/zfs;

and create an auto-compile script:

touch /usr/local/src/filesystem/zfs/update.sh;
codium update.sh;

with the following content (Root, READ IT FIRST!!!)

#!/bin/dash
sudo apt-get install -y git build-essential autoconf automake libtool gawk alien fakeroot dkms libblkid-dev uuid-dev libudev-dev libssl-dev zlib1g-dev libaio-dev libattr1-dev libelf-dev linux-headers-$(uname -r) python3 python3-dev python3-setuptools python3-cffi libffi-dev || return;

sudo rm -r zfs;
git clone --recursive --recurse-submodules --single-branch --branch master https://github.com/openzfs/zfs.git || return;
cd zfs || return;

sh autogen.sh     CFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" CPPFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" C++FLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" LDFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" || return;

./configure --prefix=/ --libdir=/lib --includedir=/usr/include --datarootdir=/usr/share  --enable-systemd CFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" CPPFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" LDFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" || return;

make -j1 deb-utils deb-dkms CFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" CPPFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" C++FLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" LDFLAGS="-m64 -march=native -mtune=native -flto -fuse-linker-plugin -O3" || return;

rm -r ../.package.temp;
mv ../package ../.package.temp;

mkdir ../package || return; 
mv *.deb ../package/ || return;
cd ../package || return;

sudo dkms remove -m zfs -v $(dkms status | grep zfs | awk -F ", " '{print $2}') --all;
sudo apt-get reinstall -y ./*.deb --allow-downgrades --allow-change-held-packages || return;
sudo dkms uninstall -m zfs -v $(dkms status | grep zfs | awk -F ", " '{print $2}') --all || return;
sudo dkms install --force -m zfs -v $(dkms status | grep zfs | awk -F ", " '{print $2}') || return;

sudo apt-mark hold libzfs2 libzfs2-devel libzpool2 python3-pyzfs zfs zfs-dkms zfs-dracut zfs-initramfs zfs-test || return;
sudo update-initramfs -u;

rm -r ../.package.old;
mv ../.package.temp ../.package.old || return;

Note that it includes a bunch of C-CPP-C++ and LDFLAGS. I added them to speed zfs up, if only a little. They might not work on your system and can be safely removed.
In fact, i recommend removing them, as i really cannot guarantee that they will not cause the script to fail. They do work on my system though, so you might give it a try.

Also note that the script creates four different folders,
zfs - compilation folder, re-downloaded with each script execution,
packages - Including Latest successfully created deb packages,
.packages-temp - renamed packages folder, backup for unexpected script failure, and
.packages.old - Previous version of deb packages, if current ones fail.

One last issue before running the command:
ZFS searches for the Linux include directory in the wrong folder structure.
We have to manually link the correct folder to its search path, or the the installation fails.
You have to do this every time your kernel changes, or zfs dkms WILL fail!

sudo ln -s /usr/src/linux-headers-5.7.0-2parrot2-common/include/linux /lib/modules/5.7.0-2parrot2-amd64/build/include/linux;

Now enter the following commands to execute the script and activate your zfs installation

./update.sh;
sudo modprobe zfs;
zpool --version;

That’s it! zfs is now installed!
Lets continue with step 5.

5: Setup System

Lets start actually setting up your new System, shall we?

You will have to format your drive. USE OF DANGEROUS TOOLS!!!.
You have to manually create the partitions that the installer would normally add, that is:

EFI ~ 256 MiB - Boot Flag efi,
Boot ~ 5Gib,
OS - Cleared Filesystem,
Swap ~ Size of RAM

Use the preinstalled tool gparted to this. Again: BE CAREFUL!!!
“What could possibly go wrong?”

gparted

Now you need to know the ID of your OS partition. You likely know its dev value eg. /dev/sda3,
but we need its own ID. Find it with the command:

ls -l /dev/disk/by-id;

The following command creates your new Root FS.
DO NOT UNDER ANY CIRCUMSTANCES ADD “-f”!!!,
as you should not be asked for it if you followed the correct partitioning layout.
YOU MIGHT OVERWRITE THE WRONG PARTITION!!!

sudo zpool create -o ashift=12 -O recordsize=1M -O mountpoint=none -O checksum=edonr -O compression=zstd -O atime=off -O devices=off -O exec=off -O readonly=on -O canmount=off -O xattr=sa -O dedup=edonr,verify -O dnodesize=auto -O acltype=posixacl -O relatime=on RootZ /dev/vda3;

Now export and re-import the new Root under /mnt2 with the following commandos:

sudo zpool export RootZ && sudo zpool import -R /mnt2 RootZ;

The following commands create a default Root Filesystem structure to allow ease of management at best settings.
Add more sub-filesystem if needed. Do not encrypt the Root itself, only sub-filesystems for ease of management.

Note: There appears to be a bug where creating and mounting too fast can cause the mount to silently fail.
You can make sure that all is right by manually mounting one at a time (sudo zfs mount RootZ/Parrot/XFCE/NAME)

sudo zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase RootZ/Parrot;
sudo zfs create -o encryption=aes-256-gcm -o keylocation=prompt -o keyformat=passphrase RootZ/Data;
sudo zfs create RootZ/Parrot/XFCE;
sudo zfs create RootZ/Parrot/XFCE/User;
sudo zfs create -o mountpoint=/home/yin -o devices=on -o exec=on -o readonly=off -o canmount=on RootZ/Parrot/XFCE/User/Yin;
sudo zfs create -o mountpoint=/home/yin/.steam -o devices=on -o exec=on -o readonly=off -o canmount=on RootZ/Parrot/XFCE/User/Yin/Steam;
sudo zfs create -o mountpoint=/ -o devices=on -o exec=on -o readonly=off -o canmount=on RootZ/Parrot/XFCE/Root;
sudo zfs create -o mountpoint=/var/lib/libvirt/images -o compression=zstd-fast -o devices=on -o exec=on -o readonly=off -o canmount=on -o sync=standard RootZ/Parrot/XFCE/Images;
sudo zfs create -o mountpoint=/usr/local/src -o devices=on -o exec=on -o readonly=off -o canmount=on RootZ/Parrot/XFCE/Sources;
sudo zfs create -o mountpoint=/.install -o compression=zstd-19 -o readonly=off -o canmount=noauto  RootZ/Parrot/XFCE/Install;
sudo zfs mount RootZ/Parrot/XFCE/Install;
sudo zfs load-key -a;
sudo zfs mount -a;

You can also allow maximum compression at you data folder, but be warned -
MASSIVE CPU SPIKES DURING ANY COPY!!!:

sudo zfs set compression=zstd-19 RootZ/Data;

Now we will make sure to transfer all required files to your Root.
First mount the original installation - Don’t forget to change the dev!:

sudo mount /dev/sda4 /mnt;

Then transfer the unmodified OS to the Install folder for Reference -
CPU SPIKE DURING INITIAL TRANSFER!!!

sudo rsync -ahAHSX --sparse --no-i-r --info=progress2 /mnt/ /mnt2/.install/;
sudo zfs set readonly=on RootZ/Parrot/XFCE/Install;

Then transfer the OS again, this time into the correct folder structure:

sudo rsync -ahAHSX --sparse --no-i-r --info=progress2 /mnt/ /mnt2/;
sudo umount /mnt;

At last copy the zfs compile directory to your new OS:

sudo rsync -ahAHSX --sparse --no-i-r --info=progress2 /usr/local/src/filesystem /mnt2/usr/local/src/

Once all is transferred, lets start you OS.
Of course, it still wont boot. Your efi and boot partitions are still empty and it cannot understand its own filesystem.
To change this we will need to mount it and enter (login) using chroot, eg. running the OS from within the live-USB.

Thanks to the following link for the required commands:

sudo mount --bind /dev /mnt2/dev;
sudo mount --bind /dev/pts /mnt2/dev/pts;
sudo mount --bind /proc /mnt2/proc;
sudo mount --bind /sys /mnt2/sys;
sudo mount --bind /run /mnt2/run;
sudo chroot /mnt2;

This is you new OS! You still have to install grub.
First update all packages:

sudo parrot-upgrade

Then mount boot and efi and install grub:

sudo mount /dev/vda2 /boot;
mkdir /boot/efi;
sudo mount /dev/vda1 /boot/efi;
sudo grub-install;

Lastly install your kernel-images to the new boot:

sudo apt reinstall linux-headers-5.7.0-2parrot2-amd64 linux-image-5.7.0-2parrot2-amd64

With that done, lets also install zfs to your new OS - exactly as before:

sudo ln -s /usr/src/linux-headers-5.7.0-2parrot2-common/include/linux /lib/modules/5.7.0-2parrot2-amd64/build/include/linux;
cd /usr/local/src/filesystem/zfs;
sudo apt update;
./update.sh;

Everything is finished, so lets umount and return:

sudo umount /boot/efi;
sudo umount /boot;
exit;

Now try (often unsuccessfully) to umount everything - Don’t worry, this does not have to work:

sudo umount /mnt2/run;
sudo umount /mnt2/sys;
sudo umount /mnt2/proc;
sudo umount /mnt2/dev/pts;
sudo umount /mnt2/dev;
sudo zpool sync RootZ;
sudo zfs umount -a;
sudo zpool export RootZ;

That’s it for part 5, so lets finally get rid of our live-USB and shutdown the system!

sudo shutdown now;

6: Finalize Setup

Once you pass the grub menu, you will be thrown into the initramfs rescue shell. Don’t be discouraged!
“I have a bad feeling about this”

This is the result of another zfs issue, which has somehow persisted for more than 5 years.
We will fix it later. For now enter the following commands:

zpool import -R /root RootZ;
zfs load-key -a;
zfs mount -a;
exit;

This will mount Root as well as all sub-partitions and allow you to Boot into the filesystem.
Congratulations! You just succeeded in Booting for the first time.

Now lets fix some of the remaining issues:

Disable your second login screen, as mentioned above.

The 5 year bug:

In short: Log into your system and open the following file with vscodium:

codium /etc/grub.d/10_linux

Replace line 94:

rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`

with the line:

rpool=`zdb -l ${GRUB_DEVICE} | grep " name:" | cut -d\' -f2`

Then enter

sudo update-grub

This will allow grub to detect your Root filesystem and boot you into a nice looking Password Prompt.
Note that this causes another issue, one that i have no solution for:

Whenever you enter the Root Password and move to your home password prompt, you will get stuck.
Entering the password will cause the prompt to first dis-, than reappear.
This is caused by Grub not properly mounting all sub-filesystems, so your Home Directory is still not mounted.
The only way to log in is to press CTRL ALT F1,

enter your password to login through the terminal,

then enter the command:
sudo zfs mount -a;

enter your password again to execute it.

Now return to the GUI with CTRL ALT F1,
and enter your password for the third and final time.

I consider it an anti-theft measure, but you might find it annoying. If so, don’t change 10_linux.
You would then be thrown into the initramfs shell, but only enter your password once.
Let me know if you find a solution to this!

Lastly, enter your boot,efi and swap partition into the file /etc/fstab,
that is, change the UUID of the following lines using ls -l (seen before)

UUID=23b79a33-a286-4d53-814a-f533423b73f8  none           swap    sw                                                                    0       0
UUID=a595037a-9679-47a2-958d-bf3a09e84597  /boot          btrfs   noatime,nodiratime,relatime,autodefrag,compress-force=lzo,ssd_spread  0       0
UUID=9BA4-DB3A                             /boot/efi      vfat    umask=0077,noatime,nodiratime,relatime                                0       0

As the very last step of this guide, lets make a backup using the second best feature of ZFS - Snapshots!

sudo zfs snapshot -r RootZ/Parrot/XFCE@Install

This simple command creates a perfect backup of your entire system in less than a second.
You can now always return to the current system state, should you accidentaly damage something.
You can even use the best feature of ZFS to copy it to a different drive - Send/Receive!

To some other zfs partition?
sudo zfs send -R --large-block --replicate --embed --backup --compressed --raw --verbose RootZ/Parrot/XFCE@Install | sudo zfs receive Backup/Parrot;

Or as a file?
sudo zfs send -R --large-block --replicate --embed --backup --compressed --raw --verbose RootZ/Parrot/XFCE@Install > ~/Parrot.zfs

Thats it! You are fine to go and use your newly designed system!!!*
Finally!!!

PS: My next guide talks about running Parrot without Systemd!!!

2 Likes

Rather amusing: I cannot edit my own post because i used too many links :sweat_smile:

Here a quick addition that should be added before compiling zfs on the live-USB:

One last issue before running the command:
ZFS searches for the Linux include directory in the wrong folder structure.
We have to manually link the correct folder to its search path, or the the installation fails.
You have to do this every time your kernel changes, or zfs dkms WILL fail!

sudo ln -s /usr/src/linux-headers-5.7.0-2parrot2-common/include/linux /lib/modules/5.7.0-2parrot2-amd64/build/include/linux;

And the Part with the unmounted home dir and missing code end block should be expanded to:

The only way to log in is to press CTRL ALT F1,

enter your password to login through the terminal,

then enter the command:
sudo zfs mount -a;

enter your password again to execute it.

Now return to the GUI with CTRL ALT F1,
and enter your password for the third and final time.

Edit:
Changed it since my trust level got updated,
just had to wait a minute :rofl:

Hey,

Great post. :slight_smile:

(moved to Tutorials forum)

Quick note: The etcher script works again.
This allowed me to find an issue that i managed to miss, please replace the firejail lines with:

if firejail --help > /dev/null; then sudo chown root:root node_modules/electron/dist/chrome-sandbox || return; fi || return;
if firejail --help > /dev/null; then sudo chmod 4755 node_modules/electron/dist/chrome-sandbox || return; fi || return;

I also missed the two lines:

cd /usr/local/src/filesystem/zfs;
chmod u+x update.sh;

directly below

touch /usr/local/src/filesystem/zfs/update.sh;

I recently detected another issue with zfs,

whereas adding too many layers of zfs datasets can lead to folders not correctly mounting/unmounting.
This guide surpasses the maximum depth with both the user and steam datasets.

If you stumble upon this issue, simply create both datasets directly under XFCE,
or use the zfs rename command to change the location of existing datasets, e.g.:

sudo zfs rename RootZ/Parrot/XFCE/User/Yin/Steam RootZ/Parrot/XFCE/Steam;
sudo zfs rename RootZ/Parrot/XFCE/User/Yin RootZ/Parrot/XFCE/Yin;