Anthony J. Martinez

Adding GNU/Emacs Info to Debian-based Systems

For reasons explained here, the standard documentation of GNU/Emacs does not ship alongside the application itself in Debian-based systems without use of the non-free repositories. Those of us wishing to have the in-depth documentation that typically ships with GNU/Emacs will need to tell apt how to get it. The process is largely identical to that described in my previous post where I enabled bluetooth on my Librem 14. In fact, I only added one more Package stanza as follows:


Package: emacs-common-non-dfsg 
Pin: origin
Pin-Priority: 501

With this added all that was left was an apt update and an apt install emacs-common-non-dfsg.

My GNU/Emacs now contains the documentation it was supposed to have to begin with.

Enabling Bluetooth on the Librem 14

As the Librem 14 ships with PureOS it lacks the non-free firmware required for the onboard Bluetooth to function out of the box. Fortunately, this is a trivial problem to solve. Purism does not control how you configure your hardware, you do.

The Process

Note: the following assumes you are running PureOS 10

  1. Create /etc/apt/sources.list.d/bullseye-nonfree.list with the following contents:
# Debian Bullseye non-free for firmware-atheros
deb bullseye non-free
deb-src bullseye non-free
  1. Create /etc/apt/preferences.d/bullseye-nonfree with the following contents:
Package: *
Pin: origin
Pin-Priority: 1

Package: firmware-atheros
Pin: origin
Pin-Priority: 501
  1. Update apt:
sudo apt update
  1. Install the non-free drivers:
sudo apt install firmware-atheros
  1. Toggle the hardware kill-switch for the WiFi/Bluetooth, and enjoy.

Life Below Gig

For the last two years or so, I have lived in The Netherlands and enjoyed gigabit downstream at home. Unfortunate family circumstances have me back home in Texas, and the step backwards having lost some 80% of my downstream is remarkable. Almost more alarming is the huge lack of upstream (6mbps vs 40mbps).

It's clear that when I return I will have to pay whatever extortion Xfinity requires as they're the monopoly power in my zip code. It's also clear that A Rust Site Engine likely needs some new features like pagination to make it easier to consume this site if you don't have super fast internet.

Software Projects @

For quite some time I have wondered how I would manage support for user input, contributions, or support in my various libre projects. With the recent addition of Spaces support to both Synapse and Element, I have created a space full of rooms for each of my projects here.

Individual rooms are as follows:

As I update these project, I will use these rooms to make appropriate announcements. For now this will be a manual project, but later I may write a bot to manage these announcements for me.

Basic Tails Setup

The following mini-guide will take you down the path to a basic Tails install with one important extra feature: support for offline USB HSM use.


Getting Tails

Here you have two choices, which are well described here, but boil down to:

HSM Support

Once you have a base Tails install the rest is quite simple.

  1. Boot your new Tails USB
  2. Connect to Tor
  3. Hit Super and start typing "Configure persistent volume"
  4. Create your passphrase to encrypt the persistent storage volume
  5. Click the Create button
  6. When the feature list appears, enable "Additional Software"
  7. Reboot
  8. Unlock your persistent storage in the Welcome Screen
  9. Under "Additional Settings" on the Welcome Screen expand the options and choose "Administration Password"
  10. Connect to Tor
  11. Open a terminal and run sudo apt update && sudo apt --yes install opensc libengine-pkcs11-openssl
  12. Tails will update and ask if you want to persist this Additional Software. Tell it yes, you want the additional software available every time you unlock your Persistent Storage

At this point, if you reboot and unlock your persistent storage your Tails system will be able to use any USB HSM supported by OpenSC. Installation of software from the persistent storage does not require an administration password, and for added security it is probably best to avoid setting one unless your workflow requires administrative rights for some reason. After your software finishes installing from persistent storage you are ready to use your HSM directly with tools like:

Signing Example

# Here 20 is the key ID of a signing key on a Nitrokey HSM 2
amnesia@amnesia:~$ openssl dgst -engine pkcs11 -keyform e -sign 20 -out special.sig special.img
engine "pkcs11" set
Enter PKCS#11 token PIN for UserPIN (MY-MAGIC-KEY):

# And now to verify the resulting signature
amnesia@amnesia:~$ openssl dgst -engine pkcs11 -keyform e -verify 20 -signature special.sig special.img
engine "pkcs11" set
Verified OK

Librem Key in Tails 4.21

While Purism has upstreamed their changes to Nitrokey libraries, those changes haven't trickled down to the masses quite yet. If, like me, you happen to have a Librem Key and also keep a Tails stick handy this little tip should let you use the two together at least as far as gpg is concerned.

  1. Boot Tails and set an admin password
  2. Create /etc/udev/rules.d/40-libremkey.rules containing
ATTR{idVendor}=="316d", ATTR{idProduct}=="4c4b", ENV{ID_SMARTCARD_READER}="1", ENV{ID_SMARTCARD_READER_DRIVER}="gnupg", GROUP+="plugdev", TAG+="uaccess"
  1. Run sudo udevadm control --reload-rules
  2. Plug in your Librem Key
  3. Verify it now shows up when you run gpg --card-status

CrossBuild Initial Release

CrossBuild Provides a Dockerfile to build a container that supports cross compilation for ARMv7 and i686 targets from x86-64 hosts.

This project was created out of a need to have a single-shot process for building a few Rust projects with stripped binaries featuring static C-runtimes for:

While cross exists, and is far more full featured, the project lags behind in updating the base OS, GCC, and QEMU versions used (as of this writing).

An example exists that handles my specific use case for one project, and might serve as inspiration for modifications to suit more complicated needs.


Using podman since I do not have docker itself installed:

# Assuming you have cloned this repository and are in the repo
$ cd docker
$ podman build -t crossbuild:dev -f ./Dockerfile


Again using podman and assuming you're in the repository:

$ podman run --rm \
	-e REPO_URL="" \
	-e BIN_NAME="connchk" -v ./example/:/opt/build \
	crossbuild:dev /opt/build/

# ...

$ tree example
├── armv7-unknown-linux-gnueabihf_connchk
├── i686-unknown-linux-gnu_connchk

$ file example/*_connchk
example/armv7-unknown-linux-gnueabihf_connchk: ELF 32-bit LSB executable, ARM, EABI5 version 1 (GNU/Linux), statically linked, BuildID[sha1]=00811fa9637b6abf243fb707a8970b0cea43ba4f, for GNU/Linux 3.2.0, stripped
example/i686-unknown-linux-gnu_connchk:        ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, BuildID[sha1]=6e1d92bb01aed5b0adb673b1dfbe5fb28cf5da18, for GNU/Linux 3.2.0, stripped

Quick update on Librem Key Usage

In another post, I noted the steps I took to get my Librem Key working for my cryptography needs within Pure OS. This covers:

One thing I noted in the last post was that if you wanted to use the pkcs11 interface of the smartcard, it was necessary to kill off gpg-agent first. What I failed to notice myself, was that on normal boot the pcks11 interface grabs the device first. Since I was not frequently booting to run straight into SSH, I was generally unplugging my Librem Key and only plugging it back in when I needed to SSH. This allowed gpg-agent to snag the device and make it seem like nothing was wrong to me.

Fast forward to today, when I'm using SSH immediately almost every time I start my machine. Since I don't wish to wear out the USB plug in short order, and since clearly software is to blame, I set forth on finding a solution:

sudo systemctl disable pcscd && sudo systemctl stop pcscd

There you have it, now your gpg-agent reigns supreme over your smartcard and you can go about your business just fine. Should you wish to use the pkcs11 interface just stop your local gpg-agent, and start pcscd:

systemctl --user stop gpg-agent && sudo systemctl start pcscd

Use the latest Rust

Given the risks associated with CVE-2021-29922 anyone using A Rust Site Engine should make sure to build with Rust 1.53 or later. Generally, ARSE should only be used behind a reverse proxy that mitigates the risks but safety is a short rustup update && cargo install arse away.

Smooth Sailing on PureOS

After some eleven weeks of full time use of my Purism Librem 14 using Qubes OS, I have decided to give the native Pure OS as shot. While this will not provide the same degree of isolation I had in Qubes OS, the primary use cases for this machine do not necessarily require that degree of separation. When I do not wish to leave a trace, or otherwise find that the native attack surface is too great, I have a Tails stick I can boot.

As I write this, my laptop has been running Pure OS smoothly for two days. Almost the entirety of that time has passed while using the USB-C hub that was giving me fits in Qubes OS. In fact, right now I've got a USB-C SSD plugged in to send a Qubes backup across the ocean to my backup server at home. This very same operation failed somewhat regularly if I used the USB-C hub in Qubes, but has not been problematic yet in Pure OS.

Migrating my data from multiple qubes, into a single yet still "reasonably secure" Pure OS install was fairly simple thanks to Borg Backup. For those maybe curious about how such a thing can be accomplished, this is what I did:

With all of this done, I can use the system just as I was previously for management of my private network and for development purposes. While I am no longer using Split GPG and Split SSH, my private key material is not directly on the system and can be accessed only when my Librem Key is plugged in and unlocked. As I noted before, if one wishes to execute cryptographic operations using the pkcs11 interface it is still necessary to first stop gpg-agent. An alternative, however, is just to encrypt to yourself using gpg directly: gpg -se -r yourname <filename>.

So far, the only thing I have installed from a non-purism repository is syncthing. This actually exists in the default repositories, but the version is extremely stale so I added the developer's repository and stable branch to my sources.list.d and pinned the package to come from there in all cases. Long time users of Debian-based sytems will not be surprised by this at all.

More on the Librem 14

While I still need to do a longer test, it seems many of the USB-C issues I have observed on my Librem 14 while running Qubes OS may be the fault of the OS. A few hours of operation using the USB-C hub in Pure OS, the OS that shipped with the laptop, did not show any of the problems I have had running in Qubes.

Given that I still have another laptop also running Qubes OS, and that I more often use it for security tasks, it might even make sense to just install Pure OS and run the Librem 14 that way as my primary system. Before I do that, I'll spend some more time running the OS as a live system off a USB stick.

The audio jack does not work in any OS, and that is a bit disappointing. Thankfully, my USB-C hub does have an audio jack that works and in Pure OS I can also pair my bluetooth headset. I am hopeful that future releases of the EC from Purism will address the audio jack.

USB-C Hub is Out

The USB-C PD hub I have been using with my Librem 14 and Lenovo X390 just does not want to behave when connected to the Librem 14. The device itself shuts down at random and while I am curious to find out exactly why that investigation is not going to be a priority for a while.

Running the Librem 14 on USB-C power directly, and using the onboard HDMI port for video output, has proven rock solid but when involving the hub things are a bit iffy. I do wish I had remembered to bring the barrel-connector power supply that came with the laptop along for the trip across the Atlantic. I do have a USB-C to DisplayPort cable, so perhaps I will see if I have any video instability issues over that port by itself.

A new theory... Librem 14 + USB-C Woes

Given that today has been a day full of compiling Buildroot on the Librem 14, and that my battery has been stable at 98% the entire time, it might just be the now-removed USB-C hub/doc causing my troubles. If it is not that, it could also be the very small amount of RAM given the sys-usb qube in my Qubes OS system.

One thing I have noticed is that sometimes the screen just shuts off while I'm using the dock, though it only ever does this on the Librem 14. This never happens on my work laptop, but my work laptop is also not running Qubes with sys-usb in play and limited to 300MB of RAM.

When my builds complete, I will bump the RAM up on my sys-usb qube and see if that makes any difference. My unusual choices bite me in the backside sometimes!

TLS Implementation Failures

By now we have all attempted to access a website in any modern browser and found ourselves reading a warning that proceeding is dangerous. These tend to pop up when one encounters self-signed certificates, which themselves are not inherently evil, rather than certificates issued by one of the many globally trusted root certificate authorities. Failures in TLS implementation are not necessarily due to the use of self-signed certificates, but could rest in a failure to add the signing certifiate to the appropriate trust store after having verified the signer is who they say they are.

Everyone verifies certificates, right? Failing to do so extinguishes any real benefit of transport layer security, and exposes an extraordinarily large attack surface in the multitude of RESTful APIs and chat services that make the world of IoT tick. If, for whatever reason, your service does not mandate client certificates how safe can you be if you are not certain your clients are checking certificates? Since it requires more work to ignore certificate checking (examples below) surely no one is goiing the extra mile to do it wrong...

Unfortunately, ignoring certificate checks is fairly normal in some circles (looking at you, IoT) and if you want to know if a device on your network is guilty the process for finding out is trivial. This, of course, also means that a malicious attack is just as easy. So is preventing such attacks: always check certificates.

Are you curious if the brand new IoT widget you just recieved is Doing It Right™? By now we know every one of these devices is constantly phoning home to the mothership about your every move, but how can you check if this is done securely? Glad you asked!

If a picture is worth a thousand words...

No time to watch an ASCII Cast?

  1. bettercap to gather information on network hosts, and ARP spoof
  2. sslsplit to forge TLS certs on the fly
  3. An iptables pre-routing NAT rule to direct TLS traffic through sslsplit
  4. tshark to inspect the raw traffic, and anything intercepted by sslsplit
  5. Five minutes of your time

Final Thoughts

If the answer to "are you verifying certificates?" is no, then you are doing it wrong and putting both sides of your communications at risk. If you are a developer, and you do not know if you are checking certificates go take a look at your libraries and find out which extra options you need to use to disable checking. Search your source for these options. If you find them, file a bug and fix it. Immediately!

Librem 14 USB-C Charging in Qubes OS - During Qubes Backup

While still plugged in, and fully charged, I kicked off a Qubes Backup. In my last post, I noted how horribly inefficient that process is so I thought it might reveal something. Clearly, the first few minutes do actually decrease the charge of my battery even while plugged in. Fortunately, it appears that with two threads at work the charging profile is able to keep the decrease to a minimum.

For my next test, I think I will do something like run a huge DispVM and compile an extra bloated Linux kernel.

Librem 14 USB-C Charging in Qubes OS - Baseline

Sometime either last week, or the week before, my Purism Librem 14 gave me a bit of a scare. While transfering a large amount of data across the internet, and plugged in, it just died. It was fairly hot here in The Netherlands, and in the loft my office is in it was even hotter. My intial belief, or really fear, was that the brand new laptop was dead. Such is my luck in most cases, and after more than a little time in combat I am conditioned to expect the worst. Heat kills, and it would not be the first time I have seen it happen.

Fortunately, the device itself was not dead as in brick but was dead as in 0% charge remaining. Even though the USB-C power supply I am using, and the USB-C hub I was powering through at the time can handle plenty of wattage, it appears that the power requested was not greater than or equal to the power consumed. As our dear friend Newton explained quite a long time ago there ain't no such thing as a free lunch. I ran out of juice. Charging at 10W might have something to do with it:

Now the above is just my first sample of the charging profile, and the data were taken while the laptop was mostly idle. The data were collected the same way I did it before, parsed with our friend Python, and plotted with Bokeh.

For the next test(s), I will throw some load at the system and see what I can determine from the battery stats. What I am looking for now is whether or not I see the indication of charging whilst the battery energy continues to decrease minute over minute at full load. What is most interesting to me so far is that I have in fact been able to use the full power of the system for hours on end, and it appears that my decision to execute a backup in Qubes (itself an extraordinarily inefficient process) with low battery life may have had some impact on the situation. It should not have mattered, but what should be and what is are rarely unified in practice.

On the upside, and as a topic to detail in another post, the scare did provide the necessary push towards actually configuring Borg Backup scripts for my most critical VMs. Now, in addition to my full VM backups there are also de-duplicated and encrypted Borg backups of those VMs that I can run in seconds to my backup server back in the US.

The Python responsible for the chart above
from glob import glob

import pandas as pd

from bokeh.plotting import figure
from bokeh.embed import autoload_static
from bokeh.resources import CDN

samples = glob("*.txt")

charge_profile = { "energy": [], "rate": [] }

for sample in samples:
    with open(sample, 'r') as f:
        for line in f:
            if "energy:" in line:
            if "energy-rate:" in line:

df = pd.DataFrame(charge_profile)

p = figure(x_axis_label="Time (m)", y_axis_label="Wh (Navy), W (Firebrick)", title="Librem 14 Charging Profile")

p.multi_line([df.index, df.index], [, df.rate], color=["navy", "firebrick"])

js, tag = autoload_static(p, CDN, '/tech/ext/librem-14-charging.js')

# Then write the js and tag out...

Librem 14 + USB-C PD Hub in Qubes OS

A little more than a week ago, I picked up a few USB-C items to try out with my Librem 14 laptop...

Librem 14 displaying 4K video over a VAVA VA-UC020 Hub

The BatPower P120B works great for powering both my Librem 14, and my work laptop. The two USB-A power ports are great as I've now freed up space on my power strip that was previously occupied by a dual-port USB-A charger. It has also lightened my load on the road, as I no longer require a laptop charger and USB chargers for other devices.

All of the features of the VAVA UC020 are working in Qubes OS. Both my USB keyboard and mouse are plugged into my 4K monitor, which is connected to one of the VAVA's USB-A ports and its HDMI port. Early in the boot process, the panel came alive and has stayed that way ever since. As soon as I plugged in my wired headset, the audio device was available for assignment.

Switching between my work laptop and the Librem 14 is now as easy as swapping one cable!

SmartCards and Fedora

Attempting to use my second Librem Key with Fedora presented some challenges in dealing with pcscd. The root cause is that polkit does not allow normal users access to pcsc or the smartcard itself. This can be resolved with a single rule:

In /etc/polkit-1/rules.d/42-pcsc.rules:

  function(action, subject) {
    if (( == "org.debian.pcsc-lite.access_pcsc" || == "org.debian.pcsc-lite.access_card") &&
        subject.isInGroup("wheel")) {
          return polkit.Result.YES;

For the subject.isInGroup condition, I used the group wheel as I am the only member of that group on the system in question. Use your own descretion here, or use an even more specific condition to allow only one user like subject.user == "foo".

Additional Points

While this does allow access through pkcs11 and pkcs15 tools or gpg, I have not yet found the magic potion that will allow me to use both. Whichever tools are used first have a monopoly on the device. That said, on a modern Linux distro just using pkcs11 ought to do the trick.

Update: 2021-06-18

You can simply kill gpg-agent if you wish to use the pkcs11 interface after gpg takes a greedy lock on the device.


Use -engine pkcs11 with openssl subcommands that support it:

openssl rsautl -engine pkcs11 -keyform e -inkey <KEY_ID> -encrypt -in <INPUT> -out <OUTPUT>


Use "pkcs11:id=%<KEY_ID>?pin-value=<PIN>" as the identity file argument for ssh either on the command line, or in an ssh_config file. You will likely wish to get the PIN value itself from somewhere so it's not just in plaintext in your history:

ssh -i "pkcs11:id=%03?pin-value=123456" user@host

Or in an ssh_config file:

Host host
  IdentityFile "pkcs11:id=%03?pin-value=123456"
  User user

Adding SSH Agent Support to Split GPG

Split GPG is a very cool feature of Qubes OS but it leaves out one critical feature: enabling SSH support so the GPG backend qube can make use of an authentication subkey. There are a few different ways to solve this, and this guide provided some of the inspiration for what follows.

The Landscape

Here are the requirements for what follows:

Qubes RPC Policy

The first step is to configure an appropriate Qubes RPC Policy. A basic, and generally sane option, is to use a default configuration that asks the user to approve all requests and allows any qube to target any other qube with such a request. In my own configuration there are explicit allow rules for specific qubes where I use SSH frequently for admin purposes.

In dom0 create /etc/qubes-rpc/policy/qubes.SshAgent:

admin personal-gpg allow
@anyvm @anyvm ask

Actions in the Split GPG VM

The following actions all take place in the qube configured to act as the GPG backend for a Split GPG configuration.

Enable SSH support for gpg-agent:

$ echo "enable-ssh-support" >> /home/user/.gnupg/gpg-agent.conf

Update .bash_profile to use the gpg-agent socket as SSH_AUTH_SOCK by appending:

if [ "${gnupg_SSH_AUTH_SOCK_by:-0}" -ne $$ ]; then
	export SSH_AUTH_SOCK="$(gpgconf --list-dirs agent-ssh-socket)"
export GPG_TTY=$(tty)
gpg-connect-agent updatestartuptty /bye >/dev/null

Create /rw/config/qubes.SshAgent with the following content, and make it executable:

# Qubes Split SSH Script

# Notification for requests
notify-send "[`qubesdb-read /name`] SSH Agent access from: $QREXEC_REMOTE_DOMAIN"

# SSH connection

Update /rw/config/rc.local appending the following:

ln -s /rw/config/qubes.SshAgent /etc/qubes-rpc/qubes.SshAgent

Sourcing .bash_profile and /rw/config/rc.local should put the qube in a state where, if available, a GPG authetication subkey will be available to ssh-agent:

Example from my system:

[user@personal-gpg ~]$ ssh-add -l
4096 SHA256:V2KMVlJjPOn86z6a2srEcnMQj78OujEXJ597PJ6+wyY (none) (RSA)

Template VM Modifications

For my tastes it made the most sense to make a systemd service available to all qubes using my f33-dev template, and then start that service from /rw/config/rc.local on qubes I want to use the new feature.

In the approprite Template VM create a service similar to the following, but replace personal-gpg with the name of your Split GPG backend qube.


Description=Qubes Split SSH

Environment="AGENT_SOCK=/run/user/1000/SSHAgent" "AGENT_VM=personal-gpg"
ExecStart=socat "UNIX-LISTEN:${AGENT_SOCK},fork" "EXEC:qrexec-client-vm ${AGENT_VM} qubes.SshAgent"


Once this has been added run the following, and shut the template qube down:

sudo systemctl daemon-reload

The Client Side

In the actual SSH client qubes, there are a few actions required to complete the loop.

Append the following to .bashrc - make sure this matches the AGENT_SOCK in your systemd service:

### Split SSH Config
export SSH_AUTH_SOCK="/run/user/1000/SSHAgent"

In /rw/config/rc.local append the following to start the service:

systemctl start split-ssh

Source .bashrc and /rw/config/rc.local and with the split GPG backend qube running test that your key is available:

[user@admin ~]$ ssh-add -l
4096 SHA256:V2KMVlJjPOn86z6a2srEcnMQj78OujEXJ597PJ6+wyY (none) (RSA)

Since my Qubes RPC policy allows the admin qubes to reach personal-gpg without my confirmation, a system notification appears stating:

[personal-gpg] SSH Agent access from: admin


With a few simple steps the power of Split GPG can be extended to include SSH Agent support. As a result, network-attached qubes used for administration of remote assets no longer directly store the private key material used for authentication and the attack surface is that much smaller. There are a few ways to get the pubkey to add to remote ~/.ssh/authorized_keys but the easiest way is probably ssh-add -L.

Librem 14, Librem Keys, and Qubes OS

With the arrival of my second Librem Key, I thought now would be a good time to go over how I use Qubes OS features along with some more products from Purism for various signing, encryption, and authentication tasks.

The Landscape

Here are the various components at play:

The base of all but one of my qubes is Fedora.

Security Device

Getting Started

The new Librem Key needs to have its PIN set, and since my Qubes OS configuration uses a USB qube it will be necessary to give my running disposable VM access to the key itself:

In dom0, where my target vm is disp4632 and my BACKEND:DEVID is sys-usb:2-1:

$ qvm-usb attach disp4632 sys-usb:2-1

In the disposable VM run:

[user@disp4632 ~]$ gpg --card-status
Reader ...........: Purism, SPC Librem Key (000000000000000000009BB1) 00 00
Application ID ...: D276000124010303000500009BB10000
Application type .: OpenPGP
Version ..........: 3.3
Manufacturer .....: ZeitControl
Serial number ....: 00009BB1
Name of cardholder: [not set]
Language prefs ...: de
Salutation .......: 
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: rsa2048 rsa2048 rsa2048
Max. PIN lengths .: 64 64 64
PIN retry counter : 3 0 3
Signature counter : 0
KDF setting ......: off
Signature key ....: [none]
Encryption key....: [none]
Authentication key: [none]
General key info..: [none]

Change the PINs

  1. gpg --card-edit in the disposable VM
  2. admin at the gpg/card> prompt
  3. passwd at the gpg/card> prompt
  4. Select 1 and follow the prompts, where the first PIN is the default: 123456
  5. Select 3 and follow the prompts, where the first Admin PIN is the default: 12345678
  6. Select q and quit.

Initialize the New Librem Key

Remove the new Librem Key

In dom0 run:

qvm-usb detach disp4632 sys-usb:2-1
Insert the original Librem Key

In dom0 run:

# Assuming you plugged the original key into the same port
qvm-usb attach disp4632 sys-usb:2-1
Insert and mount the Librem Vault

In dom0 find the appropriate block device, and attach it to the disposable VM:

qvm-block list

qvm-block attach disp4632 sys-usb:sdb1

In the disposable VM find the attached disk (likely /dev/xvdi)

[user@disp4632 ~]$ lsblk
--- SNIP ---
xvdi    202:128  1 28.9G  0 disk

Then mount the disk:

[user@disp4632 ~]$ udisksctl mount -b /dev/xvdi
Mounted /dev/xvdi at /mnt/removable

Note that I did not sudo mount /dev/xvdi /mnt/removable as the operation does not require root, and we do not use powers we do not need do we?!

Extract the encrypted backup from the Librem Vault
[user@disp4632 ~]$ cp /mnt/removable/gpg-backup/backup* .
Unmount and Remove the Librem Vault
[user@disp4632 ~]$ udisksctl unmount -b /dev/xvdi
Unmounted /dev/xvdi.

In dom0:

qvm-block detach disp4632 sys-usb:sdb1
Decrypt the backup

This assumes you have installed opensc and have pkcs15-tool and pkcs11 drivers.

First, find the Key ID for encryption key on your existing Librem Key:

[user@disp4632 ~]$ pkcs15-tool -D
Using reader with a card: Purism, SPC Librem Key (000000000000000000009BB4) 00 00
PKCS#15 Card [OpenPGP card]:
        Version        : 0
        Serial number  : 000500009bb4
        Manufacturer ID: ZeitControl
        Language       : de
        Flags          : PRN generation, EID compliant


Private RSA Key [Encryption key]
        Object Flags   : [0x03], private, modifiable
        Usage          : [0x22], decrypt, unwrap
        Access Flags   : [0x1D], sensitive, alwaysSensitive, neverExtract, local
        Algo_refs      : 0
        ModLength      : 4096
        Key ref        : 1 (0x01)
        Native         : yes
        Auth ID        : 02
        ID             : 02 <-- THIS ID
        MD:guid        : ee23dccc-fc38-2dc2-3bc8-bb5f859168d4


Now use it to decrypt the pbkdf2 key used to encrypt the GPG backup tarball itself. This hybrid encryption scheme allows securely storing data of arbitrary sizes and using pbkdf2 with randomly generated secrets and then encrypting those secrets with the Librem Key's encryption key.

Decrypting the pbkdf2 password file with the Librem Key:

[user@disp4632 ~]$ openssl rsautl -engine pkcs11 -keyform e -decrypt -inkey 02 -in backup.key.enc -out backup.key
engine "pkcs11" set.
Enter PKCS#11 token PIN for OpenPGP card (User PIN):

Decrypting the GPG backup with the pbkdf2 password file:

[user@disp4632 ~]$ openssl enc -chacha20 -pbkdf2 -pass file:backup.key -d -in backup.tar.gz.enc -out backup.tar.gz
Extract the backup
tar xf backup.tar.gz
Verify the keyring is in tact
[user@disp4632 ~]$ gpg -k
pub   rsa4096 2021-05-08 [C]
uid           [ultimate] Anthony J. Martinez <>
sub   rsa4096 2021-05-08 [S]
sub   rsa4096 2021-05-08 [E]
sub   rsa4096 2021-05-08 [A]

Remove the original Librem Key

In dom0:

qvm-usb detach disp4632 sys-usb:2-1
Insert the new Librem Key again

In dom0:

qvm-usb attach disp4632 sys-usb:2-1
Export the signing, encryption, and authentication subkeys to the Librem Key

Edit the key in expert mode:

[user@disp4632 ~]$ gpg --expert --edit-key FCBF31FDB34C8555027AD1AF0AD2E8529F5D85E1

In the gpg> prompt select each subkey and use the keytocard command.

Example, using the signing key (key 1):

gpg> key 1

sec  rsa4096/0AD2E8529F5D85E1
     created: 2021-05-08  expires: never       usage: C   
     trust: ultimate      validity: ultimate
ssb* rsa4096/A2206FDD769DBCFC <-- NOTICE THE * HERE - this key is selected
     created: 2021-05-08  expires: never       usage: S   
ssb  rsa4096/6BE6910237B3B233
     created: 2021-05-08  expires: never       usage: E   
ssb  rsa4096/FD94BDD7BED5E262
     created: 2021-05-08  expires: never       usage: A   
[ultimate] (1). Anthony J. Martinez <>
[ultimate] (2)  Anthony J. Martinez <>

gpg> keytocard
gpg> key 1 <-- this is to deselect key 1

Repeat the above for key 2 and 3.

Verify the card status
[user@disp4632 ~]$ gpg --card-edit

Reader ...........: Purism, SPC Librem Key (000000000000000000009BB1) 00 00
Application ID ...: D276000124010303000500009BB10000
Application type .: OpenPGP
Version ..........: 3.3
Manufacturer .....: ZeitControl
Serial number ....: 00009BB1
Name of cardholder: [not set]
Language prefs ...: de
Salutation .......: 
URL of public key : [not set]
Login data .......: [not set]
Signature PIN ....: forced
Key attributes ...: rsa4096 rsa4096 rsa4096
Max. PIN lengths .: 64 64 64
PIN retry counter : 3 0 3
Signature counter : 0
KDF setting ......: off
Signature key ....: C9ED 41D4 EB62 80BB E61F  0E59 A220 6FDD 769D BCFC
      created ....: 2021-05-08 11:43:52
Encryption key....: 335D C8BC E4A6 8FFF B9B5  CBEF 6BE6 9102 37B3 B233
      created ....: 2021-05-08 11:44:54
Authentication key: D157 68B9 CCCF 4FB5 6FC2  971E FD94 BDD7 BED5 E262
      created ....: 2021-05-08 11:45:39


From here the new Librem Key is configured, and the disposable VM is of no further use. Since disposable VMs are destroyed when the application they were created to run is stopped, the only cleanup necessary is to close the terminal to the disposable VM.

Additional Notes

On my system, I also have vault and personal-gpg qubes. These are both network-isolated and function much the same way the physical key does. The personal-gpg qube holds the very same subkeys as both Librem Keys, and through the use of Split GPG allows for a smartcard-like use of the qube from my other qubes. In a later post, I will detail how I use QubesRPC in personal-gpg to also serve as my ssh-agent for using the authentication subkey in things like my admin qube to prevent me from needing dozens of copies of my SSH private keys everywhere. The vault qube is home to the master secret key, and as such never has any data fed in to it.

The process used to decrypt data can be reversed to encrypt data as well. I will leave that as an exercise for the reader, but the short version is that instead of the decrypt option(s) for the openssl tools use their encrypt counterparts. If you wish to generate a random secret to use with pbkdf2 the following should do the trick:

openssl rand -base64 -out secret.key 32

Another week with the Librem 14

Another week has passed, and I am liking the Librem 14 quite a lot overall. Having now mostly pounded the keyboard into submission it is much more tolerable. The A and S keys are the most disobedient of the bunch, and the oddly located R Shift results in some random profanity when I end up a line higher in my menu or code when that is not at all what I wanted.

Most of what I have worked on was simplifying my home network. Thanks to the presence of a physical LAN port, and some of the finer points of Qubes OS NetVMs it was easy to setup multiple VMs each assigned my wired interface. This allowed me to verify tagged and untagged VLAN settings on a switch the configuraton of which I forgot long ago. All of this took place on battery with me sitting on the floor of my closet without much fear that I would soon need to race across the house to get my charger.

For much of the last week, I also tested out using awesomewm with Qubes OS. Basic tiling was fine, and helped me handle the lower 1080p resultion offered by the Librem 14. In the end, I went back to XFCE with some tiling enabled by keyboard shortcuts.

No buyer regret here. This machine does everything I need it to do and it does it much better than the machine it replaced. When I return to The Netherlands that machine will change roles and continue life, but my main machine will definitely be this Librem 14 for several years to come.

Librem 14 Battery Life While Running Qubes OS

Here is a first look at the kind of battery life one might expect while using a Purism Librem 14 as shipped with the 4-cell battery while running a normal workload in Qubes OS.

TL;DR - The Chart


Given a steamy day in Texas, one Librem 14 running Qubes OS, and a few general tasks to accomplish I set out to find out how long I might expect to spend away from a wall outlet if I:


Once the system was running, I started a loop to give me battery stats every 60s:

while true; do
    upower -i /org/freedesktop/UPower/devices/battery_BAT0 > $(date +%s)_bat-info.txt
    sleep 60

From here, I just went about my business. TemplateVMs on my system are primarily based on Fedora. As a result very nearly every boot means there are updates available. Updating was probably the most strenuous task executed during the test run. In fact, I do not recall the fans turning on for anything else throughout the day except for maybe the one time I started one of my more substantial VMs I use for development. For the most part I had at least (7) VMs running, one of which maintained a WireGuard VPN connection to my cloud environment. General tasks amounted to:


This machine lasts at least as long with light use as my previous system. No one runs towards Qubes OS with the hopes of marathon battery life, and I am pretty happy with near 5hrs of battery runtime on WiFi with a persistent VPN to my cloud resources. Next time, I may shut off WiFi and see how much purely local heavy use I can squeeze out before the battery dies.

First Impressions - Librem 14

For my birthday last year, I ordered the Purism Librem 14 to serve in place of my aging Lenovo T460s. Slightly less than a year later I got my new laptop, and a few weeks after delivery I was able to fly home on vacation to finally get started using it. To most of my friends, waiting a year for a laptop to be delivered is utter madness, but for me there are almost no new laptops I am even willing to consider. Having a physical RJ45 jack is a hard requirement for me, and today it seems this typically requires a willingness to carry a "laptop" that weighs more than a healthy newborn human. Give me power, give robust networking options, and give me the RAM I need to run Qubes OS to its fullest. The Librem 14 offers this all, on paper, and over the coming month I will find out exactly how that all pans out in reality.

First Boot

My machine was ordered with the following specs:

Since there are not any good defaults for installing Qubes OS on an OEM device one needs to install it on their own. Knowing this I performed my first boot into Pure OS using encryption passphrases and user passwords no one should ever use on system they plan to actually use. The process for using PureBoot was pretty clear and straight forward, and the Librem Key that shipped with the Librem 14 flashed green a expected when I made my first boot. It also flashed red, as expected, when I booted the second time having updated the kernel.

My time in Pure OS was limited to two tasks:

  1. Generating new GnuPG keys to store on the Librem Key
  2. Making a USB boot drive from the latest Qubes OS release

The Next 20 Boots

My initial installation of Qube was mising one critical point: an encrypted root partition. Use of PureBoot requires an unencrypted /boot and that is fine given that I am notified of any changes and can opt to sign them if they were expected after an update. Lacking encryption of the rest of the disk is not an acceptable scenario for me, so I reinstalled. And reinstalled again. And again. And again.

Since I am writing this on vacation, I will have to come back to a determination of the root cause later. For now, just know that selecting the defaults in the Qube OS installer - which regrettably includes wasting 15GB of disk on swap I will never use - was the only partition scheme that resulted in an encrypted root partition. To be clear, I do not think this has anything to do with the Librem 14. When I do find the root cause, the appropriate project will get a detailed bug report. At any rate, once I was up and running the next task was restoring my qubes (VMs) from a backup taken on my T460s right before shutting it down last. This was done from a USB3 SSD, and the speed was outstanding. The restore completed much faster than the backup was made, likely owing to both faster USB controllers and the gap between Intel's i5-6300U and i7-10710U.

Normal Use

So far, I have not done much more than restore my qubes and fix a few remote issues I caused late last week that made SELinux angry. Everything is running smoothly, and the few times I have run a CPU intensive task like Rust compilation I have been very pleased with the performance of the machine. The only thing I do not like is the keyboard. I am far from the only person who regards older Thinkpad keyboards as being top of class, and coming from such a keyboard to this does not please me much. While typing this, my backspace key (and vim motions) have been flexed. My a, s, b, and l seem to particularly hate their existence. For some, this could be grounds for a return, but I have had other machines in the past with keyboards I found infuriating at first. Usually, the force of my typing tends to smooth things out in short order. If that is the case here all will be will. If not, it will probably also be fine since most of my use is on a desk plugged in to a Drop ALT.

Qubes and extended battery life are rarely thoughts one has concurrently, and I am pleased to note that the Librem 14 managed more than 4 hours on WiFi doing a mixed load of tasks. These included spawning at least 40 Disposable VMs, upgrading all of my Template VMs, compiling two different versions of the engine rendering and serving this page several times, and for reasons I still do not understand, playing the official music video for Enya's "Only Time" which I have not linked lest anyone else inexplicably end up with it stuck in their head.

So far so good. Tomorrow, I head for the mountains. When I return, I will put the Librem 14 through its paces as I work on some personal Rust projects.

For Next Time

I will try to: