Heads 33c3
This is the draft of my 33c3 talk on Heads. The 45 minute video is available for watching.
Introduction
 |
 |
I'm Trammell Hudson with Two Sigma Investments and for the past several years I've been researching firmware vulnerabilities on laptops and server systems. One of the more important ones that I found was the Thunderstrike vulnerabilities, the first of its class against Macs that allowed a proximate attacker to overwrite the motherboard SPI flash ROM as well as documenting the organization, checksums and signatures used to protect the Mac's firmware. I presented Thunderstrike at 31c3, and the next year I collaborated with Xeno Kovah and Corey Kallenberg of Legbacore, both of whom now work on firmware security at Apple, to explore porting UEFI vulnerabilities from PCs to Macs as part of Thunderstrike 2. We found that the enormous amount of shared code between the common PC's UEFI firmware and Apple's EFI implementation made it possible to use these previously published attacks on Macs, this time allowing remote attackers to overwrite the ROMs. Xeno and I presented Thunderstrike 2 at Blackhat/DefCon and CCC.
My real passion is not just breaking systems, but taking things apart to learning how they work, documenting them and building systems that other people can extend. And that's why I'm excited to be here today to talk about Heads, a Free Software licensed firmware and Linux-based bootloader that has a goal of improving the security of laptops and servers by incorporating the research that people have been doing into firmware security.
The name is a play on the popular Tails Linux distrbution, which is great for when you need to leave no trace with a stateless system. In contrast to Tails, Heads is a way to improve the security of any data you store on the machine.
Why firmware security is so important
Let's back up for a minute and talk about WHY boot-time firmware security is so important.
Since the x86 firmware contains the initial instructions the CPU executes when it comes out of reset, it is important that it be protected against malicious modification. And since it is able to talk directly to hardware that can bypass kernel checks, it is vital that bugs in the firmware be fixable. The concern is that if the boot firmware is compromised or buggy, then the rest of the system security is built on an unstable foundation. This was well deomonstrated by Intel's Advanced Threats Research team demonstrated in their DefCon talk on guest escalation into the hypervisor, using buggy SMM.
As Joanna Rutkowska reminded us last year at 32c3 that most every device in our systems has some sort of firmware, much of which is opaque or difficult to verify and that the devices might be actively attempting to attack the boot process or the operating system. Her stateless laptop would be great if we were given a clean slate, although Heads is largely focused on what can we do with existing commodity systems.
 |
 |
These firmware attacks aren't just theoretical research vulnerabilities. We know that malicious hackers and nation states are using firmware attacks against their targets. And, once malware has taken root inside the firmware, it is very difficult to remove since it can hide inside of things like SMM and it can survive reinstallation of the OS and even replacement of the harddrive.
In fact, some vendors have even shipped “rootkits” in their firmware that reinstalls itself into the OS and circumvented the OS's security policies, so if you reformat the disk and attempt to install a clean copy, the rootkit reinstalls itself. Bugs in this vendor supplied and unreplaceable code could be taken advantage of by malicious attackers as well. Luckily in this case the vendor released a firmware update without the rootkit.
Unfortunately most firmwares are never updated since there are usually four or five companies between the UEFI source tree and your machine, which means that every party along the way needs to be motivated to QA a fix.
This situation would be unacceptable for an Operating System or application, both of which receive frequent and rapid updates when vulnerabilities are found.
Even though UEFI-the-standard is open, UEFI-the-firmware contains closed source portions that prevent users from being able to patch their own systems without vendor support.
And with UEFI comes extra complexity. The Linux community was opposed to the complexity compared to the the original BIOS, which had decades of legacy baggage but was very simple and reasonably well understood. Most systems continue to ship with BIOS compatibility modules as well, so there is now twice the attack surface and potential for bugs.
The state of the world today is that most firmware is closed source, rarely updated, inflexible and potentially full of unfixable vulnerabilities.
Philosophy of Heads
These problems with the existing firmware ecosystem convinced me that there must be a better design. My belief is that systems must be open for us to be able to audit it and trust it, as well as being flexible for us to freely adapt it to our own needs. It should be built on widely used and well tested code. And it should be reproducible so that we can have confidence that we're building the same code, which also allows us to use cryptographic measurements to ensure that we're running the same code that we flashed onto the machine when we set it up.
That's the philosphy behind Heads. It's a specialized distribution built with the Free coreboot firmware that uses Linux in ROM as a bootloader. It incorporates security research and tools to help ensure an orderly, well measured boot process. It's not a Linux system that you would run – the entire firmware, kernel and tools fit in 4MB of ROM. The goal of the Heads runtime is to establish a hardware root of trust, then decrypt, measure and verify the real OS, and finally kexec into it.
This isn't a new idea -- it has its genesis back in the 90s when the national labs were building large scale parallel machines. At that time the tightly coupled Intel Paragon and ASCI Red parallel machines were the worlds fastest computers, and we also built large Linux clusters with racks of desktop computers that were the 30th fastest in the world. These systems used PXE and TFTP to boot, but the stock firmware was very frustrating in its limitations.
Our colleague Ron Minnich at Los Alamos was building similar clusters and realized that the BIOS was duplicatating much of the work that Linux was doing during system setup, so he had the idea to flash a minimal Linux kernel and initrd into the boot ROM in place of the BIOS. He called this LinuxBIOS and it powered machines like the MRC Linux cluster to #3 on the 2003 top500.
LinuxBIOS was renamed coreboot in 2008 as part of a major codebase refactoring. Google started using it in their Chromebooks and Ron moved to Google in 2011 to continue development on the project. Chromebooks are one of the few non-UEFI systems out there and have some of the best security features, such as a physical write-protect screw to prevent accidental or malicious updates to the firmware.
coreboot simplifies the firmware part of the firmware – it doesn't have any real device drivers (other than the TPM) and the trusted computing base is roughly 1% of the size of the UEFI firmware on the same platform. There are three sections: the bootblock, which is a tiny amount of assembly involved in switching from real mode; the romstage, which runs with "cache as RAM" to setup DRAM and initialize the TPM; and the ramstage, which does a minimal amount of housekeeping and generates descriptors for the devices on the system before jumping into the payload.
Since we're able to bring up the TPM and establish our static hardware root of trust first thing in the romstage prior to initializing the memory controllers and while running out of cache, this reduces our exposure to certain types of external device attacks. There are still concerns about the EC and ME, which we'll address a bit later.
The three coreboot stages take a fraction of a second to execute, including measuring and invoking the payload. In this case the payload is Heads' Linux kernel and initrd. It can "boot" to an interactive recovery shell in less than a second. Having the full power of Linux in the ROM gives us great flexibility to customize and extend the system – we can build the boot policies and setup tools with real shell scripts and runtimes.
This flexibility also Heads' Linux kernel to support fully encrypted disks on any number of possible filesystems on any number of possible device types, in contrast to UEFI, which only supports booting from unencrypted FAT partitions on normal drives. This also potentially reduces the attack surface since Heads has only one FAT implementation, rather than depending on both Linux's and UEFI's.
Even though UEFI only has limited filesystem and device support, it is still millions of lines of code and has many features of an OS without the real-world hardening that Linux's device drivers, filesystems and networking has undergone. There is a saying that "with enough eyes all bugs are shallow", and Linux's infinite contributors have a few more eyes than opensource edk2 part combined with the various vendors development teams.
More importantly, Linux and coreboot are open source so you can fix problems, build and sign your own firmware images/kernels. You don't have to wait for your vendor to get around to patching them. You can add the device drivers, features and tools that you need in your system, rather than hoping the vendor decides to support your niche application.
Heads' 4MB runtime isn't a "real" OS that users will run; it's goal is to as quickly as possible verify and start the user's real OS with kexec. kexec was added to the kernel in 2003 by Eric Biederman for fast reboots on diskless LinuxBIOS clusters, and is very useful for using Linux as a bootloader.
This gives us significantly more flexibility than UEFI or even grub to implement policies and protections. Rather than using grub's configuration file with its limited scripting support, we can use a real shell to determine which kernel to run and the arguments to pass to it. As we'll show later, we can also have Heads do much more sophisticated things like can check PGP signatures and use TPM counters to protect against rollback attacks.
Heads features
Now that we have the basis for an open and flexible boot firmware, we can "stand on the shoulders of giants" and begin to incorporate some of the firmware research that is being done by the security community.
Before we go too far down the rabbit hole of threats and defenses, let's keep in mind that we do not all share the same threat models and that some people need very different protections or require very different functions. As steph reminds us, "your threat model is not my threat model, but that's ok" -- lucki ly open source makes it easy to adapt coreboot and Heads to fit your own needs.
Hardware hardening
The chain of trust in the boot process starts in the hardware at the very bottom. Peter Stuge at 30c3 had some very good suggestions for hardening the older thinkpads, especially since there are many devices with emebeded firmware that we can't measure or trust. Removing every programmable device might not be part of your threat model, but it is important to keep track of which embedded device you're trusting.
Right now installing Heads does involve disassembling the machine to gain access to the SPI flash chips on the mainboard to write the Heads bootblock and recovery image into the boot ROM. We set the non-volatile BP0-BP3 hardware protection bits on the flash ROM and, inspired by the chromebook writeprotect screw, suggest disconnecting the WP# pin from the motherboard entirely.
This step creates an immutable bootblock, which eliminates an entire class of software attacks (like Lighteater, Thunderstrike, Dark Jedi, Speedracer, etc) from being able to interfere with the ROM and provides the hardware root of trust that we will depend on for later measurements. It also prevents attacks via things like the Intel SVT closed chassis adapter or devices that can directly poke the chipset, such as the Mangement Engine.
The Management Engine in modern CPUs is of grave concern to the security community. The ME is a "matroshka CPU" inside the CPU that runs an undocumented opaque binary blob that is hidden from the main CPU and has access to memory, the keyboard and the network; and it is listening on the network even when the main CPU is powered down.
Most discussion of it has focused on the "root kit" capabilities of the ME and the inability to turn it off. Many BIOS menus have a "disable AMT" option, but it is unknown how much that really disables and how much is still functional.
I've done some research into how the Intel ME firmware images are signed and determined how to disable the ME through a modified firmware with only the CPU bringup module. You can flash this onto the ME's flash chip during the hardware setup process so that the "rootkit in your chipset", to the best of our knowledge, will not function since it no longer has the network device driver or java bytecode interpreter. This works on the Ivybridge x230 and Chell chromebook Skylake systems that we've tested and also makes available an additional 4 to 5 MB of space for the Heads payload.
This makes it possible to have somewhat more trustworthy systems on newer hardware than supported by libreboot, which doesn't work with any CPU with a Management Engine.
As Joanna pointed out, there might be other rogue devices attacking the boot process and early boot DMA attacks are easy to launch since the memory layout is very consistent. Yao and Zimmer at Intel wrote a guide to enabling the IOMMU to protect against early DMA attacks, but unfortunately very few UEFI systems take advanage of it. Coreboot disables bus mastering during boot and the Head's Linux kernel can turn on the IOMMU and VT-d, allowing it to protect itself from these attacks.
 |
 |
Option ROMs are another attack vector where rogue devices can attempt to influence the boot process. UEFI suggests that they be signed, although in my research I found that some vendor firmwares did not use signatures.
Snare reported this problem in 2012 at Black Hat and my Thunderstrike research showed how Option ROMs could escalate into write access to the flash. After five years, this is finally being fixed by the vendor.
coreboot users could make this sort of change on their own as soon as it became a problem. The configuration option points out that it might cause problems, but that it is important for security conscious users to disable it. Since Linux uses its own real device drivers, the Option ROMs are unlikely to be necessary with Heads. Additionally, this "Self Help" capability is important if the vendor decides to abandon a platform or if a niche device needs support in the ROM.
TPM and Encryption
Heads makes extensive use of the "Trusted Platform Module" to store the secrets for the disk encryption keys. These cryptographic coprocessors have been viewed somewhat skeptically by the free software community, but they are actually a great resource for helping to improve the security of our systems. Since coreboot and Heads control the TPM from the initial bootblock, they are able to ensure that it is setup in a user friendly way, instead of being used for enforcing DRM or other restrictions.
The main use of the TPM is to "measure" the hashes of the code and data that is executed during the boot process into special registers called PCRs. When the system is installed the disk encryption keys are "sealed" based on the hashes of the firmware and the TPM will only unseal the keys if the TPM PCRs match, along with a user provided password. The TPM also protects against theft of the system since it rate limits the password entry and has hardware enforced retry counts. An attacker with physical possession of the system can't easily extract the key unless there is some vulnerability in the TPM hardware itself.
There is a saying that there are two types of users of encrypted drives – those who have lost data due to forgetting their passphrase and those who will.
Using the TPM for storing keys adds an additional way to lose you data, since the decryption is now tied to this piece of hardware.
To help you when you lose access to the key or have to move your harddrive to another machine, Heads can generates secret shares using Shamir's Secret Sharing in addition to the master key, so that you can recover the master key given 3 out of 5 shares. You can spread these shares to friends or in different locations and an adversary would have to compromise multiple of them to recover the key.
Since Heads is built with Linux, we can use the full flexibility of shell scripts and our normal tools to do things like apply anti-evil maid techniques of measuring the LUKS cryptographic headers into the TPM PCRs. AEM is depending on the dynamic root of trust from TXT, while Heads is able to use the much simpler static root of trust since it controls all of the code in the boot path.
With these sort of modifications we hope that the system is slightly less vulnerable to different classes of attacks, but how can you tell if your system is running the boot code you flashed rather than something installed by an evil maid to steal your password?
One approach is to run the Anti-Evil-Maid toolkit, which uses the TPM to decrypt a secret phrase or image if the PCRs match, but this is subject to replay attacks.
Mathew Garret pointed out that there is a better way -- the Time-based One-time Password Algorithm used in google authenticator TOTP allows the shared secret to be used to generate a one time password that is not subject to replay attacks and that is only valid for a limited period of time. During system setup a shared secret is generated and display to the phone via QR code and then sealed based on the TPM PCRs for a clean boot.
On every subsequent boot, tpmtotp displays a six digit code based on the current time, and the user verifies that it is the same as the one that is being generated on their phone. The TPM won't unseal the shared secret to allow the system to generate the one time password unless the boot firmware and configuration matches what the user flashed onto the system, which provides a “remote attestation” for the simple humans who can't do RSA in their heads.
To subvert this would require an attacker to be proxying in real time between a stolen device and the target, a much more difficult attack.
Implementing tpmtotp in Heads is not difficult since we have the full flexibility of Linux in our bootloader. Garret's idea can be coded in a very small shell script to read the value from nvram, unseal it and compute the totp.
Code signing
This phase of the boot also marks the beginning of a transition from the TPM static root of trust chain to a user provided GPG signing key (preferably in an external hardware token). This is a significant change from UEFI, where the vendor and OEMs control the keys. It is important that updates and changes are signed by the user who installs them, not by random OEMs who may not have the best key management policies. We know that stuxnet used stolen kernel driver certificates, which were trusted by the OS.
Android introduced dm-verity as part of their verified boot process and I would love to see it become a feature in mainstream kernels to prevent malware from being able to gain persistence. Heads supports booting from a dm-verity hashes filesystem and updating the disk requires booting into a special mode and signing the hashes, preferably with a hardware token like the yubikey.
Even with encrypted disks there are still possible replay attacks against vulnerable versions. This can be mitigated by using the TPM counters and GPG signed hypervisor, kernels and initial ramdisks.
Once these next stages have been validated and measured, the encryption keys can be populated into the ram copy of the ramdisk and the real OS is executed by the kexec program. This allows Heads to invoke any multiboot compliant kernel and to pass the static root of trust as well as the GPG keys.
OS and Software Support
As we discussed earlier, firmware security is vital for system security; additionally, the operating system must take security seriously as well.
For slightly more secure laptops, we've been working on supporting the highly recommended, “reasonably secure” Qubes OS. This has involved a few changes to the Xen hypervisor to eliminate the dependencies on legacy BIOS features and working with the dom0 initramfs to insert the TPM protected encryption keys. We've also modified Qubes to be able to use a read-only root file system with the dm-verity signed hashes.
Additionally, the Qubes team recognizes that firmware security is important and is requiring systems to have open source firmware in order to be “Qubes R4 certified”.
Heads has a goal of 100% bit-for-bit reproducible builds so that if you download the tree, you should be able to generate a binary that exactly matches the one that we build. It's a goal, so there are some steps that aren't there yet, and we're working on it and pushing changes to our open source dependencies to help them become reproducible. We've also developed tools to help with building reproducible initrd files, which we hope will be adopted by other distributions to help with their reproducibility.
We also enforce cryptographically signed commits in our tree, so any attempts to tamper with it can be detected. And, since it is open source we invite everyone to read the code, no NDAs required.
What's next?
That's the tour of where Heads is today. There are several open areas of continuing research
Such as embedded controller firmware. These are already open source on the chromebooks, which is yet another reason to support them.
Porting coreboot to more modern platforms will also allow us to take advantage of things like case intrusion detection sensors to help defeat evil maid attacks, as well as CPU improvements like Intel Bootguard, as long as we can find an OEM who will fuse the CPU in Measured Boot mode, as opposed to the user hostile Verified Boot mode.
porting to coreboot to server platforms and incorporating projects like OpenBMC to help manage Heads will help us trust our hosted systems. The threat model for servers is very different from laptops and one that is really important to consider
There are interesting projects looking to use Heads and TPM root of trusts to protect cloud systems and I'm cautiously optimistic about the ability of enclaves to help protect hosted computation.
There are always issues that need assistance.
Conclusion
As I've said several dozen times during this talk, Heads is open source and I encourage you to help out with improving the boot strap process for our systems.
We'll be having an install fest in Saal ? Later this week and will help you flash your laptop as well as answer any detailed questions that are too long for this session.
And now I'd be happy to answer any questions in the time remaining.