Login   |   1.866.392.4897 |   sales@timesys.com English Japanese German French Korean Chinese (Simplified) Chinese (Traditional)
IoT security simplified with PSA Certified VigiShield

IoT security simplified with PSA Certified VigiShield

From customer expectations to cybersecurity regulations, the demand for security assurance of devices has never been greater. Yet device manufacturers find it challenging to secure Linux devices. Establishing a secure Linux platform requires deep expertise and time investment; right from selecting the processor with the necessary security features, researching and configuring the myriad of open source software modules, and having the necessary mechanisms to regularly maintain security of the device through its life. Besides securing the system software, the device manufacturers need to worry about application security and keeping end customer data confidential. Hence the road to security is a long and expensive journey.

One of the best ways to reduce the cost of security development, improve security posture and bring secure devices to market faster is by leveraging off the shelf pre-certified components. PSA certified provides a framework for security assessment and certification of processors, system software (e.g: Linux OS, RTOS) and devices evaluated by independent labs. Device manufacturers can leverage PSA Certified components (processor and system software) to quickly start with a secure platform and focus on their value add software. Optionally they can submit their devices for certification as well to give the end consumer a peace of mind. PSA certified components also align with meeting industry security standards such as NISTIR 8259A and ETSI EN 303.

 

Image credit: PSA Certified

 

Timesys has developed VigiShield Secure By Design, a PSA certified system software which device manufacturers can leverage to bring secure Linux devices to market faster. VigiShield is available in the form of a Yocto layer and is highly configurable to support custom hardware/BSP or any custom security needs. Out of the box, it includes security features such as software integrity (secure boot and chain of trust), secure storage, secure over-the-air (OTA) updates, linux kernel and system hardening, secure communication, locked hardware ports, security audit logs and more. Additionally Timesys supports manufacturing tooling required for secure software provisioning. The security layer leverages the best in class open source solutions to bring easily maintainable security feature implementation to your Yocto based Linux distro.

The other major challenge device manufacturers face is the long term security maintenance of their Linux devices. With more than 350+ new vulnerabilities reported each week, it is a daunting task to monitor their applicability to the open source components used in devices and issue security updates. Timesys Linux OS/BSP Maintenance subscription service provides long-term security updates and maintenance of your Linux OS. Using this service device manufacturers can rely on timely security updates that can be deployed to devices in the field with the secure and robust OTA update mechanism included in VigiShield. 

With security built-in, device manufacturers can focus more on innovation during the product development process and get to market faster. Schedule a VigiShield consultation today.

 

 

VigiShield Secure By Design for Yocto

VigiShield Secure By Design for Yocto

Overview

The Yocto Project is a popular choice for creating custom Linux distributions for IoT devices. When creating a custom Linux distribution one of the key challenges faced by device manufacturers is the time and expertise required to secure the distribution. Having helped multiple device manufacturers through this journey, Timesys now offers a standard product called “VigiShield Secure by Design.”  This is available as a yocto layer (meta-vigishield) that delivers out-of-the box security to your Yocto-based Linux distribution. VigiShield implements the OS security features covering various security standards such as NISTIR 8259A, ETSI EN 303 and has obtained a PSA Level 1 certification through an independent security lab evaluation. By leveraging VigiShield, device manufacturers can now focus on their value-added applications instead of worrying about security.

 

Security Features Implemented by VigiShield

 

Secure boot and chain of trust

A key aspect of security is ensuring only authentic software runs on the device (i.e. malicious software is prevented from running). To achieve this, each piece of software is digitally signed at the time of building the software and the signature is verified on the device before executing that piece of software. If the signature check fails, the software is not allowed to run. The authentication of the first piece of user software i.e. bootloader is referred to as “Secure boot” and extending the authentication to all other software including user space applications is referred to as “Chain of Trust.”
Note: To achieve secure boot, there is a dependency on the hardware needing to support it.

VigiShield supports secure boot and chain of trust on various hardware platforms. Taking the NXP i.MX8 processor series as an example, VigiShield integrates and configures the code signing tools provided by NXP to create signed bootloader images as part of the build. All pieces of software that are loaded after the bootloader are signed and verified using open source technologies. For example, the kernel, device tree and initramfs (optional) are all bundled in as a signed fitImage and verified by the bootloader. Similarly a signed read-only root filesystem is authenticated using the Linux kernel dm-verity mechanism.

Resources: Secure boot and chain of trust blog, Faster secure boot blog. Webinar

 

Secure Storage

Secure storage ensures data confidentiality (e.g: keeps customer/application data private), prevents IP-theft, and prevents cloning/counterfeiting of the device. Data is secured by means of encryption and the encryption key is typically protected by a hardware specific mechanism.

VigiShield provides full disk encryption using the Linux dm-crypt mechanism and the encryption key is kept confidential by a hardware/processor root of trust (hardware unique key). For example, the VigiShield implementation on the i.MX8 processors achieves this by creating an encrypted blob which is protected by a unique master key residing in the one-time programmable fuses of the processor. Only authorized software can request that the processor perform decryption through the hardware cryptography engine.

Resources: Encrypted Data storage blog

 

Secure updates

Software/firmware updates are required in order to provide security or feature updates to devices. Secure Updates ensures that only authentic and legitimate firmware is  updated on the device.

VigiShield provides a secure and robust over-the-air (OTA) solution. This includes OTA server authentication using certificates, secure downloads over TLS1.3, installation of signed / authenticated images using Public-key cryptography (PKCS #1.5 signature validation, using 2048 bits RSA key with SHA256) and prevention of unauthorized rollback (anti-rollback) of images. Apart from authenticating the firmware update bundle, the individual images (e.g: bootloader, kernel etc.) are authenticated as part of secure boot as described in the previous section.

VigiShield uses the SWUpdate open source solution for OTA updates with an A/B update scheme. However, through our VigiShield professional services team, we can customize OTA updates to use other solutions such as OS-Tree, Mender and RAUC; all with the same set of security features.

Resources: Designing OTA Updates webinar

 

Secure communication

Secure communication protects the data-in-transit when the device communicates with external devices (e.g: cloud servers, remote sensors, etc). Secure communication is achieved using authentication between all involved parties and encrypting the data transmitted between them.

VigiShield secures external communication to/from the device by using secure protocols such as TLS1.3, which are implemented as part of open source libraries such as openSSL. The certificates/keys used to authenticate any external communication are stored in the encrypted and authenticated secure storage, to avoid any potential tampering. VigiShield further reduces the attack surface by disabling many known weak/insecure ciphers in openSSL.

 

Security audit logs

Security audit logs record any runtime security violations/breaches on the target system. Security events such as unauthorized access to sensitive files, opening of network connections, failed login etc. are typically logged to an access controlled, integrity protected log file for further investigation.

VigiShield supports logging security violations on the target system using the Linux audit framework and open source auditd user space utility. The log file access is restricted to root users and any unauthorized access to the logs are also recorded. VigiShield comes with baseline logging capability and can be customized to monitor specific security events based on end user requirements.

Resources: Logging with auditd blog

 

Secured ports

Incorporating hardware security is critical to mitigate physical attacks on the device. VigiShield can be easily configured to disable and/or password protect interfaces such as serial port and JTAG.

 

Hardening

Security hardening is the act of reducing the attack surface of the device and making the device more difficult to hack. VigiShield implements a variety of system hardening techniques.

  • Toolchain hardening: The toolchain used to build the target device binaries has security flags to make exploiting vulnerabilities difficult. VigiShield enables toolchain options such as stack protection, buffer overflow checks, position independent executables etc. 
  • Kernel hardening: The Linux kernel hardening is a broad topic since there are a myriad of security features, hardening options, and mitigations for known exploits that can be enabled via the Linux kernel configuration. VigiShield provides a detailed kernel hardening report of the recommended hardening options based on your hardware architecture and kernel version; along with a base set of options pre-enabled.
    Resources: Kernel hardening blog, webinar
  • U-boot hardening: VigiShield can help mitigate a wide range of threats such as tampering u-boot environment variables, overriding boot commands, access of serial console, etc.
    Resources: Securing U-Boot blog
  • Userspace hardening: If software such as dropbear/openssh are included on the target device, VigiShield hardens the software by disabling root access, enforces key based authentication instead of passwords, etc.
  • Best practice cryptography: VigiShield disables a wide range of legacy / insecure OpenSSL ciphers by default.
  • Access control: Vigishield is configured to have no root login or default hardcoded passwords. As part of VigiShield, a discretionary access control report is generated detailing the permissions of each file in the filesystem. For example, details on the actions that users and groups can perform on a file: read, write, execute.
    Resources: Discretionary Access Control blog

 

SBOM and vulnerability report

Software Bill of Materials (SBOM) is an essential component for performing vulnerability or license analysis, both of which play an important role in software supply chain risk management.

VigiShield includes SBOM generation (SPDX), vulnerability (CVE) monitoring and management as part of Timesys Vigiles free tier offering.

Resources: Vigiles SBOM and vulnerability management

 

VigiShield Add-ons

Yocto/BSP and Security Customizations
Apart from the standard PSA certified VigiShield offering, we provide customizations as part of our Professional Services which covers hardware enablement, Yocto customizations, custom security feature implementations, integration with device management / IoT cloud services, and more!

Security Training
Whether you are new to security, looking for consultation to refine your security requirements or help integrating our solutions into your processes; we can help with our customized security training offerings.

Secure manufacturing assistance
We have expertise in developing the manufacturing tooling required for secure software programming and provisioning. We can help integrate your custom or 3rd party solutions for securely storing device certificates.

Trusted Applications for Secure OS
For customers seeking enhanced security, we have expertise in implementing “trusted applications” that can be deployed on a secure OS (e.g: OP-TEE, Trusty, etc) running on a trusted execution environment.
Resources: Trusted Software with OP-TEE blog, webinar

Long Term Linux OS security and maintenance
Timesys Linux OS/BSP Maintenance subscription service provides long-term security updates and maintenance of your Linux OS. Using this service, device manufacturers can rely on timely security updates that can be deployed to devices in the field with the secure and robust OTA update mechanism included in VigiShield.

 

Conclusion

The DIY route of securing a Yocto-based distribution is a tedious process and adversely affects the time-to-market for device manufacturers. Worse yet, without the right expertise and independent 3rd-party evaluation, devices might still be at risk. By leveraging off-the-shelf, security certified products such as VigiShield, device manufacturers can now bring more secure devices to the market faster and at a lower cost. To learn more, schedule a free security consultation.

Securing U-Boot: A Guide to Mitigating Common Attack Vectors

Securing U-Boot: A Guide to Mitigating Common Attack Vectors

Overview

Many embedded systems implementing software authentication (secure boot and chain of trust) use U-Boot as their bootloader. Making sure this bootloader is properly secured so that someone cannot bypass your chain of trust and boot unauthenticated software is very important. We have commonly seen field-deployed embedded systems with secure boot setups which fail to mitigate against the direct execution of unauthenticated software from U-Boot. To help prevent these sorts of attacks, we suggest considering three main categories of attack surface reduction: environmental tampering protection, software authentication, and command line access limitation.

If you are new to investigating and understanding these issues, feel free to skim over some of the more verbose example blocks and patchset information. These are not strictly necessary to gain an initial understanding.

 

Software Authentication and Signing

First and foremost, you’ll want to make sure your first bootloader (i.e. U-Boot) is signed and being authenticated by your processor’s ROM through whatever mechanism is provided. On NXP processors, this is called High Assurance Boot (HAB). We’re going to jump ahead and assume this has been completed. Once completed, we can focus on creating a complete chain of trust for the following boot stages.

Now, once this signed U-Boot has started execution, we’ll need U-Boot to properly check that any following stages are signed correctly before proceeding to boot into them. This can be processor dependent, wherein some silicon vendors provide a ROM-based API for authenticating signed binaries. Those specific mechanisms (e.g. NXP AHAB containerization) are out of the scope of this article, so we’ll assume a common approach is being used. We tend to use a Flattened Image Tree (FIT) to bundle together the following boot stages. U-Boot then has built in mainline mechanisms which can be used to sign and authenticate the entire FIT bundle before booting.

A FIT image is defined by its Image Tree Source (ITS) file. This ITS file tends to look something like:

/dts-v1/;

/ {
        description = "U-Boot fitImage for Poky (Yocto Project Reference Distro)/1.0/imx8qxp-b0-mek";
        #address-cells = <1>;

        images {
                kernel-1 {
                        description = "Linux kernel";
                        data = /incbin/("Image");
                        type = "kernel";
                        arch = "arm64";
                        os = "linux";
                        compression = "none";
                        load = <0x80200000>;
                        entry = <0x80200000>;
                        hash-1 {
                                algo = "sha1";
                        };
                };
                fdt-1 {
                        description = "Flattened Device Tree blob";
                        data = /incbin/("imx8qxp-mek-ov5640-rpmsg.dtb");
                        type = "flat_dt";
                        arch = "arm64";
                        compression = "none";
                        load = <0x83000000>;
                        entry = <0x83000000>;
                        hash-1 {
                                algo = "sha1";
                        };
                };
                ramdisk-1 {
                        description = "timesys-initramfs-imx8qxp-b0-mek.cpio.gz";
                        data = /incbin/("timesys-initramfs-imx8qxp-b0-mek.cpio.gz");
                        type = "ramdisk";
                        arch = "arm64";
                        os = "linux";
                        compression = "gzip";
                        load = <0xd0000000>;
                        entry = <0xd0000000>;
                        hash-1 {
                                algo = "sha1";
                        };
                };
        };

        configurations {
                default = "conf-1";
                conf-1 {
                        description = "Linux kernel, FDT blob, ramdisk";
                        kernel = "kernel-1";
                        fdt = "fdt-1";
                        ramdisk = "ramdisk-1";
                        hash-1 {
                                algo = "sha1";
                        };
                };
        };
};

In here, we see there is a main configuration node which contains entries for the Linux kernel image, device tree blob, and initramfs.

This ITS is then compiled into the fitImage file via uboot-mkimage:

uboot-mkimage -D "-I dts -O dtb -p 2000" -f image.its fitImage

Then, we can sign this fitImage file with:

uboot-mkimage -D "-I dts -O dtb -p 2000" -F -k "/key_directory" -r fitImage

Where /key_directory is a directory which contains your RSA key pair for signing the fitImage. These can be generated using OpenSSL by:

cd /key_directory
openssl genpkey -algorithm RSA -out dev.key -pkeyopt rsa_keygen_bits:2048 -pkeyopt rsa_keygen_pubexp:65537
openssl req -batch -new -x509 -key dev.key -out dev.crt

You’ll also need U-Boot to be set up for FIT image booting and signing, otherwise uboot-mkimage will throw an error:

#ifdef CONFIG_FIT_SIGNATURE
	fprintf(stderr,
		"Signing / verified boot options: [-k keydir] [-K dtb] [ -c <comment>] [-p addr] [-r] ...\n"
		"          -k => set directory containing private keys\n"
		"          -K => write public keys to this .dtb file\n"
		"          -G => use this signing key (in lieu of -k)\n"
		"          -c => add comment in signature node\n"
		"          -F => re-sign existing FIT image\n"
		"          -p => place external data at a static position\n"
		"          -r => mark keys used as 'required' in dtb\n"
		"          -N => openssl engine to use for signing\n");
#else
	fprintf(stderr,
		"Signing / verified boot not supported (CONFIG_FIT_SIGNATURE undefined)\n");
#endif

To do this, you can set these config options in U-Boot:

CONFIG_SECURE_BOOT=y
CONFIG_FIT=y
CONFIG_FIT_SIGNATURE=y
CONFIG_FIT_VERBOSE=y
CONFIG_DEFAULT_FDT_FILE="u-boot-signed-devicetree.dtb"

Then, once U-Boot is compiled, we can add the public key into U-Boot’s compiled DTB. To do that, we’ll first need to create a dummy FIT image using this dummy ITS file:

/dts-v1/;

/ {
    description = "U-Boot Simple fitImage";
    #address-cells = <1>;

    images {
        dummy-1 {
            description = "dummy";
            data = /incbin/("empty_placeholder_file");
            type = "kernel";
            arch = "arm";
            os = "linux";
            compression = "none";
            load = <0x80008000>;
            entry = <0x80008000>;
            hash-1 {
                algo = "sha1";
            };
        };
    };

    configurations {
        default = "conf-1";
        conf-1 {
            description = "dummy";
            dummy = "dummy-1";
			hash-1 {
				algo = "sha1";
			};
			signature-1 {
				algo = "sha1,rsa2048";
				key-name-hint = "dev";
				sign-images = "dummy";
			};
        };
    };
};

Note: it is important to have the signature-1 node inside the configuration node here, instead of putting signature nodes on each image piece. With this layout, all of the associated image hashes are contained within the configuration signature, so someone cannot swap in different signed images and perform a version rollback attack by changing one image/binary in your FIT bundle separately.

We then package this dummy ITS into a file named simpleFitImage:

uboot-mkimage -D "-I dts -O dtb -p 2000" -f simple.its simpleFitImage

Then we can sign our actual U-Boot DTB, while using the simpleFitImage as a reference (for the signature-1 node):

uboot-mkimage -D "-I dts -O dtb -p 2000" -F -k "/key_directory" -K imx8.dtb -r simpleFitImage

If we decompile the resulting, signed DTB:

dtc -I dtb -O dts -o imx8_decompiled.dts imx8.dtb

Inside, we now see this node:

signature {
    key-dev {
        required = "conf";
        algo = "sha1,rsa2048";
        rsa,r-squared = <0x26b42979 0xf91dba64 0x11c5cab5 0x8273b76e 0xdc7562f3 0xcdd3742c ... etc>;
        rsa,modulus = <0xb4aac057 0xbddc7ce8 0x3c4d48b3 0x622d6e95 0xb09eb6c6 0xafc3c9d7 ... etc>;
        rsa,exponent = <0x00 0x10001>;
        rsa,n0-inverse = <0x5a7322b9>;
        rsa,num-bits = <0x800>;
        key-name-hint = "dev";
    };
};

Now, when booting via bootm, if the signature is bad/missing you should see an error similar to this:

## Loading kernel from FIT Image at ... ...
   Using 'conf@1' configuration
   Verifying Hash Integrity ... sha1,rsa2048:dev_bad- Failed to verify required signature 'key-dev'
Bad Data Hash

If there are no errors, each node should show a correct signature check similar to this:

## Loading kernel from FIT Image at ... ...
   Using 'conf-1' configuration
   Verifying Hash Integrity ... sha1,rsa2048:dev+ OK
   Trying 'kernel-1' kernel subimage

U-Boot’s Environment Pitfalls

This is one of the reasons U-Boot is so commonly liked. The entire boot flow is setup and controlled through environmental parameters. When U-Boot boots, it runs the specified set of commands listed inside the bootcmd parameter. So, by modifiying this, we can easily boot into alternate images or quickly make minor modifications to the boot process. When it comes to security, this becomes a double-edged sword. In a field-deployed embedded system, we do not want someone to be able to tamper with the environment and arbitrarily execute whatever U-Boot commands they would like.

So, how can we prevent this? Unfortunately, U-Boot does not offer an easy way to sign/authenticate or encrypt the environment. It’s usually easier to disable the other exploitable paths someone may use.

To start we can limit access to the U-Boot command line interface (CLI) via:

  • Disabling/Password Protecting Autoboot Interrupt
  • Disabling the serial console

Once those two are done, if your U-Boot environment never needs to change, you can make sure it is not stored in nonvolatile memory by setting this in your U-Boot configuration:

CONFIG_ENV_IS_NOWHERE=y

With these measures in place, there’s little left for an attacker to turn to when trying to modify your environment. They may still be able to perform a more cumbersome attack such as RAM modification/injection via JTAG or another means, but these attack vectors should also be limited (i.e. Make sure you disable JTAG in accordance if your processor’s reference manual).

If you do need your environment to remain modifiable due to something such as software update management, this becomes a bit trickier. With the environment stored in a nonvolatile device, you’re now subject to offline tampering of the storage device. This can still be mitigated against quite well by disabling any dangerous U-Boot commands. If all of the enabled U-Boot commands are benign (i.e. properly require signed software and cannot be used to modify memory), then you’re safe from environmental tampering too.

Now, with a cursory understanding of these environment-related pitfalls, let’s look into securing them.

 

Autoboot Interruption

I’m sure you’ve seen it before… When U-Boot starts, the serial console displays a 3 second countdown. If you enter a keystroke, you’re taken to U-Boot’s command line interface. So, if this is left enabled, anyone with access to your serial pins can easily stop U-Boot’s autoboot sequence and tamper with everything that’s left available to them (environment modification, unprotected boot commands, etc).

To disable autoboot interruption entirely, you’ll want to set this in your U-Boot configuration:

CONFIG_BOOTDELAY=-2

Note: This does not entirely prevent command prompt access. If a Linux/OS boot fails, U-boot may fall into the CLI. This is why it is still important to disable the serial console entirely. Or, at least patch U-Boot so that it will not enter the CLI after a failed autoboot sequence (appending the reset command to the end of your boot sequence can sometimes work as a fall through fail-safe).

 

Autoboot Password Protection

If disabling autoboot interruption is too extreme for your use case, you can add a sha256-backed interruption password. Be sure to make this string as long as possible, to avoid brute forcing (20+ characters!).

This can be performed by enabling the following in your U-Boot configuration:

 

CONFIG_AUTOBOOT_KEYED=y
CONFIG_AUTOBOOT_ENCRYPTION=y
CONFIG_AUTOBOOT_STOP_STR_SHA256="..."

As previously mentioned, if a Linux/OS boot fails, U-boot may still open up the CLI. So, this does not necessarily offer full protection.

 

Command Line Disablement

You can also consider disabling the U-Boot command line by turning this off:

# CONFIG_CMD_CMDLINE is not set

Most customers still want some form of the CLI left enabled for configuration and software update handling, so this is not an option we see commonly used. With this disabled, when a command is entered/run, U-Boot falls into this block:

__weak int board_run_command(const char *cmdline)
{
	printf("## Commands are disabled. Please enable CONFIG_CMDLINE.\n");

	return 1;
}

Console Disablement

To entirely disable the U-Boot console, append this to your defconfig:

CONFIG_DISABLE_CONSOLE=y

You’ll then need to set this in arch_cpu_init(or another corresponding function) to turn it on:

gd->flags |= GD_FLG_SILENT | GD_FLG_DISABLE_CONSOLE;

So for example,

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Tue, 12 Apr 2022 19:04:34 -0400
Subject: [PATCH] Disable u-boot console

---
 arch/arm/cpu/armv7/sc58x/soc.c | 2 ++
 configs/sc589-ezkit_defconfig  | 1 +
 2 files changed, 3 insertions(+)

diff --git a/arch/arm/cpu/armv7/sc58x/soc.c b/arch/arm/cpu/armv7/sc58x/soc.c
index c4a4845114..87fa369de4 100644
--- a/arch/arm/cpu/armv7/sc58x/soc.c
+++ b/arch/arm/cpu/armv7/sc58x/soc.c
@@ -34,6 +34,8 @@ void v7_outer_cache_enable(void)
 
 int arch_cpu_init(void)
 {
+     gd->flags |= GD_FLG_SILENT | GD_FLG_DISABLE_CONSOLE;
+
 #ifdef CONFIG_DEBUG_EARLY_SERIAL
     return serial_early_init();
 #else
diff --git a/configs/sc589-ezkit_defconfig b/configs/sc589-ezkit_defconfig
index 7b978aeded..4da70a8f8f 100644
--- a/configs/sc589-ezkit_defconfig
+++ b/configs/sc589-ezkit_defconfig
@@ -27,3 +27,4 @@ CONFIG_SPI=y
 CONFIG_USB=y
 CONFIG_USB_MUSB_HCD=y
 CONFIG_OF_LIBFDT=y
+CONFIG_DISABLE_CONSOLE=y

Kernel Command Line Parameters

You should also make sure that the kernel command line parameters which U-Boot passes to the kernel (bootargs in U-Boot’s environment) cannot be modified to anything unexpected. If modification is allowed, someone can easily pass an unexpected argument to a driver or even set init= or rdinit= to /bin/sh to gain access to a shell.

I like to do this by checking that the bootargs match an expected string. So, if we have two sets of acceptable boot arguments, we might do:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 1 Nov 2021 16:39:01 -0400
Subject: [PATCH] Add in basic tamper detection for u-boot's bootargs variable,
 so that someone can not modify kernel boot arguments

---
 common/bootm.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/common/bootm.c b/common/bootm.c
index db4362a643..ca913ce945 100644
--- a/common/bootm.c
+++ b/common/bootm.c
@@ -524,6 +524,14 @@ int do_bootm_states(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[],
     boot_os_fn *boot_fn;
     ulong iflag = 0;
     int ret = 0, need_boot_fn;
+    static char* bootargs_a = "root=/dev/mmcblk2p2 console=ttymxc0,115200 rootwait rw";
+    static char* bootargs_b = "root=/dev/mmcblk2p3 console=ttymxc0,115200 rootwait rw";
+    char* bootargs = env_get("bootargs");
+
+    if( (strcmp(bootargs_a, bootargs) != 0) && (strcmp(bootargs_b, bootargs) != 0) ){
+        printf("\nDetected tampering of bootargs: blocking...\n");
+        while(1);
+    }
 
     images->state |= states;

Concerning Commands

Here’s a non-comprehensive list of U-Boot configuration settings which could be disabled to reduce your attack surface:

U-Boot Configuration Description
CONFIG_CMD_GO This is the equivalent of an assembly jump/branch operation. It allows an attacker to change execution to any arbitrary address.
CONFIG_CMD_BOOTI
CONFIG_CMD_BOOTZ
CONFIG_CMD_BOOTEFI
CONFIG_CMD_ELF
CONFIG_CMD_ABOOTIMG
CONFIG_CMD_ADTIMG
Assuming we’re using the signed FIT image strategy, these should be disabled (as FIT uses CMD_BOOTM only). These commands open alternate boot paths (booti, bootz, bootelf, bootvx, bootefi, android boot images)
CONFIG_CMD_MEMORY Enables memory dumping (md), memory writing (mw), and other memory operations
CONFIG_CMD_SMC
CONFIG_CMD_HVC
Enables injecting secure monitor calls. This could be concerning if you’re using ATF-A + OP-TEE
CONFIG_CMD_NET
CONFIG_CMD_USB
CONFIG_USB_STORAGE
CONFIG_CMD_BOOTP
CONFIG_CMD_TFTPBOOT
These can be used to externally load images from USB devices, network transfers, etc
CONFIG_CMD_REMOTEPROC
CONFIG_CMD_ICC
CONFIG_CMD_FPGA
Enables controlling secondary cores and FPGAs
CONFIG_CMD_IMI Enables dumping image info (iminfo)
CONFIG_CMD_I2C
CONFIG_CMD_SPI
Leaving this enabled may allow an attacker to modify your I2C/SPI/etc devices. This could give access to sorts of devices, including power management units.
CONFIG_CMD_DIAG
CONFIG_CMD_IRQ
CONFIG_CMD_BDI
and more
I would classify these as unnecessary information leakage. While not explicitly bad, they may give an attacker information you don’t want them to have (such as stack pointer locations, memory sizes, etc).

Again, this is not a complete list. In fact, it ultimately may be better to create a whitelist of known, acceptable commands and blacklist everything else. If you don’t need a command, disable it!

Also, given we’re booting via a signed FIT image, this uses the bootm command. I like to further secure this command by deleting any alternate boot paths from the code (in case someone mistakenly leaves the associated CONFIG options enabled).

To make sure bootm requires an authenticated FIT image, I do the following:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Wed, 25 Aug 2021 07:38:05 -0400
Subject: [PATCH] Make FIT the only bootm option

---
 cmd/bootm.c | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/cmd/bootm.c b/cmd/bootm.c
index 03ea3b8998..d164f71572 100644
--- a/cmd/bootm.c
+++ b/cmd/bootm.c
@@ -163,17 +163,8 @@ int do_bootm(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
 #else
 
     switch (genimg_get_format((const void *)image_load_addr)) {
-#if defined(CONFIG_LEGACY_IMAGE_FORMAT)
-    case IMAGE_FORMAT_LEGACY:
-        if (authenticate_image(image_load_addr,
-            image_get_image_size((image_header_t *)image_load_addr)) != 0) {
-            printf("Authenticate uImage Fail, Please check\n");
-            return 1;
-        }
-        break;
-#endif
-#ifdef CONFIG_ANDROID_BOOT_IMAGE
-    case IMAGE_FORMAT_ANDROID:
+#ifdef CONFIG_FIT
+    case IMAGE_FORMAT_FIT:
         /* Do this authentication in boota command */
         break;
 #endif

In newer versions of U-Boot, most this logic has moved to common/bootm.c, under boot_get_kernel() and bootm_find_os().

 

Real World Example

Our customers commonly send us boards with field deployment issues. A lot of times, these boards have been permanently secured using some form of secure boot technology. In these cases, I try to avoid asking for possession of the associated private keys, as that opens up more attack vectors for the customer (if someone were to compromise one of my machines/etc). So, I’m sometimes left in a position where it would be nice to run a new software image, but I cannot do so because it’s unsigned. Sometimes we have to wait for the customer to sign the new software image for us… and sometimes, we find another way.

Here’s a recent board I received:

U-Boot dub-2017.03-r11.2+gf9055c2 (Mar 14 2022 - 12:42:45 +0000)

CPU:   Freescale i.MX6DL rev1.3 at 792MHz
CPU:   Industrial temperature grade (-40C to 105C) at 35C
Reset cause: POR
I2C:   ready
DRAM:  512 MiB
MMC:   FSL_SDHC: 0, FSL_SDHC: 1
In:    serial
Out:   serial
Err:   serial
Model: ...
Board: ...
Boot device: MMC4
PMIC:  DA9063, Device: 0x61, Variant: 0x60, Customer: 0x00, Config: 0x56
Net:   Board Net Initialization Failed
No ethernet found.
Hit any key to stop autoboot:  0 
=> 

It has an external SD card slot… so let’s see if we could easily change the MMC boot device from its internal eMMC to the external SD card.

Can we switch between devices?

=> mmc dev 0
switch to partitions #0, OK
mmc0(part 0) is current device
=> mmc dev 1
switch to partitions #0, OK
mmc1 is current device

Yep, that worked, device 1 is our SD card.

What about changing the boot device?

=> printenv bootcmd
bootcmd=if run loadscript; then setexpr bs_ivt_offset ${filesize} - 0x4020;if hab_auth_img ${loadaddr} ${bs_ivt_offset}; then source ${loadaddr};fi; fi;
=> printenv loadscript
loadscript=load mmc ${mmcbootdev}:${mmcpart} ${loadaddr} ${script}
=> printenv mmcbootdev
mmcbootdev=0
=> editenv mmcbootdev
edit: 1
## Error: Can't overwrite "mmcbootdev"
## Error inserting "mmcbootdev" variable, errno=1

We can see U-Boot appears to be hardened against changing the MMC boot device. Interesting.

What if we just directly run a modified boot command instead of modifying the environment? I happen to have a kernel image and device tree built for this board on my SD card already:

=> fatls mmc 1:1
  5710976   zImage-imx6.bin
    51503   zImage-imx6dl-imx6.dtb
     2430   boot.scr 
=> fatload mmc 1:1 ${loadaddr} zImage-imx6.bin
reading zImage-imx6.bin
5710976 bytes read in 289 ms (18.8 MiB/s)
=> printenv loadaddr
loadaddr=0x12000000
=> fatload mmc 1:1 0x18000000 zimage-imx6dl.dtb
reading zimage-imx6dl-ccimx6-iotest.dtb
51503 bytes read in 31 ms (1.6 MiB/s)
=> bootz 0x12000000 - 0x18000000                          
Kernel image @ 0x12000000 [ 0x000000 - 0x572480 ]
## Flattened Device Tree blob at 18000000
   Booting using the fdt blob at 0x18000000
   Authenticating image from DDR location 0x18000000... FAILED!
hab entry function fail

Secure boot enabled

We can see U-Boot also appears to be hardened against booting an unsigned kernel image. What else can we try?

=> go
go - start application at address 'addr'

Usage:
go addr [arg ...]
    - start application at address 'addr'
      passing 'arg' as arguments

go was left enabled on this board and most likely does not contain any signature checking.

Let’s try to jump into a custom built U-Boot version using go. First, let’s dump some memory info.

=> bdinfo
arch_number = 0x00001323
boot_params = 0x10000100
DRAM bank   = 0x00000000
-> start    = 0x10000000
-> size     = 0x20000000
current eth = unknown
ip_addr     = 192.168.42.30
baudrate    = 115200 bps
TLB addr    = 0x2FFF0000
relocaddr   = 0x2FF4E000
reloc off   = 0x1874E000
irq_sp      = 0x2EF3DBA0
sp start    = 0x2EF3DB90
Early malloc usage: ec / 400

Okay, so they have 512MB of RAM ranging from 0x10000000 to 0x30000000. We can see most of U-Boot has also been relocated to the upper region of memory. This is important to know, as booting another instance of U-Boot requires not trampling over the current stack/bss/etc.

Lets trick U-Boot into thinking it only has 256MB of RAM and rearrange some addresses so the new instance of U-Boot will not overlap any of these regions:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Thu, 14 Apr 2022 14:28:59 -0400
Subject: [PATCH] Allow U-Boot to boot from another currently running version
 of U-Boot via 'go'

---
 arch/arm/imx-common/hab.c       | 4 ++++
 board/vendor/imx6/imx6.c      | 2 +-
 common/board_f.c                | 7 ++++---
 common/board_r.c                | 4 ++++
 include/configs/imx6_common.h | 5 ++++-
 5 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/arch/arm/imx-common/hab.c b/arch/arm/imx-common/hab.c
index fc970641d1..ec699967c8 100644
--- a/arch/arm/imx-common/hab.c
+++ b/arch/arm/imx-common/hab.c
@@ -629,6 +629,10 @@ static int validate_ivt(int ivt_offset, ulong start_addr)
 
 uint32_t authenticate_image(uint32_t ddr_start, uint32_t image_size)
 {
+    //Disable HAB security!    
+    //Always return true, even for unauthenticated images
+    return 1;
+
     ulong load_addr = 0;
     size_t bytes;
     ptrdiff_t ivt_offset = 0;
diff --git a/board/vendor/imx6/imx6.c b/board/vendor/imx6/imx6.c
index 3ec90f0369..58d580eb21 100644
--- a/board/vendor/imx6/imx6.c
+++ b/board/vendor/imx6/imx6.c
@@ -252,7 +252,7 @@ static struct imx6_variant imx6_variants[] = {
 /* 0x13 - 55001818-19 */
     {
         IMX6DL,
-        MEM_512MB,
+        MEM_256MB,




diff --git a/common/board_f.c b/common/board_f.c
index 7e40a35bb1..1e87cd1b89 100644
--- a/common/board_f.c
+++ b/common/board_f.c
@@ -359,13 +359,14 @@ static int setup_dest_addr(void)
      * thie mechanism. If memory is split into banks, addresses
      * need to be calculated.
      */
-    gd->ram_size = board_reserve_ram_top(gd->ram_size);
+    //Force the RAM size to 256MB
+    gd->ram_size = 0x10000000;
 
 #ifdef CONFIG_SYS_SDRAM_BASE
     gd->ram_top = CONFIG_SYS_SDRAM_BASE;
 #endif
-    gd->ram_top += get_effective_memsize();
-    gd->ram_top = board_get_usable_ram_top(gd->mon_len);
+    //Force the top of RAM to be at 0x20000000 instead of 0x30000000
+    gd->ram_top = 0x20000000;
     gd->relocaddr = gd->ram_top;
     debug("Ram top: %08lX\n", (ulong)gd->ram_top);
 #if defined(CONFIG_MP) && (defined(CONFIG_MPC86xx) || defined(CONFIG_E500))
diff --git a/common/board_r.c b/common/board_r.c
index 2b14e3d9f8..14747f1b4b 100644
--- a/common/board_r.c
+++ b/common/board_r.c
@@ -487,6 +487,10 @@ static int should_load_env(void)
 
 static int initr_env(void)
 {
+    //Always use the default environment -- don't read from nonvolatile storage
+    set_default_env(NULL);
+    return 0;
+
     /* initialize environment */
     if (should_load_env())
         env_relocate();
diff --git a/include/configs/imx6_common.h b/include/configs/imx6_common.h
index 7061a473d8..c21b0926ce 100644
--- a/include/configs/imx6_common.h
+++ b/include/configs/imx6_common.h
@@ -41,11 +41,14 @@
 /*
  * RAM
  */
+//Limit the amount of memory we're allowed to map to 256MB
+#define CONFIG_MAX_MEM_MAPPED       0x10000000
 #define CONFIG_LOADADDR             0x12000000
 #define CONFIG_SYS_LOAD_ADDR        CONFIG_LOADADDR
 #define CONFIG_DIGI_LZIPADDR        0x15000000
 #define CONFIG_DIGI_UPDATE_ADDR     CONFIG_LOADADDR
-#define CONFIG_SYS_TEXT_BASE        0x17800000
+//Move the starting text base to a lower region
+#define CONFIG_SYS_TEXT_BASE        0x12800000
 /* RAM memory reserved for U-Boot, stack, malloc pool... */
 #define CONFIG_UBOOT_RESERVED       (10 * 1024 * 1024)
 /* Size of malloc() pool */

Now, we build and store this U-Boot image on our SD card at an offset of 0x1000. Along with our custom Linux kernel, DTB, and file system.

U-Boot 2017.03-r11.2+gf9055c2 (Mar 14 2022 - 12:42:45 +0000)
...
=> mmc dev 1; mmc read 0x12800000 8 1000; go 0x12800000

U-Boot 2017.03-r2.3+g2002510765 (Mar 31 2022 - 22:52:18 +0000)
...
=> 

We’ve done it! We’re running an unsigned version of U-Boot!

Finishing the chain, we can boot all the way into Linux via:

U-Boot 2017.03-r2.3+g2002510765 (Mar 31 2022 - 22:52:18 +0000)
...
=> setenv mmc dev 1; setenv mmcroot /dev/mmcblk1p2; run mmcboot

Yocto 2.4-r3 imx6 /dev/ttymxc3
imx6 login: root

root@imx6:~# whoami
root

Command Whitelisting

So, what if you had a known subset of commands which were considered safe? How would you create a whitelist within U-Boot for this? A simple whitelist can be added to cmd_call() via something like this:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 15:56:23 -0400
Subject: [PATCH] Lockdown U-boot to allow only a whitelist of commands to be
 used

---
 common/command.c                  | ...

diff --git a/common/command.c b/common/command.c
index 0d8bf244be..125dabd162 100644
--- a/common/command.c
+++ b/common/command.c
@@ -13,6 +13,7 @@
 #include <console.h>
 #include <env.h>
 #include <linux/ctype.h>
+#include <u-boot/sha256.h>
 
 /*
  * Use puts() instead of printf() to avoid printf buffer overflow
@@ -556,6 +557,42 @@ int cmd_discard_repeatable(cmd_tbl_t *cmdtp, int flag, int argc,
     return cmdtp->cmd_rep(cmdtp, flag, argc, argv, &repeatable);
 }
 
+//Create a whitelist of commands that can always be run inside U-Boot
+static char * customer_whitelist_table[] = {
+    "run",
+    "echo",
+    "bmode",
+    "fastboot",
+    "setenv",
+    "saveenv",
+    "mmc",
+    "bootm",
+    "ext4load",
+    "customer_authenticate",
+};
+
+#define CUSTOMER_WHITELIST_LENGTH ARRAY_SIZE(customer_whitelist_table)
+
+extern bool imx_hab_is_enabled(void);
+
+//Check if the current function name is within the whitelist
+static int customer_cmd_whitelist(char * name)
+{
+    if (imx_hab_is_enabled()){
+        for(int i = 0; i < CUSTOMER_WHITELIST_LENGTH; i++)
+        {
+            if(strcasecmp(customer_whitelist_table[i], name) == 0)
+            {
+                return 0;
+            }
+        }
+        printf("CUSTOMER Error: Attempted to run %s while unauthenticated\r\n", name);
+        return -1;
+    }else{
+        return 0;
+    }
+}
+
 /**
  * Call a command function. This should be the only route in U-Boot to call
  * a command, so that we can track whether we are waiting for input or
@@ -571,6 +608,13 @@ int cmd_discard_repeatable(cmd_tbl_t *cmdtp, int flag, int argc,
 static int cmd_call(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[],
             int *repeatable)
 {
+    //Only execute commands if they're in the whitelist or we're authenticated
+    if(customer_authentication != 1){
+        if(customer_cmd_whitelist(cmdtp->name) != 0){
+            return -1;
+        }
+    }
+
     int result;
 
     result = cmdtp->cmd_rep(cmdtp, flag, argc, argv, repeatable);

Note: imx_hab_is_enabled() is checking if secure boot is enabled on an NXP processor and will vary for you. Also, customer_authentication is another command that I will discuss more below. It’s used to give our customers the ability to disable the whitelist if a password is entered.

In cases where our customers do not want the entire CLI disabled, this will allow them to enter the CLI and then run ‘customer_authenticate password’ in order to bypass the whitelist and unlock all of U-Boot’s commands.

To do this, we’ll first enable autoboot password interruption again:

CONFIG_AUTOBOOT_KEYED=y
CONFIG_AUTOBOOT_ENCRYPTION=y
CONFIG_AUTOBOOT_STOP_STR_SHA256="..."

We then modify the passwd_abort_sha256 function to allow us to externally hook into it by passing in a password string. This string will be what is sent in from the password portion of ‘customer_authenticate password’.

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 15:56:23 -0400
Subject: [PATCH] Modify passwd_abort_sha256 so we can pass in an arbitrary password for verification

---
 common/autoboot.c                 | ...

diff --git a/common/autoboot.c b/common/autoboot.c
index 0a59b81ae2..13ea153531 100644
--- a/common/autoboot.c
+++ b/common/autoboot.c
@@ -75,7 +75,7 @@ static int slow_equals(u8 *a, u8 *b, int len)
  * @etime: Timeout value ticks (stop when get_ticks() reachs this)
  * @return 0 if autoboot should continue, 1 if it should stop
  */
-static int passwd_abort_sha256(uint64_t etime)
+static int passwd_abort_sha256(uint64_t etime, char * password)
 {
     const char *sha_env_str = env_get("bootstopkeysha256");
     u8 sha_env[SHA256_SUM_LEN];
@@ -109,32 +109,57 @@ static int passwd_abort_sha256(uint64_t etime)
      * generate the sha256 hash upon each input character and
      * compare the value with the one saved in the environment
      */
-    do {
-        if (tstc()) {
-            /* Check for input string overflow */
-            if (presskey_len >= MAX_DELAY_STOP_STR) {
-                free(presskey);
-                free(sha);
-                return 0;
-            }
 
-            presskey[presskey_len++] = getc();
+    if(password != NULL){
+        //This adds in ability to verify an arbitrary password string
 
-            /* Calculate sha256 upon each new char */
-            hash_block(algo_name, (const void *)presskey,
-                   presskey_len, sha, &size);
+        /* Calculate sha256 upon each new char */
+        hash_block(algo_name, (const void *)password,
+               strlen(password), sha, &size);
 
-            /* And check if sha matches saved value in env */
-            if (slow_equals(sha, sha_env, SHA256_SUM_LEN))
-                abort = 1;
-        }
-    } while (!abort && get_ticks() <= etime);
+        /* And check if sha matches saved value in env */
+        if (slow_equals(sha, sha_env, SHA256_SUM_LEN))
+            abort = 1;
+    }else{
+        do {
+            if (tstc()) {
+                /* Check for input string overflow */
+                if (presskey_len >= MAX_DELAY_STOP_STR) {
+                    free(presskey);
+                    free(sha);
+                    return 0;
+                }
+
+                presskey[presskey_len++] = getc();
+
+                /* Calculate sha256 upon each new char */
+                hash_block(algo_name, (const void *)presskey,
+                       presskey_len, sha, &size);
+
+                /* And check if sha matches saved value in env */
+                if (slow_equals(sha, sha_env, SHA256_SUM_LEN))
+                    abort = 1;
+            }
+        } while (!abort && get_ticks() <= etime);
+    }
 
     free(presskey);
     free(sha);
+
+    //1 = Authentication successful
+    //0 = Authentication failed
+    customer_authentication = abort;
+
     return abort;
 }
 
+//New function to allow us to hook pre-existing password
+//verification infrastructure with a passed string pointer
+int passwd_abort_sha256_string(char * password)
+{
+    passwd_abort_sha256(0, password);
+}
+

On this NXP processor, I also modified the autoboot interruption password to only be enabled while secure boot (HAB) is enabled. So, during development, you can still easily interrupt U-Boot with a single keystroke.

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 15:56:23 -0400
Subject: [PATCH] Only check the autoboot password if HAB is enabled

 common/autoboot.c                 | ...

...

 /**
  * passwd_abort_key() - check for a key sequence to aborted booting
  *
@@ -189,6 +214,7 @@ static int passwd_abort_key(uint64_t etime)
      */
     do {
         if (tstc()) {
+            return 1; //Abort if any key is pressed (until HAB fuses are burned)
             if (presskey_len < presskey_max) {
                 presskey[presskey_len++] = getc();
             } else {
@@ -220,6 +246,8 @@ static int passwd_abort_key(uint64_t etime)
     return abort;
 }
 
+extern bool imx_hab_is_enabled(void);
+
 /***************************************************************************
  * Watch for 'delay' seconds for autoboot stop or autoboot delay string.
  * returns: 0 -  no key string, allow autoboot 1 - got key string, abort
@@ -236,9 +264,8 @@ static int abortboot_key_sequence(int bootdelay)
      */
     printf(CONFIG_AUTOBOOT_PROMPT, bootdelay);
 #  endif
-
-    if (IS_ENABLED(CONFIG_AUTOBOOT_ENCRYPTION))
-        abort = passwd_abort_sha256(etime);
+    if (imx_hab_is_enabled() && IS_ENABLED(CONFIG_AUTOBOOT_ENCRYPTION))
+        abort = passwd_abort_sha256(etime, NULL);
     else
         abort = passwd_abort_key(etime);
     if (!abort)

And finally, we can add in the customer_authenticate command via this patch:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 15:56:23 -0400
Subject: [PATCH] Add in customer_authenticate

---
 cmd/Makefile                      | ...
 cmd/customer.c                    | ...
 include/u-boot/sha256.h           | ...

diff --git a/cmd/Makefile b/cmd/Makefile
index 7c62e3becf..9a2836d8fb 100644
--- a/cmd/Makefile
+++ b/cmd/Makefile
@@ -155,6 +155,10 @@
 obj-$(CONFIG_CMD_FASTBOOT) += fastboot.o
 obj-$(CONFIG_CMD_FS_UUID) += fs_uuid.o
 
 obj-$(CONFIG_CMD_USB_MASS_STORAGE) += usb_mass_storage.o
+
+# Customer - Customer Custom Commands
+obj-y += customer.o
+
 obj-$(CONFIG_CMD_USB_SDP) += usb_gadget_sdp.o
 obj-$(CONFIG_CMD_THOR_DOWNLOAD) += thordown.o
 obj-$(CONFIG_CMD_XIMG) += ximg.o
diff --git a/cmd/customer.c b/cmd/customer.c
new file mode 100644
index 0000000000..d02e0bd4ae
--- /dev/null
+++ b/cmd/customer.c
@@ -0,0 +1,31 @@
+/*
+ * Copyright 2000-2009
+ * Wolfgang Denk, DENX Software Engineering, wd@denx.de.
+ *
+ * SPDX-License-Identifier:    GPL-2.0+
+ */
+
+#include <common.h>
+#include <command.h>
+#include <u-boot/sha256.h>
+
+uint8_t customer_authentication = 0;
+
+static int do_customer_authenticate(cmd_tbl_t *cmdtp, int flag, int argc, char * const argv[])
+{
+    if (argc > 1){
+        passwd_abort_sha256_string(argv[1]);
+        if(customer_authentication){
+            printf("Customer: Authentication Successful\r\n");
+        }else{
+            printf("Customer: Authentication Failed\r\n");
+        }
+    }
+    return 0;
+}
+
+U_BOOT_CMD(
+    customer_authenticate,    2,    1,    do_customer_authenticate,
+    "Command Authentication for Customer",
+    ""
+);

diff --git a/include/u-boot/sha256.h b/include/u-boot/sha256.h
index 6fbf542f67..6ae12193dc 100644
--- a/include/u-boot/sha256.h
+++ b/include/u-boot/sha256.h
@@ -5,6 +5,7 @@
 #define SHA256_DER_LEN    19
 
 extern const uint8_t sha256_der_prefix[];
+extern uint8_t customer_authentication;
 
 /* Reset watchdog each time we process this many bytes */
 #define CHUNKSZ_SHA256    (64 * 1024)
@@ -25,4 +26,7 @@ void sha256_csum_wd(const unsigned char *input, unsigned int ilen,
 void sha256_hmac(const unsigned char *key, int keylen,
         const unsigned char *input, unsigned int ilen,
         unsigned char *output);
+
+extern int passwd_abort_sha256_string(char * password);
+
 #endif /* _SHA256_H */

On this particular board, fastboot was left enabled as well. So, we’ll want to further lock down fastboot by incorporating our customer_authenticate mechanism:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Mon, 20 Sep 2021 16:27:05 -0400
Subject: [PATCH] Lockdown fastboot commands as well

---
 drivers/fastboot/fb_command.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/fastboot/fb_command.c b/drivers/fastboot/fb_command.c
index 3c4acfecf6..1fdb7544c0 100644
--- a/drivers/fastboot/fb_command.c
+++ b/drivers/fastboot/fb_command.c
@@ -108,6 +108,15 @@ int fastboot_handle_command(char *cmd_string, char *response)
 
     for (i = 0; i < FASTBOOT_COMMAND_COUNT; i++) {
         if (!strcmp(commands[i].command, cmd_string)) {
+
+            //If not authenticated, this disables all commands except UCmd.
+            //UCmd must remain available to allow for "ucmd customer_authenticate <password>" authentication
+            if(customer_authenticate != 1){
+                if(strcasecmp(commands[i].command, "UCmd:") != 0){
+                    break;
+                }
+            }
+
             if (commands[i].dispatch) {
                 commands[i].dispatch(cmd_parameter,
                             response);

Conclusion

I hope this post was a helpful tool for getting you started with securing your embedded U-Boot implementations. Security is a constant battle, and some of these suggestions and guidelines will most definitely morph and evolve as U-Boot development continues in the open source world. Nevertheless, this post should provide a good foundation for hardening many different versions of U-Boot. If you find a version of U-Boot in which some of these suggestions appear to be not applicable, feel free to ask us where they’ve gone.

 

Help!

As always, we offer professional engineering services to help you with your project. Whether you wish for us to integrate our security feature implementation, called VigiShield, into your project or just need another set of hands to help with general embedded software engineering and security, we’re available to help!

DM-Verity Without an Initramfs

DM-Verity Without an Initramfs

Summary

On Linux-based embedded systems implementing software authentication (secure boot and chain of trust), the file system verification is generally performed using an Initial RAM Filesystem (initramfs). Using an initramfs is more straight forward and flexible, as you can more easily adjust or calculate your verification arguments from the initramfs. For older kernels (< 5.4) which do not have in-kernel roothash signature verification, an initramfs is also essential if you wish to perform roothash signature verification from Linux during the boot sequence. However, as you might expect, using an initramfs may add undesirable boot time and storage requirements to the system. If your embedded system is operating with very strict boot timing or storage requirements, skipping the initramfs may be beneficial to you.  This can be done by using a technique called early device mapping.

 

Background Principles

The Linux kernel has a driver subsystem for device mapping, where a hardware storage device can be remapped through a target driver to produce a new, transparent, virtual storage device. This subsystem contains a few target drivers. One of these is called Verity, which is used for filesystem verification. Crypt is another one we commonly use, which provides filesystem encryption. We colloquially refer to these as DM-Verity and DM-Crypt. DM-Verity is what we will be using in this post. Let’s begin with a simple initramfs-based DM-Verity example.

Starting with an ext4 rootfs partition, we can generate the verity metadata from a build system via:

DATA_BLOCK_SIZE="4096"
HASH_BLOCK_SIZE="4096"
PART_SRC_EXT4=example.ext4
METADATA_HASH_DEV=example.ext4.metadata

veritysetup format --data-block-size=${DATA_BLOCK_SIZE} \
    --hash-block-size=${HASH_BLOCK_SIZE} \
    ${PART_SRC_EXT4} ${METADATA_HASH_DEV}

The output from this command looks like:

VERITY header information for example.ext4.metadata
UUID:               e17b33f3-ce02-4d9b-a0a8-90c85ebe3240
Hash type:          1
Data blocks:        16384
Data block size:    4096
Hash block size:    4096
Hash algorithm:     sha256
Salt:               2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b
Root hash:          b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941

Then when we mount and switch into the verified root filesystem from the initramfs during boot, we would do this from the target device:

MAPPER_NAME="verity_example"
PART_SRC_EXT4="/dev/mmcblk0p1"
METADATA_HASH_DEV="/dev/mmcblk0p2"
ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"

veritysetup \
    --restart-on-corruption --ignore-zero-blocks \
    create ${MAPPER_NAME} ${PART_SRC_EXT4} \
    ${METADATA_HASH_DEV} ${ROOTHASH}

mkdir -p /newroot
mount /dev/mapper/${MAPPER_NAME} /newroot
switch_root /newroot /sbin/init

Now you’re done! Every block of data contained in /newroot will now verified by the kernel before it is used. However, part of the chain of trust is still missing from this example. To understand that, we’ll need to know what the DM-Verity root hash used above is. So, what is the root hash? It’s basically the last/top node of special a binary hash tree called a Merkle Tree. Each block of data is hashed and stored at the bottom level of this binary tree. Pairs of hashes are then subsequently hashed together until a final top hash is derived.

This Merkle Tree is also what’s contained inside the Verity metadata partition or file, along with some additional header information.

 

So, before running veritysetup to create the mountable Verity-backed /dev/mapper entry, the root hash needs to be verified too. Otherwise someone could break your chain of trust by replacing your entire root file system, metadata file system, and root hash input. You can verify this root hash from your bootloader via some form of trusted/secure boot like UEFI. However, an easy way to perform signature verification isn’t always available in many embedded bootloaders, so we opt to do it from within the kernel. Traditionally, this is done inside the initramfs using a library like OpenSSL.

So for example, from your build system, you could create an RSA-signed roothash via:

ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"
PRIVATE_KEY="verity_private.pem"
PUBLIC_KEY="verity_public.pem"

#Generate RSA Key Pair
openssl genpkey -algorithm RSA -out ${PRIVATE_KEY} -pkeyopt rsa_keygen_bits:2048
openssl rsa -pubout -in ${PRIVATE_KEY} -out ${PUBLIC_KEY}

#Sign the root hash
echo ${ROOTHASH} | tr -d '\n' > roothash.txt
openssl rsautl -sign -inkey ${PRIVATE_KEY} -out ${ROOTHASH}.signed -in ${ROOTHASH}

Then on your embedded target, you would store the public key and signed root hash and check the authenticity via your initramfs:

#If this is correctly signed, it will return the roothash from the signed roothash file
ROOTHASH=$(openssl rsautl -verify -in "roothash.txt.signed" -inkey ${PUBLIC_KEY} -pubin)

#Result = b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941
echo ${ROOTHASH}

You can then use this resulting verified roothash to run “veritysetup create”, as shown above.

 

Linux Kernel-space Signature Verification

Signature verification in the Linux kernel is performed with CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG=y. This is a kernel configuration option which was added in 5.4. It adds the ability to do roothash signature verification in Linux’s kernel-space without any external tools or libraries.

This is utilized by creating an RSA certificate from the build system:

PRIVATE_KEY="verity_key.pem"
CERT="verity_cert.pem"
openssl req -x509 -newkey rsa:1024 -keyout ${PRIVATE_KEY} \
    -out ${CERT} -nodes -days 365 -set_serial 01 -subj /CN=example.com

Then you must add the certificate into your kernel build system with the appropriate config options:

CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG=y
CONFIG_SYSTEM_TRUSTED_KEYRING=y
CONFIG_SYSTEM_TRUSTED_KEYS="verity_cert.pem"

Then we create the signed root hash from the build system:

ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"
PRIVATE_KEY="verity_key.pem"
CERT="verity_cert.pem"
echo ${ROOTHASH} | tr -d '\n' > roothash.txt
openssl smime -sign -nocerts -noattr -binary -in roothash.txt \
    -inkey ${PRIVATE_KEY} -signer ${CERT} -outform der -out roothash.txt.signed

Don’t forget to require signatures on the embedded target’s kernel command-line parameters by appending:

dm_verity.require_signatures=1

Then, finally, we can mount the file system from our initramfs:

MAPPER_NAME="verity_example"
PART_SRC_EXT4="/dev/mmcblk0p1"
METADATA_HASH_DEV="/dev/mmcblk0p2"
ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"

veritysetup \
    --restart-on-corruption --ignore-zero-blocks --root-hash-signature=roothash.txt.signed \
    create "${MAPPER_NAME}" "${PART_SRC_EXT4}" \
    "${METADATA_HASH_DEV}" "${ROOTHASH}"

mkdir -p /newroot
mount /dev/mapper/${MAPPER_NAME} /newroot
switch_root /newroot /sbin/init

How can I do this without an initramfs? CONFIG_DM_INIT=y

CONFIG_DM_INIT is a kernel mechanism which allows you to pass a device mapper table to the kernel during boot, from the kernel command-line parameters.

So, from a command line perspective, the user-space command transforms from this:

veritysetup --ignore-zero-blocks \
    create "dm-0" "/dev/mmcblk0p1" "/dev/mmcblk0p2" \
    "b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"

To this after we manually pass the DM table information into the kernel:

dm-mod.create="verity,,,ro,0 131072 verity 1 /dev/mmcblk0p1 /dev/mmcblk0p2 4096 4096 16384 1 sha256 \
               b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941 \
               2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b 1 ignore_zero_blocks"

To break this down a bit further:

dm-mod.create="<name>,<uuid>,<minor>,<flags>,[dm_table_params {dm_verity_params}]"

Where dm_table_params="<start_sector> <num_sectors> <target_type> <dm_verity_params>"

And dm_verity_params="<version> <dev> <hash_dev> <data_block_size> <hash_block_size> <num_data_blocks> \
                      <hash_start_block> <algorithm> <digest> <salt> [<#opt_params> <opt_params>]"

So, in our example:

name="verity"
uuid="unused/unset"
minor="unused/unset"
flags="ro"
dm_table_params=
    start_sector="0"
    num_sectors="131072"
    target_type="verity"
    target_args="dm_verity_params"
        version="1"
        dev="mmcblk0p1"
        hash_dev="mmcblk0p2"
        data_block_size="4096"
        hash_block_size="4096"
        num_data_blocks="16384"
        hash_start_block="1"
        algorithm="sha256"
        digest="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"
        salt="2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b"

From the “veritysetup format” command above, we can see how a lot of these parameters are derived:

VERITY header information for example.ext4.metadata
UUID:               e17b33f3-ce02-4d9b-a0a8-90c85ebe3240
Hash type:          1
Data blocks:        16384
Data block size:    4096
Hash block size:    4096
Hash algorithm:     sha256
Salt:               2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b
Root hash:          b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941

For the number of sectors, that is calculated via Verity Data Blocks * Verity Data Block Size / Sector Size. Assuming your sector size is 512 (most common), we then have 16384*4096/512 = 131072.

From U-Boot, I like to set all of this up similarly to:

setenv DATA_BLOCKS 16384
setenv DATA_BLOCK_SIZE 4096
setenv DATA_SECTORS 131072
setenv HASH_BLOCK_SIZE 4096
setenv HASH_ALG sha256
setenv SALT 2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b
setenv ROOT_HASH b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941
setenv DATA_DEV mmcblk0p1
setenv DATA_META_DEV mmcblk0p2

setenv bootargs ${bootargs} dm-mod.create="verity,,,ro,0 \${DATA_SECTORS} verity 1 /dev/\${DATA_DEV}
    /dev/\${DATA_META_DEV} \${HASH_BLOCK_SIZE} \${DATA_BLOCK_SIZE} \${DATA_BLOCKS} 1 \${HASH_ALG}
    \${ROOT_HASH} \${SALT} 1 ignore_zero_blocks" root=/dev/dm-0 dm_verity.require_signatures=1

Note: /dev/dm-0 is the first device mapper device that is created by this early device mapping procedure. If this is your rootfs, then you must also set root=/dev/dm-0, as we have above.

 

Wait, what about early root hash signature verification? Where did that go?

Good catch, we can’t use the OpenSSL+initramfs method anymore. Unfortunately, DM_VERITY_VERIFY_ROOTHASH_SIG also does not support early mapping, as it uses the kernel keyring system and does not provide a way to setup the keyring via the kernel command-line arguments.

Alright, fine, let’s fix it. We’ll add a new verity table parameter called “root_hash_sig_hex”, where we can set the incoming root hash signature, without the keyring.

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Wed, 30 Mar 2022 10:57:25 -0400
Subject: [PATCH 1/1] DM-Verity: Add root_hash_sig_hex parameter to early
 verity device mapping, so that we can pass a roothash signature in via
 /proc/cmdline.  This enables the ability to use DM_VERITY_VERIFY_ROOTHASH_SIG
 alongside early device mapping

---

 drivers/md/dm-verity-target.c     |  8 +++-
 drivers/md/dm-verity-verify-sig.c | 63 +++++++++++++++++++++++++++++++
 drivers/md/dm-verity-verify-sig.h | 19 +++++++++-
 3 files changed, 87 insertions(+), 3 deletions(-)

diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c
index 3fb02167a590..047dd9a10264 100644
--- a/drivers/md/dm-verity-target.c
+++ b/drivers/md/dm-verity-target.c
@@ -930,7 +930,13 @@ static int verity_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
             if (r)
                 return r;
             continue;
-
+        } else if (verity_verify_is_hex_sig_opt_arg(arg_name)) {
+            r = verity_verify_hex_sig_parse_opt_args(as, v,
+                                 verify_args,
+                                 &argc, arg_name);
+            if (r)
+                return r;
+            continue;
         }
 
         ti->error = "Unrecognized verity feature request";
diff --git a/drivers/md/dm-verity-verify-sig.c b/drivers/md/dm-verity-verify-sig.c
index 919154ae4cae..906e5a2034b5 100644
--- a/drivers/md/dm-verity-verify-sig.c
+++ b/drivers/md/dm-verity-verify-sig.c
@@ -28,6 +28,12 @@ bool verity_verify_is_sig_opt_arg(const char *arg_name)
                 DM_VERITY_ROOT_HASH_VERIFICATION_OPT_SIG_KEY));
 }
 
+bool verity_verify_is_hex_sig_opt_arg(const char *arg_name)
+{
+    return (!strcasecmp(arg_name,
+                DM_VERITY_ROOT_HASH_VERIFICATION_OPT_HEX_SIG_KEY));
+}
+
 static int verity_verify_get_sig_from_key(const char *key_desc,
                     struct dm_verity_sig_opts *sig_opts)
 {
@@ -64,6 +70,33 @@ static int verity_verify_get_sig_from_key(const char *key_desc,
     return ret;
 }
 
+static int verity_verify_get_sig_from_hex(const char *key_desc,
+                    struct dm_verity_sig_opts *sig_opts)
+{
+    int ret = 0, i = 0, j = 0;
+    uint8_t byte[3] = {0x00, 0x00, 0x00};
+    long result;
+
+    sig_opts->sig = kmalloc(strlen(key_desc)/2, GFP_KERNEL);
+    if (!sig_opts->sig) {
+        ret = -ENOMEM;
+        goto end;
+    }
+
+    sig_opts->sig_size = strlen(key_desc)/2;
+
+    for(i = 0, j = 0; i < strlen(key_desc)-1; i+=2, j+=1){
+        byte[0] = key_desc[i];
+        byte[1] = key_desc[i+1];
+        kstrtol(byte, 16, &result);
+        sig_opts->sig[j] = result;
+    }
+
+end:
+
+    return ret;
+}
+
 int verity_verify_sig_parse_opt_args(struct dm_arg_set *as,
                      struct dm_verity *v,
                      struct dm_verity_sig_opts *sig_opts,
@@ -93,6 +126,36 @@ int verity_verify_sig_parse_opt_args(struct dm_arg_set *as,
     return ret;
 }
 
+int verity_verify_hex_sig_parse_opt_args(struct dm_arg_set *as,
+                     struct dm_verity *v,
+                     struct dm_verity_sig_opts *sig_opts,
+                     unsigned int *argc,
+                     const char *arg_name)
+{
+    struct dm_target *ti = v->ti;
+    int ret = 0;
+    const char *sig_key = NULL;
+
+    if (!*argc) {
+        ti->error = DM_VERITY_VERIFY_ERR("Signature key not specified");
+        return -EINVAL;
+    }
+
+    sig_key = dm_shift_arg(as);
+    (*argc)--;
+
+    ret = verity_verify_get_sig_from_hex(sig_key, sig_opts);
+    if (ret < 0)
+        ti->error = DM_VERITY_VERIFY_ERR("Invalid key specified");
+
+    v->signature_key_desc = kstrdup(sig_key, GFP_KERNEL);
+    if (!v->signature_key_desc)
+        return -ENOMEM;
+
+    return ret;
+}
+
+
 /*
  * verify_verify_roothash - Verify the root hash of the verity hash device
  *                 using builtin trusted keys.
diff --git a/drivers/md/dm-verity-verify-sig.h b/drivers/md/dm-verity-verify-sig.h
index 19b1547aa741..2be00114becc 100644
--- a/drivers/md/dm-verity-verify-sig.h
+++ b/drivers/md/dm-verity-verify-sig.h
@@ -10,6 +10,7 @@
 
 #define DM_VERITY_ROOT_HASH_VERIFICATION "DM Verity Sig Verification"
 #define DM_VERITY_ROOT_HASH_VERIFICATION_OPT_SIG_KEY "root_hash_sig_key_desc"
+#define DM_VERITY_ROOT_HASH_VERIFICATION_OPT_HEX_SIG_KEY "root_hash_sig_hex"
 
 struct dm_verity_sig_opts {
     unsigned int sig_size;
@@ -23,11 +24,13 @@ struct dm_verity_sig_opts {
 int verity_verify_root_hash(const void *data, size_t data_len,
                 const void *sig_data, size_t sig_len);
 bool verity_verify_is_sig_opt_arg(const char *arg_name);
-
+bool verity_verify_is_hex_sig_opt_arg(const char *arg_name);
 int verity_verify_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
                     struct dm_verity_sig_opts *sig_opts,
                     unsigned int *argc, const char *arg_name);
-
+int verity_verify_hex_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
+                    struct dm_verity_sig_opts *sig_opts,
+                    unsigned int *argc, const char *arg_name);
 void verity_verify_sig_opts_cleanup(struct dm_verity_sig_opts *sig_opts);
 
 #else
@@ -45,6 +48,11 @@ bool verity_verify_is_sig_opt_arg(const char *arg_name)
     return false;
 }
 
+bool verity_verify_is_hex_sig_opt_arg(const char *arg_name)
+{
+    return false;
+}
+
 int verity_verify_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
                     struct dm_verity_sig_opts *sig_opts,
                     unsigned int *argc, const char *arg_name)
@@ -52,6 +60,13 @@ int verity_verify_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
     return -EINVAL;
 }
 
+int verity_verify_hex_sig_parse_opt_args(struct dm_arg_set *as, struct dm_verity *v,
+                    struct dm_verity_sig_opts *sig_opts,
+                    unsigned int *argc, const char *arg_name)
+{
+    return -EINVAL;
+}
+
 void verity_verify_sig_opts_cleanup(struct dm_verity_sig_opts *sig_opts)
 {
 }

So, this patch allows the kernel to take the roothash signature in as a hex-formatted string from the command-line arguments and converts it back into raw data when it is copied into sig_opts->sig. This is done inside the verity_verify_get_sig_from_hex() section of the patch.

Now that our kernel is patched, we can do the following from our build system:

ROOTHASH="b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941"
PRIVATE_KEY="verity_key.pem"
CERT="verity_cert.pem"
echo ${ROOTHASH} | tr -d '\n' > roothash.txt
openssl smime -sign -nocerts -noattr -binary -in roothash.txt \
    -inkey ${PRIVATE_KEY} -signer ${CERT} -outform der -out roothash.txt.signed

xxd -p sign.txt | tr -d '\n'

#This will show a hexdump of a signature that looks like:
# 3081f906092a864886f70d010702a081eb3081e8020101310f300d06096086480165030402010500300
# b06092a864886f70d0107013181c43081c1020101301b30163114301206035504030c0b6578616d706c65
# 2e636f6d020101300d06096086480165030402010500300d06092a864886f70d01010105000481806f196
# a3081d941d22de98b34fc56f5c1b7ffc827ccd1307be9017bb6773da49026ef556668185c68b30562a60c
# ec635bbe0a52ad92b878dac9e4ad146f5d36101b48ec5f522d12772b8e915524586598c8659494fba427e
# e46c02043f30f45e096a1b9a987fc200b815f43bb48d42ad2d64b3f632f5332e6f74890b4541b467d

Then from our target system boot arguments, we can append the new “root_hash_sig_hex” parameters:

setenv DATA_BLOCKS 16384
setenv DATA_BLOCK_SIZE 4096
setenv DATA_SECTORS 131072
setenv HASH_BLOCK_SIZE 4096
setenv HASH_ALG sha256
setenv SALT 2a4c7638f03b92bdb92d7284a742e0c4407c9ef65fdf2a7ea78ed02fde4a518b
setenv ROOT_HASH b96a69664f9279857931dbf64f942caf909076e40fd5bd5ed8d30b53ff922941
setenv DATA_DEV mmcblk0p1
setenv DATA_META_DEV mmcblk0p2
setenv VERITY_SIGNATURE 3081f906092a864886f70d010702a081eb3081e8020101310f300d06096086480165030402010500300
    b06092a864886f70d0107013181c43081c1020101301b30163114301206035504030c0b6578616d706c652e636f6d020101300d
    06096086480165030402010500300d06092a864886f70d010101050004818071fcaaf1b252a56448438e0a9350b7380a407b1e9
    0ae869ec5062466b0eb6cc5358e253a9d57086c358220745bc60c2a6d8dbc30c02fb1714c9c98f10e0679b87deb0c19929675c8
    fcf89f37c684f043583fca52729ffb6e928eb29b7ee0c9eab3a3b0809a4463f3c8d6d458745c9116a7df1677c707df6352f2323
    13a62ce20

setenv bootargs ${bootargs} dm-mod.create="verity,,,ro,0 \${DATA_SECTORS} verity 1 /dev/\${DATA_DEV}
    /dev/\${DATA_META_DEV} \${HASH_BLOCK_SIZE} \${DATA_BLOCK_SIZE} \${DATA_BLOCKS} 1 \${HASH_ALG}
    \${ROOT_HASH} \${SALT} 3 ignore_zero_blocks root_hash_sig_hex \${VERITY_SIGNATURE}"
    root=/dev/dm-0 dm_verity.require_signatures=1

Now you’re really done! Your kernel will verify the DM-Verity file system’s roothash and boot without any requirement for an initramfs.

 

Other Considerations

1) Here are all of the kernel config options which are related to device mapping and verity:

CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_MD=y
CONFIG_BLK_DEV_DM=y
CONFIG_BLK_DEV_MD=y
CONFIG_DM_INIT=y
CONFIG_DM_VERITY=y
CONFIG_DM_VERITY_VERIFY_ROOTHASH_SIG=y
CONFIG_SYSTEM_TRUSTED_KEYRING=y
CONFIG_SYSTEM_TRUSTED_KEYS="verity_cert.pem"

2) Your verified file system is only as good as your chain of trust. Every stage in your boot process needs to be properly secured. Our secure boot article outlines this in more detail here.

3) If your U-Boot environment can be tampered with, such that dm_verity.require_signatures can be disabled, then your root file system verification can easily be defeated. You might consider forcefully enabling this with a kernel patch as such:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Wed, 30 Mar 2022 10:58:20 -0400
Subject: [PATCH 1/1] DM-Verity: Require dm-verity roothash signatures, with no
 ability to disable via /proc/cmdline

---
 drivers/md/dm-verity-verify-sig.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/md/dm-verity-verify-sig.c b/drivers/md/dm-verity-verify-sig.c
index 906e5a2034b5..b979e8e6b498 100644
--- a/drivers/md/dm-verity-verify-sig.c
+++ b/drivers/md/dm-verity-verify-sig.c
@@ -14,10 +14,12 @@
 
 #define DM_VERITY_VERIFY_ERR(s) DM_VERITY_ROOT_HASH_VERIFICATION " " s
 
-static bool require_signatures;
+static bool require_signatures = true;
+/*
 module_param(require_signatures, bool, 0444);
 MODULE_PARM_DESC(require_signatures,
        "Verify the roothash of dm-verity hash tree");
+*/
 
 #define DM_VERITY_IS_SIG_FORCE_ENABLED() \
    (require_signatures != false)

4) Small amounts of data corruption can be automatically corrected with CONFIG_DM_VERITY_FEC. If you’re interested in this, consider enabling this configuration option as well.

5) While booting from an MMC device, I observed that the MMC partition enumeration was out of sync with the early device mapping driver’s device lookup, resulting in a race condition. That is, it would attempt to find the enumerated MMC partitions (mmcblk0p1, mmcblk0p2) before they were instantiated and fail. I fixed this with another patch:

From: Nathan Barrett-Morrison <nathan.morrison@timesys.com>
Date: Wed, 30 Mar 2022 10:54:01 -0400
Subject: [PATCH 1/3] DM-Verity: Wait up to 10 seconds for eMMC/SD partitions
 to show up before failing dm_get_device()

---
 drivers/md/dm-verity-target.c | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c
index 711f101447e3..3fb02167a590 100644
--- a/drivers/md/dm-verity-target.c
+++ b/drivers/md/dm-verity-target.c
@@ -18,6 +18,7 @@
 #include "dm-verity-verify-sig.h"
 #include <linux/module.h>
 #include <linux/reboot.h>
+#include <linux/delay.h>
 
 #define DM_MSG_PREFIX            "verity"
 
@@ -998,10 +999,17 @@ static int verity_ctr(struct dm_target *ti, unsigned argc, char **argv)
     }
     v->version = num;
 
-    r = dm_get_device(ti, argv[1], FMODE_READ, &v->data_dev);
-    if (r) {
-        ti->error = "Data device lookup failed";
-        goto bad;
+    //Wait up to 10 seconds for devices to become available --
+    //wait_for_device_probe() sort of handles this, but the eMMC/SD probe finishes
+    //and dm_get_device() fails before the eMMC/SD partitions are found
+    for(i = 0; i <= 100; i++){
+        r = dm_get_device(ti, argv[1], FMODE_READ, &v->data_dev);
+        if (r && i < 100) {
+            msleep_interruptible(100);
+        }else if(r){
+            ti->error = "Data device lookup failed";
+            goto bad;
+        }
     }
 
     r = dm_get_device(ti, argv[2], FMODE_READ, &v->hash_dev);

 

Boot Time Comparisons

Testing on a relatively slower ARM processor (ADSP-SC589, Single core 500Mhz ARMv7) shows some major improvements. Keep in mind that this processor is favorable for showing the time difference here, as it has a slower initramfs loading and unpacking time. Newer processors with faster eMMC, DDR, and cores will narrow this margin.

On this processor, there is a total boot time difference of 29.19 seconds.

Using initramfs Not using initramfs Difference
Total boot time 53.33 s 24.14 s 29.19 s
U-Boot: Load initramfs from MMC 2.88 s 0.00 s 2.88 s
U-Boot: Verify initramfs 4.07 s 0.00 s 4.07 s
U-Boot: Relocate initramfs 3.81 s 0.00 s 3.81 s
Linux: Unpack initramfs 18.26 s 0.00 s 18.26 s
Linux: Run initramfs, mount partition, start systemd 1.41 s 1.09 s 0.32 s

On a faster, multi-core ARMv8+ processor, I believe you would still see a significant 1-2+ second difference in boot time. Early device mapping can be a very important tool if you are trying to hit a boot time goal of under 10 seconds.

 

Help!

As always, we offer professional engineering services to help you with your project. Whether you wish for us to integrate our security feature implementation, called VigiShield, into your project or just need another set of hands to help with general embedded software engineering and security, we’re available to help!

Yocto Security: Logging with auditd

Yocto Security: Logging with auditd

The Linux audit framework is a security system which can provide accurate information about almost all security-relevant actions running processes may take on a system. The logs created using the audit framework can be used to investigate potential security incidents.

Loggable actions include, but are not limited to:

  • accessing/changing sensitive files
  • execution of specific binaries
  • opening network connections
  • loading kernel modules

Audit is typically used together with SELinux, however this article discusses configuring the audit framework on its own within a Yocto project and some example of auditing rules.

 

Enabling the auditing subsystem in the kernel

Before proceeding with the userspace daemon auditd, you need to enable kernel support for auditing. This can be done via menuconfig, e.g.

bitbake virtual/kernel -c menuconfig

or via explicit config fragments:

CONFIG_AUDIT=y

Some kernels (like those from meta-arm-bsp) may already enable auditing by default. If that is the case, you do not need to do anything for this step.

 

Getting auditd

Honister and later (3.4+)
Users on Honister or a later Yocto release do not need any special layer configuration to get auditd, as it is included in meta-oe.
Hardknott and Dunfell (3.1, 3.3)
Users on Dunfell or Hardknott should add the meta-selinux layer at git://git.yoctoproject.org/meta-selinux to their source tracking system and BBLAYERS variable, pinning the appropriate revision for their Yocto release.

For example, using repo with Hardknott, we can use this manifest entry and this BBLAYERS assignment:

  <project name="meta-selinux" path="sources/meta-selinux" remote="yocto" revision="8b94f828a292d0e61d83aeeeeb4001c7cde08721" upstream="hardknott" />
BBLAYERS += " ${BSPDIR}/sources/meta-selinux"

Earlier releases
Projects on earlier releases of Yocto or OpenEmbedded Classic no longer benefit from regular security updates at the time of writing. The instructions provided may or may not still work, but this is not recommended.

 

Installing auditd

To install auditd, add auditd to the IMAGE_INSTALL list of packages for your image. This installs the auditd userspace daemon and init scripts to launch it automatically on boot.

IMAGE_INSTALL_append = " auditd"

If you are not using systemd, then start-stop-daemon is also needed for audit’s rules to load correctly.

IMAGE_INSTALL_append = " auditd dpkg-start-stop"

auditd in action

With auditd installed and our firmware image deployed, we can now see the two components of the auditing system in the list of running processes:

  • The kernel task kauditd
  • The userspace auditd daemon
root@imx8qxp-b0-mek:~# ps | grep audit
   31 root         0 SW   [kauditd]
  598 root     11304 S<   /sbin/auditd

If we check the audit log in /var/log/audit/audit.log, we see a message that the system has started:

root@imx8qxp-b0-mek:~# cat /var/log/audit/audit.log
type=DAEMON_START msg=audit(1520598905.876:992): op=start ver=3.0.1
  format=enriched kernel=5.10.69+g54bd61ba352b auid=4294967295 pid=598 uid=0
  ses=4294967295 res=successAUID="unset" UID="root"

However, since we just installed it, we are not logging anything yet. To log an action like reading from /etc/passwd, we can add a new auditing rule with auditctl.

root@imx8qxp-b0-mek:~# auditctl -w /etc/passwd -p warx -k passwd

Explanation of the options:

Option Description
-w /etc/passwd Adds a new watch for the file at the given path, /etc/passwd. This option only works with existing files and directories, not wildcards.
-p warx Specifies the type of watch to perform. In this case, Write, Attribute change, Read, and eXecute will be logged when performed on the given file.
-k passwd Specifies a group name for this watch. For example, we may also want to watch /etc/shadow with a group named passwd.

After doing this, we can read from /etc/passwd:

root@imx8qxp-b0-mek:~# cat /etc/passwd > /dev/null

And check the audit log again:

type=PATH msg=audit(1677197344.196:4): item=0 name="/etc/passwd" inode=176
  dev=fc:01 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0
  cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0OUID="root" OGID="root"

The file access was successfully written to the audit log.

 

Using rules files

A rules file is any file used with auditd that adds functionality like the above. Rules files consist of a format identical to the command-line format of auditctl, but also allow extra whitespace and comments. An example is shown below.

30-passwd.rules
# Watch writes to passwd file
-w /etc/passwd -p wa -k passwd

# Watch all accesses to shadow file
-w /etc/shadow -p warx -k passwd

# Watch for groups changes
-w /etc/group -p wa -k passwd

There are a number of sample rules that come with auditd that you may like to use. A full list of these rules can be found in the sources.

 

Installing rules

To use a rule, copy its file into /etc/audit/rules.d and it will be loaded when audit starts.

To simplify the process, you can use a custom-audit-rules.bb recipe like the following:

SUMMARY = "Custom rules for audit"
DESCRIPTION = "Adds additional audit rules"

LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/${LICENSE};md5=0835ade698e0bcf8506ecda2f7b4f302"

# Audit rules files are placed in the custom-audit-rules directory adjacent to the recipe location.
AUDIT_RULES := " 30-pci-dss-v31.rules 40-local.rules 41-containers.rules"
RDEPENDS_${PN} := "audit"

python () {
    d.setVar('SRC_URI', ' '.join(f'file://{path}' for path in d.getVar('AUDIT_RULES').split()))
}

do_install() {
    RULES_DIR=${D}${sysconfdir}/etc/audit/rules.d
    install -d $RULES_DIR

    for rule in ${AUDIT_RULES}; do
        install -m 0644 ${S}/$rule $RULES_DIR/$rule
    done
}

FILES_${PN} := "${sysconfdir}/audit/rules.d/*"

For a directory structure like the following,

└── recipes-security
    └── audit
        ├── custom-audit-rules
        │   └── 30-passwd.rules
        └── custom-audit-rules.bb

This recipe will install any specified rules under custom-audit-rules.

 

Conclusion

Using auditd is a simple way to enhance the security posture of your IoT device which can help actively monitor and investigate potential security breaches.

Learn more about our VigiShield offering which implements a number of security best practices, including audit logs.

Liam White McShane, Junior Embedded Systems Engineer, has over a year of experience in designing and debugging embedded systems. He specializes in providing customers with software and firmware services for their devices. Liam holds a bachelor’s degree in Computer Science from the University of Pittsburgh.
Yocto Security: Automating compliance using OpenSCAP

Yocto Security: Automating compliance using OpenSCAP

The Security Content Automation Protocol (SCAP) is a method of using certain interoperable security standards to automate evaluating policy compliance of deployed systems.

In order to scan a system for compliance or vulnerabilities, there are two components involved:

  • SCAP content, typically consists of a list of rules to audit
  • SCAP scanning tool that can interpret the SCAP content to perform the check on the system and optionally remediate

To illustrate using a practical example, a security requirement can mandate not storing any plain text passwords in a file. This requirement can be encoded as a rule using one of the specifications under SCAP, for example Open Vulnerability and Assessment Language (OVAL). Then to perform a scan of the system to assess compliance, a SCAP compatible tool such as OpenSCAP scanner can be used to read the SCAP content (e.g. OVAL file) and report the assessment in a standardized format.

The SCAP content is generally developed by software vendors and/or device manufacturers, sometimes in conjunction with authorities such as Center for Internet Security (CIS), Defense Information Systems Agency (DISA), National Institute of Standards and Technology (NIST) etc. There is a publicly available repository of pre-existing compliance checks and profiles based on industry standards such as PCI-DSS, HIPAA, DISA STIG, NIST SP 800-53 etc. This can be used as a baseline and any additional security requirements can be layered on top of the existing checks and/or profiles. While there are a lot of guides available on how to leverage SCAP on commercial / open source Linux distributions, the information regarding usage with custom Linux distributions is sparse. The next section is intended as a getting started guide for SCAP on embedded / IoT Linux devices using Yocto project as an example.
 

Getting started with OpenSCAP on Yocto

Yocto has a meta-security-compliance layer under meta-security that packages both SCAP scanner (OpenSCAP) and SCAP content (SCAP Security Guide). Using these packages a security audit of the target device (QEMU in this case) can be performed. 

To begin, follow the setup instructions on the OE-Core wiki page. Note: If you are using Poky or any other reference distro, there are some variables that need to be tweaked which will be covered later in this article.

Step 1: Get source code

git clone git://git.openembedded.org/openembedded-core oe-core
git clone git://git.yoctoproject.org/meta-security
git clone git://git.openembedded.org/meta-openembedded
git clone git://git.openembedded.org/bitbake oe-core/bitbake

Step 2: Build setup

source ./oe-core/oe-init-build-env
//Add the below lines to bblayers.conf
BBLAYERS += "${TOPDIR}/../meta-openembedded/meta-oe"
BBLAYERS += "${TOPDIR}/../meta-openembedded/meta-python"
BBLAYERS += "${TOPDIR}/../meta-security/meta-security-compliance"
//Add the below line to local.conf
IMAGE_INSTALL:append = " os-release openembedded-release oe-scap scap-security-guide "
//Build
bitbake core-image-minimal

SCAP for configuration checks

The scap-security-guide package includes a basic security profile which can be used to audit the system. This is a very minimal security rule set and is to be used for illustrative purposes only. Follow the below instructions to run the scan with the basic profile. 

Step 3: Run openscap scanner

//Start a qemu instance of the built image
runqemu
//Run oscap on the qemu target console
# oscap xccdf eval --report basic-embedded.html --profile xccdf_org.ssgproject.content_profile_basic-embedded /usr/share/xml/scap/ssg/content/ssg-openembedded-ds.xml
Title   Verify All Account Password Hashes are Shadowed
Rule    xccdf_org.ssgproject.content_rule_accounts_password_all_shadowed
Result  pass
…………………
Title   Limit the Number of Concurrent Login Sessions Allowed Per User
Rule    xccdf_org.ssgproject.content_rule_accounts_max_concurrent_login_sessions
Result  notapplicable

Interpreting the report

The tool generates reports in multiple formats including a human friendly html file which can be viewed directly on the device or copied over to a host PC for viewing. Below is a screenshot of the sample report and if you want to play around with the actual report visit this link. 

The summary section gives a big picture view of the compliance results along with the severity of any failed checks. The rules overview section lets you filter based on the result status and search for specific rules. The other nice part is to filter based on a profile group, for example if you are trying to achieve a particular compliance e.g. HIPAA, you can pick that group and the results will be rearranged to automatically map the rules to the relevant section of the security standard. 

The “Result Details” section provides detailed information on each rule along with any applicable references to various compliance requirements mapping. The results section also provides details on what exactly was checked for and/or if a check was skipped.

 

SCAP for vulnerabilities assessment

Apart from using SCAP for configuration compliance checks, OpenSCAP can also be used to check if the software installed on the system is patched for known vulnerabilities (typically CVEs). Distributions such as Ubuntu provide an OVAL feed containing data about patched versions of software which can be used to audit the system. In case of “build your own distro” such as Yocto, BuildRoot, OpenWRT the team responsible for creating the distro would have to create such a feed. It is a cumbersome process and hence not widely used. However if one wishes to create such a feed, for Yocto users, the oe-scap package in the meta-security-compliance layer provides a reference example. Below is an illustration where the oscap tool is used to check if CVE-2017-7502 is patched in the nss package on the system. This can be further automated using the openscap daemon which can run on a periodic schedule.

# oscap xccdf eval /usr/share/oe-scap/OpenEmbedded_nodistro_0.xccdf.xml
Title   CPE-2017:1365: nss security and bug fix update (Important)
Rule    oval-com.redhat.rhsa-def-20171365
Ident   CVE-2017-7502
Result  pass

Customizing SCAP content

A good starting point for creating custom SCAP content is by leveraging the work already done by the community. The Compliance As Code project has pre-existing rules/checks/profiles available for Linux systems and a “new product” guide for creating custom content. The project already has profiles that map to various security standards that can be inherited for the new product being created as illustrated in this patch. Another place to look for public SCAP content is the NIST National Checklist Program.

 

Debugging failures

A commonly faced issue when running oscap is checks return as “notapplicable”. The most likely reason for this is because of a VERSION_ID mismatch. The default expected VERSION_ID is nodistro.0 as specified in the /etc/os-release file. The best way to debug such scenarios is to pass in the “–verbose DEVEL” argument when calling an oscap application on the target device. Depending on your Yocto version you might need this patch for correctly mapping the VERSION_ID. 

# cat /etc/os-release 
ID=nodistro
NAME="OpenEmbedded"
VERSION="nodistro.0"
VERSION_ID=nodistro.0
PRETTY_NAME="OpenEmbedded nodistro.0"

 The SCAP content can link multiple different specs within each other, hence it is useful to understand the linkage/dependencies. To inspect SCAP content, use the oscap info command and then trace the linked checklists/checks/dictionaries for any possible errors. The below example inspects a SCAP source data stream (DS) which encapsulates links to rule checklists (XCCDF), compliance check definitions (OVAL) and dictionaries to identify software (CPE).

# oscap info /usr/share/xml/scap/ssg/content/ssg-openembedded-ds-1.3.xml
Document type: Source Data Stream
Imported: 2018-03-09T12:34:56
Stream: scap_org.open-scap_datastream_from_xccdf_ssg-openembedded-xccdf-1.2.xml
Generated: (null)
Version: 1.3
Checklists:
	Ref-Id: scap_org.open-scap_cref_ssg-openembedded-xccdf-1.2.xml
		Status: draft
		Generated: 2019-07-06
		Resolved: true
		Profiles:
	           Title: Basic Profile for Embedded Systems
			Id: xccdf_org.ssgproject.content_profile_basic-embedded
		Referenced check files:
		   ssg-openembedded-oval.xml
			system: http://oval.mitre.org/XMLSchema/oval-definitions-5
		   ssg-openembedded-ocil.xml
			system: http://scap.nist.gov/schema/ocil/2
Checks:
	Ref-Id: scap_org.open-scap_cref_ssg-openembedded-oval.xml
	Ref-Id: scap_org.open-scap_cref_ssg-openembedded-ocil.xml
	Ref-Id: scap_org.open-scap_cref_ssg-openembedded-cpe-oval.xml
Dictionaries:
	Ref-Id: scap_org.open-scap_cref_ssg-openembedded-cpe-dictionary.xml

Conclusion

Whether you are looking to embrace security automation or trying to meet regulatory compliance requirements, SCAP can play an important part of your security toolbox. Leveraging the OpenSCAP project, one can get a jumpstart in building secure devices that can be easily audited in an automated fashion. 

Learn more about Timesys security solutions for out of the box hardened OS profiles and vulnerability monitoring to automate your DevSecOps and meet compliance.

Akshay Bhat is CTO, Embedded Products and Services at Timesys. He oversees the strategic direction of Timesys’ technology roadmap. With more than 16 years of industry experience with embedded systems software development and security, Akshay’s focus is on Timesys solutions that transform the software development lifecycle for embedded and enable the development of embedded system products with stronger security. Akshay has authored numerous embedded Linux and industry articles and delivered several embedded systems security presentations. He received his MS in Electrical Engineering from NYU Polytechnic University.
Securing your Linux Configuration (Kernel Hardening)

Securing your Linux Configuration (Kernel Hardening)

 

 

This article discusses the process by which your kernel’s configuration can be strengthened to protect against common security exploits. This is sometimes referred to as hardening, or specifically in this context, kernel configuration hardening.

 

Preface

A Linux kernel configuration is a file which defines all of the enabled (or disabled) options which are compiled in to your kernel. If you have not seen one before, they generally reside in the kernel’s build directory with a filename of “.config”. They are sometimes collapsed in to a defconfig (default config) file which only shows options which were not already selected by default.

This discussion will present many configuration tables in the form of:

Expected Conditional Architectures Kernel Versions Note
OPTION1=Y Architecture to which this option applys (X86 and/or ARM) Kernel versions in which OPTION1 exists A description of OPTION1
OPTION2=is not set Architecture to which this option applys (X86 and/or ARM) Kernel versions in which OPTION2 exists A description of OPTION2

In this case, it is recommended that the fictional option, OPTION1, is selected (CONFIG_OPTION1=y) in your kernel’s .config or defconfig file. It is also recommended that OPTION2 is disabled (CONFIG_OPTION2=is not set) in your kernel’s .config or defconfig file.

The option descriptions are sometimes difficult to interpret and may require further research. If you find an option that you’d like to know more information about, you may have to inspect the kernel source, search LWN, search Patchwork, or use your web search engine of choice.

 

Understanding Configuration Selections

When Linux kernel hardening, there are generally four categories of reason for which a configuration item may be enabled or disabled:

  • Adding an additional level of protection against a known exploit by enabling a configuration item. For example:
    • Mitigating Spectre attacks with CONFIG_RETPOLINE=y (which traps the processor’s speculative execution paths for indirect address calls by using a return trampoline).
  • Disabling a configuration item or subsystem which is known to be exploitable. For example:
    • Disabling the USB networking subsystem (USB_USBNET=is not set), so that applications using network-based IPC(Inter-Processor Communication) mechanisms may not be inadvertently exposed through a USB port.
    • Disabling /dev/mem access to physical memory (DEVMEM=is not set), so that physical memory cannot be easily modified.
  • Enabling general security strengthening features in the kernel (which may necessarily protect against a presently known attack). For example:
    • Reducing the risk of memory page leakage by enabling page poisoning (PAGE_POISONING=Y) to overwrite any potentially sensitive information upon freeing.
    • Enabling Security-Enhanced Linux
  • Disabling any unused kernel configuration options. If you don’t need it, disable it. As an added bonus, doing so may also improve your boot times.
Expected Conditional Architectures Kernel Versions Note
RETPOLINE=Y X86_32, X86_64 4.15-4.20, 5.0-5.17 Avoid speculative indirect branches in kernel (Spectre Mitigation)
USB_USBNET=is not set ARM, ARM64, X86_32, X86_64 2.6.22-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 If enabled, this adds USB networking subsystem support to the kernel
DEVMEM=is not set ARM, ARM64, X86_32, X86_64 4.0-4.20, 5.0-5.17 Do not allow access to physical memory via /dev/mem
PAGE_POISONING=Y ARM, ARM64, X86_32, X86_64 4.6-4.20, 5.0-5.17 Fill the pages with poison patterns after free_pages() and verify the patterns before alloc_pages. The filling of the memory helps reduce the risk of information leaks from freed data. This must be enabled from the boot cmdline with page_poison=1

The latter two options can help to protect against exploits which have not yet been discovered or released into the public domain (e.g. zero-day exploits) by reducing your kernel’s exploitable attack surface.

Analyzing, understanding, and modifying the kernel configuration with these tasks in mind is not trivial. Furthermore, in a large project it may not be clear exactly who is using which kernel configuration option. This can result in iterative backstepping until you arrive at a final configuration which works for your entire team.

There’s also a maintenance burden once you have created your final configuration. When you upgrade your kernel, many of the configuration options will have been removed and renamed. This will require another assessment and configuration period. Keeping your kernel up to date is extremely important, as security features are continuously added and revised in newer kernels. Starting a project with a long term stable (LTS) kernel is recommended, as an LTS kernel provides smaller (and sometimes backported) version increments to resolve security flaws (e.g upgrading from 5.0.x to 5.0.y). Such patches are provided until the long term stable support period ends.

 

Cost-Benefit Analysis

Not all configuration items provide the same cost-benefit as others. To some extent, most options will have an impact in these key areas:

  • Compile Time
    • This is especially true for security features which are checked by the compiler during compile time. For example, GCC_PLUGIN_STACKLEAK=Y will block uninitialized stack variable attacks through the use of a compiler plugin which searches for and initializes such variables.
    • This should not be considered a concern. Added compilation time is well worth the added security.
  • Kernel Binary Size
    • This is usually not a concern. Adding and removing features may change the kernel size by a few megabytes, which is generally negligible on modern systems.
  • Boot time
    • For example, DM_CRYPT=y, which adds drive encryption capabilities to your kernel, will add additional non-neglibile boot time to your system. This is because it may require booting in to an Initial Ram Filesystem, retreiving your disk encryption key, and ultimately mounting the encrypted disk through the device mapper subsystem. All of these steps add time to the boot process.
  • Processor Load
    • This is perhaps the most concerning option and must be determined by trial and error. If every available security option is enabled on a slower ARM processor, there may be too much overhead to reliably run your processes (at a reasonable latency).
    • For example, erasing memory with PAGE_POISONING=Y will use CPU cycles. The amount of overhead added will be dependent upon factors such as your CPU and RAM speed.
Expected Conditional Architectures Kernel Versions Note
GCC_PLUGIN_STACKLEAK=Y ARM64, X86_32, X86_64 5.2-5.17 This blocks most uninitialized stack variable attacks, with the performance impact being driven by the depth of the stack usage, rather than the function calling complexity. The performance impact on a single CPU system kernel compilation sees a 1% slowdown.
DM_CRYPT=Y ARM, ARM64, X86_32, X86_64 2.6.4-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Filesystem Hardening (Block Level Encryption via dm-crypt) – Requires userspace support
PAGE_POISONING=Y ARM, ARM64, X86_32, X86_64 4.6-4.20, 5.0-5.17 Fill the pages with poison patterns after free_pages() and verify the patterns before alloc_pages. The filling of the memory helps reduce the risk of information leaks from freed data. This must be enabled from the boot cmdline with page_poison=1

Some configuration items will provide a better cost-benefit. Configuration items which have minimal impact on processor load are most valuable. For example, these add virtually no CPU overhead:

  • Disabling DEBUG_BUGVERBOSE, which will help ensure that sensitive backtrace information is not leaked upon a kernel BUG() condition.
  • Enabling ARCH_HAS_ELF_RANDOMIZE, which will make repeat exploits much more difficult by randomizing certain memory locations.

While these will add CPU overhead to some degree:

  • Enabling DEBUG_VIRTUAL will enable some sanity checking in virt_to_page translation at the cost of CPU cycles.
  • Enabling INIT_ON_ALLOC_DEFAULT_ON or INIT_ON_FREE_DEFAULT_ON will protect against heap memory leaks by erasing the regions after use, at the cost of erase time.
Expected Conditional Architectures Kernel Versions Note
DEBUG_BUGVERBOSE=is not set ARM, ARM64, X86_32, X86_64 2.6.9-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Make sure this is not enabled, as it could provide an attacker sensitive kernel backtrace information on BUG() conditions
ARCH_HAS_ELF_RANDOMIZE=Y ARM, ARM64, X86_32, X86_64 4.1-4.20, 5.0-5.17 Randomized locations for stack, mmap, brk, and ET_DYN
INIT_ON_FREE_DEFAULT_ON=Y ARM, ARM64, X86_32, X86_64 5.3-5.17 More expensive form of INIT_ON_ALLOC_DEFAULT_ON. The primary difference is that data lifetime in memory is reduced, as anything freed is wiped immediately, making live forensics or cold boot memory attacks unable to recover freed memory contents.
INIT_ON_ALLOC_DEFAULT_ON=Y ARM, ARM64, X86_32, X86_64 5.3-5.17 All page allocator and slab allocator memory will be zeroed when freed, eliminating many kinds of “uninitialized heap memory” flaws, especially heap content exposures.
DEBUG_VIRTUAL=Y ARM, ARM64, X86_32, X86_64 2.6.28-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Enable some costly sanity checks in virtual to page code. This can catch mistakes with virt_to_page() and friends.

Using multiple kernel configurations (development and production) can be an option. While it is wise to develop on a prototype system that most closely resembles a production system (as to not cause unforseen bugs in changing timing conditions and loads between configurations), some security options be too cumbersome to reasonably develop on. If you’re writing a kernel driver and you need to use a tracing tool or read a core dump, then certainly enable them while developing.

 

Categories of Linux Kernel Hardening

In the Timesys Kernel Hardening Analysis Tool, the kernel security options have been divided into various groups. This categorization is by no means a definitive separation; some options could be further categorized or applied to multiple categories.

Memory Protection

Memory exploits are classes of attack in which an entity is able to retrieve or modified privileged information about the system. These can be further categorized into:

  • Stack Overflow Protections: These are security features which seek to prevent access to and tampering of stack variables in memory. A stack canary (an arbitrary value sitting at the top of the stack, which, if modified, alerts the kernel of tampering) is sometimes mentioned in these protections.
Expected Conditional Architectures Kernel Versions Note
INIT_STACK_ALL_ZERO=Y ARM, ARM64, X86_32, X86_64 5.9-5.17 Initializes everything on the stack with a zero value. This is intended to eliminate all classes of uninitialized stack variable exploits and information exposures, even variables that were warned to have been left uninitialized. (Strongest, safest)
GCC_PLUGIN_ARM_SSP_PER_TASK=Y ARM 5.2-5.17 Generates a separate stack canary value for each task, so if one task’s canary value is leaked it does not cause all other tasks to become vulnerable.
STACKPROTECTOR=Y ARM, ARM64, X86_32, X86_64 4.18-4.20, 5.0-5.17 This option turns on the “stack-protector” GCC feature. This feature puts, at the beginning of functions, a canary value on the stack just before the return address, and validates the value just before actually returning. Stack based buffer overflows (that need to overwrite this return address) now also overwrite the canary, which gets detected and the attack is then neutralized via a kernel panic.
STACKPROTECTOR_STRONG=Y ARM, ARM64, X86_32, X86_64 4.18-4.20, 5.0-5.17 Adds the CONFIG_STACKPROTECTOR canary logic to additional conditions related to variable assignment.
STACKPROTECTOR_PER_TASK=Y ARM, ARM64 5.0-5.17 Use a different stack canary value for each task
VMAP_STACK=Y ARM64 4.9-4.20, 5.0-5.17 Enable this if you want the use virtually-mapped kernel stacks with guard pages. This causes kernel stack overflows to be caught immediately.
SCHED_STACK_END_CHECK=Y ARM, ARM64, X86_32, X86_64 3.18-3.19, 4.0-4.20, 5.0-5.17 Additional validation check on commonly targeted structure. Detect stack corruption on calls to schedule()
STACKLEAK_METRICS=is not set ARM64, X86_32, X86_64 5.2-5.17 If this is set, STACKLEAK metrics for every task are available in the /proc file system.
STACKLEAK_RUNTIME_DISABLE=is not set ARM64, X86_32, X86_64 5.2-5.17 If set, allows runtime disabling of kernel stack erasing
GCC_PLUGIN_STACKLEAK=Y ARM64, X86_32, X86_64 5.2-5.17 This blocks most uninitialized stack variable attacks, with the performance impact being driven by the depth of the stack usage, rather than the function calling complexity. The performance impact on a single CPU system kernel compilation sees a 1% slowdown.
  • Heap Overflow: These are security features which seek to prevent heap memory exposure and modification.
Expected Conditional Architectures Kernel Versions Note
STRICT_KERNEL_RWX=Y ARM, ARM64, X86_32, X86_64 4.11-4.20, 5.0-5.17 If this is set, kernel text and rodata memory will be made read-only, and non-text memory will be made non-executable. This provides protection against certain security exploits (e.g. executing the heap or modifying text)
SLAB_FREELIST_HARDENED=Y ARM, ARM64, X86_32, X86_64 4.14-4.20, 5.0-5.17 Harden slab freelist metadata: Many kernel heap attacks try to target slab cache metadata and other infrastructure. This options makes minor performance sacrifices to harden the kernel slab allocator against common freelist exploit methods. Some slab implementations have more sanity-checking than others. This option is most effective with CONFIG_SLUB.
SLAB_FREELIST_RANDOM=Y ARM, ARM64, X86_32, X86_64 4.7-4.20, 5.0-5.17 Randomizes the freelist order used on creating new pages. This security feature reduces the predictability of the kernel slab allocator against heap overflows.
COMPAT_BRK=is not set ARM, ARM64, X86_32, X86_64 2.6.25-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not disable heap randomization
INET_DIAG=is not set ARM, ARM64, X86_32, X86_64 2.6.14-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not allow INET socket monitoring interface. Assists heap memory attacks
  • User Copy Protection: These are security features which seek to prevent memory exploitation during kernel and userspace memory transfer transactions.
Expected Conditional Architectures Kernel Versions Note
HARDENED_USERCOPY=Y ARM, ARM64, X86_32, X86_64 4.8-4.20, 5.0-5.17 This option checks for obviously wrong memory regions when copying memory to/from the kernel (via copy_to_user() and copy_from_user() functions) by rejecting memory ranges that are larger than the specified heap object, span multiple separately allocated pages, are not on the process stack, or are part of the kernel text. This kills entire classes of heap overflow exploits and similar kernel memory exposures.
HARDENED_USERCOPY_FALLBACK=is not set ARM, ARM64, X86_32, X86_64 4.16-4.20, 5.0-5.15 This is a temporary option that allows missing usercopy whitelists to be discovered via a WARN() to the kernel log, instead of rejecting the copy, falling back to non-whitelisted hardened usercopy that checks the slab allocation size instead of the whitelist size.
HARDENED_USERCOPY_PAGESPAN=is not set ARM, ARM64, X86_32, X86_64 4.8-4.20, 5.0-5.17 When a multi-page allocation is done without __GFP_COMP, hardened usercopy will reject attempts to copy it. There are, however, several cases of this in the kernel that have not all been removed. This config is intended to be used only while trying to find such users.
HAVE_HARDENED_USERCOPY_ALLOCATOR=Y ARM, ARM64, X86_32, X86_64 4.8-4.20, 5.0-5.17 The heap allocator implements __check_heap_object() for validating memory ranges against heap object sizes.
  • Information exposure: Options which are selected to limit exposure to privileged information.
Expected Conditional Architectures Kernel Versions Note
X86_UMIP=Y X86_32, X86_64 5.5-5.17 If enabled, a general protection fault is issued if the SGDT, SLDT, SIDT, SMSW or STR instructions are executed in user mode. These instructions unnecessarily expose information about the hardware state.
PROC_PAGE_MONITOR=is not set ARM, ARM64, X86_32, X86_64 2.6.28-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not expose process memory utilization via /proc interfaces
PROC_VMCORE=is not set ARM, ARM64, X86_32, X86_64 2.6.37-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not export the dump image of crashed kernel
DEBUG_FS=is not set ARM, ARM64, X86_32, X86_64 2.6.11-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not enable debugfs, as it may expose vulnerabilities
  • Kernel Address Space Layout Randomization (KASLR): A security method by which kernel memory structures are randomized in order to prevent repeat or replay-style attacks.
Expected Conditional Architectures Kernel Versions Note
ARCH_HAS_ELF_RANDOMIZE=Y ARM, ARM64, X86_32, X86_64 4.1-4.20, 5.0-5.17 Randomized locations for stack, mmap, brk, and ET_DYN
RANDOMIZE_BASE=Y ARM64, X86_32, X86_64 4.7-4.20, 5.0-5.17 In support of Kernel Address Space Layout Randomization (KASLR), this randomizes the physical address at which the kernel image is decompressed and the virtual address where the kernel image is mapped, as a security feature that deters exploit attempts relying on knowledge of the location of kernel code internals.
RANDOMIZE_MEMORY=Y X86_64 4.8-4.20, 5.0-5.17 Randomizes the base virtual address of kernel memory sections (physical memory mapping, vmalloc & vmemmap). This security feature makes exploits relying on predictable memory locations less reliable.
SLAB_FREELIST_RANDOM=Y ARM, ARM64, X86_32, X86_64 4.7-4.20, 5.0-5.17 Randomizes the freelist order used on creating new pages. This security feature reduces the predictability of the kernel slab allocator against heap overflows.
GCC_PLUGIN_RANDSTRUCT=Y ARM, ARM64, X86_32, X86_64 4.13-4.20, 5.0-5.17 Randomizes layout of sensitive kernel structures

Reducing Attack Surface

These are configuration options which can be selected to reduce the potential for exposure to unknown zero-day attacks by limiting the attack surface as much as we can. These are options that reduce the amount of information exposure and compiled-firmware attack surface (Again: If you don’t need it, disable it).

  • Kernel Replacement Attacks: Methods in which a kernel binary could be replaced during runtime.
Expected Conditional Architectures Kernel Versions Note
HIBERNATION=is not set ARM, ARM64, X86_32, X86_64 2.6.23-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not support hibernation. Allows replacement of running kernel.
KEXEC=is not set ARM, ARM64, X86_32, X86_64 2.6.16-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not allow system to boot another Linux kernel
KEXEC_FILE=is not set ARM, ARM64, X86_32, X86_64 3.17-3.19, 4.0-4.20, 5.0-5.17 Do not allow system to boot another Linux kernel
  • Module Security Attacks: These are attacks which can be performed by loading a tainted, custom, module in to a system or maliciously modifying a pre-existing module’s memory. The mitigations for this mostly consist of restricting execution regions, making such regions read-only, and signature checking prior to loading modules.
Expected Conditional Architectures Kernel Versions Note
MODULES=is not set ARM, ARM64, X86_32, X86_64 You should not allow for modules to be loaded unless you have the proper signing and signature checks enabled. Allowing the kernel to load unsigned modules can be dangerous
STRICT_MODULE_RWX=Y ARM, ARM64, X86_32, X86_64 4.11-4.20, 5.0-5.17 Module text and rodata memory will be made read-only, and non-text memory will be made non-executable. This provides protection against certain security exploits (e.g. writing to text)
MODULE_SIG=Y ARM, ARM64, X86_32, X86_64 3.7-3.19, 4.0-4.20, 5.0-5.17 Enable module signature verification
MODULE_SIG_ALL=Y ARM, ARM64, X86_32, X86_64 3.9-3.19, 4.0-4.20, 5.0-5.17 Automatically sign all modules during modules_install (so we don’t have to do this manually)
MODULE_SIG_SHA512=Y ARM, ARM64, X86_32, X86_64 3.7-3.19, 4.0-4.20, 5.0-5.17 Sign modules with SHA-512 algorithm
MODULE_SIG_FORCE=Y ARM, ARM64, X86_32, X86_64 3.7-3.19, 4.0-4.20, 5.0-5.17 Require modules to be validly signed
DEBUG_SET_MODULE_RONX=Y ARM, ARM64, X86_32, X86_64 Varies depending on architecture Helps catch unintended modifications to loadable kernel module’s text and read-only data. It also prevents execution of module data.
  • Reducing Syscall Exposure: Syscalls are interfaces in which user-space and kernel-space can communicate and access each other. Some legacy syscalls may be exploitable and are generally not required on modern systems. Disabling syscalls when possible is a good way to reduce your attack surface.
Expected Conditional Architectures Kernel Versions Note
SECCOMP=Y ARM, ARM64, X86_32, X86_64 Varies depending on architecture This kernel feature is useful for number crunching applications that may need to compute untrusted bytecode during their execution. By using pipes or other transports made available to the process as file descriptors supporting the read/write syscalls, it’s possible to isolate those applications in their own address space using seccomp. Once seccomp is enabled via prctl(PR_SET_SECCOMP), it cannot be disabled and the task is only allowed to execute a few safe syscalls defined by each seccomp mode.
USELIB=is not set ARM, ARM64, X86_32, X86_64 4.5-4.20, 5.0-5.17 If enabled, this allows the libc5 and earlier dynamic linker usblib syscall. Should no longer be needed.
MODIFY_LDT_SYSCALL=is not set X86_32, X86_64 4.3-4.20, 5.0-5.17 Linux can allow user programs to install a per-process x86 Local Descriptor Table (LDT) using the modify_ldt(2) system call. This is required to run 16-bit or segmented code such as DOSEMU or some Wine programs. It is also used by some very old threading libraries. Enabling this feature adds a small amount of overhead to context switches and increases the low-level kernel attack surface. Disabling it removes the modify_ldt(2) system call.
LEGACY_VSYSCALL_NONE=Y X86_32, X86_64 4.4-4.20, 5.0-5.17 There will be no vsyscall mapping at all. This will eliminate any risk of ASLR bypass due to the vsyscall fixed address mapping. Attempts to use the vsyscalls will be reported to dmesg, so that either old or malicious userspace programs can be identified.
X86_VSYSCALL_EMULATION=is not set X86_32, X86_64 3.19, 4.0-4.20, 5.0-5.17 If set, this enables emulation of the legacy vsyscall page.
  • Security Policy Attacks: These are attacks which attempt to gain elevated (root) privileges within a system, generally through the use or execution of a misconfigured binary or file. Mitigations for this mostly rely on Linux Security Modules (LSMs) which extend discretionary access control (DAC) or implement mandatory access control (MAC, Security-Enhanced Linux).
Expected Conditional Architectures Kernel Versions Note
SECURITY=Y ARM, ARM64, X86_32, X86_64 2.5.50-2.5.75, 2.6.0-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 This allows you to choose different security modules to be configured into your kernel.
SECURITY_YAMA=Y ARM, ARM64, X86_32, X86_64 3.4-3.19, 4.0-4.20, 5.0-5.17 This selects Yama, which extends DAC support with additional system-wide security settings beyond regular Linux discretionary access controls. Currently available is ptrace scope restriction. Like capabilities, this security module stacks with other LSMs. Further information can be found in Documentation/admin-guide/LSM/Yama.rst.
SECURITY_WRITABLE_HOOKS=is not set ARM, ARM64, X86_32, X86_64 4.12-4.20, 5.0-5.17 If SECURITY_SELINUX_DISABLE must be set, make sure this is not set. Subsequent patches will add RO hardening to LSM hooks, however, SELinux still needs to be able to perform runtime disablement after init to handle architectures where init-time disablement via boot parameters is not feasible. Introduce a new kernel configuration parameter CONFIG_SECURITY_WRITABLE_HOOKS, and a helper macro __lsm_ro_after_init, to handle this case.
SECURITY_SELINUX_DISABLE=is not set ARM, ARM64, X86_32, X86_64 2.6.6-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 Do not allow NSA SELinux runtime disable
SECURITY_LOCKDOWN_LSM=Y ARM, ARM64, X86_32, X86_64 5.4-5.17 Enables the lockdown LSM, which enables you to set the lockdown=integrity or lockdown=confidentiality modes during boot. Integrity attempts to block userspace from modifying the running kernel, while confidentiality also restricts reading of confidential material.
SECURITY_LOCKDOWN_LSM_EARLY=Y ARM, ARM64, X86_32, X86_64 5.4-5.17 Enable lockdown LSM early in init
LOCK_DOWN_KERNEL_FORCE_CONFIDENTIALITY=Y ARM, ARM64, X86_32, X86_64 5.4-5.17 The kernel runs in confidentiality mode by default. Features that allow the kernel to be modified at runtime or that permit userland code to read confidential material held inside the kernel are disabled.
SECURITY_SAFESETID=Y ARM, ARM64, X86_32, X86_64 5.1-5.17 SafeSetID is an LSM module that gates the setid family of syscalls to restrict UID/GID transitions from a given UID/GID to only those approved by a system-wide whitelist. These restrictions also prohibit the given UIDs/GIDs from obtaining auxiliary privileges associated with CAP_SET{U/G}ID, such as allowing a user to set up user namespace UID mappings.
SECURITY_LOADPIN=Y ARM, ARM64, X86_32, X86_64 4.7-4.20, 5.0-5.17 Any files read through the kernel file reading interface (kernel modules, firmware, kexec images, security policy) can be pinned to the first filesystem used for loading. When enabled, any files that come from other filesystems will be rejected. This is best used on systems without an initrd that have a root filesystem backed by a read-only device such as dm-verity or a CDROM.
SECURITY_LOADPIN_ENFORCE=Y ARM, ARM64, X86_32, X86_64 4.20, 5.0-5.17 If selected, LoadPin will enforce pinning at boot. If not selected, it can be enabled at boot with the kernel parameter “loadpin.enforce=1”.

 

System Architecture

Many security features are architecture specific because of a specific hardware level reason (differing instruction set, caches, branch predictors, and more) or merely because they have not been implemented on a specific architecture. Looking at DEBUG_SET_MODULE_RONX, we find that it was a relatively recent addition for ARM and ARM64 architectures.

Expected Conditional Architectures Kernel Versions Note
DEBUG_SET_MODULE_RONX=Y X86_32, X86_64 2.6.38-2.6.39, 3.0-3.19, 4.0-4.10 Helps catch unintended modifications to loadable kernel module’s text and read-only data. It also prevents execution of module data.
DEBUG_SET_MODULE_RONX=Y ARM64 3.18-3.19, 4.0-4.10
DEBUG_SET_MODULE_RONX=Y ARM 3.14-3.19, 4.0-4.10

Looking at the Spectre and Meltdown variants, there are differing options depending on architecture as well:

Expected Conditional Architectures Kernel Versions Note
HARDEN_BRANCH_PREDICTOR=Y ARM, ARM64 4.16-4.20, 5.0-5.17 (Spectre related) Speculation attacks against some high-performance processors rely on being able to manipulate the branch predictor for a victim context by executing aliasing branches in the attacker context. Such attacks can be partially mitigated against by clearing internal branch predictor state and limiting the prediction logic in some situations.
RETPOLINE=Y X86_32, X86_64 4.15-4.20, 5.0-5.17 Avoid speculative indirect branches in kernel (Spectre Mitigation)

 

Timesys Kernel Hardening Analysis Tool

This tool is available as part of the meta-vigishield layer, learn more about VigiShield here.

This Yocto-based tool can perform some security-minded analysis of your kernel configuration. The tool generates a report that shows the status of many configuration items which we have assessed as being security related.

The output from the Timesys Kernel Hardening Analysis Tool is formatted as a Comma Separated List (CSV). As an example, here are the first few lines from a sample report are:

Detected Kernel Version: 5.0.19
Detected Architecture: ARM
Detected configuration at: /mnt/Projects/Yocto/build/tmp/work-shared/qemuarm-uboot/kernel
Report Generated At: 2022-02-01 12:22:07.088485
Expected Conditional Status    Priority Kernel Versions Category Note
GCC_PLUGIN_RANDSTRUCT=Y FAILED 3 (High) 4.13-4.20, 5.0-5.17 gcc_plugin Randomizes layout of sensitive kernel structures
GCC_PLUGIN_ARM_SSP_PER_TASK=Y SKIPPED (Version mismatch) 3 (High) 5.2-5.17 gcc_plugin Generates a separate stack canary value for each task, so if one task’s canary value is leaked it does not cause all other tasks to become vulnerable.
GCC_PLUGIN_STRUCTLEAK=Y SKIPPED (Version mismatch) 3 (High) 5.2-5.17 gcc_plugin This plugin is available to identify and zero-initialize stack variables that may have passed through uninitialized
STACKPROTECTOR=Y PASSED 3 (High) 4.18-4.20, 5.0-5.17 stack_canary This option turns on the “stack-protector” GCC feature. This feature puts, at the beginning of functions, a canary value on the stack just before the return address, and validates the value just before actually returning. Stack based buffer overflows (that need to overwrite this return address) now also overwrite the canary, which gets detected and the attack is then neutralized via a kernel panic.
STACKPROTECTOR_STRONG=Y PASSED 3 (High) 4.18-4.20, 5.0-5.17 stack_canary Adds the CONFIG_STACKPROTECTOR canary logic to additional conditions related to variable assignment.
INIT_ON_ALLOC_DEFAULT_ON=Y SKIPPED (Version mismatch) 3 (High) 5.3-5.17 memory_protection All page allocator and slab allocator memory will be zeroed when freed, eliminating many kinds of “uninitialized heap memory” flaws, especially heap content exposures.
INIT_ON_FREE_DEFAULT_ON=Y SKIPPED (Version mismatch) 3 (High) 5.3-5.17 memory_protection More expensive form of INIT_ON_ALLOC_DEFAULT_ON. The primary difference is that data lifetime in memory is reduced, as anything freed is wiped immediately, making live forensics or cold boot memory attacks unable to recover freed memory contents.
STRICT_KERNEL_RWX=Y PASSED 4.11-4.20, 5.0-5.17 3 (High) memory_protection If this is set, kernel text and rodata memory will be made read-only, and non-text memory will be made non-executable. This provides protection against certain security exploits (e.g. executing the heap or modifying text)
(MODULE_SIG_FORCE=Y) OR (MODULES=is not set) FAILED
DESCRIPTION: MODULES FAILED 1 (Low) 2.5.45-2.5.75, 2.6.0-2.6.39, 3.0-3.19, 4.0-4.20, 5.0-5.17 module_security You should not allow for modules to be loaded unless you have the proper signing and signature checks enabled. Allowing the kernel to load unsigned modules can be dangerous
DESCRIPTION: MODULE_SIG_FORCE FAILED 3 (High) 3.7-3.19, 4.0-4.20, 5.0-5.17 module_security Require modules to be validly signed

In this case, GCC_PLUGIN_RANDSTRUCT was not enabled in the kernel configuration file, so the return status is, “FAILED.

GCC_PLUGIN_ARM_SSP_PER_TASK, GCC_PLUGIN_STRUCTLEAK, INIT_ON_ALLOC_DEFAULT_ON, STRICT_KERNEL_RWX, and INIT_ON_FREE_DEFAULT_ON are all options which were available after the 5.0.19 kernel version, so those have been skipped with a, “SKIPPED (Version mismatch)” message.

The rest of the options were set appropriately and passed (STACKPROTECTOR and STACKPROTECTOR_STRONG).

Modules are also enabled without any forced signature checking, so the OR conditional [(MODULE_SIG_FORCE=Y) OR (MODULES=is not set)] for that has also failed.

Click here to get a free guide containing the entire list of recommended kernel security configurations to consider for hardening.

 

Nathan Barrett-Morrison, Senior Embedded Systems Engineer, has 8+ years of experience in designing and debugging embedded systems. In the past, he has been a software technical lead on ultrasonic-based digital processing systems and direct thermal printing solutions. He specializes in providing customers with lower level board support packages, drivers, and numerous other services. Nathan holds a bachelor’s degree in Electrical and Computer Engineering from OSU (Ohio State University).

About Timesys

Timesys has extensive experience with embedded system development and lifecycle management. Timesys has been instrumental in working with global leader semiconductor manufacturers with smart, quick and quality solutions for highly complex systems with accelerated product innovation and multiple product variants.

Discretionary Access Control (DAC) Hardening

Discretionary Access Control (DAC) Hardening

 

 

Discretionary Access Control hardening can further improve your embedded system’s security by limiting userspace access to proprietary intellectual property, exploitable binaries, and privileged information. The example permissions shown here are defaults produced during a demonstration Yocto build.

In Linux, a file has the following relevant parameters (when listing a file with the “ls” command):

File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
/bin/tar.tar root root r w x r x r x

Where:

r Readable
w Writable
x Executable
No permissions
s Executable + Set User ID or Group ID

To fully secure your system and limit your vulnerable attack surface, these permissions should be restricted as much as possible.

 

User Accounts

You’ll want to ensure that your applications are well isolated from one another through the use of strict user accounting. Many embedded systems run everything under the root account. If security is a concern, this is not acceptable. If an application running as root becomes subject to an exploit (zero-day or otherwise), an attacker will be able to gain root access to your system. Compartmentalizing and restricting access between applications is critical.

This is why applications like NTPD, Apache2, Avahi, and DHCPD change their user after being initially launched as root. If someone were to successfully perform an injection-style attack on one of these network-based applications, root privileges are not exposed and, ideally, only the attacked application’s settings and files are modifiable.

Proprietary applications and intellectual property should also be executed under a separate, non-root, account. Alternatively, you can also promptly drop root permissions and change users after launching if desired. These applications are usually not as heavily tested for security vulnerabilities, as they’re not open source. This makes keeping them isolated from root access especially important, as they may be easier to attack.

 

SUID/SGID Protections

Also check for “Set User ID” and “Set Group ID” bits in your file system permissions. These have been commonly exploited in the past and can lead to unexpected root privilege escalation. Consider this CVE that is related to exploiting the commonly used Shadow utility (groupadd, groupdel, groupmod, useradd, userdel, and usermod).

  • In an embedded system, you may not need to add/remove/modify users after building the BSP. If so, consider deleting these tools from your system altogether.

When using modern versions of Busybox, the binary has been split in to two pieces. One with SUID permissions and one without.

In this demonstration build, the SUID binary is used (via symbolic links) for the following:

/usr/bin/passwd     -> /bin/busybox.suid
/usr/bin/traceroute -> /bin/busybox.suid
/usr/bin/vlock      -> /bin/busybox.suid
/bin/ping           -> /bin/busybox.suid
/bin/ping6          -> /bin/busybox.suid
/bin/login          -> /bin/busybox.suid
/bin/su             -> /bin/busybox.suid

You may not need some (or any) of these features in your field-ready systems. They should be disabled via the Busybox configuration whenever possible.

File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
/bin/busybox.suid root root r w s r x r x
/bin/busybox.nosuid root root r w x r x r x

 

Read Permissions

Restricting group and other users from read access on any system files which may expose vulnerable information is another way to improve security. By default, Yocto sets most files to be user and group readable. This is largely unnecessary, as not all users should need read access on configuration files and applications which they do not have execution privileges for.

For example, consider fw_env.config. This file is used to control the address and size of the bootloader›s environment information (from within Linux userspace). While removing the read permissions on it may not prevent a motivated attacker from finding the addresses manually, why make it easier for them? There’s most likely no reason every user in your system needs read access to this file.

root@board:~# cat /etc/fw_env.config

# [Device]      [Offset]    [Size]   [Sector Size]   [Sector Count]
/dev/mmcblk1    0xC0000     0x2000      0x2000             1

 

File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
/etc/fw_env.config root root r w r r

Properly adjusting read permissions may also be important for clone and binary analysis protections. Consider that a malicious entity is targeting your system in an attempt to copy out your proprietary binary files. The ARMv7 and ARMv8 architectures are so common now, it may be exceptionally easy to transplant these files on to a replica board. They may also disassemble said binary and attempt to find security flaws in it. If you can prevent read access to the application, it becomes more difficult for an attacker to obtain a binary copy.

Contemplate a system with three accounts named: root, user, and proprietary_app. Your system initialization manager will start up as root and launch your application under the separate proprietary_app account (or the application itself will quickly do so). At this point, if you have the permissions adjusted accordingly:

  • The application does not expose root privileges if an attacker finds an exploit on it, as it’s running under a separate account.
  • The application can only be read by two accounts (root and proprietary_app). If these two accounts are then passwordless, obtaining a binary dump is more difficult (bruteforce attacks via serial/SSH are not possible). It would also protect against offline cracking of the /etc/shadow file, although if an attacker is able to obtain this then they’ve already escalated to root privileges and that most likely becomes a moot point.
File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
/usr/bin/some_proprietary_app proprietary_app proprietary_app r w x

 

Execution Permissions

This is another avenue through which you can help protect your system. When looking at the executable binaries in your system, you may find group and user executable bits which do not need to be set. Considering U-Boot’s environment tools (from Linux userspace) again, the tool binaries are also unnecessarily executable by group and other users. This may give an attacker an easy method of tampering with the bootloader (if it has not been adequately hardened).

File Info Owner Group Users Other Users
Name Owner Group Read Write Execute Read Write Execute Read Write Execute
./sbin/fw_printenv root root r w x r x r x
./sbin/fw_setenv root root r w x r x r x

 

Write Permissions

During the demonstration build with Yocto, only the owner appears to be given write access to files. There does not appear to be any improvement necessary here. Your build system’s results may vary. If you find a file with inappropriate write permissions then you will want to fix it.

 

Timesys DAC Reporting Tool

Timesys has a discretionary access control review tool which will generate a comma-separated value (CSV) file during your Yocto build process. This file contains a list of all your binaries, user and group ownerships, and their various read/write/execute permissions. This enables you to more quickly scan your file system for potential conflicts. The tool is part of Timesys VigiShield offering. To view a sample report generated by the tool, click here.

 

Nathan Barrett-Morrison, Senior Embedded Systems Engineer, has 8+ years of experience in designing and debugging embedded systems. In the past, he has been a software technical lead on ultrasonic-based digital processing systems and direct thermal printing solutions. He specializes in providing customers with lower level board support packages, drivers, and numerous other services. Nathan holds a bachelor’s degree in Electrical and Computer Engineering from OSU (Ohio State University).

About Timesys

Timesys has extensive experience with embedded system development and lifecycle management. Timesys has been instrumental in working with global leader semiconductor manufacturers with smart, quick and quality solutions for highly complex systems with accelerated product innovation and multiple product variants.

5 Lessons Learned From the Log4j Vulnerability…and How the Embedded Industry Can Be Better Prepared for the Next One

5 Lessons Learned From the Log4j Vulnerability…and How the Embedded Industry Can Be Better Prepared for the Next One

 

 

Log4j has set the security world ablaze. With the first vulnerability (CVE-2021-44228) ranked with a CVSS score of 10 — as high as the scale goes — everyone is paying attention.

It’s true that the embedded world appears to be largely unaffected: having reviewed nearly 50,000 Software Bill of Materials (SBOMs), we found that less than .05% of those reviewed use log4j. That said, there are still some major lessons to be learned from this historic attack.

  1. Have an accurate SBOM
    An accurate Software Bill of Materials (SBOM) is your best friend when working in vulnerability management. When news of a new CVE breaks, the quickest way to know whether your device has been affected is to have an accurate SBOM and scan it to determine if you need to take action. And it’s not just a nice-to-have — providing a purchaser with an SBOM is included as part of an executive order from President Biden earlier this year to improve the United States’ cybersecurity.
  2. Track vulnerability lists
    The National Vulnerability Database (NVD) is the largest source for vulnerability tracking, but it’s not the only one, nor is it always quickly updated. Tracking and cross-referencing multiple vulnerability lists is the best way to stay ahead of CVEs. Additional places to track CVE information include but are not limited to: Upstream mailing list, issue trackers, security bulletins, Debian/Ubuntu/RedHat security trackers, and SoC vendor advisories.
  3. Monitor consistently
    An accurate SBOM and knowing which lists to follow are all well and good, but only if you monitor the lists consistently. With approximately 350 new CVEs every week, they must be constantly monitored, ideally with alerts set up for the most critical CVEs.

  4. Consider specialized tools to give you an advantage
    Once you know you’ve been affected by a CVE, the work has just begun. How critical is it? Is there a patch? What CVEs need to be taken care of first? You can find a wide variety of tools for tracking, filtering, triaging, and even remediating vulnerabilities — all of which are needed to keep you one step ahead of cyber attacks. You can find an excellent list of Software Composition Analysis (SCA) tools with real customer reviews from Gartner here.
  5. Have a response plan ready
    Log4j caught a lot of companies by surprise. If this vulnerability has taught us anything, it’s that we need to be ready to respond when — not if — the next one strikes. When the news of the next major vulnerability hits, will you be scrambling for a solution, or will an early alert from your system mean you’re already applying fixes and protecting your customers?

There are plenty of security scanning tools available on the market, but Timesys Vigiles is the only vulnerability monitoring and remediation tool optimized for embedded. With Vigiles’ curated CVE database, continuous security feed based on your SBOM, powerful filtering, and easy triage tools, it puts you ahead of the curve and poised to take action.

Don’t get blindsided by the next major CVE — try a free 30 day trial of Vigiles Prime today.

 

Leah Simoncelli is the Digital & Community Engagement Manager at Timesys. Additionally, she runs a global pitch competition for hardware startups with Innovation Works, one of the most active seed stage investors in the country. She has over a decade of experience in marketing, management and communication and holds a BA from American University in Washington, DC.

About Timesys

Timesys has extensive experience with embedded system development and lifecycle management. Timesys has been instrumental in working with global leader semiconductor manufacturers with smart, quick and quality solutions for highly complex systems with accelerated product innovation and multiple product variants.

The many challenges of Linux OS / BSP security maintenance

The many challenges of Linux OS / BSP security maintenance

 

 

The “upstream first” strategy is only half of the story for embedded devices

A recent blog post by Kees Cook, a Linux security expert and Google security engineer, illustrates the challenges in maintaining the security of the Linux kernel. One of the main takeaways from the blog is: “If you’re not using the latest kernel, you don’t have the most recently added security defenses (including bug fixes).”

Going the “upstream first” route is the absolute best way of keeping the kernel secure. However, it is only part of the story. The challenges faced by device manufacturers running on Linux on embedded devices is vastly different.

(more…)

Click to Hide Advanced Floating Content

Keep your IoT device secure throughout its lifecycle