Bootable Breakout

I’m currently (at the time of writing this) performing a security assessment of an ESP32-based embedded device for a client. While this assessment is not purely hardware focused, gaining access to the device’s firmware is a primary goal as reversing firmware is often an easy way to identify other vulnerabilities.

The ESP32 supports JTAG and the target device routes the JTAG pins to a client-proprietary breakout. While it’d be completely possible to solder leads onto the breakout’s pads, I was having trouble with shorting due to how close the pads were to each other. Additionally, the ESP32 is a QFN form-factor meaning that, while externally accessible, the pins on the chip are tiny and I’m nowhere near good enough to solder to them directly.

My next thought was something along the lines of “It’d be really nice if this were on a breakout board.” The problem with a breakout board is that, while I could (assuming it’s enabled) get JTAG to work, the ESP32’s peripherals wouldn’t be there including any external storage (in this case, an EEPROM).

There are solutions where I could dump the ESP32’s internal storage via JTAG and external storage via SPI, but I would potentially miss out on any interesting information in RAM as the ESP32 wouldn’t boot correctly without the EEPROM. Or I could breakout everything to breadboards and connect them all up, but that’d take some time and would be subject to smoke-letting should I make certain mistakes.

Instead I found another solution using a standard time-vs-cost trade off… Why spend a lot of time breadboarding everything when I could spend a little money on a development board and replace the necessary components.

Enter the Widora-air ESP32 development shield: https://amzn.to/2H2Ly7s (sorry, I got the last one available on Prime for the moment). This little guy had the primary things I needed in a breakout: A spot for the ESP32, a spot for the EEPROM, and all the ESP32 pins broken out. Additionally, it gave me USB to serial so I didn’t have to wire up UART to monitor what was going on.

I removed the ESP32 and EEPROM from my target device and replaced them onto the Widora-air then wired everything up to my JTAG adapter. I also cloned Espressif’s fork of OpenOCD, which had full support for the ESP32. (I also spent about 10 minutes trying to take this picture).

Widora-air + JLink

Everything worked exactly as I hoped as I was able to start carving out ROM and RAM for some vulnerability hunting.

OpenOCD output

While, I got everything working well enough, there are still missing peripherals and potentially some interesting bits of memory that I’m missing because of that. But due to this little trick, I have a head start and have confirmed JTAG is enabled. Knowing JTAG is enabled makes the prospect of some painstaking soldering to the proprietary breakout feel less daunting.

Modding MiniPro To Support Unsupported Variants

Several months ago, during a hardware device assessment, I encountered a TC58NVG2S0HBAI4 NAND EEPROM which hosted the firmware of the target device. The problem I had was that my MiniPro Universal Programmer did not support this specific variant. It did however, support several similar variants from the same vendor.

For those who may be unaware of the MiniPro, it is a cost effective universal programmer with support for 10’s of thousand of chips and variants. The latest MiniPro, the TL866II+, is available for around $50-$60 but also can come packaged with numerous adapters and sockets for around $100-$120.

The following is a quick on how to modify the MiniPro’s chip info .dll to change the necessary information to support this variant. Couple things to note first though. This method does not “add” support so much as it replaces an existing configuration meaning you’ll end up removing support for the source chip. Also, I make no guarantees that the offsets used here work for other chips, but the guess-and-check methodology should still apply. Lastly, I’ll refer to all TL866 models as MiniPro, same for the software. Where differences exist, I’ll note that then. Oh, and make backups, lots of them.

Modding the .dll

Before we get started, we need a few things. We need to find the .dll to modify and we need the datasheets for our source and target chip variants. For older TL866 models you’ll find InfoIC.dll in your MiniPro installation directory. For the newer TL866II+, you find InfoIC2plus.dll in the Xgpro installation directory. In my case, I installed both C:/. The datasheets for the chips are:

According to the first sentences of each datasheet, the primary differences were the sizes of pages and blocks. I also figured it was safe to assume that the pinouts were identical. The source chip info in MiniPro matched the datasheet.

Source Chip Info in Xgpro

Source Chip:

  • Page Size: 2048
  • Spare Size: 64
  • Pages Per Block: 64
  • Blocks: 4096

Target Chip:

  • Page Size: 4096
  • Spare Size: 256
  • Pages Per Block: 64
  • Blocks: 2048

I used IDA Pro, but any hex editor should do. Upon opening the .dll in IDA, I noted a very large data section. I opened the strings few to quickly find the section I wanted to modify.

IDA Strings View of Source Chip

Each chip’s info section appeared to be 73 bytes long and started with the chip name in ASCII. Here is the unmodified source chip:

	 x0 x1 x2 x3 x4 x5 x6 x7  x8 x9 xA xB xC xD xE xF
	 ------------------------------------------------------------------
0x00|54 43 35 38 4E 56 47 32  53 33 20 40 54 53 4F 50  TC58NVG2S3 @TSOP
0x10|34 38 00 00 00 00 00 00  00 00 00 00 00 00 00 00  48..............
0x20|00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 21  ...............!
0x30|00 00 00 00 00 00 00 00  A0 00 00 00 00 21 40 08  ........á....!@.
0x40|08 00 00 00 00 08 00 00  00 10 00 00 40 00 01 01  ............@...
0x50|98 DC 00 00 00 00 00 00  02 00 00 00 40 00 00 00  ÿ_..........@...
0x60|08 00 00 00 F8 00 00 00  13 00 00 00 00 00 00 00  ....°...........
0x70|01 00 00 00 									  ....

After some experimentation, I determined that the sizes were stored in little endian, in the following locations:

0x44-0x45: Page Size
0x48-0x49: Blocks
0x4C-0x4D: Spare Size
0x5C-0x5D: Pages per Block

If we check the locations in the chip info, we see that 0x44-0x45 equals 0x0800 or 2048. 0x48-0x49 and 0x4C-0x4D equal 0x40 or 64. And 0x5C-0x5D equal 0x1000 or 4096.

Modified each of those section to set the correct values for my target chip and successfully dumped the contents.

Reading target chip, contents not shown due to protect client

Methodology TL;DR

The methodology here is pretty straight forward. If you want to “add” support for a chip, simply look for another chip variant or at least something that would have the same pinout or physical form factor to use as your source chip.

Once you have a source chip, determine the differences by comparing datasheets.

Take your source values and convert to little endian base 16 and look for for those numbers in the source chip’s section of the MiniPro .dll. Be sure to make a backup first.

Save your edits, open mini pro, and confirm your changes are correct.

Dump contents of your target chip and profit or whatever.

UART-to-Root: The (Slightly) Harder Way

Quick Note: This post assumes some knowledge of UART and U-boot, and touches slightly on eMMC dumping.

Many familiar with hardware hacking know that UART can be a quick and easy way to find yourself with a shell on a target device. Often times, especially in older home routers and the like, you’ll be automatically logged in as root or be able to log in with an easily-guessed or default password.

In other circumstances, you may need to edit some boot arguments in the bootloader to trigger a shell (such as adding a 1 for single-user mode or adding init=/bin/sh). With this initial shell, you can dump and crack passwords or modify the firmware to grant access without the modified bootargs (change password).

Recently, I came head-to-head with a device that had a slightly more complicated boot process with many environment variables setting other environment variables that eventually called a boot script from an eMMC that did more of the same.

Some Background

My target device was being driven by a cl-som-imx6; an off-the-shelf, bolt-on System on Module from Compulab. My target version of the cl-som-imx6 utilized a 16 gig eMMC for firmware storage that had two partitions: a FAT boot partition (in addition to U-boot on an EEPROM) and an EXT4 Linux filesystem.

cl-som-imx6 with eMMC removed

My first goal for this device was to get an active shell on the device while it was fully booted. Since I had multiple copies of my target device, I went for a quick win and removed eMMC then dumped it’s contents with the hope of recovering and cracking password hashes. While I was able to get the hashes from /etc/shadow, I was disappointed to see they were hashed with sha512crypt ($6$) and have yet been unable to crack them.

Without valid credentials, my next goal was to modify boot args to bypass authentication and drop me directly into a root shell, with the hope of being able to change the password. The classic init=/bin/sh trick.

It’s important to note that when modifying the bootargs with init=/bin/sh, the device will not go through its standard boot process, therefore it will not kick off any scripts or applications that would normally fire on boot. So, while you may have a root shell, you will not be interacting with the device in its normal state. It is also temporary and will not persist after reboot.

The Problem

This is where it started getting a bit tricker. In my experience, U-boot usually has an environment variable called bootargs that passes necessary information to the kernel. In this case, there were several variables that set bootargs under different circumstances. I attempted to modify every instance where bootargs were getting set (to add init=/bin/sh) to no avail.

# binwalk part1.bin 

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
28672         0x7000          Linux kernel ARM boot executable zImage (little-endian)
34516         0x86D4          LZO compressed data
34884         0x8844          LZO compressed data
35489         0x8AA1          device tree image (dtb)
1322131       0x142C93        SHA256 hash constants, little endian
3218379       0x311BCB        mcrypt 2.5 encrypted data, algorithm: "5o", keysize: 12292 bytes, mode: "A",
3982809       0x3CC5D9        device tree image (dtb)
4273569       0x4135A1        Unix path: /var/run/L
4932888       0x4B4518        xz compressed data
5359334       0x51C6E6        LZ4 compressed data, legacy
5513216       0x542000        uImage header, header size: 64 bytes, header CRC: 0x665C5745, created: 2018-09-26 16:36:26, image size: 2397 bytes, Data Address: 0x0, Entry Point: 0x0, data CRC: 0x9F621F80, OS: Linux, CPU: ARM, image type: Script file, compression type: none, image name: "boot script"
5517312       0x543000        device tree image (dtb)
...

During this time, I also discovered that there appeared to be some sort of watch-dog active that would completely reset the device after about 2 minutes of playing around in U-boot’s menu options.

As a note: I don’t believe this was an intended “Security” function but rather an unintended effect caused by the rest of the device (attached to the cl-som-imx6) after it failed to fully boot after X time.

After an hour or so reading the UART output during boot and attempting to understand the logic flow of the environment variables, I discovered that U-boot was calling a boot script before it touched any of my edited boot args. Luckily for me, this boot script was being called from the eMMC’s boot partition, which I had dumped previously.

Binwalk quickly identified the boot script’s location, but failed to extract it. Using the offset of the script as a starting point and the offset of the following signature as the end point, I used dd to extract the script. As luck would have it, the script was actually a script (plaintext) and not a binary.

# dd if=partition1.bin of=boot.script skip=5513216 count=4096 bs=1
4096+0 records in
4096+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0173901 s, 236 kB/s

The script was exactly 80 lines and contained several if/else statements, but most importantly, it had only one line setting the bootargs. At this point, my theory was that the only environment variables that mattered were being set by this script. I needed to modify this script to add init=/bin/sh.

# cat boot.script 
setenv loadaddr 0x10800000
setenv fdt_high 0xffffffff
setenv fdt_addr 0x15000000
setenv bootm_low 0x15000000
setenv kernel_file zImage
setenv vmalloc vmalloc=256M
setenv cma cma=384M
setenv dmfc dmfc=3
setenv console ttymxc3,115200
setenv env_addr 0x10500000
setenv env_file boot.env
setenv ext_env ext_env=empty
...
setenv setup_args 'setenv bootargs console=${console} root=${rootdev} rootfstype=ext4 rw rootwait ${ext}'
...

The next hurdle was that I didn’t have a direct way of modifying the contents of the eMMC without removing it and that’s the easy part. Getting it back on the SOM would have been tougher work than I was willing to tackle at the time.

The Solution

Without a simple way to modify the boot script, I decided to try to manually copy and paste each line of the script into the U-boot menu shell and, if necessary, remove all other environment variables.

I ran into two problems with this approach. First, any line over 34 characters that I tried to paste got truncated to 34. This was likely just caused by the ft232h buffer or something else with the serial connection. Second, and more annoying, was the watch-dog reset. There was simply no way I was going to paste in 80 lines (especially as many would require multiple c/p’s due to the 34 char limit). Even after removing as much as possible

My only answer was to automate the process. I had previously been playing around with the idea of bruteforcing simple 4 digit security codes, so I already had the outline of a script ready.

I modified the script to read lines from an input file and write them to the serial device, where any line over 32 (to be safe) characters would be chucked up. To ensure data was sent at the correct time, I put made sure the script waited for the shell prompt to return before sending the next line, with an additional .5 second sleep for good measure. Also, since the script would take over my ft232h, I needed to make sure it stopped autoboot at the correct time to enter the U-Boot shell.

This approach worked perfectly and I was dropped into a /bin/sh shell as root. I then took control of my ft232h again so I could interact manually. With a quick passwd, I changed the root password and rebooted. As the modified environment variables didn’t persist through reboot, the device booted as normal and presented me with a login prompt. I entered my newly set password and I was in.

Serial and script output ending in shell
Changing password

I’d post a screenshot of the final successful login after full boot, but I’d have to redact too much stuff that it doesn’t make any sense.

As a note: So I could keep an eye on everything, I used a second ft232h to watch the target’s TX pin and since it echoed everything back, I could also see my script’s input. Also, the watch-dog was still in effect since the device didn’t boot as it should have, therefore I had to be quick on the passwd.

The Script

Below is the script exactly as I used it. With a touch of modification to the until1 and until2 vars, it should be useable for other targets.

#!/usr/bin/env python
# By Mike Kelly
# exfil.co
# @lixmk

import serial
import sys
import argparse
import re
from time import sleep

# Key words 
until1 = "Hit any key to stop autoboot:"
until2 = "SOM-iMX6 #"

# Read from device until prompt identifier
# Not using resp in this, but you can 
def read_until(until):
    resp = ""
    while until not in resp:
        resp += dev.read(1)
    return resp

def serialprint():
    # Get to U-Boot Shell
    read_until(until1)
    dev.write("\n")
    
    # Wait for U-Boot Prompt
    read_until(until2)
    sleep(.5)
    dev.write("\n")
    with open(infile) as f:
        lines = f.readlines()
        for line in lines:
            # Lines < 32
            if len(line) < 32:
                read_until(until2)
                sleep(.5)
                print "Short Line: "+line.rstrip("\n")
                dev.write(line)
            # Break up longer lines
            else:
                read_until(until2)
                sleep(.5)
                for chunk in re.findall('.{1,32}', line):
                    print "Long Line: "+chunk.rstrip("\n")
                    dev.write(chunk)
                    sleep(.5)
                dev.write("\n")
        print "" 
        print "Done... Got root?"
        exit()

if __name__ == '__main__':
    # Argument parsing
    parser = argparse.ArgumentParser(usage='./setenv.py -d /dev/ttyUSB0 -b 115200 -f infile.txt')
    parser.add_argument('-d', '--device', required=True, help='Serial Device path ie: /dev/ttyUSB0')
    parser.add_argument('-b', '--baud', required=True, type=int, help='Serial Baud rate')
    parser.add_argument('-f', '--infile', type=str, help="Input file")
    args = parser.parse_args()
    device = args.device
    baud = args.baud
    infile = args.infile
    
    # Configuring device
    dev = serial.Serial(device, baud, timeout=5)
    # Executing
    serialprint()