Taking things apart
This is an annotated transcript of a talk I gave at HackRice about reverse engineering and some of the fun projects/hacks they could do with reverse engineering skills. Some of the results were reported in the news.
The Joy of Taking Things Apart
When you hear "Reverse Engineering", you not might think of all of the good things you can use these skills for. You might not think of yourself as a reverse engineer, but if you do any sort of programming, you are probably already one.
We’ve all encountered this sort of "documentation" for APIs where the comments are useless, outdated or contribute to negative understanding of the system. Sometimes the only course of action is to read the source to try to figure out what sort of processing this function is doing on the Foo object.
Frequently you don’t have access to the source, so it becomes necessary to turn to other techniques, like endless searching stackoverflow for answers. Other times more invasive techniques are required, like black box testing with various inputs, or even using an interactive disassembler to make sense of the machine code.
Knowing how to do this sort of reverse engineering makes you a better programmer because you must build a mental model of what is going on inside of this system. This helps you learn better debugging skills on your own projects, because you can think through what must be happening inside of code.
Reverse engineering also allows you to add the features that you feel are important. As an example of this, in 2009 I started making short films with the Canon 5D Mark 2, but was very frustrated with some of the limitations in the camera's firmware. It was missing lots of basic functionality and this frustration led me to attempt to add the features that I wanted.
Many of these projects start with a simple goal: find out if your code is running in the system. This is the first screenshot of my success with the Canon 5D: I’ve modified the firmware version number to display three extra vanity letters. Re-writing strings is a good easy technique for figuring out if you have "won" and your code is running on the system.
This little hack turned into Magic Lantern, the open source programming environment for Canon DSLR cameras. It's one of my most popular reverse engineering projects; at this point there are tens of thousands of users on all seven continents and maybe even in space.
These are all features that the users of the camera wanted, not necessarily what the original manufacturer intended. The big ones are manual control of all camera parameters, as well as raw video recording. One of the best parts of reverse engineering is that it allows you to make a system work the way you want.
Another useful reverse engineering skills is creating interoperable programs or devices. Games are a popular place to start with this sort of thing.
One of my colleagues at Two Sigma proposed a challenge to automate Dr Mario on the Wii. He had done some impressive work on the computer vision to detect the corners of the board, ortho-rectify it and detect the game components. Next he needed a way to talk to the console from his program.
Dr Mario uses the classic controller, which plugs into the Wiimote using a special connector. Thanks to someone else figuring out and writing down how the i2c pins are connected and the protocol that it speaks, we were able to build an Arduino interface.
Using this reverse engineered USB to Wiimote adapter that would interoperate with the stock game and console, we put together a fun programming contest at Two Sigma. The contestants wrote AIs that watched the video, generated strategies and sent the commands to the interface. For a short while one of our AIs was the top rated player on the online game.
You can also use reverse engineering to do things like personalize the games. Mike Mika modified the Donkey Kong ROM for his daughter to have Mairo held captive and rescued by Pauline. This required finding the sprites in the ROM and copying them around to swap the roles of the various characters.
Sometimes you just want to make art. Corey Arcangel created a beautiful serene art piece in which he modified Super Mario to have no enemies, no world and no mario: just the clouds endlessly drifting by. This required reverse engineering enough of the game using a 6502 disassembler to figure out what parts were involved in running the game and replace them with NOPs.
We ended up with some surplus robots with no documentation and no interfaces, so it was necessary to reverse engineering the connectors and cabling. Step one was lots of contiunity testing with the multimeter - put it in beep mode and probe until all of the wires were mapped. And then we used the oscilloscope to figure out the quadrature position encoders while we rotated the joints by hand. We published these findings along with a six DOF inverse kinesmatics controller that we wrote to allow precise movement of the arms.
This reverse engineering project turned into another Two Sigma programming contest -- we provided a REST API to retrieve the images from overhead cameras and to command the arm. The teams wrote AI bots to play shuffle board against each other.
And the reverse engineered robot was pretty good!
At our hackerspace, NYC Resistor, we have a laser cutter that can cut paper, woods, acrylics and also etch some metals. But it has a windows only printer driver that doesn't work for many of our members who want to use OS X or Linux. We knew that we could fix that limitation with reverse engineering of the communication between their driver and the laser cutter.
A packet capture of the communication between the windows driver and the laser showed that it was HPGL formatted data sent over the normal Unix LPD network port. We can find those two specifications and implement them! Armed with this insight, we were able to write a full command-line tool that interoperates with the laser cutter and allows anyone on any platform to send their jobs to the laser, thanks to reverse engineering.
Old computers are easier to reverse engineering to adapt them to the modern era. They tend to come from an era when everyone was expected to know how to read a schematic and be handy with a soldering iron.
Luckily these skills have not been totally lost and are easily to pickup again. You can build a quick proof-of-concept with a bit of messy breadboard, a prototype system like the Arduino or Teensy and the schematic,
And once you have something that sort of works, it is fairly easy to transfer it to a PCB or a permaprotoboard and repurpose these devices into new uses. A fun project is to put a Raspberry Pi or BeagleBone Black into the device to replace the old motherboard, like this TRS80 Model 100 that I interfaced to with a Teensy++.
Even high voltage items like the CRT in this antique [[Mac-SE video|Macintosh SE] can be repurposed by using the schematic for the analog video interface.
CPU speed have increased enough that we can write a software video card that runs in the BeagleBone Black to replace the custom video chips used along with the 68k. This Mac now runs Linux -- hardware reverse engineering frequently seems to have an end goal of putting Linux in everything.
And sometimes it also uncovers fun easter eggs, like these photos of the team at Apple who worked on the Mac SE system. After increasing the ROM from 64 KB to 256 KB, they had a few extra kilobytes and stored these four images for the future to find.
Reverse engineering on new machines can serve many purposes, most commonly for security reasons. On a recent project I was looking into reports of a security vulnerability in MacBooks that allowed rootkits to be installed via Option ROMs on Thunderbolt devices.
This ROM dump lead to some interesting findings related to the previously undocumented format of Apple’s EFI ROM. I documented these so that other researchers could decode the ROMs to look for security issues.
I replicated snare's 2012 work, in which a malicious Thunderbolt device can install a rootkit that logs the user’s FileVault or firmware password and exfiltrate it. I have since changed the password, so don't try to break into my machine with this one.
I also found a few significant security issues of my own. One of the issues that I identified is that there is no hardware validation of the contents of the boot ROM, which opens up a significant vulnerability.
To demonstrate the risk of this vulnerability, I wrote a proof of concept exploit named Thunderstrike that used Thunderbolt devices to rewrite the boot ROM. This allows an evil maid to install malware given a few seconds alone with the laptop. This is a very dangerous place to have untrusted code -- the boot ROM controls the system from the very first instruction and gives it the ability to hide from detection, backdoor the OS and prevent any attempts to remove it with software.
The Thunderstrike vulnerability was significant enough that Apple issued a security update to fix it. This is one way that reverse engineering leads to safer systems.
Freedom to Tinker
The Electronic Frontier Foundation (EFF) has a comprehensive list of the frequently asked questions regarding reverse engineering, especially where it intersects with copyright. I highly recommend that you read through the list and ask them if you have any questions.
The "Freedom to Tinker" is an important one: it allows you to create interoperable products, extend features, reuse old systems and use your devices in the way that you want to use them.
Reverse engineering tools
Some of the simplest tools are ones that you probably already have
on your Mac or Linux machine.
/usr/bin/xxd for hex dumps
/usr/bin/strings to look for human readable
strings are part of the normal Unix toolkit. (Windows users might need to
install cygwin to get them).
Search engines are one of the best reverse engineering tools out there. When you encounter random hex constants, it’s amazing how quickly they can be matched to other sources. In this case I was able to avoid reverse engineering a huge block of code by recognizing that the lookup table were the crc32 polynomials.
For decoding packet based protocols, Wireshark is free and great. And it’s not just ethernet -- it can also decode USB, Bluetooth and exotic formats like SS7. At 31C3 there was a fascinating demo using wireshark and the HackRF SDR to decrypt SMS communications being sent to cell phones.
The interactive disassembler Hopper isn’t free, but it is well worth the modest investment. This sort of tool makes it possible to quickly get a feel for what a program is doing, name functions and variables, jump into called functions, track cross references, etc.
If you’re doing any sort of hardware work, a multimeter is an inexpensive purchase that will let you measure voltages and trace wires to figure out what connects to where. In my opinion this "beep mode" for continuity testing is the most important feature: it should respond instantly and audibly so that you don't have to take your eyes off the PCB you're testing.
For more complicated electronics work, an oscilloscope is slightly more expensive but well worth learning how to use. Being able to see the signals, like the ADB output from this Apple keyboard, makes it possible to understand what is going on.
The bus pirate from Dangerous Prototypes is another inexpensive tool that is useful to have in the box. It is a combination logic analyzer / interface swiss-army knife for low-speed protocols. Unlike the multimeter and oscilloscope, it can inject signals into a circuit to pretend to be different types of devices.
Once you figure something out...
Write it down
Once you've figured something out, write it down on your blog or somewhere that it can be found (i.e., not Facebook, G+ or twitter). Writing down what you learn serves many purposes: it ensures that you have a record of what you've done for later reference and it makes it possible for others to reproduce your work.
Teach someone else
Teaching what you've learned to others also cements your knowledge. I was priviledged to be invited to present my Thunderstrike reverse engineering work at CCC and hopefully helped lots of people learn how to get started with reverse engineering, learn something about EFI boot ROMs, and learn about boot time security vulnerabilities.
Smaller workshops and classes are great way to share what you’ve learned with your friends and community. Local hackerspaces love to hold classes on how-to topics and are also a great place to meet similar minded people.
We regularly hold workshops at Two Sigma on topics and try to help our colleagues learn how to get started with electronics. It is frequently nice for programmers who deal with large memory machines to step back to think about how something can be implemented in a few bytes of RAM.
Now Make something great!
My challenge to you this weekend is to make something great. Even if you think you’re not reverse engineering, you very well might be -- everytime you encounter a poorly documented API and have to look at the source, or send in different inputs to black box it: that’s reverse engineering. The skills you learn from figuring out how other pieces work will help you be a better programmer in almost all ways.