Skip to content

Illegal and undocumented instructions

Illegal and undocumented instructions are not a new thing. The Commodore 64 CPU, a 6502 with a few additional I/O lines, was known to have them, and since on current CPUs we can completely VLSI simulate a 6502 in Javascript we also understand where they come from. Pagetable.com has a wonderful article on this.

So how about current CPUs? Modern CPUs are vastly bigger and more complicated than a 6502, and they are also set up very differently. So simulation is not taking us anywhere, but we can fuzz.

Sandsifter is such a CPU fuzzer: it generates every conceivable instruction byte combination and then tries to observe what happens.

Sandsifter has uncovered secret processor instructions from every major vendor; ubiquitous software bugs in disassemblers, assemblers, and emulators; flaws in enterprise hypervisors; and both benign and security-critical hardware bugs in x86 chips.

The findings have been summarized in a whitepaper (PDF), which also describes how to effectively search the instruction space of a CPU that has variable length instructions from 1 to 15 bytes in length. A crafty way of using page faults to determine the length of privileged instructions while running unprivileges is shown.

The strategy implemented reduces the search space from some 10E36 instructions down to about 100 million, which is a manageable size on modern CPUs. Known instructions (things that some reference disassembler knows and correctly predicts) are eliminated, the rest is interesting and deserves attention.

Published inComputer ScienceHackerterrorcybercyber

4 Comments

    • kris kris

      The 6502 has actual HCF instructions, anything that ends in a “2” nibble but A2 and one other instruction leads to a lockup.

      Stop program counter (processor lock up).
      Status flags: -
      
      Addressing  |Mnemonics  |Opc|Sz | n
      ------------|-----------|---|---|---
      Implied     |KIL        |$02| 1 | -
      Implied     |KIL        |$12| 1 | -
      Implied     |KIL        |$22| 1 | -
      Implied     |KIL        |$32| 1 | -
      Implied     |KIL        |$42| 1 | -
      Implied     |KIL        |$52| 1 | -
      Implied     |KIL        |$62| 1 | -
      Implied     |KIL        |$72| 1 | -
      Implied     |KIL        |$92| 1 | -
      Implied     |KIL        |$B2| 1 | -
      Implied     |KIL        |$D2| 1 | -
      Implied     |KIL        |$F2| 1 | -
      
      • kris kris

        $A2 is a regular instruction, 2 byte: LDX #. It’s the only regular instruction ending in a 2 nibble.

        $82 and $C2 are not locking up the box, but are also undocumented/undecoded artifacts. They act as a NOP with an immediate argument (which does nothing).

    • Ralf Ertzinger

      They did 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *