Reversing NyanKonton

You can grab the binary here and follow along. Hit the floppy disk icon on CyberChef to save the output, and you have your binary downloaded.

The challenge dropped a single file. I ran file on it first thing.

file nyancat
# nyancat: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>

Mach-O, ARM64. Compiled for macOS Apple Silicon. I’m on Linux x86_64 so running it was never an option: Mach-O uses macOS syscalls, and QEMU user-mode only handles Linux ABI. The whole solve had to be static.

Loading it in Ghidra

I dropped the binary into Ghidra. It detected the format and architecture without any configuration. After auto-analysis, I opened the Symbol Tree and navigated to entry, the program’s entry point.

Understanding What the Binary Does

Reading through the decompiled entry(), the program turned out to be an interactive nyancat animation. It sets up raw terminal mode via tcgetattr / tcsetattr to capture individual keypresses, seeds the RNG with srand(0x4d2), builds a scrolling star field, then enters a main loop rendering frames and waiting for input.

Every time you press up or down, it stores the move as a single byte in a global array _g_moves and increments _g_mcount. Up arrow maps to 0x55, down arrow to 0x44.

Once _g_mcount exceeds 0x17 (24 recorded moves), the program stops accepting input and computes a hash over those 24 bytes. That hash is compared against a hardcoded target. Correct hash: flag. Wrong hash:

Nyancat soars into the void. The portal stays closed.

Reversing the Hash

The hash routine in the decompiled output:

Cleaned up, it comes down to this:

uVar5 = 0xbadc0ffee0ddf00d;
for (i = 0; i < 0x18; i++) {
    uVar5 = (uVar5 >> 0x33 | (uVar5 ^ g_moves[i]) << 0x0d) * 0x100000001b3;
    uVar5 ^= 0xcafebabedeadbeef;
}

A custom hash: each iteration XORs in the move byte, rotates, multiplies by the FNV-1a prime (0x100000001b3), then XORs with a constant. The target value to hit was 0xc5b0bb2a46a6cfc.

Bruteforcing the Right Sequence

The search space is 2^24 (16 million combinations of 0x55 and 0x44 over 24 positions). I started with Python and it was too slow. I rewrote it in Rust with rayon to parallelize across all cores.

I set up the project:

cargo new nyansolve
cd nyansolve

Cargo.toml:

[package]
name = "nyansolve"
version = "0.1.0"
edition = "2024"

[dependencies]
rayon = "1.10"

src/main.rs:

use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use rayon::prelude::*;

const TARGET: u64 = 0xc5b0bb2a46a6cfc;

fn calc_hash(moves: &[u8]) -> u64 {
    let mut h: u64 = 0xbadc0ffee0ddf00d;
    for &m in moves {
        h = h.wrapping_shr(0x33) | (h ^ m as u64).wrapping_shl(0x0d);
        h = h.wrapping_mul(0x100000001b3);
        h ^= 0xcafebabedeadbeef;
    }
    h
}

fn main() {
    let found = Arc::new(AtomicBool::new(false));
    (0u32..(1 << 24)).into_par_iter().for_each(|i| {
        if found.load(Ordering::Relaxed) { return; }
        let moves: Vec<u8> = (0..24)
            .map(|b| if (i >> b) & 1 == 0 { 0x55 } else { 0x44 })
            .collect();
        if calc_hash(&moves) == TARGET {
            let seq: String = moves.iter()
                .map(|&c| if c == 0x55 { 'U' } else { 'D' })
                .collect();
            println!("Found: {}", seq);
            found.store(true, Ordering::Relaxed);
        }
    });
}

Ran in release mode:

cargo run -r

Found: UDUDUDUDUUDUDUUUUUUDDDUU

Done in 0.02s.

Decrypting the Flag

With the sequence known, I went back to Ghidra to look at the decryption block. The seed is 0xc5b0bb2a46a6cfc, the hash target itself. Every 8 iterations a keyblock is derived from the current state by extracting bytes at 8-bit intervals. The state then advances via a left rotation by 7 and a multiply by 0x9e3779b97f4a7c15 (Fibonacci hashing constant). Each byte of the ciphertext at _NK_FLAG_CT gets XORed against the corresponding keyblock byte.

I extracted the 48 ciphertext bytes from Ghidra at offset 0x100003f68:

b9 0f 05 d3 d3 78 18 58 38 45 51 53 14 96 fc 46
dd e8 0b bd a4 8e 01 86 48 28 3e 08 52 03 76 03
20 0d e4 f0 db 61 c3 5f e6 e0 72 28 69 9a 89 6d

Then wrote the decrypt in Python:

ct = [
    0xb9, 0x0f, 0x05, 0xd3, 0xd3, 0x78, 0x18, 0x58,
    0x38, 0x45, 0x51, 0x53, 0x14, 0x96, 0xfc, 0x46,
    0xdd, 0xe8, 0x0b, 0xbd, 0xa4, 0x8e, 0x01, 0x86,
    0x48, 0x28, 0x3e, 0x08, 0x52, 0x03, 0x76, 0x03,
    0x20, 0x0d, 0xe4, 0xf0, 0xdb, 0x61, 0xc3, 0x5f,
    0xe6, 0xe0, 0x72, 0x28, 0x69, 0x9a, 0x89, 0x6d
]

uVar9 = 0xc5b0bb2a46a6cfc
mask = 0xffffffffffffffff

flag = []
for i in range(0x30):
    if (i & 7) == 0:
        keyblock = [(uVar9 >> s) & 0xff for s in range(0, 64, 8)]
        uVar9 = ((uVar9 >> 0x39) | (uVar9 << 7)) & mask
        uVar9 = (uVar9 * 0x9e3779b97f4a7c15) & mask
    flag.append(keyblock[i & 7] ^ ct[i])

print(''.join(chr(c) for c in flag))

That printed the flag.

Why the static solve worked

The binary never ran once. Two things made that possible. The hash was simple enough to reimplement byte for byte, and the decryption seed turned out to be the hash target itself. So the moment you bruteforce the sequence, you already hold everything you need to decrypt the flag offline.