So the UP5K-based SoC is working, but the routing fabric on the ICE40 "UltraPlus" (lol) series is much slower than on the HX line. The SoC generally closes timing at around Fmax=23MHz; I'm running it at 20MHz for simplicity.
The prototype using an ICE40HX was at 40MHz and boy howdy I sure can feel the difference. Display repaints are visible. Recompiling the system takes 1m25s (admittedly with the slowest combination of options, using the software emulator).
From past experience with the ICE40UP parts, I'm unlikely to get this faster than 24 MHz, so I guess I'll just need to be patient if I want the convenient QFN package with built-in SRAM.