home.social

Search

2 results for “infosecstoic”

  1. Joshua Saxe argues that "solving alignment" and "solving interpretability" are the wrong frame for AI safety and security. Fourteen years of interpretability work hasn't produced the reliable, scalable understanding that frame needs, and capabilities are pulling away from it.

    I'd go further. The problem isn't only that these systems are too complex to understand yet; in general it can't be done at all. "Will this agent ever take a harmful action?" is a question about what a program does, and Rice's theorem, Turing's halting problem in its general form, says no procedure can answer questions like that for arbitrary programs. Security learned this in 1984, when Fred Cohen proved perfect virus detection is impossible. We didn't stop doing antivirus. We stopped expecting it to be the whole defence.

    That makes Saxe's alternative the only design on offer. His diagram (attached) puts "align the model weights" at the centre of ten layers, wrapped in tool-call monitoring, human confirmation, least privilege, sandboxing, rapid response, regulation and international agreements. We forecast imperfectly, monitor what actually happens, limit how much power we hand a system when confidence is low, layer redundant controls, and plan for those controls to fail.

    Security made the same shift when "prevent every breach" gave way to resilience: reduce the probability of material impact rather than chase certainty. AI safety is on the same road, and we have a long reference class to learn from.

    joshuasaxe181906.substack.com/

  2. I can no longer tell how many of you are AI!

Share on Mastodon

Enter the server where you have an account.