… but of course the motivators for the likes of Anthropic are to simply have fewer Watts-per-token in order to make profit. That might lead to enough technological progress to shrink an RTX6000 down to a device that costs under £1k and uses less than 50W so we can have one in our PCs and laptops.
I also hope in 10-15 years everyone is able to look back at this period and be ashamed/shocked at the wasteful and obvious mistakes being made.