#grimmer — Public Fediverse posts

Risky Giant Steps Can Solve #Optimization Problems Faster | #QuantaMagazine

"#Grimmer found that the fastest sequences always had one thing in common: The middle step [in a sequence of repeating #GradientDescent steps] was always a big one. Its size depended on the number of steps in the repeating sequence. For a three-step sequence, the big step had length 4.9. For a 15-step sequence, the algorithm recommended one step of length 29.7. And for a 127-step sequence, the longest one tested, the big central leap was a whopping 370. At first that sounds like an absurdly large number, Grimmer said, but there were enough total steps to make up for that giant leap, so even if you blew past the bottom, you could still make it back quickly."

https://www.quantamagazine.org/risky-giant-steps-can-solve-optimization-problems-faster-20230811/