#roundingerror — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #roundingerror, aggregated by home.social.
-
I haven't looked too closely, but the "only sometimes it's short a nickel" screams #rounding error to me. You're using #floats, so floating-point #precision issues can also creep in, though with small values that's less likely.
You might use #Decimal values instead of floats (in the decimal module). They're pretty standard for #currency calculations.
[...]
-
@grajohnt
Here you go, sections 7.4, 7.5
They have different rules for rounding for Imperial and metric.
•4.77 m is rounded down to a posted 4.6 m. That is 15′ 8″ and gets rounded and posted as 15′ 3″
•4.68 m is rounded down to a posted 4.6 m. That is 15′4″ and gets rounded down and posted as 15′0 ″