#numericprecision — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #numericprecision, aggregated by home.social.
-
0.1 + 0.2 In SQL = 0.30000000000000004?!
SQL floating point precision is broken! Adding 0.1 + 0.2 creates a number with precision errors. This breaks financial calculations! Watch!
#sql #sqltricks #database #sqltutorial #floatingpoint #precision #sqlquiz #codingchallenge #sqlshorts #sqlbugs #numericprecision #sqlwtf
-
0.1 + 0.2 In SQL = 0.30000000000000004?!
SQL floating point precision is broken! Adding 0.1 + 0.2 creates a number with precision errors. This breaks financial calculations! Watch!
#sql #sqltricks #database #sqltutorial #floatingpoint #precision #sqlquiz #codingchallenge #sqlshorts #sqlbugs #numericprecision #sqlwtf
-
0.1 + 0.2 In SQL = 0.30000000000000004?!
SQL floating point precision is broken! Adding 0.1 + 0.2 creates a number with precision errors. This breaks financial calculations! Watch!
#sql #sqltricks #database #sqltutorial #floatingpoint #precision #sqlquiz #codingchallenge #sqlshorts #sqlbugs #numericprecision #sqlwtf
-
Today I learned combined with Public Service Announcement:
In Python, you can use format strings with variables not just as the values that need to be represented, but also as parameters for the values!
I.e., if you want to print a given number in scientific notation, you can do:
```python
> value = 314159265
> print(f”{value:6.2e}“)
3.14e+06
```But if you want to personalize the printing in significant digits and floating point digits you can do:
```python
> digits = 12 # total number of digits
> fl_digits = 4 # for the floating point part
> value = 314159265
> print(f"{value:{digits}.{fl_digits}e}")
3.1416e+06
```
#Python #precision #NumericPrecision #NumberFormatting #DynamicFormatting #DynamicNumericPrecision #TIL #PSA #TodayILearned #PublicServiceAnnouncement