home.social

#floating-point — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #floating-point, aggregated by home.social.

fetched live
  1. Alright, future engineers!
    **Round-off Error:** Error from storing exact numbers with limited precision in computers.
    Ex: 1/3 stored as 0.33333.
    Pro-Tip: Use double precision & re-order ops to minimize accumulation in long calculations!
    #NumericalAccuracy #FloatingPoint #STEM #StudyNotes

  2. “In 1980, #Intel released the #Intel8087 #FloatingPoint coprocessor, a chip that could make math up to 100 times faster. As well as arithmetic and square roots, the 8087 computed #transcendental functions including #tangent, #exponentiation, and #logarithms.

    But it all depended on a 69-bit adder: "The arithmetic heart of the floating-point execution unit is centered about a #nanomachine comprised of the adder and its related #registers, #shifters and control circuitry," as the patent describes it”

    #Silicon / #EE / #Computing <righto.com/2026/06/intel-8087->

  3. “In 1980, #Intel released the #Intel8087 #FloatingPoint coprocessor, a chip that could make math up to 100 times faster. As well as arithmetic and square roots, the 8087 computed #transcendental functions including #tangent, #exponentiation, and #logarithms.

    But it all depended on a 69-bit adder: "The arithmetic heart of the floating-point execution unit is centered about a #nanomachine comprised of the adder and its related #registers, #shifters and control circuitry," as the patent describes it”

    #Silicon / #EE / #Computing <righto.com/2026/06/intel-8087->

  4. Ah yes, another riveting tale of microcontrollers—because who doesn't love a deep dive into floating point #arithmetic on a #hobbyist chip? 🙄 Spoiler: if you enjoy endless links leading to nowhere and cryptic jargon, this post is your Disneyland. 🎢 Go ahead, embrace the thrill of being hopelessly confused! 🤓
    danielmangum.com/posts/floatin #microcontrollers #floatingpoint #techhumor #confusion #HackerNews #ngated

  5. Ah yes, another riveting tale of microcontrollers—because who doesn't love a deep dive into floating point #arithmetic on a #hobbyist chip? 🙄 Spoiler: if you enjoy endless links leading to nowhere and cryptic jargon, this post is your Disneyland. 🎢 Go ahead, embrace the thrill of being hopelessly confused! 🤓
    danielmangum.com/posts/floatin #microcontrollers #floatingpoint #techhumor #confusion #HackerNews #ngated

  6. 🚀✨ Behold, the thrilling tale of querying 3 billion vectors—a journey where Vicki Boykis heroically attempts to decode Jeff Dean's cryptic wisdom on #mapreduce. Spoiler: It's basically a nerdy treasure hunt for semantically similar items, but with more floating-point numbers than your brain can handle. 💻🧠
    vickiboykis.com/2026/02/21/que #HackerNews #VickiBoykis #treasureHunt #techJourney #floatingPoint #HackerNews #ngated

  7. 🚀✨ Behold, the thrilling tale of querying 3 billion vectors—a journey where Vicki Boykis heroically attempts to decode Jeff Dean's cryptic wisdom on #mapreduce. Spoiler: It's basically a nerdy treasure hunt for semantically similar items, but with more floating-point numbers than your brain can handle. 💻🧠
    vickiboykis.com/2026/02/21/que #HackerNews #VickiBoykis #treasureHunt #techJourney #floatingPoint #HackerNews #ngated

  8. 0.1 + 0.2 In SQL = 0.30000000000000004?!

    SQL floating point precision is broken! Adding 0.1 + 0.2 creates a number with precision errors. This breaks financial calculations! Watch!

    #sql #sqltricks #database #sqltutorial #floatingpoint #precision #sqlquiz #codingchallenge #sqlshorts #sqlbugs #numericprecision #sqlwtf

    youtube.com/watch?v=C53FQIhge3U

  9. 0.1 + 0.2 In SQL = 0.30000000000000004?!

    SQL floating point precision is broken! Adding 0.1 + 0.2 creates a number with precision errors. This breaks financial calculations! Watch!

    #sql #sqltricks #database #sqltutorial #floatingpoint #precision #sqlquiz #codingchallenge #sqlshorts #sqlbugs #numericprecision #sqlwtf

    youtube.com/watch?v=C53FQIhge3U

  10. Learn about numeric data types (and the number 998,001) in Python

    The value 1 / 998,001 has an interesting property, but to see it in Python you need to learn about numeric data types.

    rodstephensbooks.com/many_digi

    #Python #FloatingPoint #Decimal #Precision

  11. Learn about numeric data types (and the number 998,001) in Python

    The value 1 / 998,001 has an interesting property, but to see it in Python you need to learn about numeric data types.

    rodstephensbooks.com/many_digi

    #Python #FloatingPoint #Decimal #Precision

  12. Проектирование FPU блока, работающего на частоте 7 ГГц

    Данный проект представляет собой современный вычислительный блок, предназначенный для обработки научных и графических данных на сверхвысокой частоте 7 ГГц . Архитектура построена на базе 128-битных векторов и 5-стадийного конвейера.

    habr.com/ru/articles/993270/

    #floatingpoint #floating_point

  13. Il faudrait un diplôme avant d'autoriser l'utilisation des nombres à virgule flottante.

    #floatingpoint #ieee754

  14. @linuxgal

    Note that this isn't Python-specific behaviour. This is required by the IEEE-754 floating-point standard, which Python (and most other runtimes) conform to.

    NaN, Inf, -Inf, comparison rules, exceptions ... it's all in there.

    #Python #IEEE754 #FloatingPoint #math #standard

  15. What kind of mishegus is this?
    "There are ARM processors that have mixed-endian floating-point representation for double-precision numbers: each of the two 32-bit words is stored as little-endian, but the most significant word is stored first."

    #floatingpoint #endianness

  16. Thank you for looking it up and for the pointer.
    Yes, Henry Baker has done many good deeds.

    There's a nice little puzzle about this format in the MIT HAKMEM.

    I ask not to publish a solution, even if obvious, so it can be used to try AI.
    (AI couldn't solve it about a year ago; I haven't tried it since.)

    Find a non-zero number whose two internal representations as (1) a machine integer and (2) a floating-point value have the same bits.

    Continued.

    #Bitwise
    #FloatingPoint
    #Puzzle

    @dougmerritt