home.social

#pandas — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #pandas, aggregated by home.social.

  1. Profile raw CSVs from ETL or APIs: this 282-line pandas script flags outliers via z-score (std thresh) and outputs an HTML report. Intermediate level, ready to run. valtersit.com/python/pandas-da #python #data #pandas

  2. If I had to explain my job... #IOER_FDz .. I'd just show this workflow from the last two days. It's a great example of building reproducible, automated documentation. 🚀

    1. Start: A project schema, drafted by colleagues in Excel.

    2. Problem: Excel isn't reproducible and is hard to use as a single source of truth for a database.
    Solution: I migrated the schema to the universal Protobuf format, making it version-controlled and language-neutral. [1]

    3. Problem: Protobuf definitions aren't easy for everyone to read.
    Solution: Using an existing #CI/#CD workflow from [3] (cheers @mcnesium), I put the schema on a documentation website. Now it's accessible and legible. [4]

    4. Problem: Colleagues still needed a familiar template for data collection.
    Solution: Two Python scripts using #pandas now auto-generates both `.xlsx` and open-source `.ods` schema [5] and templates [6] directly from the Protobuf single-point-of-truth, and make all of these available for download, too.

    5. Problem: The text-based schema wasn't visual enough.
    Solution: Added a script [7] to the CI that transforms the Protobuf files into a #MermaidJS class diagram, rendered directly on the docs site. [8]

    6. Problem: The diagram was too narrow in the site's layout.
    Solution: The script now generates a second, wide-format version of the diagram for a dedicated fullscreen view. [9]

    7. Problem: The static diagram was hard to edit or restyle.
    Solution: The CI now generates a shareable link to the Mermaid Live Editor [10]. It reads our latest diagram file, compresses it (using pako/zlib), and bakes it into the URL. Thanks to a code snippet from a friendly stranger on GitHub [11], anyone can now open the *latest* version of the schema in the editor with a single click.

    From a static Excel file to version-controlled, multi-format, visual, and interactive documentation, all fully automated. That's the job.

    #Automation #GitLab #Python #Protobuf #OpenData #Documentation

    @ioer

    [1]: gitlab.vgiscience.de/caserepor
    [2]: docs.casereports.fdz.ioer.info/
    [3]: lbsn.vgiscience.org
    [4]: docs.casereports.fdz.ioer.info
    [5]: gitlab.vgiscience.de/caserepor
    [6]: gitlab.vgiscience.de/caserepor
    [7]: gitlab.vgiscience.de/caserepor
    [8]: docs.casereports.fdz.ioer.info
    [9]: docs.casereports.fdz.ioer.info
    [10]: mermaidlive.com/play
    [11]: github.com/mermaid-js/mermaid-

  3. A zoo in the city of Kobe said Wednesday that it will delay the return of a giant female panda to China by another year as the animal undergoes medical treatment. japantimes.co.jp/news/2023/12/ #japan #kobe #pandas #tantan

  4. #Introduction: This account is all about #earlymodern #diplomacy. We toot about the ups & downs of editing a #handbook on #emdiplomacy & give insights into the #LifeOfTheEditors. We introduce our great authors, reveil #emdiplomacysSecrets, present #emdiplomacysFaces, discuss #emdiplomacysSources. We share reading recommendations & #emdiplomacy related research. We love to connect (with) #researchers & #histodons from the field of #emdiplomacy. And there will also be some #Pandas, of course.

  5. #Introduction: This account is all about #earlymodern #diplomacy. We toot about the ups & downs of editing a #handbook on #emdiplomacy & give insights into the #LifeOfTheEditors. We introduce our great authors, reveil #emdiplomacysSecrets, present #emdiplomacysFaces, discuss #emdiplomacysSources. We share reading recommendations & #emdiplomacy related research. We love to connect (with) #researchers & #histodons from the field of #emdiplomacy. And there will also be some #Pandas, of course.

  6. #Introduction: This account is all about #earlymodern #diplomacy. We toot about the ups & downs of editing a #handbook on #emdiplomacy & give insights into the #LifeOfTheEditors. We introduce our great authors, reveil #emdiplomacysSecrets, present #emdiplomacysFaces, discuss #emdiplomacysSources. We share reading recommendations & #emdiplomacy related research. We love to connect (with) #researchers & #histodons from the field of #emdiplomacy. And there will also be some #Pandas, of course.

  7. #Introduction: This account is all about #earlymodern #diplomacy. We toot about the ups & downs of editing a #handbook on #emdiplomacy & give insights into the #LifeOfTheEditors. We introduce our great authors, reveil #emdiplomacysSecrets, present #emdiplomacysFaces, discuss #emdiplomacysSources. We share reading recommendations & #emdiplomacy related research. We love to connect (with) #researchers & #histodons from the field of #emdiplomacy. And there will also be some #Pandas, of course.

  8. #Introduction: This account is all about #earlymodern #diplomacy. We toot about the ups & downs of editing a #handbook on #emdiplomacy & give insights into the #LifeOfTheEditors. We introduce our great authors, reveil #emdiplomacysSecrets, present #emdiplomacysFaces, discuss #emdiplomacysSources. We share reading recommendations & #emdiplomacy related research. We love to connect (with) #researchers & #histodons from the field of #emdiplomacy. And there will also be some #Pandas, of course.