home.social

#parquet — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #parquet, aggregated by home.social.

fetched live
  1. peaoPerdido @[email protected] ·

    Heard about geoparquet a while ago and today I gave it a try. I discovered that I can create geoparquet files using geopandas.to_parquet() 😎 But I also discovered that gdal from ubuntuGis does not recognize the file type 😢 So I can´t look at the file in Qgis nor do ogrinfo on it.

    I'm installing gdal from ubuntugis-unstable. Am I doing something wrong? Or GDAL / Qgis from ubuntuGIS / debianGIS does not support parquet?

    #gdal #gis #ubuntu #parquet #qgis

    #gdal #gis #ubuntu #parquet #qgis
  2. Thomas @[email protected] ·

    I've made a new release for the PHP DuckDB PDO driver:

    - Update to DuckDB 1.5.5
    - Support for PHP 8.6
    - Support for TrueAsync and Quack protocol
    - More examples in the readme:
    - ascii chart rendering with SQL
    - LLM integration
    - fetch public data
    - direct data copy from MySQL/MariaDB

    Binary releases are available on GitHub and can be installed or updated with PIE:

    pie install thomas-0816/pdo-duckdb-php

    github.com/thomas-0816/pdo-duc

    #php #duckdb #pdo #analytics #bigdata #parquet

    #php #duckdb #pdo #analytics #bigdata #parquet
  3. Ryan Kennedy @[email protected] ·

    Spent the past couple of nights working up a #Golang utility to read the central XML file (exports.xml) from an #AppleHealth export and convert the <Record/> elements to #Parquet.

    Finally got that working tonight and was able to load the Parquet file into #DuckDB to begin doing some querying.

    #golang #applehealth #parquet #duckdb
  4. Hacker News @[email protected] ·

    Paging Through a Parquet File in DuckDB: File_row_number or Offset?

    rusty.today/blog/paging-parque

    Comments: news.ycombinator.com/item?id=4

    #HackerNews #DuckDB #Parquet #Paging #FileRowNumber #Offset #DataProcessing

    #hackernews #duckdb #parquet #paging #filerownumber #offset
  5. Sylvain Lesage @[email protected] ·

    The "Awesome Parquet" list keeps growing.

    github.com/severo/awesome-parq

    The newest projects:

    1. GeoParquet Viewer 🗺️: A table and map viewer for GeoParquet files in the browser, by @mmohr.

    geoparquet.info/

    2. Hardwood ☕: A minimal dependency Java implementation of Apache Parquet, by @gunnarmorling.

    hardwood.dev/

    3. LakeQL 🚣: Pure JavaScript duck-compatible SQL query engine for Parquet and Iceberg data in object storage.

    github.com/earonesty/lakeql

    🧵

    #Parquet

    #parquet
  6. dusoft @dusoft ·

    Do you use Adminer? It's the best PHP database management, since PHPMyAdmin has enshittified.

    But there has not been DuckDB / Parquet support, so here you go:
    ambience.sk/adminer-support-fo

    Both work in read-only mode and require external dependencies, but they work with Adminer!

    #adminer #php #db #database #duckdb #parquet
  7. Jakob Miksch @[email protected] ·

    #til about #GeoParquet files naming:

    It is RECOMMENDED to use .parquet as the file extension for a GeoParquet file. This provides the best interoperability with existing Parquet tools. The file extension .geoparquet SHOULD NOT be used

    #parquet #gis #geo #gischat

    geoparquet.org/releases/v1.1.0/

    #til #geoparquet #parquet #gis #geo #gischat
  8. Rod2ik 🇪🇺 🇨🇵 🇪🇸 🇨🇱 🇺🇦 🇨🇦 🇬🇱☮🕊️ @[email protected] ·

    Jérôme #Barella mis en examen pour le #viol et le #meurtre de #Lyhanna, annonce le #parquet d' #Agen

    franceinfo.fr/faits-divers/dis

    #barella #viol #meurtre #lyhanna #parquet #agen
  9. Hacker News @[email protected] ·

    Postgres data stored in Parquet on S3: LTAP architecture explained

    databricks.com/blog/lakebase-l

    #HackerNews #Postgres #Parquet #S3 #LTAP #architecture #dataengineering

    #hackernews #postgres #parquet #s3 #ltap #architecture
  10. LabPlot (labplot.org) @[email protected] ·

    #LabPlot (upcoming release) just got even more powerful.

    The latest merge request introduces native support for importing data from the following file formats:

    👉 #Apache #Parquet
    👉 #Apache #ORC
    👉 #Arrow #IPC (#Feather)

    —three of the most efficient and widely used columnar storage formats in data science.

    #DataScience #OpenSource #FOSS #FLOSS #KDE #DataViz #BigData #Data #DataAnalysis #Python #R #OpenData #HackerNews #TechNews #OpenData #Science #Research #Analytics #Tech #IT

    #labplot #apache #parquet #orc #arrow #ipc
  11. disouzam_bh @[email protected] ·

    Parquet file format. Talk by Boudewijn Braams on Spark+AI Summit Europe 2019

    #parquet #dataprocessing

    youtu.be/1j8SdS7s_NY?si=rfH4cN

    #parquet #dataprocessing
  12. Rod2ik 🇪🇺 🇨🇵 🇪🇸 🇨🇱 🇺🇦 🇨🇦 🇬🇱☮🕊️ @[email protected] ·

    BFM: Dominique de #Villepin visé par une enquête du #parquet #financier concernant des #statuettes #reçues en #cadeau lorsqu'il était au #Quai d' #Orsay

    Le #mec, il reçoit un cadeau de 125000€ en nature et il ne se dit pas qu'il va y avoir un #problème #éthique et #moral, voire #juridique...

    bfmtv.com/police-justice/domin

    #villepin #parquet #financier #statuettes #recues #cadeau
  13. Alessandra Bilardi @[email protected] ·

    When does #Iceberg beat #Parquet+projection on #AWSGlue, and when doesn't ?

    An end-to-end #ETL PoC on #AWS to find out: producer, #Kinesis, two #Firehose paths, two #Glue jobs, #Athena.

    🔮 Spoiler: how the data is read is the key to the choice.

    In the article: every choice with its why, plus a few gems from some Glue experience 😄

    alessandra.bilardi.net/diary/a

    #DiaryOfALazyDeveloper

    #iceberg #parquet #awsglue #etl #aws #kinesis
  14. Habr @[email protected] ·

    Гайд: Как работать с форматом PARQUET

    В прошлом году мы начали публиковать данные в каталоге «Если быть точным» в формате Parquet . Его придумали инженеры Twitter и Cloudera в 2013 году, и сегодня он стал стандартом хранения аналитических данных — его используют Google, Amazon, Netflix и большинство современных data-платформ. В этом гайде мы расскажем, как эффективно работать с данными в формате Parquet с помощью Python.

    habr.com/ru/articles/1013604/

    #parquet #python #анализ_данных

    #анализ_данных #python #parquet
  15. N-gated Hacker News @[email protected] ·

    🐒 Ah, yes, the holy grail of nerd bragging rights: a 47M+ item #archive of Hacker News, now in the culinary delight format of #Parquet for all your "data chef" needs. 🍽️ Updated every 5 minutes, because clearly, what's more riveting than a play-by-play of techie's daily musings? Oh wait, I forgot—🥱 anything else.
    huggingface.co/datasets/open-i #HackerNews #DataChef #TechieBraggingRights #DailyUpdates #HackerNews #ngated

    #archive #parquet #hackernews #datachef #techiebraggingrights #dailyupdates
  16. Hacker News @[email protected] ·

    Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m

    huggingface.co/datasets/open-i

    #HackerNews #HackerNews #Archive #Parquet #Data #47MItems #UpdatedEvery5m

    #hackernews #archive #parquet #data #47mitems #updatedevery5m
  17. Roberto Guido @[email protected] ·

    Ho provato a riversare un dump #Wikidata in #Parquet e ad interrogarlo con #DuckDB: ci mette meno di un'ora ad estrapolare tutte le 19.939.182 entità che rappresentano persone, incluse le sottoclassi di wdt:Q5.
    Decisamente meglio del mio deserializzatore implementato in Go, che per fare la stessa cosa ci mette quasi 8 ore.

    #wikidata #parquet #duckdb