#dataframe — Public Fediverse posts

Release of Polars 2.0

https://pola.rs/posts/release-polars-2/

Comments: https://news.ycombinator.com/item?id=49977177

#HackerNews #Polars2.0 #DataScience #DataAnalysis #DataFrame #Performance

Narwhals provides a lightweight and extensible compatibility layer between various DataFrame libraries: https://www.python4data.science/en/latest/performance/parallelise-pandas.html#narwhals
#Python #pandas #DataFrame

Oh look, another "groundbreaking" #blog post about #DuckDB from a self-proclaimed data wizard. 🙄 Apparently, the limitations of basic text queries are just too much for our hero, who bravely delves into the wild world of Full-Text Search. 🌟 Spoiler alert: it's as thrilling as watching paint dry on a data frame. 🥱
https://peterdohertys.website/blog-posts/full-text-search-w-duckdb.html #DataWizard #FullTextSearch #DataFrame #HackerNews #ngated

Giới thiệu các tính năng độc đáo của thư viện C++ DataFrame dành cho phân tích dữ liệu hiệu năng cao. Đây là công cụ mã nguồn mở mạnh mẽ, hỗ trợ xử lý dữ liệu dạng bảng (table), chuỗi thời gian (time-series) với tốc độ vượt trội của C++. Thư viện cung cấp nhiều thuật toán thống kê, tài chính và khả năng thao tác dữ liệu linh hoạt tương tự như Pandas nhưng tối ưu hơn về bộ nhớ.

#Cpp #DataFrame #OpenSource #Programming #DataAnalysis #LapTrinh #PhanTichDuLieu #MaNguonMo

https://www.reddit.com/r/o

🌍🔧 "SedonaDB: #The 'revolutionary' #geospatial #DataFrame #library that no one asked for, written in #Rust because why not? 😒 It's like they took a detour through the desert to reinvent the wheel—again. 🚗💨"
https://sedona.apache.org/latest/blog/2025/09/24/introducing-sedonadb-a-single-node-analytical-database-engine-with-geospatial-as-a-first-class-citizen/ #SedonaDB #reinventing #wheel #tech #news #HackerNews #ngated

SedonaDB: A new geospatial DataFrame library written in Rust

https://sedona.apache.org/latest/blog/2025/09/24/introducing-sedonadb-a-single-node-analytical-database-engine-with-geospatial-as-a-first-class-citizen/

#HackerNews #SedonaDB #Geospatial #Rust #DataFrame #Library #DataScience

Spark Connect. А нужны ли перемены?

Привет, Хабр! Я Станислав Габдулгазиев, архитектор департамента поддержки продаж Arenadata. Apache Spark давно и прочно занял место одного из ключевых инструментов в арсенале инженеров и дата-сайентистов, работающих с большими данными. Его способность быстро обрабатывать огромные объёмы информации, гибкость за счёт поддержки множества языков (Python, Scala, Java, SQL) и возможность решать самые разнообразные задачи — от сложных ETL до машинного обучения и стриминга — делают его незаменимым инструментом в мире анализа данных.

https://habr.com/ru/companies/arenadata/articles/921246/

#spark_connect #apache #datalake #lakehouse #платформа_данных #bigdata #dataframe #интеграция_сервисов #apache_arrow #spark

Polars — «убийца Pandas» на максималках

Всем привет! Меня зовут Александр Андреев, я инженер данных. Сегодня я хочу рассказать вам о библиотеке Polars - потенциальной замене Pandas, любимой у большинства дата-инженеров и дата-саентистов библиотеки для работы с данными. В своей статье я последовательно пройдусь от истории библиотеки Polars до примеров кода, технических аспектов ее производительности и в конце дам ссылки на все бенчмарки, обучающие материалы и дополнительные статьи, которые использовались для написания данного обзора-туториала по этой замечательной библиотеке.

https://habr.com/ru/articles/946788/

#polars #pandas #data_engineering #data_science #data_analysis #dataframe #library #python #rust #dataset

How to generate dataframe summaries with python and AI for a type of dataset #datascience #dataframe #pandas #llm #Ollama #mistral #dev (https://fundor333.com/post/2025/generate-dataframe-summaries-with-python/)

I was annoyed that there is no "expand_grid()" function in :python: #Python as in :rstats: #RStats #tidyverse

So I just published a small package on #PyPI !

Introducing polarsgrid
https://pypi.org/project/polarsgrid/

Using the excellent #polars 🐻‍❄️ package, easily create a table with product of factors:

from polarsgrid import expand_grid
expand_grid(a=[1, 2, 3], b=["x", "y"])

Yields all combinations of its inputs as a #DataFrame

It can also produce a #LazyFrame for streaming extra-big tables to disk

Nuevo post en el blog de #juncotic! 💪

Seguimos con #python de la mano de @andrea_navarro

¿Han usado #Pandas para trabajar con datos?

Hoy Andrea nos explica cómo usarlo para ordenar columnas de un DataFrame, con ejemplos prácticos, y un CSV descargable para jugar con los datos 😃

Pueden leerlo acá: 👇

https://juncotic.com/ordenamiento-de-columnas-con-pandas/

Espero que les guste y sirva! 🙂

#python #pandas #dataframe #datascience #data

I've talked about creating data.frames and tibbles before, but it is an important topic so I have covered it again. This time specifically from the perspective of creating them from vectors. Post: www.spsanderson.com/steveondata/... #R #RStats #tibble #dplyr #tidyverse #dataframe #baseR #blog

Fahmatrix – A Lightweight, Pandas-Like DataFrame Library for Java

https://github.com/moustafa-nasr/fahmatrix

#HackerNews #Fahmatrix #Pandas #Java #DataFrame #Lightweight #Library

Computing travel time matrices in r⁵py from @geopandas #DataFrame is two lines of code:

(1) create an r5py.TransportNetwork from @openstreetmap and #GTFS data

(2) turn it into an r5py.TravelTimeMatrix()

Try it out in #binder: https://r5py.readthedocs.io/stable/user-guide/user-manual/quickstart.html

[備忘録] GradioでExcel風インターフェースを実装してみる
https://qiita.com/Tadataka_Takahashi/items/24c8f519d963d7dc9dae?utm_campaign=popular_items&utm_medium=feed&utm_source=popular_items

#qiita #Python #DataFrame #gradio

OPEN SOURCE 🚀

The Problem❔

There have been many instances where I needed to compare two dataframes and analyze their differences. To address this need, I created a fast Python library called "data_fingerprint" that does exactly that.

Check it out and let me know what you think! 🕵‍♂️
https://github.com/SimpleSimpler/data_fingerprint

#datascience #python #pandas #dataanalytics #dataengineering #dataframe #data

Parsing CSV with units in the header · Issue #166 · hgrecco/pint-pandas

https://github.com/hgrecco/pint-pandas/issues/166

Now we can read a #csv file with a header like `time / s,mass / g` into #pandas and call `.pint.quantify()` to get a #dataframe in which the columns have #units as in #Pints !

Handy for CSV restricted to single-row headers, as in Confluence Databases and Microsoft Lists.

I’ve written article about #pandera - #Python package used for #pandas #DataFrame validation: https://www.linkedin.com/pulse/do-you-use-dataframes-your-production-environment-schema-molinski-9swaf

(Sorry that this is on LinkedIn, but I’m trying to reach general audience there - my main goal is to promote the package and @pyOpenSci , and LinkedIn has larger community than my personal blog 😊)

[Перевод] 7 продвинутых приемов pandas для науки о данных

Pandas — это основная библиотека для работы с данными. Вот несколько приёмов, которые я использую, чтобы быстрее и проще выполнять повторяющиеся задачи по работе с данными.

https://habr.com/ru/articles/858894/

#pandas #datascience #numpy #matplotlib #анализ_данных #dataframe

Data Exploration With pandas
#Python #Pycharm #Datascience #Dataframe #Pandas #Pandasdataframes

https://blog.jetbrains.com/pycharm/2024/10/data-exploration-with-pandas/

Pandas НЕ для анализа данных

В среде питонистов библиотека Pandas пользуется большой популярностью и по большей мере известна в контексте DataSciense и анализа данных. DataFrame пандас позволяет не только всячески манипулировать данными, но и выводить их в нужном формате, предоставляя широкие возможности для кастомизации. Например, использовали ли вы объекты класса Styler , входящего в состав Pandas? Мне показалось интересным взглянуть на Pandas с этой стороны.

https://habr.com/ru/articles/822793/

#python #pandas #django #dataframe #html #server_side_rendering

Why Ibis?

Ibis defines a #Python #dataframe #API that executes on any query engine – the #frontend for any #backend data platform, with 20+ backends today. This allows Ibis to have excellent performance – as good as the backend it is connected to – with a consistent user experience.

https://ibis-project.org/

Interesting quasi replacement for pandas (?). Worth looking into. #polars #dataframe #rust #pandas : "Polars — Why we have rewritten the string data type"(https://pola.rs/posts/polars-string-type/)

Interesting but so high level that I am not sure it is super useful. #polars #rust #dataframe #architecture #explainer #query #query-optimization : "Polars — A bird's eye view of Polars"(https://pola.rs/posts/polars_birds_eye_view/)

A Step-by-Step Guide to Performing Data Analysis With Kotlin DataFrame.

#dataframe #java #kotlin

https://blog.jetbrains.com/kotlin/2024/04/a-step-by-step-guide-to-performing-data-analysis-with-kotlin-dataframe/?utm_medium=erik.in&utm_source=mastodon

Spark. План запросов на примерах

Всем привет! В этой статье возьмем за основу пару таблиц и пройдемся по планам запросов по нарастающей: от обычного селекта до джойнов, оконок и репартиционирования. Посмотрим, чем отличаются виды планов друг от друга, что в них изменяется от запроса к запросу и разберем каждую строчку на примере партиционированной и непартиционированной таблицы.

https://habr.com/ru/articles/807421/

#apache_spark #pyspark #sql #python #bigdata #data_engineering #explain #execution_plan #план_запроса #dataframe

From Structured Query Languages (SQL) to Dataframe Languages (DL)
#sql #dataframe comparison
https://zuinnote.eu/blog/?p=2545 #bigdata #dataflows

Literally me: "Search on this online store sucks and I need a way to compare a lot of product details, if only there were a way to download this data to a spreadsheet or something....
WAIT A MINUTE - I KNOW PYTHON!"

A Few Hours Later, I'm saving a Pandas DataFrame to CSV

#python #webScraping #nerd #NEERRRRRDDDDDD #pandas #dataFrame

ChatGPT Coding - Creating a Data Frame in R!

Check out this helpful R tutorial for initializing a dataframe with ChatGPT!

#rstats #RStudio #ChatGPT #MachineLearning #dataframe

ChatGPT Coding - Removing Bottom Row from Python Data Frame!

Check out this useful ChatGPT Python tutorial for removing a bottom row from a dataframe!

#python #pythoncode #ChatGPT #AI #dataframe #openai

ChatGPT Coding - Creating a Data Frame in Python!

Check out this useful Python tutorial for initializing a new dataframe with ChatGPT!

#python #pythoncode #ChatGPT #AI #dataframe

No Mess in R Data Frames!

How to clean the column names of a data frame in the R programming language.

#rstats #dataframe #datacleaning #programming

To review the (draft) update of national #SSP projections for #GDP and #population, you can...

1. View the data via our interactive #ScenarioExplorer hosted by the #IIASA #ScenarioServices team
2. Download the full dataset as xlsx
3. Access the data via our #Python or #Rstats API

And with our #opensource package #pyam_iamc, it's as easy as the snippet below...
... to get something that's just like a #pandas #DataFrame - but better!

#ReadTheDocs at https://pyam-iamc.readthedocs.io

Don't Make This MISTAKE with R Data Classes !!!

How to convert factors to the numeric data type without changing the factor level numbers in the R programming language.

#rstats #datatypes #programming #dataframe

Great TRICK for DATAFRAMES in Python!!!

How to compare two pandas DataFrames and find differences between rows in the Python programming language.

#python #dataframe #programming #programmer

Python Indices Can Really Be Used Like This???

This video explains how to insert a new row at a certain index position of a pandas DataFrame. We have to slice our data!!!

#python #pythonprogramming #index #dataframe