#dataframe — Public Fediverse posts

Narwhals provides a lightweight and extensible compatibility layer between various DataFrame libraries: https://www.python4data.science/en/latest/performance/parallelise-pandas.html#narwhals
#Python #pandas #DataFrame

Narwhals provides a lightweight and extensible compatibility layer between various DataFrame libraries: https://www.python4data.science/en/latest/performance/parallelise-pandas.html#narwhals
#Python #pandas #DataFrame

Oh look, another "groundbreaking" #blog post about #DuckDB from a self-proclaimed data wizard. 🙄 Apparently, the limitations of basic text queries are just too much for our hero, who bravely delves into the wild world of Full-Text Search. 🌟 Spoiler alert: it's as thrilling as watching paint dry on a data frame. 🥱
https://peterdohertys.website/blog-posts/full-text-search-w-duckdb.html #DataWizard #FullTextSearch #DataFrame #HackerNews #ngated

Oh look, another "groundbreaking" #blog post about #DuckDB from a self-proclaimed data wizard. 🙄 Apparently, the limitations of basic text queries are just too much for our hero, who bravely delves into the wild world of Full-Text Search. 🌟 Spoiler alert: it's as thrilling as watching paint dry on a data frame. 🥱
https://peterdohertys.website/blog-posts/full-text-search-w-duckdb.html #DataWizard #FullTextSearch #DataFrame #HackerNews #ngated

🌍🔧 "SedonaDB: #The 'revolutionary' #geospatial #DataFrame #library that no one asked for, written in #Rust because why not? 😒 It's like they took a detour through the desert to reinvent the wheel—again. 🚗💨"
https://sedona.apache.org/latest/blog/2025/09/24/introducing-sedonadb-a-single-node-analytical-database-engine-with-geospatial-as-a-first-class-citizen/ #SedonaDB #reinventing #wheel #tech #news #HackerNews #ngated

🌍🔧 "SedonaDB: #The 'revolutionary' #geospatial #DataFrame #library that no one asked for, written in #Rust because why not? 😒 It's like they took a detour through the desert to reinvent the wheel—again. 🚗💨"
https://sedona.apache.org/latest/blog/2025/09/24/introducing-sedonadb-a-single-node-analytical-database-engine-with-geospatial-as-a-first-class-citizen/ #SedonaDB #reinventing #wheel #tech #news #HackerNews #ngated

SedonaDB: A new geospatial DataFrame library written in Rust

https://sedona.apache.org/latest/blog/2025/09/24/introducing-sedonadb-a-single-node-analytical-database-engine-with-geospatial-as-a-first-class-citizen/

#HackerNews #SedonaDB #Geospatial #Rust #DataFrame #Library #DataScience

SedonaDB: A new geospatial DataFrame library written in Rust

https://sedona.apache.org/latest/blog/2025/09/24/introducing-sedonadb-a-single-node-analytical-database-engine-with-geospatial-as-a-first-class-citizen/

#HackerNews #SedonaDB #Geospatial #Rust #DataFrame #Library #DataScience

Spark Connect. А нужны ли перемены?

Привет, Хабр! Я Станислав Габдулгазиев, архитектор департамента поддержки продаж Arenadata. Apache Spark давно и прочно занял место одного из ключевых инструментов в арсенале инженеров и дата-сайентистов, работающих с большими данными. Его способность быстро обрабатывать огромные объёмы информации, гибкость за счёт поддержки множества языков (Python, Scala, Java, SQL) и возможность решать самые разнообразные задачи — от сложных ETL до машинного обучения и стриминга — делают его незаменимым инструментом в мире анализа данных.

https://habr.com/ru/companies/arenadata/articles/921246/

#spark_connect #apache #datalake #lakehouse #платформа_данных #bigdata #dataframe #интеграция_сервисов #apache_arrow #spark

Polars — «убийца Pandas» на максималках

Всем привет! Меня зовут Александр Андреев, я инженер данных. Сегодня я хочу рассказать вам о библиотеке Polars - потенциальной замене Pandas, любимой у большинства дата-инженеров и дата-саентистов библиотеки для работы с данными. В своей статье я последовательно пройдусь от истории библиотеки Polars до примеров кода, технических аспектов ее производительности и в конце дам ссылки на все бенчмарки, обучающие материалы и дополнительные статьи, которые использовались для написания данного обзора-туториала по этой замечательной библиотеке.

https://habr.com/ru/articles/946788/

#polars #pandas #data_engineering #data_science #data_analysis #dataframe #library #python #rust #dataset

How to generate dataframe summaries with python and AI for a type of dataset #datascience #dataframe #pandas #llm #Ollama #mistral #dev (https://fundor333.com/post/2025/generate-dataframe-summaries-with-python/)

I was annoyed that there is no "expand_grid()" function in :python: #Python as in :rstats: #RStats #tidyverse

So I just published a small package on #PyPI !

Introducing polarsgrid
https://pypi.org/project/polarsgrid/

Using the excellent #polars 🐻‍❄️ package, easily create a table with product of factors:

from polarsgrid import expand_grid
expand_grid(a=[1, 2, 3], b=["x", "y"])

Yields all combinations of its inputs as a #DataFrame

It can also produce a #LazyFrame for streaming extra-big tables to disk

I was annoyed that there is no "expand_grid()" function in :python: #Python as in :rstats: #RStats #tidyverse

So I just published a small package on #PyPI !

Introducing polarsgrid
https://pypi.org/project/polarsgrid/

Using the excellent #polars 🐻‍❄️ package, easily create a table with product of factors:

from polarsgrid import expand_grid
expand_grid(a=[1, 2, 3], b=["x", "y"])

Yields all combinations of its inputs as a #DataFrame

It can also produce a #LazyFrame for streaming extra-big tables to disk

Nuevo post en el blog de #juncotic! 💪

Seguimos con #python de la mano de @andrea_navarro

¿Han usado #Pandas para trabajar con datos?

Hoy Andrea nos explica cómo usarlo para ordenar columnas de un DataFrame, con ejemplos prácticos, y un CSV descargable para jugar con los datos 😃

Pueden leerlo acá: 👇

https://juncotic.com/ordenamiento-de-columnas-con-pandas/

Espero que les guste y sirva! 🙂

#python #pandas #dataframe #datascience #data

Nuevo post en el blog de #juncotic! 💪

Seguimos con #python de la mano de @andrea_navarro

¿Han usado #Pandas para trabajar con datos?

Hoy Andrea nos explica cómo usarlo para ordenar columnas de un DataFrame, con ejemplos prácticos, y un CSV descargable para jugar con los datos 😃

Pueden leerlo acá: 👇

https://juncotic.com/ordenamiento-de-columnas-con-pandas/

Espero que les guste y sirva! 🙂

#python #pandas #dataframe #datascience #data

I've talked about creating data.frames and tibbles before, but it is an important topic so I have covered it again. This time specifically from the perspective of creating them from vectors. Post: www.spsanderson.com/steveondata/... #R #RStats #tibble #dplyr #tidyverse #dataframe #baseR #blog

I've talked about creating data.frames and tibbles before, but it is an important topic so I have covered it again. This time specifically from the perspective of creating them from vectors. Post: www.spsanderson.com/steveondata/... #R #RStats #tibble #dplyr #tidyverse #dataframe #baseR #blog

Fahmatrix – A Lightweight, Pandas-Like DataFrame Library for Java

https://github.com/moustafa-nasr/fahmatrix

#HackerNews #Fahmatrix #Pandas #Java #DataFrame #Lightweight #Library

Fahmatrix – A Lightweight, Pandas-Like DataFrame Library for Java

https://github.com/moustafa-nasr/fahmatrix

#HackerNews #Fahmatrix #Pandas #Java #DataFrame #Lightweight #Library

Computing travel time matrices in r⁵py from @geopandas #DataFrame is two lines of code:

(1) create an r5py.TransportNetwork from @openstreetmap and #GTFS data

(2) turn it into an r5py.TravelTimeMatrix()

Try it out in #binder: https://r5py.readthedocs.io/stable/user-guide/user-manual/quickstart.html

[備忘録] GradioでExcel風インターフェースを実装してみる
https://qiita.com/Tadataka_Takahashi/items/24c8f519d963d7dc9dae?utm_campaign=popular_items&utm_medium=feed&utm_source=popular_items

#qiita #Python #DataFrame #gradio

OPEN SOURCE 🚀

The Problem❔

There have been many instances where I needed to compare two dataframes and analyze their differences. To address this need, I created a fast Python library called "data_fingerprint" that does exactly that.

Check it out and let me know what you think! 🕵‍♂️
https://github.com/SimpleSimpler/data_fingerprint

#datascience #python #pandas #dataanalytics #dataengineering #dataframe #data

OPEN SOURCE 🚀

The Problem❔

There have been many instances where I needed to compare two dataframes and analyze their differences. To address this need, I created a fast Python library called "data_fingerprint" that does exactly that.

Check it out and let me know what you think! 🕵‍♂️
https://github.com/SimpleSimpler/data_fingerprint

#datascience #python #pandas #dataanalytics #dataengineering #dataframe #data

Parsing CSV with units in the header · Issue #166 · hgrecco/pint-pandas

https://github.com/hgrecco/pint-pandas/issues/166

Now we can read a #csv file with a header like `time / s,mass / g` into #pandas and call `.pint.quantify()` to get a #dataframe in which the columns have #units as in #Pints !

Handy for CSV restricted to single-row headers, as in Confluence Databases and Microsoft Lists.

Parsing CSV with units in the header · Issue #166 · hgrecco/pint-pandas

https://github.com/hgrecco/pint-pandas/issues/166

Now we can read a #csv file with a header like `time / s,mass / g` into #pandas and call `.pint.quantify()` to get a #dataframe in which the columns have #units as in #Pints !

Handy for CSV restricted to single-row headers, as in Confluence Databases and Microsoft Lists.

I’ve written article about #pandera - #Python package used for #pandas #DataFrame validation: https://www.linkedin.com/pulse/do-you-use-dataframes-your-production-environment-schema-molinski-9swaf

(Sorry that this is on LinkedIn, but I’m trying to reach general audience there - my main goal is to promote the package and @pyOpenSci , and LinkedIn has larger community than my personal blog 😊)

I’ve written article about #pandera - #Python package used for #pandas #DataFrame validation: https://www.linkedin.com/pulse/do-you-use-dataframes-your-production-environment-schema-molinski-9swaf

(Sorry that this is on LinkedIn, but I’m trying to reach general audience there - my main goal is to promote the package and @pyOpenSci , and LinkedIn has larger community than my personal blog 😊)

[Перевод] 7 продвинутых приемов pandas для науки о данных

Pandas — это основная библиотека для работы с данными. Вот несколько приёмов, которые я использую, чтобы быстрее и проще выполнять повторяющиеся задачи по работе с данными.

https://habr.com/ru/articles/858894/

#pandas #datascience #numpy #matplotlib #анализ_данных #dataframe

Data Exploration With pandas
#Python #Pycharm #Datascience #Dataframe #Pandas #Pandasdataframes

https://blog.jetbrains.com/pycharm/2024/10/data-exploration-with-pandas/

Data Exploration With pandas
#Python #Pycharm #Datascience #Dataframe #Pandas #Pandasdataframes

https://blog.jetbrains.com/pycharm/2024/10/data-exploration-with-pandas/

Pandas НЕ для анализа данных

В среде питонистов библиотека Pandas пользуется большой популярностью и по большей мере известна в контексте DataSciense и анализа данных. DataFrame пандас позволяет не только всячески манипулировать данными, но и выводить их в нужном формате, предоставляя широкие возможности для кастомизации. Например, использовали ли вы объекты класса Styler , входящего в состав Pandas? Мне показалось интересным взглянуть на Pandas с этой стороны.

https://habr.com/ru/articles/822793/

#python #pandas #django #dataframe #html #server_side_rendering

Why Ibis?

Ibis defines a #Python #dataframe #API that executes on any query engine – the #frontend for any #backend data platform, with 20+ backends today. This allows Ibis to have excellent performance – as good as the backend it is connected to – with a consistent user experience.

https://ibis-project.org/

A Step-by-Step Guide to Performing Data Analysis With Kotlin DataFrame.

#dataframe #java #kotlin

https://blog.jetbrains.com/kotlin/2024/04/a-step-by-step-guide-to-performing-data-analysis-with-kotlin-dataframe/?utm_medium=erik.in&utm_source=mastodon

A Step-by-Step Guide to Performing Data Analysis With Kotlin DataFrame.

#dataframe #java #kotlin

https://blog.jetbrains.com/kotlin/2024/04/a-step-by-step-guide-to-performing-data-analysis-with-kotlin-dataframe/?utm_medium=erik.in&utm_source=mastodon

Spark. План запросов на примерах

Всем привет! В этой статье возьмем за основу пару таблиц и пройдемся по планам запросов по нарастающей: от обычного селекта до джойнов, оконок и репартиционирования. Посмотрим, чем отличаются виды планов друг от друга, что в них изменяется от запроса к запросу и разберем каждую строчку на примере партиционированной и непартиционированной таблицы.

https://habr.com/ru/articles/807421/

#apache_spark #pyspark #sql #python #bigdata #data_engineering #explain #execution_plan #план_запроса #dataframe

From Structured Query Languages (SQL) to Dataframe Languages (DL)
#sql #dataframe comparison
https://zuinnote.eu/blog/?p=2545 #bigdata #dataflows

Literally me: "Search on this online store sucks and I need a way to compare a lot of product details, if only there were a way to download this data to a spreadsheet or something....
WAIT A MINUTE - I KNOW PYTHON!"

A Few Hours Later, I'm saving a Pandas DataFrame to CSV

#python #webScraping #nerd #NEERRRRRDDDDDD #pandas #dataFrame

ChatGPT Coding - Creating a Data Frame in R!

Check out this helpful R tutorial for initializing a dataframe with ChatGPT!

#rstats #RStudio #ChatGPT #MachineLearning #dataframe

ChatGPT Coding - Removing Bottom Row from Python Data Frame!

Check out this useful ChatGPT Python tutorial for removing a bottom row from a dataframe!

#python #pythoncode #ChatGPT #AI #dataframe #openai

ChatGPT Coding - Creating a Data Frame in Python!

Check out this useful Python tutorial for initializing a new dataframe with ChatGPT!

#python #pythoncode #ChatGPT #AI #dataframe

No Mess in R Data Frames!

How to clean the column names of a data frame in the R programming language.

#rstats #dataframe #datacleaning #programming

To review the (draft) update of national #SSP projections for #GDP and #population, you can...

1. View the data via our interactive #ScenarioExplorer hosted by the #IIASA #ScenarioServices team
2. Download the full dataset as xlsx
3. Access the data via our #Python or #Rstats API

And with our #opensource package #pyam_iamc, it's as easy as the snippet below...
... to get something that's just like a #pandas #DataFrame - but better!

#ReadTheDocs at https://pyam-iamc.readthedocs.io

At least there are couple of options when we are working with Multi-index data that can be particularly helpful for slicing or filtering data:

#python #pandas #datascience #dataframe

- pandas.DataFrame.xs
- pandas.IndexSlice

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.xs.html

https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.IndexSlice.html?highlight=indexslice#pandas-indexslice

Don't Make This MISTAKE with R Data Classes !!!

How to convert factors to the numeric data type without changing the factor level numbers in the R programming language.

#rstats #datatypes #programming #dataframe

Great TRICK for DATAFRAMES in Python!!!

How to compare two pandas DataFrames and find differences between rows in the Python programming language.

#python #dataframe #programming #programmer

Python Indices Can Really Be Used Like This???

This video explains how to insert a new row at a certain index position of a pandas DataFrame. We have to slice our data!!!

#python #pythonprogramming #index #dataframe