#chicagocrimes — Public Fediverse posts

Devs and data scientists really like our #ChicagoCrimes EDA public scripts, notebooks 📚 and data snapshots repository we created last October. That sample data/demo repository covers many different tools, libraries and notebooks to parse #LargeData:

⭐️ ⤑ https://github.com/RandomFractals/chicago-crimes

📜 ⤑ https://twitter.com/search?q=(%23ChicagoCrimes)%20(from%3ATarasNovak)&src=typed_query

#DataTools 🛠️ ...

Quick demo of our new #DuckDBSqlTools vscode extension loading and querying 7,687,725 #ChicagoCrimes recorded in 2001 through the end of November 2022 from a large 1.68 GB CSV data file in seconds ... See demo gif at:

📰 https://github.com/RandomFractals/chicago-crimes#with-duckdb-sql-tools

#DuckDB #SqlTools #VSCode #DataTools 💎💎💎

Our new #DuckDBSQLTools VSCode extension is almost ready for prime time.

You'll be able to load remote CSV and #parquet data files via httpfs extension and create in-memory #DuckDB instances too.

See demo gif of loading #ChicagoCrimes parquet data from a GitHub repository into memory, creating a CrimeReports table, and querying it on twitter:

https://twitter.com/TarasNovak/status/1617542770184577024

#VSCode #SQLTools / #DataTools 🔬💎💎💎...

📢 New 2022 #ChicagoCrimes #MalloyData #Fiddle app you can run in a browser to learn Malloy model and query semantics, nesting, and simple charting 📊📈 options:

📰 https://randomfractals.github.io/chicago-crimes/apps/malloy-fiddle

📓 https://github.com/RandomFractals/chicago-crimes#with-malloy-fiddle

🧐 https://github.com/RandomFractals/chicago-crimes/blob/main/docs/images/chicago-crime-reports-malloy-fiddle-app.gif

#DAX #dataApps #dataTools 🔬 ...

📢 New 2022 #ChicagoCrimes #MalloyData #Fiddle app you can run in a browser to learn Malloy model and query semantics, nesting, and simple charting 📊📈 options:

📰 https://randomfractals.github.io/chicago-crimes/apps/malloy-fiddle

📓 https://github.com/RandomFractals/chicago-crimes#with-malloy-fiddle

🧐 https://github.com/RandomFractals/chicago-crimes/blob/main/docs/images/chicago-crime-reports-malloy-fiddle-app.gif

#DAX #dataApps #dataTools 🔬 ...

📢 New 2022 #ChicagoCrimes #MalloyData #Fiddle app you can run in a browser to learn Malloy model and query semantics, nesting, and simple charting 📊📈 options:

📰 https://randomfractals.github.io/chicago-crimes/apps/malloy-fiddle

📓 https://github.com/RandomFractals/chicago-crimes#with-malloy-fiddle

🧐 https://github.com/RandomFractals/chicago-crimes/blob/main/docs/images/chicago-crime-reports-malloy-fiddle-app.gif

#DAX #dataApps #dataTools 🔬 ...

📢 New 2022 #ChicagoCrimes #MalloyData #Fiddle app you can run in a browser to learn Malloy model and query semantics, nesting, and simple charting 📊📈 options:

📰 https://randomfractals.github.io/chicago-crimes/apps/malloy-fiddle

📓 https://github.com/RandomFractals/chicago-crimes#with-malloy-fiddle

🧐 https://github.com/RandomFractals/chicago-crimes/blob/main/docs/images/chicago-crime-reports-malloy-fiddle-app.gif

#DAX #dataApps #dataTools 🔬 ...

Updated #ChicagoCrimes #PyScript #dataApp with gzipped CSV (~3.25MB). The app now loads 215,551 crime reports with #pyodide in a browser in about 8 seconds total for the #Python runtime, data transformation with #pandas 🐼 & charting with #Altair 📊📈

https://randomfractals.github.io/chicago-crimes/apps/pyscript/

Running some quick data summary queries with #Malloy on a 2001-2022 #ChicagoCrimes parquet data file that is 533MB, created form a larger 1.66GB CSV data, without any compression. Very responsive and fast query execution thanks to #DuckDB and Malloy #VSCode extension.

View those queries in action in this GIF: https://twitter.com/TarasNovak/status/1601650935402725376

#dataTools 🛠️ ...

Our #DataPreview 🈸 for #vscode now has over 350,000 installs. You can load large CSV files, sort & graph results with aggregate functions, and much more.

See an example of loading 48MB of #ChicagoCrimes CSV data: https://twitter.com/TarasNovak/status/1600439658810585088

Note: change data.preview.theme to light. See: https://github.com/RandomFractals/vscode-data-preview#configuration

📥 https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.vscode-data-preview

#dataViz 📊📈 #dataTools 🛠️ for #dataScientists ...

📢 New 2022 #ChicagoCrimes #dataApp with #Malloy #Composer is out. See:

gif -> https://github.com/RandomFractals/chicago-crimes/blob/main/docs/images/chicago-crimes-malloy-composer.gif

code -> https://github.com/RandomFractals/chicago-crimes/tree/main/apps/malloy-composer

app -> https://randomfractals.github.io/chicago-crimes/apps/malloy-composer

Try it in your browser. Have fun!

#dataTools/#dataApps 🔬 ...

so cool! :)
---
RT @TarasNovak
Created a web page with #PyScript loading 2022 #ChicagoCrimes CSV data with #pandas and visualizing that data with #Altair charting lib:
https://github.com/RandomFractals/chicago-crimes/blob/main/apps/pyscript/index.html
#dataApps ...
https://twitter.com/TarasNovak/status/1597379498756173824

so cool! :)
---
RT @TarasNovak
Created a web page with #PyScript loading 2022 #ChicagoCrimes CSV data with #pandas and visualizing that data with #Altair charting lib:
https://github.com/RandomFractals/chicago-crimes/blob/main/apps/pyscript/index.html
#dataApps ...
https://twitter.com/TarasNovak/status/1597379498756173824

so cool! :)
---
RT @TarasNovak
Created a web page with #PyScript loading 2022 #ChicagoCrimes CSV data with #pandas and visualizing that data with #Altair charting lib:
https://github.com/RandomFractals/chicago-crimes/blob/main/apps/pyscript/index.html
#dataApps ...
https://twitter.com/TarasNovak/status/1597379498756173824

so cool! :)
---
RT @TarasNovak
Created a web page with #PyScript loading 2022 #ChicagoCrimes CSV data with #pandas and visualizing that data with #Altair charting lib:
https://github.com/RandomFractals/chicago-crimes/blob/main/apps/pyscript/index.html
#dataApps ...
https://twitter.com/TarasNovak/status/1597379498756173824

so cool! :)
---
RT @TarasNovak
Created a web page with #PyScript loading 2022 #ChicagoCrimes CSV data with #pandas and visualizing that data with #Altair charting lib:
https://github.com/RandomFractals/chicago-crimes/blob/main/apps/pyscript/index.html
#dataApps ...
https://twitter.com/TarasNovak/status/1597379498756173824

2001-2022 #ChicagoCrimes data loaded from a #parquet file and summarized with #pandas #dataFrames and simple #Altair charts 📊📈 in a #JupyterNotebook:

👀 https://github.com/RandomFractals/chicago-crimes/blob/main/docs/images/chicago-crimes-altair-charts-ipynb.gif

📓 https://github.com/RandomFractals/chicago-crimes/blob/main/notebooks/chicago-crimes-altair.ipynb

#dataNotebooks 📚 ...

Created a web page with #PyScript loading 2022 #ChicagoCrimes CSV data with #pandas and visualizing that data with #Altair 📉📊 charting lib:

🧐 https://github.com/RandomFractals/chicago-crimes/blob/main/apps/pyscript/index.html

#dataApps ...

Created a web page with #PyScript loading 2022 #ChicagoCrimes CSV data with #pandas and visualizing that data with #Altair 📉📊 charting lib:

🧐 https://github.com/RandomFractals/chicago-crimes/blob/main/apps/pyscript/index.html

#dataApps ...

So, I ported my old #ChicagoCrimes #pandas data wrestling with #matplotlib plots to new repo. See this #JupyterNotebook with many crime data summaries and plots from 2001 to present:

🧐 https://github.com/RandomFractals/chicago-crimes/blob/main/notebooks/chicago-crimes-pandas.ipynb

#dataNotebooks 📚 ...

Hey #dataNerds 🤓, good news:

#DuckDB v0.6.0 brings reading #CSV data on par with #PyArrow & #Polars and loads 1.66 GB of #ChicagoCrimes data in 1.9s with 12 cores/24 threads when experimental parallel CSV reader & unordered insertion are enabled.

🧐 https://github.com/RandomFractals/chicago-crimes#with-duckdb

#dataTools 🔬 ...

Hey #dataNerds 🤓, good news:

#DuckDB v0.6.0 brings reading #CSV data on par with #PyArrow & #Polars and loads 1.66 GB of #ChicagoCrimes data in 1.9s with 12 cores/24 threads when experimental parallel CSV reader & unordered insertion are enabled.

🧐 https://github.com/RandomFractals/chicago-crimes#with-duckdb

#dataTools 🔬 ...

Displaying #ChicagoCrimes parquet data with #Malloy charts, imported table data source, measures, reusable queries, limits, nested grouping and bar chart renderer:

🔬 https://github.com/RandomFractals/chicago-crimes#with-malloy-charts

#VSCode #DataVis 📊 #DataTools ...

Updated #ChicagoCrimes with #MalloyData tools example & more info in docs:

🧐 https://github.com/RandomFractals/chicago-crimes#with-malloy-data

Clone that data repo & install Malloy #vscode extension to try it out:

📥 https://marketplace.visualstudio.com/items?itemName=malloydata.malloy-vscode

#dataTools 🔬 ...

I've decided to try #MalloyData today.

Here is a quick example of loading #ChicagoCrimes 2022 #parquetData with #DuckDB and Malloy queries for some rough counts and data summaries:

https://github.com/RandomFractals/chicago-crimes/issues/24

#dataTools 🛠️ ...