#netcdf — Public Fediverse posts

It seems that #forth can be a good language for quick one liners that do data processing. I basically needed something like an RPN calculator but for matrices stored as files in disk. #python #numpy scripts were used before but they were quite verbose. I can also get #netcdf data, process that as a forth pipeline, then dump to window for quick animation.

#programming #dataprocessing

@hannorein That would be a bad idea especially if someone wants to run REBOUND on an #HPC cluster.
Dealing with simulations that create tons of files will put a lot of extra load on the shared file systems.
On your own little cluster, you probably wouldn't notice, but on the Alliance's national systems, on which thousands of users are working at any given time, this all stacks up.

Why not use something based on #HDF4 or #NETCDF?

How to Download and Visualize NASA-ISRO NISAR Data in QGIS | Step-by-Step Tutorial [shared]
--
https://youtu.be/oRTplHPf_T0 <-- shared video tutorial
--
https://github.com/opengeos/qgis-nasa-earthdata-plugin <-- shared NASA Earthdata Plugin for QGIS
--
https://science.nasa.gov/mission/nisar/ <-- shared #NASA NISAR mission home page
--
H/T Qiusheng Wu
#remotesensing #earthobservation #tutorial #training #onlinelearning #download #visualisation #NISAR #opendata #satellite #NASA #Earthdata #QGIS #plugin #SAR #icemovement #ecosystems #vegetation #biomass #naturalhazard #risk #hazard #NetCDF #spatialvisualisation
@QGIS | @nasa

An experimental #duckdb #netcdf extension

https://codeberg.org/cgenie/duckdb-netcdf

#gis #cpp

I am really looking forward to a time when scientific data analysis is less of a constant fuckaround and fight with technical bullshit. I'd *really* like

- #netCDF natively supporting complex numbers
- #Python #xarray and #pandas to natively support physical units (#pint is great on its own but the integrations leave a LOT to be desired)
- #Jupyter notebooks to suck less (crashes, glitches, widget plots not saved statically, an effing BUILTIN formatter, etc.)
- proper data pipeline systems
...

𝗖𝗙 𝗠𝗲𝘁𝗮𝗱𝗮𝘁𝗮 𝗖𝗼𝗻𝘃𝗲𝗻𝘁𝗶𝗼𝗻𝘀 support #OpenScience by automating processes via metadata in #NetCDF and #Zarr files. They establish a unified language for the #weather, #climate, #ocean, and #EO community. https://buff.ly/rGPuoku
#remotesensing #EarthObs

GSPy - A New Toolbox And Data Standard For Geophysical Datasets
--
https://doi.org/10.3389/feart.2022.907614 <-- shared paper
--
https://doi.org/10.5066/P9XNQVGQ | https://code.usgs.gov/g3sc/gspy <-- shared code repository
--
[an older paper, but code is in active and ongoing development/evolution]
#GIS #spatial #mapping #geophysics #geophysical #NetCDF #datatypes #code #opensource #library #dataformats #standardisation #standardization #openstandard #portable #metadata #Python #package #GSPy #methods #workflows #xarray #CRS #opendata #architecture #toolbox

🎉 Generate spreadsheet templates for recording data and metadata for CF-NetCDF files! 🚀

The latest update adds a sheet for variable attributes—metadata describing each variable.

🔗 https://www.nordatanet.no/aen/template-generator/

Templates bring structure and enable automated CF-NetCDF creation with scripts. In the future, I plan to develop a converter tool ion top of this—upload a template, download a CF-NetCDF!

#NetCDF #CFConventions #FAIRData

Excited to share that I'm now a member of the CF Conventions Committee! 🎉

CF (Climate and Forecast) conventions standardise how scientific data and metadata are stored in NetCDF files, ensuring consistency and machine readability. Looking forward to contributing and collaborating with the community to shape its future! 🌍

#CFConventions #NetCDF #FAIRData #ScientificData #DataScience

🚀 Excited to share my new FREE video tutorial on creating CF-NetCDF files using R! 🌟

In this video, you'll learn to:
• Ensure compliance with CF and ACDD conventions
• Work with 1D, 2D, and 3D data
• Manage irregular grids and moving instruments

🔗 Watch it here: https://youtu.be/IZDygRjfMIg

📖 Plus, check out the accompanying Jupyter book for code and explanations: https://nordatanet.github.io/NetCDF_in_R_from_beginner_to_pro/04_creating_a_cfnetcdf_file.html

#CFConventions #NetCDF #RProgramming #RStats #DataScience #ClimateData #FAIRData

I’m pleased to share that my NetCDF in Python and R tutorials are now available on NorDataNet. 🎉

Access them here:

Python: https://www.nordatanet.no/NetCDF-in-Python-from-beginner-to-pro
R: https://www.nordatanet.no/NetCDF-in-R-from-beginner-to-pro

These tutorials cover everything from basic data extraction to creating your own NetCDF files, ensuring they meet CF and ACDD standards for FAIR data.

NorDataNet enhances access to environmental datasets, supporting long-term preservation and interoperability. 🌍📊

#NetCDF #FAIRData #Python #R #NorDataNet

Using Mesh Data In QGIS [including accessing NetCDF data]
--
https://courses.gisopencourseware.org/course/view.php?id=72 <-- shared tutorial
--
https://youtu.be/OBi4wuxLiAE?si=_5DCAcsy8qjhsA-k <-- tutorial as a video
--
“Mesh data represents an unstructured grid that can include temporal and other components. It consists of vertices, edges, and faces that form a spatial structure in 2D or 3D space. Each vertex can store multiple datasets, which can also have a temporal dimension, making mesh data highly versatile for various applications such as meteorology, hydrodynamics, and environmental modeling…
This course is designed to equip you with the knowledge and skills needed to effectively work with mesh data within the QGIS environment…"
#QGIS #GIS #mesh #datavisualisation #NetCDF #climate #animation #openaccess #onlinelearning #learning #spatial #mapping #tutorial

🌍 Heading to the 2024 CF Workshop in Norrköping, Sweden! 🇸🇪 CF conventions are vital for ensuring data adheres to the FAIR principles. While NetCDF files are great, CF standards make data truly interoperable across disciplines.

I'll be sharing ideas on introducing CF to new disciplines like biology and geology, helping them unlock the full potential of their data. Excited to learn how CF conventions evolve and meet the community! 🌟

#FAIRData #CFConventions #Interoperability #NetCDF #OpenScience

🌍📊 Want to work with NetCDF files in Python? My tutorial series covers everything from opening and plotting NetCDF data to creating CF-compliant files for FAIR data publication.

Whether you're new to NetCDF or looking to enhance your skills, I've got you covered! 🚀 Check it out: https://lhmarsden.github.io/NetCDF_in_Python_from_beginner_to_pro

Topics include:
🔸Extracting data 📝
🔸Plotting 📈
🔸Creating CF-compliant files 🌐
🔸Granularity 🖥️
🔸CF & ACDD 🖥️

Suggestions? Let me know! #Python #DataScience #NetCDF #xarray #FAIRData #ClimateData

🚀 New Video Alert! 🚀

I’ve just released a tutorial on how to convert NetCDF data to CSV or XLSX using R. Whether you’re dealing with 1D, 2D, or 3D variables, this video will guide you through the process step-by-step.

🎥 Watch here: https://youtu.be/9-EDaRQ8Aps

#RStats #NetCDF #DataScience #FAIRdata #CSV #XLSX #Tutorial #YouTube