#apptainer — Public Fediverse posts

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

For those doing #HPC #DL model trainings, I need some suggestions. I want to use #horovod in a multi-gpu, multimode setting, but using #Apptainer (or #Docker) containers, due to cluster policy issues. Any reference to share? Issue here: github.com/horovod/horo...

Galaxy can be adjusted for sensitive data.
In the Netherlands, a GDPR-compliant Galaxy clone runs in an isolated network, converting Toolshed tools to Apptainer containers—secure, auditable, and reproducible.
Read more: https://galaxyproject.org/news/2025-01-20-esg-w-p5/

#UseGalaxy #GalaxyProject #UniFreiburg #EOSC #EuroScienceGateway #SensitiveData #GDPR #PrivacyByDesign #ReproducibleResearch #Bioinformatics #Apptainer #DataGovernance #Netherlands

@gnulinux Danke für den podcast zu #Apptainer :) Gab mir als früher intensiver Nutzer von #hpc einen informativen Einblick, was aus Singularity entstanden ist, und zu den aktuellen Möglichkeiten mit containern und wissenschaftlichem Rechnen :awesome:

My head hurts. Doing a deep-dive into containerization/VM with #Jenkins. So many options. I think I'll give #LXD a shot. But frankly, since my #OpenMPI program is already containerized via #Apptainer, I'll just go ahead and build on the (fairly powerful) Jenkins host for now, just to get the pipeline built as a proof of concept.

Time for an adult beverage.

🎩✨ Behold, the marvel of Apptainer: the magical name change from #Singularity that will revolutionize your Linux container universe by... being exactly like every other container system but with more labels. 🌟🔒 Rejoice as your software components remain safely ensconced in their digital dungeons, as you navigate the labyrinthine #GitHub to inspect your secured, portable future. 🤖🔍
https://apptainer.org/ #Apptainer #LinuxContainers #SoftwareDevelopment #HackerNews #ngated

Apptainer: Application Containers for Linux

https://apptainer.org/

#HackerNews #Apptainer #Application #Containers #Linux #Technology #OpenSource

@beardedtechguy Doing some literature review for a grant application and trying to figure out how to do SCIF apps in #Apptainer. They're jackhammering the hell out of the pavement in front of my office, so I sought refuge in the student center's Starbucks.

#Python #poetry (and #uv I guess) is NOT a comprehensive tool to pin your environment! It is JUST a tool to help you manage some downstream Python packages, that's it. Use something more powerful like :nixos: #nix flakes or #apptainer​s or whatever floats your boat to do proper pinning of an environment - where you can pin your package manager itself.

@beardedtechguy Welcome back! And I guess congratulations are in order

Grading and accreditation work are done here, too, for now. I’m currently picking up my NASA work again, building an #Apptainer container to be shipped to Oxford University. Also getting ready to upgrade my #HamRadio license and working on improving my wireless communications class for next semester.

@cliffwade Morning, Cliff. We're getting a new roof as we speak. Hopefully will be done by 6pm tonight. I’ll be shipping some NASA code via #Apptainer to Oxford University after tracking down some bugs in both apptainer and the code itself. I'll spend the weekend studying up a bit to upgrade my #HamRadio license and thinking about how to improve my wireless communications class for next semester. There will be the occasional beer sprinkled in for relaxation.

Building an #Apptainer container for our colleagues in Oxford. The task is made much more pleasant by the awesome weather and a bottle of #Jever Pilsener. It’s very nice outside sitting on the deck.

Today I tried out #AMD #Instinct #MI300a for my existing Deep Learning pipeline. Good news: It worked out of the box. Bad news: For some reason it could not beat my local #Nvidia #1080ti...
After trying all sorts of #ROCM installation methods via prebuild wheels, #apptainer images etc I tried #nanogpt by @karpathy and sure enought: The gpt code ran approx 2x faster than on a #a100 ... I hope that this is due to my programming skills. Not AMD prefering #transformers over #CNNs ...

Today I tried out #AMD #Instinct #MI300a for my existing Deep Learning pipeline. Good news: It worked out of the box. Bad news: For some reason it could not beat my local #Nvidia #1080ti...
After trying all sorts of #ROCM installation methods via prebuild wheels, #apptainer images etc I tried #nanogpt by @karpathy and sure enought: The gpt code ran approx 2x faster than on a #a100 ... I hope that this is due to my programming skills. Not AMD prefering #transformers over #CNNs ...

Today I tried out #AMD #Instinct #MI300a for my existing Deep Learning pipeline. Good news: It worked out of the box. Bad news: For some reason it could not beat my local #Nvidia #1080ti...
After trying all sorts of #ROCM installation methods via prebuild wheels, #apptainer images etc I tried #nanogpt by @karpathy and sure enought: The gpt code ran approx 2x faster than on a #a100 ... I hope that this is due to my programming skills. Not AMD prefering #transformers over #CNNs ...

Apptainer で OpenFold を実行してタンパク質の立体構造予測を試してみた
https://dev.classmethod.jp/articles/getting-started-with-openfold-using-apptainer/

#dev_classmethod #Bioinformatics #Amazon_EC2 #AWS #Apptainer

#ThisMonthInFluiddyn - Nov 2024 edition

* #FluidDyn, #FluidSim, #FluidImage now works with #numpy 2 and #mpi4py 4.0.

* #Spack packaging recipes

https://foss.heptapod.net/fluiddyn/fluiddyn/-/tree/branch/default/misc/spack?ref_type=heads

* #Apptainer and #Guix packaging for FluidSim

https://foss.heptapod.net/fluiddyn/fluidsim/-/tree/branch/default/doc/apptainer?ref_type=heads

https://foss.heptapod.net/fluiddyn/fluidsim/-/blob/7e52e7bed240665823e04e9f5f8a1fae7d787d06/doc/examples/clusters/gricad/README.md

* Basic support for #Jax is part of #Transonic with v0.7.x

* #FluidImage v0.5.x has a number of improvements, such as work towards matching #UVmat (the MATLAB #PIV suite), new executors and a command `fluidimage-mean`

Apparently you can apptainer/singularity build docker-daemon://some-image to force a sif container to be built from your local docker installation rather than pulling it from the internet!

In the rare case when you have both docker and apptainer on the same machine, anyway.

#apptainer #singularity #docker

The challenge today is to make #genAI converting a Dockerfile to an Apptainer/Singularity file.

I think I could abuse it quite heavily 😅

#HPC #SLURM #apptainer #singularity #Docker #dockerfile #challengeaccepted #devops #datascience #OpenScience

Apptainer - Docker der Wissenschaft

Zur Replikation und Kapselung von Laufzeitumgebungen ist Apptainer die Lösung innerhalb der Computerwissenschaft.

#Apptainer #Wissenschaft #Linux #Replikation #Container #Docker #Linux

https://gnulinux.ch/apptainer-docker-der-wissenschaft

1. Built and shipped an #Apptainer #HPC container from scratch in record time.

2. Kentucky beer cheese and pretzels for dinner.

3. Washing it down with a Kentucky Bourbon Barrel Ale.

#ThreeGoodThings
@threegoodthings

Trinity のパフォーマンスモニタリング機能を Apptainer で動く Trinity コンテナで試してみた
https://dev.classmethod.jp/articles/performance-monitoring-in-trinity-with-apptainer-containers/

#dev_classmethod #Bioinformatics #Apptainer #Trinity #HPC #AWS_ParallelCluster_3 #AWS

Trinity 中間ファイルの保存先ストレージの違いで処理時間に影響あるのか確認してみた
https://dev.classmethod.jp/articles/how-storage-affects-processing-time-for-trinity-intermediate-files/

#dev_classmethod #Bioinformatics #HPC #AWS_ParallelCluster_3 #Apptainer #Trinity #Amazon_EFS #AWS

Just released: Apptainer Cheat Sheet by cbohn4

Download it free at http://www.cheatography.com/cbohn4/cheat-sheets/apptainer/?utm_source=mastodon

Here's their description of it: A quick reference guide for using Apptainer

@cheatsheets #CheatSheet #CheatSheets #hpc #apptainer

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

Trinity インプットデータの保存先に因る影響を処理時間で比較してみた （S3 vs インスタンスストア）
https://dev.classmethod.jp/articles/comparing-trinity-input-data-storage-s3-vs-instance-store/

#dev_classmethod #Bioinformatics #HPC #AWS_ParallelCluster_3 #Apptainer #Trinity #Mountpoint_for_Amazon_S3 #AWS

AWS ParallelCluster で Apptainer と Moutpoint for Amazon S3 の組み合わせて Trinity を実行してみた
https://dev.classmethod.jp/articles/combining-apptainer-and-s3-to-run-trinity-on-aws-parallelcluster/

#dev_classmethod #Bioinformatics #HPC #AWS_ParallelCluster_3 #AWS #Mountpoint_for_Amazon_S3 #Apptainer

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Idea: bundling your cli $EDITOR (:vim: #vim :emacs: #emacs whatever) into a one-file static binary (#Apptainer #Appimage whatever) that you can take with you and will work everywhere, with all plugins and configs already set up. Alternatively as a :nixos: #nix #flake of course, though that's not so easily portable to systems where you don't have root access and the admins don't want to install #Nix

@kylewritescode Good morning Kyle.

I already had a productive morning. I finally figured out how to let my hybrid #Apptainer #OpenMPI container for our NASA code talk to #UKY's #HPC cluster’s InfiniBand backend. Now the code scales beautifully. Next up, getting it to run on our collaborators’ cluster at #UMN.

I think I’ll have my coffee now.

I think :nixos: #nix is the most elegant way to get #GnuPG and #pass to work on #SailfishOS. I have tried many ways including using random packages from others on OpenRepos and the like, manual compilation of GnuPG (omg was that painful 😩 but it worked eventually), #Apptainer (worked okayish but meh containers...).

Here's how to do it:

https://forum.sailfishos.org/t/installing-the-nix-package-manager-on-sailfishos/17503/2?u=nobodyinperson

@Anachron Nah, termux is suboptimal on #SailfishOS, at least in my experience. If I have the chance of not using any VM/containerization/whatever weird stuff, I'll take it. nix does exactly that. I also messed around a lot with #Apptainer, that worked quite well but is - again - containers, so meh... Let's see how far I come with nix on SailfishOS. So far, hledger and tmux work, this is all I need right now. 🙂

BTW, SailfishOS is a Linux distribution for phones, if you're unaware.

High Performace Software Foundation to be formed by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-to-Form-High-Performance-Software-Foundation #HPC #LinuxFoundation #Spack #Kokkos #Apptainer #WarpX #HPSF #portable #SoftwareStack

However, a :nixos: #nixshell for "quickly give me :docker: #Docker for a minute" is not possible, right? It wouldn't start the daemon.

In this case #Apptainer is a cool alternative: no root or daemon required to run software from containers.

Heh, I really like the principle of using :nixos: nix-shell as a shebang on non-#NixOS platforms. This script just launches #KBibTeX (the #TexLaTeX #BibTeX bibliography editor). Helpful in a synced :gitannex: #gitAnnex repo with multiple participants across different OSs.

In this case, the environment and programs (e.g. PDF viewer) installed in the OS stay accessible - a great advantage in comparison to container solutions like #Apptainer's for example.

https://nixos.wiki/wiki/Nix-shell_shebang

#nix #NixOS

Heh, I really like the principle of using :nixos: nix-shell as a shebang on non-#NixOS platforms. This script just launches #KBibTeX (the #TexLaTeX #BibTeX bibliography editor). Helpful in a synced :gitannex: #gitAnnex repo with multiple participants across different OSs.

In this case, the environment and programs (e.g. PDF viewer) installed in the OS stay accessible - a great advantage in comparison to container solutions like #Apptainer's for example.

https://nixos.wiki/wiki/Nix-shell_shebang

#nix #NixOS

Heh, I really like the principle of using :nixos: nix-shell as a shebang on non-#NixOS platforms. This script just launches #KBibTeX (the #TexLaTeX #BibTeX bibliography editor). Helpful in a synced :gitannex: #gitAnnex repo with multiple participants across different OSs.

In this case, the environment and programs (e.g. PDF viewer) installed in the OS stay accessible - a great advantage in comparison to container solutions like #Apptainer's for example.

https://nixos.wiki/wiki/Nix-shell_shebang

#nix #NixOS

Heh, I really like the principle of using :nixos: nix-shell as a shebang on non-#NixOS platforms. This script just launches #KBibTeX (the #TexLaTeX #BibTeX bibliography editor). Helpful in a synced :gitannex: #gitAnnex repo with multiple participants across different OSs.

In this case, the environment and programs (e.g. PDF viewer) installed in the OS stay accessible - a great advantage in comparison to container solutions like #Apptainer's for example.

https://nixos.wiki/wiki/Nix-shell_shebang

#nix #NixOS