#apptainer — Public Fediverse posts

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

Which new features did I like the most?

Well, there are so many, it merits a preprint for which @egonw already lend tremendous support.

Anyway, here are my favourites:
- #Snakemake tracks all #metadata during the workflow execution. This caused many(!) file access requrests. Now, we have a SQLite-DB for that purpose, thereby lifting quite some overhead.
- containerizing workflows to a Dockerfile was possible for long time. With "--containerize apptainer" there is direct support for #apptainer
- when a workflow is aborted abruptly it cannot delete its lockfile. Now running `--unlock` will not require calculating the DAG any more.

Oh, there is much, much more. But the Changelog is already linked.

#SnakemakeHackathon2026

For those doing #HPC #DL model trainings, I need some suggestions. I want to use #horovod in a multi-gpu, multimode setting, but using #Apptainer (or #Docker) containers, due to cluster policy issues. Any reference to share? Issue here: github.com/horovod/horo...

Galaxy can be adjusted for sensitive data.
In the Netherlands, a GDPR-compliant Galaxy clone runs in an isolated network, converting Toolshed tools to Apptainer containers—secure, auditable, and reproducible.
Read more: https://galaxyproject.org/news/2025-01-20-esg-w-p5/

#UseGalaxy #GalaxyProject #UniFreiburg #EOSC #EuroScienceGateway #SensitiveData #GDPR #PrivacyByDesign #ReproducibleResearch #Bioinformatics #Apptainer #DataGovernance #Netherlands

My head hurts. Doing a deep-dive into containerization/VM with #Jenkins. So many options. I think I'll give #LXD a shot. But frankly, since my #OpenMPI program is already containerized via #Apptainer, I'll just go ahead and build on the (fairly powerful) Jenkins host for now, just to get the pipeline built as a proof of concept.

Time for an adult beverage.

🎩✨ Behold, the marvel of Apptainer: the magical name change from #Singularity that will revolutionize your Linux container universe by... being exactly like every other container system but with more labels. 🌟🔒 Rejoice as your software components remain safely ensconced in their digital dungeons, as you navigate the labyrinthine #GitHub to inspect your secured, portable future. 🤖🔍
https://apptainer.org/ #Apptainer #LinuxContainers #SoftwareDevelopment #HackerNews #ngated

Building an #Apptainer container for our colleagues in Oxford. The task is made much more pleasant by the awesome weather and a bottle of #Jever Pilsener. It’s very nice outside sitting on the deck.

Today I tried out #AMD #Instinct #MI300a for my existing Deep Learning pipeline. Good news: It worked out of the box. Bad news: For some reason it could not beat my local #Nvidia #1080ti...
After trying all sorts of #ROCM installation methods via prebuild wheels, #apptainer images etc I tried #nanogpt by @karpathy and sure enought: The gpt code ran approx 2x faster than on a #a100 ... I hope that this is due to my programming skills. Not AMD prefering #transformers over #CNNs ...

#ThisMonthInFluiddyn - Nov 2024 edition

* #FluidDyn, #FluidSim, #FluidImage now works with #numpy 2 and #mpi4py 4.0.

* #Spack packaging recipes

https://foss.heptapod.net/fluiddyn/fluiddyn/-/tree/branch/default/misc/spack?ref_type=heads

* #Apptainer and #Guix packaging for FluidSim

https://foss.heptapod.net/fluiddyn/fluidsim/-/tree/branch/default/doc/apptainer?ref_type=heads

https://foss.heptapod.net/fluiddyn/fluidsim/-/blob/7e52e7bed240665823e04e9f5f8a1fae7d787d06/doc/examples/clusters/gricad/README.md

* Basic support for #Jax is part of #Transonic with v0.7.x

* #FluidImage v0.5.x has a number of improvements, such as work towards matching #UVmat (the MATLAB #PIV suite), new executors and a command `fluidimage-mean`

Apptainer - Docker der Wissenschaft

Zur Replikation und Kapselung von Laufzeitumgebungen ist Apptainer die Lösung innerhalb der Computerwissenschaft.

#Apptainer #Wissenschaft #Linux #Replikation #Container #Docker #Linux

https://gnulinux.ch/apptainer-docker-der-wissenschaft

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

High Performance Software Foundation launched by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-Forms-High-Performance-Software-Foundation
#HPC #LinuxFoundation #OpenSource #Apptainer #E4S #HPCToolkit #Kokkos #Spack #Viskores #HPSF

Trinity インプットデータの保存先に因る影響を処理時間で比較してみた （S3 vs インスタンスストア）
https://dev.classmethod.jp/articles/comparing-trinity-input-data-storage-s3-vs-instance-store/

#dev_classmethod #Bioinformatics #HPC #AWS_ParallelCluster_3 #Apptainer #Trinity #Mountpoint_for_Amazon_S3 #AWS

AWS ParallelCluster で Apptainer と Moutpoint for Amazon S3 の組み合わせて Trinity を実行してみた
https://dev.classmethod.jp/articles/combining-apptainer-and-s3-to-run-trinity-on-aws-parallelcluster/

#dev_classmethod #Bioinformatics #HPC #AWS_ParallelCluster_3 #AWS #Mountpoint_for_Amazon_S3 #Apptainer

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

Hi @kev @mike, could you add these icons?

:scons: https://scons.org/images/SCons-Bricks.png
:datalad: https://avatars.githubusercontent.com/u/8927200?s=48&v=4
:hledger: https://hledger.org/images/coins2-248.png
:apptainer: https://github.com/apptainer/apptainer/blob/main/docs/logos/apptainer_v3.png

Thanks! 💛

@datalad #DataLad #Apptainer #SCons #hledger

@kylewritescode Good morning Kyle.

I already had a productive morning. I finally figured out how to let my hybrid #Apptainer #OpenMPI container for our NASA code talk to #UKY's #HPC cluster’s InfiniBand backend. Now the code scales beautifully. Next up, getting it to run on our collaborators’ cluster at #UMN.

I think I’ll have my coffee now.

High Performace Software Foundation to be formed by @linuxfoundation https://www.admin-magazine.com/News/Linux-Foundation-to-Form-High-Performance-Software-Foundation #HPC #LinuxFoundation #Spack #Kokkos #Apptainer #WarpX #HPSF #portable #SoftwareStack

However, a :nixos: #nixshell for "quickly give me :docker: #Docker for a minute" is not possible, right? It wouldn't start the daemon.

In this case #Apptainer is a cool alternative: no root or daemon required to run software from containers.

What tool to use to defrag #ext3 #partition images (e.g. #Apptainer #Singularity overlays)?

- e2defrag (https://launchpad.net/e2defrag): ultra old, doesn't compile (undefined reference to `HOLE_BLKADDR')
- shake (https://github.com/unbrice/shake): in userpace, not for image files (?)
- defrag (https://aur.archlinux.org/packages/defrag): weird script, apparently equally inappropriate
- e4defrag (https://e2fsprogs.sourceforge.net/) doesn't work for ext3 partitions

🤔

@datalad #containerization