home.social

#elasticsearch — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #elasticsearch, aggregated by home.social.

  1. Девять испытаний роста нагрузки: от стартапа к приложению для 25 миллионов пользователей

    Эта статья совсем не технический анализ, а увлекательный рассказ о том, как маленький, но очень перспективный стартап стал топовым приложением, а также о том, какие сложности встали на пути команды разработки, DevOps и тестирования X5 Tech. Мы сразу заложили основные принципы нагруженного приложения: микросервисы как основа всего, полное покрытие метриками, асинхронность, кэширование на максималках. Какую-то функциональность разрабатывали сами, где-то задействовали сервисы других техкоманд из X5, а где-то и сторонние решения с рынка. Весь код писали на Python, использовали FastAPI и другие популярные на тот момент фреймворки и технологии.

    habr.com/ru/companies/X5Tech/a

    #highload #микросервисы #latency #postgresql #elasticsearch #kubernetes #hpa #балансировка_нагрузки #нагрузочное_тестирование #observability

  2. Девять испытаний роста нагрузки: от стартапа к приложению для 25 миллионов пользователей

    Эта статья совсем не технический анализ, а увлекательный рассказ о том, как маленький, но очень перспективный стартап стал топовым приложением, а также о том, какие сложности встали на пути команды разработки, DevOps и тестирования X5 Tech. Мы сразу заложили основные принципы нагруженного приложения: микросервисы как основа всего, полное покрытие метриками, асинхронность, кэширование на максималках. Какую-то функциональность разрабатывали сами, где-то задействовали сервисы других техкоманд из X5, а где-то и сторонние решения с рынка. Весь код писали на Python, использовали FastAPI и другие популярные на тот момент фреймворки и технологии.

    habr.com/ru/companies/X5Tech/a

    #highload #микросервисы #latency #postgresql #elasticsearch #kubernetes #hpa #балансировка_нагрузки #нагрузочное_тестирование #observability

  3. Девять испытаний роста нагрузки: от стартапа к приложению для 25 миллионов пользователей

    Эта статья совсем не технический анализ, а увлекательный рассказ о том, как маленький, но очень перспективный стартап стал топовым приложением, а также о том, какие сложности встали на пути команды разработки, DevOps и тестирования X5 Tech. Мы сразу заложили основные принципы нагруженного приложения: микросервисы как основа всего, полное покрытие метриками, асинхронность, кэширование на максималках. Какую-то функциональность разрабатывали сами, где-то задействовали сервисы других техкоманд из X5, а где-то и сторонние решения с рынка. Весь код писали на Python, использовали FastAPI и другие популярные на тот момент фреймворки и технологии.

    habr.com/ru/companies/X5Tech/a

    #highload #микросервисы #latency #postgresql #elasticsearch #kubernetes #hpa #балансировка_нагрузки #нагрузочное_тестирование #observability

  4. Девять испытаний роста нагрузки: от стартапа к приложению для 25 миллионов пользователей

    Эта статья совсем не технический анализ, а увлекательный рассказ о том, как маленький, но очень перспективный стартап стал топовым приложением, а также о том, какие сложности встали на пути команды разработки, DevOps и тестирования X5 Tech. Мы сразу заложили основные принципы нагруженного приложения: микросервисы как основа всего, полное покрытие метриками, асинхронность, кэширование на максималках. Какую-то функциональность разрабатывали сами, где-то задействовали сервисы других техкоманд из X5, а где-то и сторонние решения с рынка. Весь код писали на Python, использовали FastAPI и другие популярные на тот момент фреймворки и технологии.

    habr.com/ru/companies/X5Tech/a

    #highload #микросервисы #latency #postgresql #elasticsearch #kubernetes #hpa #балансировка_нагрузки #нагрузочное_тестирование #observability

  5. 🔍 Spark + Elasticsearch Debugging 🧵

    Building a cybersecurity analytics platform. Hit 2 blockers:

    ❌ JAR path mismatch → Fixed absolute path
    ❌ No data nodes (single-node Docker ES) → Added es.nodes.wan.only=true

    ✅ Result: 89 records loaded. Working pipeline!

    Lesson: Verify JAR paths + disable node discovery for single-node ES.

    #PySpark #Elasticsearch #DataEngineering #CyberSecurity #Debugging

  6. 🔍 Spark + Elasticsearch Debugging 🧵

    Building a cybersecurity analytics platform. Hit 2 blockers:

    ❌ JAR path mismatch → Fixed absolute path
    ❌ No data nodes (single-node Docker ES) → Added es.nodes.wan.only=true

    ✅ Result: 89 records loaded. Working pipeline!

    Lesson: Verify JAR paths + disable node discovery for single-node ES.

    #PySpark #Elasticsearch #DataEngineering #CyberSecurity #Debugging

  7. How to Install #Zammad on #AlmaLinux #VPS (10-Minute Quick-Start Guide) This article provides a guide on how to install Zammad on #AlmaLinux VPS.

    If you're looking to run a robust open-source #helpdesk on your AlmaLinux VPS, Zammad is a great pick. It's modern, feature-packed, and supports everything from ticketing to chat and reporting. This guide ...
    Continued 👉 blog.radwebhosting.com/how-to- #elasticsearch #selfhosted #certbot #ticket #selfhosting #customersupportplatform #opensource #letsencrypt

  8. How to Install #Zammad on #AlmaLinux #VPS (10-Minute Quick-Start Guide) This article provides a guide on how to install Zammad on #AlmaLinux VPS.

    If you're looking to run a robust open-source #helpdesk on your AlmaLinux VPS, Zammad is a great pick. It's modern, feature-packed, and supports everything from ticketing to chat and reporting. This guide ...
    Continued 👉 blog.radwebhosting.com/how-to- #elasticsearch #selfhosted #certbot #ticket #selfhosting #customersupportplatform #opensource #letsencrypt

  9. How to Install #Zammad on #AlmaLinux #VPS (10-Minute Quick-Start Guide) This article provides a guide on how to install Zammad on #AlmaLinux VPS.

    If you're looking to run a robust open-source #helpdesk on your AlmaLinux VPS, Zammad is a great pick. It's modern, feature-packed, and supports everything from ticketing to chat and reporting. This guide ...
    Continued 👉 blog.radwebhosting.com/how-to- #elasticsearch #selfhosted #certbot #ticket #selfhosting #customersupportplatform #opensource #letsencrypt

  10. How to Install #Zammad on #AlmaLinux #VPS (10-Minute Quick-Start Guide) This article provides a guide on how to install Zammad on #AlmaLinux VPS.

    If you're looking to run a robust open-source #helpdesk on your AlmaLinux VPS, Zammad is a great pick. It's modern, feature-packed, and supports everything from ticketing to chat and reporting. This guide ...
    Continued 👉 blog.radwebhosting.com/how-to- #selfhosting #selfhosted #elasticsearch #ticket #letsencrypt #opensource #certbot #customersupportplatform

  11. How to Install #Zammad on #AlmaLinux #VPS (10-Minute Quick-Start Guide) This article provides a guide on how to install Zammad on #AlmaLinux VPS.

    If you're looking to run a robust open-source #helpdesk on your AlmaLinux VPS, Zammad is a great pick. It's modern, feature-packed, and supports everything from ticketing to chat and reporting. This guide ...
    Continued 👉 blog.radwebhosting.com/how-to- #selfhosting #selfhosted #elasticsearch #ticket #letsencrypt #opensource #certbot #customersupportplatform

  12. How to Install #Zammad on #AlmaLinux #VPS (10-Minute Quick-Start Guide) This article provides a guide on how to install Zammad on #AlmaLinux VPS.

    If you're looking to run a robust open-source #helpdesk on your AlmaLinux VPS, Zammad is a great pick. It's modern, feature-packed, and supports everything from ticketing to chat and reporting. This guide ...
    Continued 👉 blog.radwebhosting.com/how-to- #selfhosting #selfhosted #elasticsearch #ticket #letsencrypt #opensource #certbot #customersupportplatform

  13. How to Install #Zammad on #AlmaLinux #VPS (10-Minute Quick-Start Guide) This article provides a guide on how to install Zammad on #AlmaLinux VPS.

    If you're looking to run a robust open-source #helpdesk on your AlmaLinux VPS, Zammad is a great pick. It's modern, feature-packed, and supports everything from ticketing to chat and reporting. This guide ...
    Continued 👉 blog.radwebhosting.com/how-to- #selfhosting #selfhosted #elasticsearch #ticket #letsencrypt #opensource #certbot #customersupportplatform

  14. Mastodon Incident Report / Root cause analysis:

    Earlier today, users experienced timeouts with Search, Hashtags, and Autocomplete.

    Root Cause: Our setup separates the Mastodon frontend VPS (Hetzner) from backend services (for example Elasticsearch) via an OPNSense firewall. Suricata (our IPS) triggered a false-positive on internal traffic and aggressively blocked the VPS IP, severing the connection to the search database.

    Resolution: We identified the false-positive, added the frontend IP to the whitelist, and traffic immediately normalized. Everything is back to green!

    #mastodon #mastoadmin #burningboard #elasticsearch #firewall #opnsense #suricata #oopsie

  15. Mastodon Incident Report / Root cause analysis:

    Earlier today, users experienced timeouts with Search, Hashtags, and Autocomplete.

    Root Cause: Our setup separates the Mastodon frontend VPS (Hetzner) from backend services (for example Elasticsearch) via an OPNSense firewall. Suricata (our IPS) triggered a false-positive on internal traffic and aggressively blocked the VPS IP, severing the connection to the search database.

    Resolution: We identified the false-positive, added the frontend IP to the whitelist, and traffic immediately normalized. Everything is back to green!

    #mastodon #mastoadmin #burningboard #elasticsearch #firewall #opnsense #suricata #oopsie

  16. Mastodon Incident Report / Root cause analysis:

    Earlier today, users experienced timeouts with Search, Hashtags, and Autocomplete.

    Root Cause: Our setup separates the Mastodon frontend VPS (Hetzner) from backend services (for example Elasticsearch) via an OPNSense firewall. Suricata (our IPS) triggered a false-positive on internal traffic and aggressively blocked the VPS IP, severing the connection to the search database.

    Resolution: We identified the false-positive, added the frontend IP to the whitelist, and traffic immediately normalized. Everything is back to green!

    #mastodon #mastoadmin #burningboard #elasticsearch #firewall #opnsense #suricata #oopsie

  17. Mastodon Incident Report / Root cause analysis:

    Earlier today, users experienced timeouts with Search, Hashtags, and Autocomplete.

    Root Cause: Our setup separates the Mastodon frontend VPS (Hetzner) from backend services (for example Elasticsearch) via an OPNSense firewall. Suricata (our IPS) triggered a false-positive on internal traffic and aggressively blocked the VPS IP, severing the connection to the search database.

    Resolution: We identified the false-positive, added the frontend IP to the whitelist, and traffic immediately normalized. Everything is back to green!

    #mastodon #mastoadmin #burningboard #elasticsearch #firewall #opnsense #suricata #oopsie

  18. Mastodon Incident Report / Root cause analysis:

    Earlier today, users experienced timeouts with Search, Hashtags, and Autocomplete.

    Root Cause: Our setup separates the Mastodon frontend VPS (Hetzner) from backend services (for example Elasticsearch) via an OPNSense firewall. Suricata (our IPS) triggered a false-positive on internal traffic and aggressively blocked the VPS IP, severing the connection to the search database.

    Resolution: We identified the false-positive, added the frontend IP to the whitelist, and traffic immediately normalized. Everything is back to green!

    #mastodon #mastoadmin #burningboard #elasticsearch #firewall #opnsense #suricata #oopsie

  19. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #opensource #reverseproxy #elasticsearch #selfhosted #forumsoftware #selfhosting #rockylinux #rubyonrails #rubygems #redis #forum #sidekiq #postgresql #letsencrypt

  20. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #opensource #reverseproxy #elasticsearch #selfhosted #forumsoftware #selfhosting #rockylinux #rubyonrails #rubygems #redis #forum #sidekiq #postgresql #letsencrypt

  21. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #opensource #reverseproxy #elasticsearch #selfhosted #forumsoftware #selfhosting #rockylinux #rubyonrails #rubygems #redis #forum #sidekiq #postgresql #letsencrypt

  22. CW: release notes for Malcolm v26.05.0, a network traffic analysis tool suite for network security monitoring

    Malcolm v26.05.0 delivers a mix of feature improvements, performance improvements, bug fixes, dependency updates, and deployment refinements across Malcolm and Hedgehog for both Docker- and Kubernetes-based workflows.

    If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

    github.com/idaholab/Malcolm/co

    • ✨ Features and enhancements
      • #726 — use hierarchical structure for NetBox device roles
        • Expanded/reworked NetBox preloaded device roles into a hierarchical taxonomy (thanks Crubumble)
      • #867 — examine large chown'ed directories in container images and see if they can be reduced
      • #954 — allow users to provide custom netbox scripts to be automatically registered on startup (thanks PrudhviChanda)
        • Added NetBox custom script support in the container/runtime and docs, including bind-mounting ./netbox/custom-scripts and automatic script registration at startup
        • Renamed NetBox startup/control scripts from netbox/scripts to netbox/control-scripts
      • Added file.strings extraction/indexing/search support across Strelka → Logstash → OpenSearch templates (wildcard field mapping type) → Arkime/WISE
      • Added configurable Zeek file analyzer timeout via ZEEK_FILE_ANALYZER_TIMEOUT_SEC
      • netdev users in ISO-installed environment can run nmcli and nmtui to configure network interfaces.
      • the malcolm_appliance_packager.sh script that creates a tarball of Malcolm images can now package for both Malcolm and Hedgehog profiles.
    • ✅ Component version updates
    • 🐛 Bug fixes
      • #757 — multiple OpenSearch nodes (using Malcolm-Helm) fail to communicate with each other due to self-signed certs (thanks scott-jeffery)
        • OpenSearch post-start setup now supports configurable default replica counts instead of always forcing single-node replicas to 0
        • OpenSearch self-signed internal cert generation can now be skipped when external/preexisting certs are being used
      • #827 — Fix raspberry pi build which is broken since v25.12.0 Hedgehog/Malcolm platform unification
        • Updated Hedgehog Raspberry Pi docs and first-boot behavior/documentation
        • Hedgehog Raspberry Pi image now forces password change for sensor on first login and disables direct root password login by default
        • Refactored Raspberry Pi GitHub Actions build into reusable workflow .github/workflows/raspi-build-push.yml
      • #878 — Arkime capture Fails to Start on Hedgehog When WISE Web Config Is Enabled
        • Arkime RBAC role-mapping injection is now only applied when role-based access control is enabled
        • Arkime WISE configuration initialization now handles missing/empty persistent config files more robustly
        • Arkime live capture now normalizes WISE URLs better, follows redirects when probing, and avoids some bad URL construction edge cases
      • #957 — configuration script can disable ICS parsers unintentionally
      • #959 — Arkime sessions view attempts to load PCAP for Zeek and Suricata logs (which don't have PCAP) (see also arkime/arkime#3934)
      • Fixed one-off cleanup of interrupted Zeek intel files during stop --wipe
    • 🧹 Code and project maintenance
      • Documentation improvements
      • #913 — replace ingress-nginx which is EOL
        • Switched Kubernetes ingress example/docs from ingress-nginx to Traefik and replaced the old Vagrant example with a new RKE2/Traefik-based environment
        • Fixed malformed indentation in kubernetes/01-volumes-nfs.yml.example for the filescan volume section
        • Removed deprecated Kubernetes example files for ingress-nginx and the old separate NFS-server Vagrant setup
        • opensearch is no longer part of the hedgehog Docker Compose profile, and some depends_on relationships were adjusted accordingly
      • #942 - Fixed mutable default argument usage in Zeek threat feed helper functions (thanks @stef41)
      • #917 — develop IronBank (US DoD) images for Malcolm
    • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
      • Added ZEEK_FILE_ANALYZER_TIMEOUT_SEC (default 5) to zeek.env. This is the default amount of time a file can be inactive before the file analysis gives up and discards any internal state related to the file.
      • ZEEK_CLUSTER_BACKEND can be specified in zeek.env to specify the Zeek cluster backend (ZeroMQ vs Broker).
    • ❌ Errata
      • Under NetBox → Plugins → NetBox HealthCheck Plugin → HealthCheck the error "unavailable: Unable to connect to Redis: Connection Error" is displayed. This is a side effect of #882 and does not actually indicate a problem with NetBox or its connection to Valkey. This will be fixed in the next release.

    Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

    Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

    Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

    As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

    #Malcolm #HedgehogLinux #Zeek #Arkime #Strelka #NetBox #OpenSearch #Elasticsearch #Suricata #PCAP #NetworkTrafficAnalysis #networksecuritymonitoring #OT #ICS #icssecurity #CyberSecurity #Cyber #Infosec #INL

  23. CW: release notes for Malcolm v26.05.0, a network traffic analysis tool suite for network security monitoring

    Malcolm v26.05.0 delivers a mix of feature improvements, performance improvements, bug fixes, dependency updates, and deployment refinements across Malcolm and Hedgehog for both Docker- and Kubernetes-based workflows.

    If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

    github.com/idaholab/Malcolm/co

    • ✨ Features and enhancements
      • #726 — use hierarchical structure for NetBox device roles
        • Expanded/reworked NetBox preloaded device roles into a hierarchical taxonomy (thanks Crubumble)
      • #867 — examine large chown'ed directories in container images and see if they can be reduced
      • #954 — allow users to provide custom netbox scripts to be automatically registered on startup (thanks PrudhviChanda)
        • Added NetBox custom script support in the container/runtime and docs, including bind-mounting ./netbox/custom-scripts and automatic script registration at startup
        • Renamed NetBox startup/control scripts from netbox/scripts to netbox/control-scripts
      • Added file.strings extraction/indexing/search support across Strelka → Logstash → OpenSearch templates (wildcard field mapping type) → Arkime/WISE
      • Added configurable Zeek file analyzer timeout via ZEEK_FILE_ANALYZER_TIMEOUT_SEC
      • netdev users in ISO-installed environment can run nmcli and nmtui to configure network interfaces.
      • the malcolm_appliance_packager.sh script that creates a tarball of Malcolm images can now package for both Malcolm and Hedgehog profiles.
    • ✅ Component version updates
    • 🐛 Bug fixes
      • #757 — multiple OpenSearch nodes (using Malcolm-Helm) fail to communicate with each other due to self-signed certs (thanks scott-jeffery)
        • OpenSearch post-start setup now supports configurable default replica counts instead of always forcing single-node replicas to 0
        • OpenSearch self-signed internal cert generation can now be skipped when external/preexisting certs are being used
      • #827 — Fix raspberry pi build which is broken since v25.12.0 Hedgehog/Malcolm platform unification
        • Updated Hedgehog Raspberry Pi docs and first-boot behavior/documentation
        • Hedgehog Raspberry Pi image now forces password change for sensor on first login and disables direct root password login by default
        • Refactored Raspberry Pi GitHub Actions build into reusable workflow .github/workflows/raspi-build-push.yml
      • #878 — Arkime capture Fails to Start on Hedgehog When WISE Web Config Is Enabled
        • Arkime RBAC role-mapping injection is now only applied when role-based access control is enabled
        • Arkime WISE configuration initialization now handles missing/empty persistent config files more robustly
        • Arkime live capture now normalizes WISE URLs better, follows redirects when probing, and avoids some bad URL construction edge cases
      • #957 — configuration script can disable ICS parsers unintentionally
      • #959 — Arkime sessions view attempts to load PCAP for Zeek and Suricata logs (which don't have PCAP) (see also arkime/arkime#3934)
      • Fixed one-off cleanup of interrupted Zeek intel files during stop --wipe
    • 🧹 Code and project maintenance
      • Documentation improvements
      • #913 — replace ingress-nginx which is EOL
        • Switched Kubernetes ingress example/docs from ingress-nginx to Traefik and replaced the old Vagrant example with a new RKE2/Traefik-based environment
        • Fixed malformed indentation in kubernetes/01-volumes-nfs.yml.example for the filescan volume section
        • Removed deprecated Kubernetes example files for ingress-nginx and the old separate NFS-server Vagrant setup
        • opensearch is no longer part of the hedgehog Docker Compose profile, and some depends_on relationships were adjusted accordingly
      • #942 - Fixed mutable default argument usage in Zeek threat feed helper functions (thanks @stef41)
      • #917 — develop IronBank (US DoD) images for Malcolm
    • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
      • Added ZEEK_FILE_ANALYZER_TIMEOUT_SEC (default 5) to zeek.env. This is the default amount of time a file can be inactive before the file analysis gives up and discards any internal state related to the file.
      • ZEEK_CLUSTER_BACKEND can be specified in zeek.env to specify the Zeek cluster backend (ZeroMQ vs Broker).
    • ❌ Errata
      • Under NetBox → Plugins → NetBox HealthCheck Plugin → HealthCheck the error "unavailable: Unable to connect to Redis: Connection Error" is displayed. This is a side effect of #882 and does not actually indicate a problem with NetBox or its connection to Valkey. This will be fixed in the next release.

    Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

    Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

    Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

    As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

    #Malcolm #HedgehogLinux #Zeek #Arkime #Strelka #NetBox #OpenSearch #Elasticsearch #Suricata #PCAP #NetworkTrafficAnalysis #networksecuritymonitoring #OT #ICS #icssecurity #CyberSecurity #Cyber #Infosec #INL

  24. CW: release notes for Malcolm v26.05.0, a network traffic analysis tool suite for network security monitoring

    Malcolm v26.05.0 delivers a mix of feature improvements, performance improvements, bug fixes, dependency updates, and deployment refinements across Malcolm and Hedgehog for both Docker- and Kubernetes-based workflows.

    If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

    github.com/idaholab/Malcolm/co

    • ✨ Features and enhancements
      • #726 — use hierarchical structure for NetBox device roles
        • Expanded/reworked NetBox preloaded device roles into a hierarchical taxonomy (thanks Crubumble)
      • #867 — examine large chown'ed directories in container images and see if they can be reduced
      • #954 — allow users to provide custom netbox scripts to be automatically registered on startup (thanks PrudhviChanda)
        • Added NetBox custom script support in the container/runtime and docs, including bind-mounting ./netbox/custom-scripts and automatic script registration at startup
        • Renamed NetBox startup/control scripts from netbox/scripts to netbox/control-scripts
      • Added file.strings extraction/indexing/search support across Strelka → Logstash → OpenSearch templates (wildcard field mapping type) → Arkime/WISE
      • Added configurable Zeek file analyzer timeout via ZEEK_FILE_ANALYZER_TIMEOUT_SEC
      • netdev users in ISO-installed environment can run nmcli and nmtui to configure network interfaces.
      • the malcolm_appliance_packager.sh script that creates a tarball of Malcolm images can now package for both Malcolm and Hedgehog profiles.
    • ✅ Component version updates
    • 🐛 Bug fixes
      • #757 — multiple OpenSearch nodes (using Malcolm-Helm) fail to communicate with each other due to self-signed certs (thanks scott-jeffery)
        • OpenSearch post-start setup now supports configurable default replica counts instead of always forcing single-node replicas to 0
        • OpenSearch self-signed internal cert generation can now be skipped when external/preexisting certs are being used
      • #827 — Fix raspberry pi build which is broken since v25.12.0 Hedgehog/Malcolm platform unification
        • Updated Hedgehog Raspberry Pi docs and first-boot behavior/documentation
        • Hedgehog Raspberry Pi image now forces password change for sensor on first login and disables direct root password login by default
        • Refactored Raspberry Pi GitHub Actions build into reusable workflow .github/workflows/raspi-build-push.yml
      • #878 — Arkime capture Fails to Start on Hedgehog When WISE Web Config Is Enabled
        • Arkime RBAC role-mapping injection is now only applied when role-based access control is enabled
        • Arkime WISE configuration initialization now handles missing/empty persistent config files more robustly
        • Arkime live capture now normalizes WISE URLs better, follows redirects when probing, and avoids some bad URL construction edge cases
      • #957 — configuration script can disable ICS parsers unintentionally
      • #959 — Arkime sessions view attempts to load PCAP for Zeek and Suricata logs (which don't have PCAP) (see also arkime/arkime#3934)
      • Fixed one-off cleanup of interrupted Zeek intel files during stop --wipe
    • 🧹 Code and project maintenance
      • Documentation improvements
      • #913 — replace ingress-nginx which is EOL
        • Switched Kubernetes ingress example/docs from ingress-nginx to Traefik and replaced the old Vagrant example with a new RKE2/Traefik-based environment
        • Fixed malformed indentation in kubernetes/01-volumes-nfs.yml.example for the filescan volume section
        • Removed deprecated Kubernetes example files for ingress-nginx and the old separate NFS-server Vagrant setup
        • opensearch is no longer part of the hedgehog Docker Compose profile, and some depends_on relationships were adjusted accordingly
      • #942 - Fixed mutable default argument usage in Zeek threat feed helper functions (thanks @stef41)
      • #917 — develop IronBank (US DoD) images for Malcolm
    • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
      • Added ZEEK_FILE_ANALYZER_TIMEOUT_SEC (default 5) to zeek.env. This is the default amount of time a file can be inactive before the file analysis gives up and discards any internal state related to the file.
      • ZEEK_CLUSTER_BACKEND can be specified in zeek.env to specify the Zeek cluster backend (ZeroMQ vs Broker).
    • ❌ Errata
      • Under NetBox → Plugins → NetBox HealthCheck Plugin → HealthCheck the error "unavailable: Unable to connect to Redis: Connection Error" is displayed. This is a side effect of #882 and does not actually indicate a problem with NetBox or its connection to Valkey. This will be fixed in the next release.

    Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

    Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

    Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

    As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

    #Malcolm #HedgehogLinux #Zeek #Arkime #Strelka #NetBox #OpenSearch #Elasticsearch #Suricata #PCAP #NetworkTrafficAnalysis #networksecuritymonitoring #OT #ICS #icssecurity #CyberSecurity #Cyber #Infosec #INL

  25. CW: release notes for Malcolm v26.05.0, a network traffic analysis tool suite for network security monitoring

    Malcolm v26.05.0 delivers a mix of feature improvements, performance improvements, bug fixes, dependency updates, and deployment refinements across Malcolm and Hedgehog for both Docker- and Kubernetes-based workflows.

    If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

    github.com/idaholab/Malcolm/co

    • ✨ Features and enhancements
      • #726 — use hierarchical structure for NetBox device roles
        • Expanded/reworked NetBox preloaded device roles into a hierarchical taxonomy (thanks Crubumble)
      • #867 — examine large chown'ed directories in container images and see if they can be reduced
      • #954 — allow users to provide custom netbox scripts to be automatically registered on startup (thanks PrudhviChanda)
        • Added NetBox custom script support in the container/runtime and docs, including bind-mounting ./netbox/custom-scripts and automatic script registration at startup
        • Renamed NetBox startup/control scripts from netbox/scripts to netbox/control-scripts
      • Added file.strings extraction/indexing/search support across Strelka → Logstash → OpenSearch templates (wildcard field mapping type) → Arkime/WISE
      • Added configurable Zeek file analyzer timeout via ZEEK_FILE_ANALYZER_TIMEOUT_SEC
      • netdev users in ISO-installed environment can run nmcli and nmtui to configure network interfaces.
      • the malcolm_appliance_packager.sh script that creates a tarball of Malcolm images can now package for both Malcolm and Hedgehog profiles.
    • ✅ Component version updates
    • 🐛 Bug fixes
      • #757 — multiple OpenSearch nodes (using Malcolm-Helm) fail to communicate with each other due to self-signed certs (thanks scott-jeffery)
        • OpenSearch post-start setup now supports configurable default replica counts instead of always forcing single-node replicas to 0
        • OpenSearch self-signed internal cert generation can now be skipped when external/preexisting certs are being used
      • #827 — Fix raspberry pi build which is broken since v25.12.0 Hedgehog/Malcolm platform unification
        • Updated Hedgehog Raspberry Pi docs and first-boot behavior/documentation
        • Hedgehog Raspberry Pi image now forces password change for sensor on first login and disables direct root password login by default
        • Refactored Raspberry Pi GitHub Actions build into reusable workflow .github/workflows/raspi-build-push.yml
      • #878 — Arkime capture Fails to Start on Hedgehog When WISE Web Config Is Enabled
        • Arkime RBAC role-mapping injection is now only applied when role-based access control is enabled
        • Arkime WISE configuration initialization now handles missing/empty persistent config files more robustly
        • Arkime live capture now normalizes WISE URLs better, follows redirects when probing, and avoids some bad URL construction edge cases
      • #957 — configuration script can disable ICS parsers unintentionally
      • #959 — Arkime sessions view attempts to load PCAP for Zeek and Suricata logs (which don't have PCAP) (see also arkime/arkime#3934)
      • Fixed one-off cleanup of interrupted Zeek intel files during stop --wipe
    • 🧹 Code and project maintenance
      • Documentation improvements
      • #913 — replace ingress-nginx which is EOL
        • Switched Kubernetes ingress example/docs from ingress-nginx to Traefik and replaced the old Vagrant example with a new RKE2/Traefik-based environment
        • Fixed malformed indentation in kubernetes/01-volumes-nfs.yml.example for the filescan volume section
        • Removed deprecated Kubernetes example files for ingress-nginx and the old separate NFS-server Vagrant setup
        • opensearch is no longer part of the hedgehog Docker Compose profile, and some depends_on relationships were adjusted accordingly
      • #942 - Fixed mutable default argument usage in Zeek threat feed helper functions (thanks @stef41)
      • #917 — develop IronBank (US DoD) images for Malcolm
    • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
      • Added ZEEK_FILE_ANALYZER_TIMEOUT_SEC (default 5) to zeek.env. This is the default amount of time a file can be inactive before the file analysis gives up and discards any internal state related to the file.
      • ZEEK_CLUSTER_BACKEND can be specified in zeek.env to specify the Zeek cluster backend (ZeroMQ vs Broker).
    • ❌ Errata
      • Under NetBox → Plugins → NetBox HealthCheck Plugin → HealthCheck the error "unavailable: Unable to connect to Redis: Connection Error" is displayed. This is a side effect of #882 and does not actually indicate a problem with NetBox or its connection to Valkey. This will be fixed in the next release.

    Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

    Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

    Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

    As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

    #Malcolm #HedgehogLinux #Zeek #Arkime #Strelka #NetBox #OpenSearch #Elasticsearch #Suricata #PCAP #NetworkTrafficAnalysis #networksecuritymonitoring #OT #ICS #icssecurity #CyberSecurity #Cyber #Infosec #INL

  26. CW: release notes for Malcolm v26.05.0, a network traffic analysis tool suite for network security monitoring

    Malcolm v26.05.0 delivers a mix of feature improvements, performance improvements, bug fixes, dependency updates, and deployment refinements across Malcolm and Hedgehog for both Docker- and Kubernetes-based workflows.

    If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

    github.com/idaholab/Malcolm/co

    • ✨ Features and enhancements
      • #726 — use hierarchical structure for NetBox device roles
        • Expanded/reworked NetBox preloaded device roles into a hierarchical taxonomy (thanks Crubumble)
      • #867 — examine large chown'ed directories in container images and see if they can be reduced
      • #954 — allow users to provide custom netbox scripts to be automatically registered on startup (thanks PrudhviChanda)
        • Added NetBox custom script support in the container/runtime and docs, including bind-mounting ./netbox/custom-scripts and automatic script registration at startup
        • Renamed NetBox startup/control scripts from netbox/scripts to netbox/control-scripts
      • Added file.strings extraction/indexing/search support across Strelka → Logstash → OpenSearch templates (wildcard field mapping type) → Arkime/WISE
      • Added configurable Zeek file analyzer timeout via ZEEK_FILE_ANALYZER_TIMEOUT_SEC
      • netdev users in ISO-installed environment can run nmcli and nmtui to configure network interfaces.
      • the malcolm_appliance_packager.sh script that creates a tarball of Malcolm images can now package for both Malcolm and Hedgehog profiles.
    • ✅ Component version updates
    • 🐛 Bug fixes
      • #757 — multiple OpenSearch nodes (using Malcolm-Helm) fail to communicate with each other due to self-signed certs (thanks scott-jeffery)
        • OpenSearch post-start setup now supports configurable default replica counts instead of always forcing single-node replicas to 0
        • OpenSearch self-signed internal cert generation can now be skipped when external/preexisting certs are being used
      • #827 — Fix raspberry pi build which is broken since v25.12.0 Hedgehog/Malcolm platform unification
        • Updated Hedgehog Raspberry Pi docs and first-boot behavior/documentation
        • Hedgehog Raspberry Pi image now forces password change for sensor on first login and disables direct root password login by default
        • Refactored Raspberry Pi GitHub Actions build into reusable workflow .github/workflows/raspi-build-push.yml
      • #878 — Arkime capture Fails to Start on Hedgehog When WISE Web Config Is Enabled
        • Arkime RBAC role-mapping injection is now only applied when role-based access control is enabled
        • Arkime WISE configuration initialization now handles missing/empty persistent config files more robustly
        • Arkime live capture now normalizes WISE URLs better, follows redirects when probing, and avoids some bad URL construction edge cases
      • #957 — configuration script can disable ICS parsers unintentionally
      • #959 — Arkime sessions view attempts to load PCAP for Zeek and Suricata logs (which don't have PCAP) (see also arkime/arkime#3934)
      • Fixed one-off cleanup of interrupted Zeek intel files during stop --wipe
    • 🧹 Code and project maintenance
      • Documentation improvements
      • #913 — replace ingress-nginx which is EOL
        • Switched Kubernetes ingress example/docs from ingress-nginx to Traefik and replaced the old Vagrant example with a new RKE2/Traefik-based environment
        • Fixed malformed indentation in kubernetes/01-volumes-nfs.yml.example for the filescan volume section
        • Removed deprecated Kubernetes example files for ingress-nginx and the old separate NFS-server Vagrant setup
        • opensearch is no longer part of the hedgehog Docker Compose profile, and some depends_on relationships were adjusted accordingly
      • #942 - Fixed mutable default argument usage in Zeek threat feed helper functions (thanks @stef41)
      • #917 — develop IronBank (US DoD) images for Malcolm
    • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.
      • Added ZEEK_FILE_ANALYZER_TIMEOUT_SEC (default 5) to zeek.env. This is the default amount of time a file can be inactive before the file analysis gives up and discards any internal state related to the file.
      • ZEEK_CLUSTER_BACKEND can be specified in zeek.env to specify the Zeek cluster backend (ZeroMQ vs Broker).
    • ❌ Errata
      • Under NetBox → Plugins → NetBox HealthCheck Plugin → HealthCheck the error "unavailable: Unable to connect to Redis: Connection Error" is displayed. This is a side effect of #882 and does not actually indicate a problem with NetBox or its connection to Valkey. This will be fixed in the next release.

    Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

    Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

    Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

    As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

    #Malcolm #HedgehogLinux #Zeek #Arkime #Strelka #NetBox #OpenSearch #Elasticsearch #Suricata #PCAP #NetworkTrafficAnalysis #networksecuritymonitoring #OT #ICS #icssecurity #CyberSecurity #Cyber #Infosec #INL

  27. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #postgresql #forum #opensource #rockylinux #letsencrypt #selfhosted #rubyonrails #rubygems #reverseproxy #selfhosting #sidekiq #elasticsearch #forumsoftware #redis

  28. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #postgresql #forum #opensource #rockylinux #letsencrypt #selfhosted #rubyonrails #rubygems #reverseproxy #selfhosting #sidekiq #elasticsearch #forumsoftware #redis

  29. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #postgresql #forum #opensource #rockylinux #letsencrypt #selfhosted #rubyonrails #rubygems #reverseproxy #selfhosting #sidekiq #elasticsearch #forumsoftware #redis

  30. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #postgresql #forum #opensource #rockylinux #letsencrypt #selfhosted #rubyonrails #rubygems #reverseproxy #selfhosting #sidekiq #elasticsearch #forumsoftware #redis

  31. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #selfhosting #postgresql #elasticsearch #rubygems #rubyonrails #letsencrypt #forum #opensource #reverseproxy #selfhosted #rockylinux #sidekiq #redis #forumsoftware

  32. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #selfhosting #postgresql #elasticsearch #rubygems #rubyonrails #letsencrypt #forum #opensource #reverseproxy #selfhosted #rockylinux #sidekiq #redis #forumsoftware

  33. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #selfhosting #postgresql #elasticsearch #rubygems #rubyonrails #letsencrypt #forum #opensource #reverseproxy #selfhosted #rockylinux #sidekiq #redis #forumsoftware

  34. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #postgresql #rubygems #selfhosted #forumsoftware #rubyonrails #opensource #redis #forum #elasticsearch #rockylinux #sidekiq #letsencrypt #reverseproxy #selfhosting

  35. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #postgresql #rubygems #selfhosted #forumsoftware #rubyonrails #opensource #redis #forum #elasticsearch #rockylinux #sidekiq #letsencrypt #reverseproxy #selfhosting

  36. How to Deploy #Forem on Rocky Linux #VPS This article provides a guide detailing how to deploy Forem on Rocky Linux VPS.
    What is Forem?
    Forem is a robust, open-source platform for building communities like DEV.to. Deploying it on Rocky Linux involves setting up dependencies like PostgreSQL, ...
    Continued 👉 blog.radwebhosting.com/deploy- #postgresql #rubygems #selfhosted #forumsoftware #rubyonrails #opensource #redis #forum #elasticsearch #rockylinux #sidekiq #letsencrypt #reverseproxy #selfhosting