home.social

#elasticsearch — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #elasticsearch, aggregated by home.social.

  1. CW: release notes for Malcolm v26.08.0, a network traffic analysis tool suite for network security monitoring

    Malcolm v26.08.0 adds a NetBox purdue_zone custom field that propagates ICS/OT network zone classifications to devices, prefixes, and virtual machines (and automatically to autopopulated devices from their containing prefix); Raspberry Pi 5 support for Hedgehog Linux; and, configurable Strelka scanner and disabled-Suricata-SID lists. This release also fixes five security vulnerabilities: an nginx RBAC bypass via percent-encoded, case-varied, or slash-doubled request paths; an archive-bomb bypass affecting raw-stream and lzip-compressed uploads; a case-variant path bypass of the nginx auth gate exposing the Arkime backend to forged identity headers; an Arkime authentication gap on sensor nodes that fell back to digest instead of enforcing s2s; and a CSRF vulnerability in the kiosk /script_call endpoint allowing unauthenticated data-destructive operations. Arkime, Zeek, NetBox, OpenSearch, OpenSearch Dashboards, Logstash, Filebeat, Keycloak, and other components have been updated as well. Several other bug fixes and general improvements are also included.

    github.com/idaholab/Malcolm/co

    See the Release Notes for the full set of new features, enhancements, bug fixes, and component version updates.

    Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

    Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

    Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

    As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

    #Malcolm #HedgehogLinux #Zeek #Arkime #Strelka #NetBox #OpenSearch #Elasticsearch #Suricata #PCAP #NetworkTrafficAnalysis #networksecuritymonitoring #OT #ICS #icssecurity #CyberSecurity #Cyber #Infosec #INL

  2. CW: release notes for Malcolm v26.07.1, a network traffic analysis tool suite for network security monitoring

    Malcolm v26.07.1 adds a few minor changes on top of Malcolm v26.07.0, the most notable being a fix for a crash in the strelka-backend container on arm64 platforms. Malcolm v26.07.0 added IEC 60870-5-104 (IEC 104) protocol support using CERT.LV's Zeek plugin, including Logstash parsing, ECS normalization, Arkime fields, and a new OpenSearch Dashboards dashboard. This release also fixes three archive extraction and authentication security vulnerabilities; improves NetBox enrichment configuration; and addresses PostgreSQL major version upgrade, custom CA certificate for KeyCloak, container health check, privilege-drop signal chaining, and configuration script issues. Arkime, Zeek, Fluent Bit, Filebeat, Logstash, Supercronic, and Alpine-based images have been updated as well.

    If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

    github.com/idaholab/Malcolm/co

    • Features and enhancements

      • Add IEC 60870-5-104 (IEC 104) support using the CERT.LV spicy-iec104 Zeek plugin, including Zeek log ingestion, ECS field mapping, Arkime fields, and an IEC 104 dashboard #939
      • Make LOGSTASH_NETBOX_ENRICHMENT_DATASETS more flexible: it now accepts default, ics/ot, all, explicit provider.dataset values, and combinations such as default,ics #1037
      • Allow LOGSTASH_NETBOX_ENRICHMENT_DATASETS to be configured through checkboxes in the configuration TUI #1033
      • Improve ./scripts/start error messages by listing missing or invalid authentication-related files instead of reporting only a generic authentication setup failure #865
      • Have system-quickstart detect and prepopulate existing time synchronization settings when rerun #992
    • 🛡️ Security Remediation & Hardening

      • Fix an RBAC bypass caused by URI normalization differences between Nginx location matching and the Lua authorization layer CVE-2026-63177 #1042
      • Fix path traversal in archive extraction directory handling by validating resolved paths and using libarchive's secure extraction flags CVE-2026-63134 #1040
      • Limit archive entry count, nesting depth, and total expanded size to prevent inode- and resource-exhaustion denial of service during extraction CVE-2026-63133 #1041
      • Mark OpenID Connect session cookies as secure and improve handling of externally forwarded HTTPS schemes
    • 🐛 Bug fixes

      • Co-installation of opencv-python and opencv-contrib-python corrupts cv2.abi3.so, segfaulting strelka-backend at import on arm64 #1046 (fix)
      • Allow the configuration TUI to reset supported variables back to empty values after installation #1024, #1030
      • Fix the broken signal chain in docker-uid-gid-setup.sh so signals reach the final process after dropping privileges #1039 to ensure clean shutdown of containers
      • Fix PostgreSQL being reported unhealthy after a major-version upgrade, improve upgrade-state handling, and perform required post-upgrade extension and collation maintenance #1038
      • Fix the Nginx Lua/OpenID Connect helper not honoring user-provided CA certificates for KeyCloak when KEYCLOAK_SSL_VERIFY=true #1035
      • Restore curl to the the htadmin container for use by the health check script #1029
      • Reduce the size of the OpenSearch Dashboards image by copying only the permissions data needed from its upstream image layer #1031
      • Fix JSON handling of several Zeek fields whose names contain dots by normalizing them to underscore-separated field names
      • Fix additional Zeek and Suricata field normalization and ECS mapping inconsistencies found while updating dashboards and index templates
    • Component version updates

    • 🧹 Code and project maintenance

      • Broad spelling, grammar, naming consistency, and documentation cleanup across scripts, configuration, dashboards, and documentation #990
      • Expand and restructure documentation to provide better project context for developers and LLM-assisted code analysis #964
      • Improve installer validation, environment-variable mapping tests, and configuration item metadata
      • Refresh dashboards, index templates, field mappings, protocol documentation, and navigation links
      • Minor improvements to the Hedgehog Raspberry Pi image build process.
    • 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.

      • LOGSTASH_NETBOX_ENRICHMENT_DATASETS in logstash.env now defaults to default and may contain default, ics/ot, all, explicit provider.dataset values, or a comma-separated combination of these values
      • ZEEK_DISABLE_ICS_IEC104 in zeek.env controls whether the IEC 104 Zeek plugin is disabled
      • SAFE_EXTRACT_MAX_ENTRIES, SAFE_EXTRACT_MAX_DEPTH, and SAFE_EXTRACT_MAX_BYTES in upload-common.env set archive extraction resource limits for uploaded archive files (e.g., containing Zeek logs for processing); their defaults are 5,000 entries, 20 directory levels, and 4 GiB of expanded data

    Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

    Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

    Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

    As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

    #Malcolm #HedgehogLinux #Zeek #Arkime #Strelka #NetBox #OpenSearch #Elasticsearch #Suricata #PCAP #NetworkTrafficAnalysis #networksecuritymonitoring #OT #ICS #icssecurity #CyberSecurity #Cyber #Infosec #INL

  3. So, another #FioTechRant about #Elasticsearch #APM with Batch - yes, I saw it's not supporting natively on Java Spring Batch / Boot process or any batches like Quarkus - but is there any way to monitor those? Maybe using different agent? Or any settings?

    Still looking on how to set it up properly... Been using ELK extensively, team even succeeded in set up RUM agent for react-native. But we'll need to test it out.

  4. So.

    I'm doing interesting PoC here - syncing data from #PostgreSQL to #Elasticsearch cluster, so we can call API directly from elasticsearch without doing SP to call the query on postgre (data is about 3-5 million rows, and returning 6-10k data each call).

    So far I can get 1-3 seconds per 1st call and < 1 second after Elastic API cached. I am wondering if there's a way to make it near-real time... It's a good improvement though, Postgre manual call takes too long.

    #FioTechRant