Malcolm v2.0.0-pre1

This is a major release containing the following new features:

idaholab/Malcolm@v1.8.1...v2.0.0-pre1

Network map baseline comparison (issue #2)
Improving on Moloch’s connections view, you can now compare current logical network topology against a
previous time frame, using any of Malcolm’s 900+ fields as references for the graph’s source and destination nodes.
Network changes are easily visualized with icons for new ( ✨ ) and removed (🚫 ) nodes. The graph of connections can be switched on the fly between all nodes, actual nodes (i.e., nodes in the specified query time frame), baseline nodes (i.e., nodes in the specified and baseline query time frames), new nodes only and baseline nodes only.
This feature makes it easy to answer questions like:
• “Are there any hosts in my network this week that didn’t exist last month?”
• “Are hosts in my OT network making any new DNS queries compared to last quarter?”
• “Does my network contain any hardware from new vendors not accounted for the last time inventory was
taken?”
This connections report can be accessed visually in the web browser (see screenshot) or programatically via REST API.

Security overview dashboards
Two new “security overview” Kibana dashboards have been created to bring potential network security issues to
the forefront for IT and OT networks:
• Security Overview (issue #108)
◦ Zeek notices by category
◦ AV signatures triggered by files carved from network traffic
◦ Clear-text transmission of passwords
◦ Outdated/insecure application protocols (e.g., TLSv1.0, SMBv1)
◦ Inbound external traffic by country (i.e., traffic where the source is a publicly routable IP and the
destination is an internal/private IP)
◦ Outbound internal traffic by country (i.e., traffic where the source is an internal/private IP and the
destination is a publicly routable IP)
◦ Summary of file types observed in file downloads/transfers
◦ External remote access over time (i.e., use of “remote access” protocols such as SSH, RDP, VNC, etc.
where either end of the connection is a publicly routable IP address)
◦ DNS queries by randomness (for identifying domain generation algorithms (DGA) used by some
malware)
• ICS/IoT Security Overview (issue #109)
◦ Log count by ICS/IoT protocol
◦ Traffic over time by ICS/IoT protocol
◦ ICS/IoT external traffic (i.e., any use of ICS/IoT protocols where either end of the connection is a publicly
routable IP address)
◦ ICS/IoT action summary
◦ Non-ICS/IoT protocols observed (for identifying IT protocols in OT networks)
◦ Source and destination IP summaries for ICS/IoT traffic
◦ File types by transport

Character frequency/entropy analysis (issue #107)
Malcolm can now optionally employ character frequency analysis to detect domain generation algorithm (DGA) hostnames often used by malware. Currently Malcolm employs this technique on DNS queries and SSL certificate servers. This makes it easier to find suspect domains (e.g., fqoxibdvbycnsappxc.nu) vs. common ones (e.g., example.org).

User interface for defining host and subnet name assignment (see issue #118)

Track user access to Malcolm web interfaces
All access to Malcolm’s web interfaces (e.g., Moloch, Kibana, PCAP upload, etc.) requires authentication by a valid account. These accesses to Malcolm’s own interfaces can now be logged and viewed in Kibana dashboards built for that purpose.

ISO (live USB and installed) improvements
Both Malcolm and Hedgehog Linux can be installed using a standard ISO file image on systems supporting UEFI
boot. Hedgehog Linux can also be run in live USB mode, effectively turning any commodity hardware into an ad-
hoc network sensor. Improvements have been made to the base OS, including:
• improved hardening for both Malcolm and Hedgehog Linux
• installations should now detect virtual environments (VMWare and VirtualBox) and install the correct guest mode drivers for changing video resolution on the fly, shared folders, etc.
• many more minor fixes and improvements

Component version updates
Updated the following components to their latest stable released versions for security updates, bug fixes,
performance improvements and new features
• Elastic stack (Elasticsearch, Kibana, Logstash and Beats) 7.6.1
• Moloch 2.2.3
• Zeek 3.0.3

Miscellaneous fixes and improvements
• Fixed cross-platform compatibility of control scripts (#103)
• Fixed offline region maps (#112 and #84)
• Fixed intermittent failure when uploading very large PCAP files (#101)
• Fixed /upload URL incorrect redirect without trailing slash (#104)
• Fixed MANAGE_PCAP_FILES not working (#114)
• and more