Thank you Jeremy for putting all this information together. Some important comments inline. On Thu, 13 Jan 2022 at 19:49, Jeremy Stanley <fungi@yuggoth.org> wrote:
Thanks to the excellent feedback from Radosław Piliszek and Pierre Riteau, the list of things for operators to look out for has grown a bit. Is anyone else aware of other, similar situations where OpenStack is commonly installed alongside Java software using Log4j in vulnerable ways? I think this list is becoming extensive enough we could consider publishing it in a security note (OSSN)...
Kolla-Ansible Central Logging -----------------------------
If you're deploying with Kolla-Ansible and have enabled central logging, then it's installing a copy of Elasticsearch (7.13.4 currently, which includes Log4j 2.11.1). According to a statement from Elastic's developers, the relevant risks can be mitigated by passing "-Dlog4j2.formatMsgNoLookups=true" on the JVM's command line. All images built after December 21, 2021 have this workaround applied, with the exception of images for Train which did not get that patch merged until January 7, 2021. The statement from Elastic about the workaround can be found here: https://xeraa.net/blog/2021_mitigate-log4j2-log4shell-elasticsearch/
This part has several issues: - Xena and Wallaby use Elasticsearch OSS 7.x, the latest available package being version 7.10.2 from January 2021, which includes Log4j 2.11.1. Unfortunately it doesn't look like it will ever be updated, so we only have the formatMsgNoLookups to help. The Kolla community is planning to move to OpenSearch as a replacement for Elasticsearch. - Victoria and earlier releases use Elasticsearch OSS 6.x which still gets updated packages. Latest Kolla images for these releases use version 6.8.22, which includes Log4j 2.17.0. I see that Elasticsearch 6.8.23 was released today, which includes Log4j 2.17.1. This should get picked up by Kolla image builds in the coming days. - The formatMsgNoLookups mitigation is applied through Kolla Ansible and doesn't depend on when Kolla images were built. The dates you mentioned above are when the commits merged in Kolla Ansible.
CloudKitty, Monasca, and OSProfiler -----------------------------------
If you're deploying CloudKitty, Monasca, or OSProfiler, you may be using Elasticsearch as a storage back-end for these services. Make sure you update it or put a suitable mitigation in place. Anyone deploying one or more of these services with Kolla-Ansible is running Elasticsearch, but should be covered so long as they update to the latest available images for their release series, as noted above.
As noted in the previous comment, for Xena and Wallaby we don't have a fix using new images, only the mitigation applied through Kolla Ansible is available. Additional Monasca components that include Log4j: - Logstash: similar vulnerability pattern to Elasticsearch, i.e. no RCE out of the box - Kafka: uses an older Log4j 1.x which is only vulnerable to CVE-2021-4104 if configured to use JMSAppender (not the default), according to https://kafka.apache.org/cve-list - Zookeeper: uses Log4j 1.x, not affected by CVE-2021-44228 according to https://blogs.apache.org/security/entry/cve-2021-44228, not sure about the others - Storm: possibly vulnerable? Pull requests in github.com/apache/storm have bumped Log4j versions, but no new release has been issued yet. Kolla uses version 1.2.2. I am looking at adding a mitigation for CVE-2021-45046 based on removing the JndiLookup class from the classpath.
Networking-ODL --------------
Neutron's Networking-ODL driver relies on the Java-based OpenDaylight service, which should be updated if used: https://access.redhat.com/solutions/6586821
SUSE OpenStack --------------
The "storm" component of SUSE OpenStack seems to be impacted: https://www.suse.com/c/suse-statement-on-log4j-log4shell-cve-2021-44228-vuln...
Sovereign Cloud Stack ---------------------
An Elasticsearch component in Sovereign Cloud Stack is affected: https://scs.community/security/2021/12/13/advisory-log4j/
-- Jeremy Stanley