Update Part Two: This information in this blog post is now obsolete. I have kept the original content as-is for historical reference purposes. As of April 2018, there is a newer WebLogic Server 10.3.6 patch, designated patch # "GFWX" 27395085. To fully remediate the security issues address by this patch, you need (as per Oracle):
"Supported versions of Java SE are: JDK6u191, JDK7u181 and JDK8u172. Your version must be one of these versions or higher."
There was a bit of buzz on the Internet a few weeks ago concerning a zero-day exploit discovered within Apache Struts 2. I did some sleuthing around in my EPM 11.1.2.4 on-premises lab and found evidence of Struts within several WebLogic processes (Calculation Manager, EPM Foundation, and more).
This past Friday, September 22, Oracle issued a Security Alert they've named CVE-2017-9805. This includes a fix for Apache Struts 2 within WebLogic 10.3.6, which every EPM 11.1.2.3 and 11.1.2.4 system on the face of the planet uses behind the scenes.
The patch number to download for WebLogic 10.3.6 is 26835212.
The procedure to apply this patch is different from how we normally apply EPM patches. On each server in the environment where you have a folder named \Oracle\Middleware\wlserver_10.3, you will want to edit this file:
Windows:
\Oracle\Middleware\utils\bsu\bsu.cmd
UNIX:
/Oracle/Middleware/utils/bsu/bsu.sh
Modify the MEM_ARGS line to be as so:
set MEM_ARGS=-Xms2048m -Xmx2048m
If you don't do this, the patch utility runs for a long time and then fails with an OutOfMemory exception error.
Next, copy the unzipped contents of the patch into this folder:
\Oracle\Middleware\utils\bsu\cache_dir
You would then shutdown EPM web services and execute the bsu script you edited earlier, and examine the output.
I would take things 1 step further and blow away the /cache and /tmp folders for each WebLogic Managed Server underneath \Oracle\Middleware\user_projects\domains\EPMSystem\servers
Stay safe out there!
Update: I recommend using the command-line patch interface rather than the graphical interface. On a 4.0GHz machine with solid state disk, it takes 12 minutes just for the GUI to render. Another 12 minutes to process the patch after you are given the opportunity to click. With the command-line interface, you can eliminate the first 12 minutes.
Also, this patch has a conflict with the April 2017 critical patch "RVBS". RVBS needs to be rolled back first. This adds another 12 minutes.
Hi Dave,
ReplyDeleteHow long does the Patch take to apply?
Also I do not have the cache_dir directory, am assuming I just create it?
Thanks,
Tej.
Hi Tej. I'm about to take an outage in my lab environment to find out! The last WebLogic patch only took 10-15 minutes to apply. It has to be done on each server while EPM services are offline.
DeleteYes, you create the cache_dir directory if it is missing. Because that directory is missing, this tells me the environment is missing patch 25388747, which was the April 2017 critical weblogic patch for a similar issue.
Correction! I'm applying the patch now and it says there is a conflict with the older patch 25388747 / "RVBS". It says we must uninstall the older patch first.
ReplyDeleteUpdate: I recommend using the command-line patch interface rather than the graphical interface. On a 4.0GHz machine with solid state disk, it takes 12 minutes just for the GUI to render. Another 12 minutes to process the patch after you are given the opportunity to click. With the command-line interface, you can eliminate the first 12 minutes.
ReplyDeleteAlso, this patch has a conflict with the April 2017 critical patch "RVBS". RVBS needs to be rolled back first. This adds another 12 minutes.
what is command line to apply patch. Is following line correct?
ReplyDeletebsu -prod_dir=/opt/essbase/Oracle/Middleware/wlserver_10.3 -patchlist=UZCY.jar -verbose -install
Have you tried runing the patch in verbose mode? The only files changed are related to Weblogic sample apps, which is what was stated in the CVE files too. As long as sample apps are not deployed, the typical EPM environment is not vulnerable. I think you might be confusing Struts and Struts2...
ReplyDeleteI would strongly recommend not to rollback any other Weblogic patch to apply this superfluous one.
I ran command in verbose mode and noticed that only files are changing is under samples folder. When I run find command on my EPM Shared Services folder I get following struts* files.
Delete./Oracle/Middleware/wlserver_10.3/server/lib/consoleapp/APP-INF/lib/struts.jar
./Oracle/Middleware/wlserver_10.3/server/lib/consoleapp/consolehelp/WEB-INF/lib/struts.jar
./Oracle/Middleware/wlserver_10.3/server/lib/consoleapp/consolehelp/WEB-INF/lib/struts-adapter.jar
./Oracle/Middleware/wlserver_10.3/server/lib/consoleapp/webapp/WEB-INF/lib/struts.jar
./Oracle/Middleware/wlserver_10.3/server/lib/consoleapp/webapp/WEB-INF/lib/struts-adapter.jar
./Oracle/Middleware/wlserver_10.3/samples/server/examples/src/examples/spring/sconfig/lib/struts2-spring-plugin-2.3.34.jar
./Oracle/Middleware/wlserver_10.3/samples/server/examples/src/examples/spring/sconfig/lib/struts-extras-1.3.9.jar
./Oracle/Middleware/wlserver_10.3/samples/server/examples/src/examples/spring/sconfig/lib/struts2-core-2.3.34.jar
./Oracle/Middleware/wlserver_10.3/samples/server/examples/src/examples/spring/sconfig/lib/struts-taglib-1.3.9.jar
./Oracle/Middleware/wlserver_10.3/samples/server/examples/src/examples/spring/sconfig/lib/struts-core-1.3.9.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/FoundationServices0/tmp/_WL_user/struts/nbe464/WEB-INF/lib/struts.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/FoundationServices0/tmp/_WL_user/struts/nbe464/.tlds/.tld_cache/struts.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/AdminServer/tmp/_WL_internal/consoleapp/z97wi8/.tld_cache/struts.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/AdminServer/tmp/_WL_internal/consoleapp/z97wi8/.tld_cache/struts-adapter.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/AdminServer/tmp/_WL_internal/consoleapp/jxhze9/.tld_cache/struts.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/AdminServer/tmp/_WL_internal/consoleapp/jxhze9/.tld_cache/struts-adapter.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/EpmaWebReports0/tmp/_WL_user/struts/3o0lfn/WEB-INF/lib/struts.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/EpmaWebReports0/tmp/_WL_user/struts/3o0lfn/.tlds/.tld_cache/struts.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/CalcMgr0/tmp/_WL_user/struts/cfgzhy/WEB-INF/lib/struts.jar
./Oracle/Middleware/user_projects/domains/EPMSystem/servers/CalcMgr0/tmp/_WL_user/struts/cfgzhy/.tlds/.tld_cache/struts.jar
./Oracle/Middleware/EPMSystem11R1/common/templates/was/epm_struts_11.1.2.4.jar
./Oracle/Middleware/EPMSystem11R1/common/templates/applications/epm_struts_11.1.2.4.jar
./Oracle/Middleware/EPMSystem11R1/common/misc/11.1.2.0/struts.jar
are any of those files are vulnerable to attack?
No, the EPM ones are Struts 1.x, which is not affected by this CVE. Struts 1 is unmaintained since 2013 and might well have its own issues, but not this particular one. Similarly for Struts 1 used by the Weblogic Admin Console, as stated in KM 2255054.1.
DeleteAfter researching, you must apply the July 2017 patch first (cumulative). Also, from what I can ascertain, this patch and the July patch for WL (25869650) as it relates to EPM may not be correct (agreeing with Giacomo) - see Doc ID 2261562.1. The patches state they fix JDK 1.7 SSL issues (which EPM cannot use JDK 1.7) and WL Samples. If you chase down the Samples path, yes there are samples installed (under templates\applications), but these do not match the samples at risk (Doc ID 2255054.1) which you can just remove them anyway I believe. This is clear as mud...I feel like I am figuring out the hereditary of the British crown. Please correct me if I have made I mistake.
ReplyDeleteI opened an SR with Oracle about this patch to ask if our EPM Cloud instances are impacted, since they're built on top of the same technology, and so far it has been stuck in "internal review" for over a week. I'll update this post with more information if I manage to receive any detailed information.
ReplyDeleteThis is what they said to me -
ReplyDeleteLet's take a step back and start over in regards to CVE-2017-9805:
1. Ignore the 3rd party blog. While the information may be helpful, it is not the official answer from Oracle
2. In a nutshell, for Hyperion customers, there is nothing that needs to be done . They can check to see if the samples exists per Note 2255054.1 and delete them.
3. Customers should apply the latest WebLogic patch, WebLogic Server 10.3.6.0 home - WLS PSU 10.3.6.0.170718 Patch 25869650
If customers are concerned about Java vulnerabilities, the latest patches available are:
Note 2271677.1 (Critical Patch Update July 2017 Patch Availability Document for Oracle Java SE)
Java SE Bundled With Oracle Products - JDK/JRE 6 Update 161: Patch 9553040
JRockit Bundled With Oracle Products - JRockit R28.3.15 (JDK6) Patch 25951569
Follow steps 1 - 5 in Note 1538740.1 (How to replace the JDK used by Oracle EPM System with a JDK of a higher patch level?)
After downloading the Java patches, extract to Oracle_Home\Middleware
Rename the folders below where xx equals the current version of jdk / jrockit installed
jdk160_xx to jdk160_xx_old
jrockit_160_xx to jrockit_160_xx_old
jdk160_151 to jdk160_xx
jrockit_160_151 to jrockit_160_xx
Interesting. Thanks for posting this.
DeleteI have updated the top of the original post to point to the information contained within this useful comment. Thanks again.
DeleteCJRatliff: Sir, I owe you a beer. Feel free to collect it if you are ever in the Detroit area. Many thanks!!
ReplyDeleteGlad I could help!
ReplyDeleteAnd thank you Dave for posting this thread... You are owed two beers!
ReplyDelete