Thursday, March 24, 2022

March 2022 Update

March 2022 is shaping up to be an interesting month, and the show isn't over yet!

One customer went live a few weeks ago on Hyperion / Oracle EPM 11.2.7.  It was a complex upgrade from EPM but we got it done on time.

This week I upgraded another customer's EPM from 11.2.3 to 11.2.8.  The in-place upgrade wasn't easy and posed challenges... mostly due to many references on the back-end to jdk1.8_181, which no longer exists as a folder starting in 11.2.5.  Now it is simply Oracle\Middleware\jdk\.  Some of my NTT colleagues have hit similar issues, so I'm helping them troubleshoot.

A very fun little project was helping one of the co-founders of EPMware by installing Essbase 21.3 on one of his AWS images.  He wants to test if his software works against the Essbase 21c Java APIs.  He gave me full admin access and I turned it around in 1 day because he didn't need to integrate it with EPM, so it was a very simple install.  I'm anxious to hear the results of his testing!

I've been working on my own sandbox and am attempting to integrate Essbase 21.3 with EPM 11.2.8.  The 11.2.8 README says this type of configuration is certified now.  So far I've managed to get Planning to talk to Essbase 21c and successfully deploy the sample "Vision" application.

When integrated with EPM's 11.2.8 Shared Services, Essbase 21.3 registers itself smoothly so we have a familiar interface to provision users and groups to it.  The problem is you can't use LCM in EPM to work with Essbase 21.3 applications... you have to use the Essbase 21c migration tools instead.  One caveat is if you want to migrate a pre-existing Essbase / 11.2.x app into 21c, it must come over as Unicode.  So some folks who want to attempt this may have some conversions to do.

Another weird thing I'm seeing in 21c is I can create Essbase apps just fine in the 21c EAS Console (it is the thin web client), but you cannot see the Planning apps in it!  I'm rebuilding my sandbox a slightly different way to see if I can figure it out.... the 21c online guides on Oracle's documentation site don't really address this topic.

Finally, I've got an upcoming project on my calendar to move a customer from OBIEE to OBIEE  This version of OBIEE isn't available on eDelivery when I last checked (Oracle says they're working on tightening up a few things), and I believe know why.  It is possible to get it, though.  Just do a web search for Oracle Business Intelligence Enterprise Edition.  With a bit of clicking, you can make your way to the OBIEE download page.

One thing I'm unhappy about is OBIEE requires an out-of-date Fusion Middleware Infrastructure: FMW instead of  I hope this gets fixed soon.

My head is spinning!  Be well,


Wednesday, March 2, 2022

Catch-Up Post for February 2022

Have I been slacking on updating this blog?

Yes.  Yes I have.  👎

Here's a quick few updates about what I've been working on.

1. Went live with Hyperion / Oracle EPM 11.2.7 for a customer.  Hypercare ends in a few days.  Things are really looking great!  We've got end-to-end SSL, Okta SSO, standalone ODI, load balancing... the works!

2. Installed EPM 11.2.8 and integrated it with Essbase 21.3.  Crazy stuff.  This experiment is worthy of its own blog post.

3. Oracle Business Intelligence  Not yet on Oracle eDelivery and I believe I know why this is so.  It just isn't ready for prime-time.  The thing wants us to install Oracle Fusion Middleware  Why????  I started pounding my face to my desk.
(Disclaimer:  this blog post was originally written on March 2, 2022.  The above paragraph will eventually be invalidated once Oracle corrects the issue.)

4. Log4J.  The Oracle KB article concerning the Log4J vulnerability was updated just this week.  Here's how to find it.  You need an active Oracle Support subscription to access this:

Apache Log4j Security Alert CVE-2021-44228 also referencing CVE-2021-45046 Mitigation on Oracle Enterprise Performance Management (Doc ID 2828262.1)

I think this is just about enough.  And today's just Wednesday.....

Sunday, February 20, 2022

EPM 11.2.x and Quarterly Java Update - Be Careful!

So far I've discovered 2 issues with respect to the Quarterly Oracle "Critical Patch Update" (abbreviated "CPU" henceforth) and Hyperion / Oracle EPM 11.2.x.

The problem seems to have been introduced with the October 2021 CPU and continues to be present in the January 2022 CPU.

So far, I've observed 2 symptoms:

  1. The legacy frconfig utility fails to launch.  No error message is provided.  The javaw process appears briefly in your process list and then closes.
  2. The EAS Console screen that is supposed to display a graphical chart when viewing ASO Essbase aggregate information is blank.
Here's how you can definitively know if you have the issue, and what you can do about it.

With respect to item #1 above, Oracle does have a Knowledge Base article about it.  It takes a bit of digging to find.  Essentially, Oracle acknowledges that the Quarterly CPU delivers an incomplete Java 8.

So let's say you blow away \Oracle\Middleware\jdk's content and replace it with the latest Java CPU.  After all, that's what we've been doing for years, so why change our techniques?

Don't do that anymore!

Whether you are in Linux or Windows, you need to count the number of files in the jdk folder before and after you apply the CPU.  In Windows you can right-click the folder and hit Properties.  Look at the file count.  If the number of files is below 900 after applying the CPU (typically I've seen it is in the mid-300's), you got hit with the problem.

The better way forward is to follow this procedure:
  1. Back up your Middleware\jdk folder.
  2. Download the Java 8 quarterly CPU.  Don't run the installer.  In the case of Windows, use 7-zip to unzip the included .gz file and then again use 7-zip to unpack the .tar file included therein.  Linux nerds like me know what to do on the Linux side of the fence.
  3. MERGE the files extracted from the tar into your Middleware\jdk.  This will replace your JDK binaries that WebLogic cares about, while also preserving the JRE files that include JavaFX (EAS Console wants these) and the stuff frconfig needs.
I am hoping this condition is temporary.  Oracle knows about it because they published the KB article about it, although the workaround they suggest is something I don't agree with.

The last KB article I read said to uninstall Java 8 (how? Java 8 is bundled with EPM and there's no uninstaller in the Windows Control Panel for it - I'm speaking Windows here as that is what the bulk of my customers use nowadays), reinstall the base Java 8 release, and then install the CPU on top of it as an "in-place upgrade".  How?  Old EPM Infrastructure dinosaurs like me can figure it out, but there are many folks new to EPM 11.2.x and trying to unpack all of this knowledge is a daunting task.

The above is my own opinion, so your mileage may vary.

Saturday, February 12, 2022

EPM 11.2.8 - Online!

My loyal readers just KNEW this blog post was coming.  😁

Hyperion / Oracle EPM was released on Friday, February 11, 2022.  I have it online now and here are initial observations.

(Disclaimer: the above screenshot is from the sample "Vision" application & associated financial data that ships with Hyperion Planning.)

Get Ready To Patch!

<Feb 12, 2022 4:52:31 PM CST> <Info> <Management> <BEA-141107> <Version: WebLogic Server Thu Sep 12 04:04:29 GMT 2019 1974621>

F:\Oracle\Middleware\jdk\bin>.\java -version
java version "1.8.0_301"
Java(TM) SE Runtime Environment (build 1.8.0_301-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.301-b09, mixed mode)

Tip:  If you want to use the old "FRConfig" utility to configure the default POV prompt on/off setting or HFR batch bursting external folders, you will want to back up the pre-delivered \Oracle\Middleware\jdk\ folder structure before applying the latest Quarterly Critical Patch Update ("CPU") for Java.  

Depending upon the method you use to patch/replace JDK 8, FRconfig won't launch and you won't see an error message explaining why.  The root cause is the Java quarterly CPU may be missing hundreds of files.  This issue may have started with the October 2021 CPU.  I noticed it also applies in the January 2022 CPU.  Oracle has a Knowledge Base article about this problem, but you have to dig a little bit to find it.  I'll probably need to write a separate blog on this specific topic alone.

Essbase 64-bit  - Release 11.1.2 (ESB11.

F:\Oracle\Middleware\OPatch>opatch version
OPatch Version:

f:\oracle\middleware\jdk\bin\java -jar f:\oracle\middleware\coherence\lib\coherence.jar
Oracle Coherence Version Build 74888
(File is dated Sept 12, 2019)

Planning "Vision" Sample Application is Still Old

Vision still has a start year of FY09.  The good news is it extends out to FY27, so you have 5 more years to play with before you need to add more.

Essbase Stills Logs in TRACE Mode

Edit your $ARBORPATH\bin\logging.xml to fix this.  While you're at it, edit your $EPM_ORACLE_INSTANCE\config\OPMN\opmn\opmn.xml if you, like me, tire of seeing the opmn ping messages in Essbase.log every 20 seconds... 24x7.

You have to insert the line highlighted in yellow exactly where shown.  OPMN can be picky about the order of statements in opmn.xml apparently:

    <restart retry="2" timeout="600"/>
    <ping timeout="20" interval="0"/>
    <port id="essbase-port-range" range="32768-33768"/>
    <process-set id="AGENT" numprocs="1" restart-on-death="false">
Those of you who've worked with me in the past know I prefer restart-on-death="false" rather than the default "true" setting.  This permits a graceful shutdown of Essbase via MaxL, with error notification (if your shutdown script is clever) if an application is loading data, calculating, or restructuring.

Premier Support Extended

The 11.2.8 README says EPM 11.2 Premier Support now runs through "at least 2032".  So we have an additional 2 years than we originally expected when 11.2.0 came out.  Oracle Safe Harbor applies here of course.

Log4J Updated

Log4J is now version 2.17.1.  Consult your IT Security folks to verify this satisfies their vulnerability concerns.

MSAD 2019 Certified

EPM 11.2.8 now speaks with MSAD 2019.  This should make IT a bit happier.

Upgrade Leapfrog - GONE!!!!

Quite simply: this one is huge.  Prior to, we could only upgrade in-place from an older 11.2.x release if we were, at most, 1-2 dot releases behind.  Now we may jump directly to 11.2.8 from any 11.2.x release, provided we are coming from at least 11.2.2.  If you're still on, you will be standing up new server infrastructure and you'll jump directly to 11.2.8 and do a bunch of LCM, HFMCopyApplication, and EAS Console migrations.

This screenshot from the 11.2.8 README says it clearly:

Essbase 21c Now Certified for Planning & Profitability

This is another big one.

Many of my customers have been holding back on kicking the Essbase 21c tires because they use Hyperion Planning.  Now is the time to take a serious look at Essbase 21c.  The last I looked, Essbase (which is the default Essbase version in EPM 11.2.x) support runs through the end of 2022.

My knowledge on this Essbase support expiration topic may be a bit out of date, so I'll need to prowl around on Oracle's website to see if they're changed their position on this specific point or not.

Happy Upgrading!

2021 was very busy for me doing non-stop upgrades to 11.2.x for many customers.  2022 is shaping up to be the same situation, with my first 11.2.7 going live in Production one week from now.

If you've never touched EPM 11.2.x before, be sure to get some professional consulting help from someone who has successfully done it and gone live in Production.  There are important differences between standing up EPM 11.2.x versus EPM 11.1.2.x.