EPM 11.2, RCU and You!

----------------
Feb 8, 2020 Update:  There's a follow up blog entry concerning EPM 11.2 Distributed Environments but please first read through the blog entry below, as it contains per-requisite knowledge.  Thank you!

----------------

There's a venerable utility for Oracle Fusion Middleware that has been available for quite some time.  Those of us who configured Oracle Data Integrator in the pre-FDMEE days, Hyperion Financial Close Management Suite 11.1.2.1 through 11.1.2.4.2xx, or Oracle Business Intelligence know what I'm about to discuss.

Oracle Repository Creation Utility, or "RCU" for short.

In EPM 11.1.2.4 and prior, you didn't need to worry about running RCU if you weren't using the modules I mentioned within my first paragraph.

Those days are behind us now.  EPM 11.2 requires that you run through the RCU process before you attempt to create your Oracle WebLogic domain and deploy your EPM Foundation web service.

R2D2 says: "Technically, EPM Foundation is an Oracle WebLogic Managed Server."
C3P0 responds: "Don't get technical with me [further insults added here]."

So let's talk about what has changed in EPM 11.2.  Your actions will depend upon if you're using Microsoft SQL Server or an Oracle database.

If you're using Microsoft SQL Server, the steps are a little more complex because your DBA needs to create an empty database for you.  Furthermore, the DBA needs to run these queries against that database before you may continue and run RCU.  Let's assume the DBA has named the RCU database "epmrcu":

ALTER DATABASE epmrcu Set READ_COMMITTED_SNAPSHOT ON
GO
ALTER DATABASE epmrcu Set ALLOW_SNAPSHOT_ISOLATION ON
GO
ALTER DATABASE epmrcu COLLATE LATIN1_GENERAL_CS_AS
GO

Note the COLLATE option is slightly different for RCU's database vs. everything else EPM wants to use.

Source:  SQL Server Database Creation Requirements

The above steps aren't needed if you're using an Oracle Database; RCU creates the schemas and associated users for you.

Now for both MS SQL Server or Oracle Database, you need to fire up Oracle/Middleware/oracle_common/bin/rcu.bat and follow the instructions here:
Using RCU 

Unless you're a DBA, you will need a DBA's assistance.  The RCU utility requires someone with elevated rights to type in their credentials for a DBA superuser account ("sa" for MS SQL or "sys as sysdba" for Oracle).  There's the option to save the commands to a SQL query so your DBA can run it on their own, but I've experienced mixed results with this in the past.

So you now have your RCU database initialized.  There's just one more manual step to do.

On each server where any Oracle WebLogic components run, you need to manually edit this file:

Oracle\Middleware\EPMSystem11R1\common\config\11.1.2.0\RCUSchema.properties

Here's the fun part:  passwords are exposed in plain text until you run through configTool on the server, at which time the passwords become encrypted.
Source: Updating RCU.properties File

(Edit: I blew away my MS Server 2016 VM and spun up MS Server 2019, after being advised by Oracle that Server 2016 hasn't been certified yet for EPM 11.2.  After running through RCU, I inspected the RCU.properties file and noticed encrypted passwords.  I still had to update the JDBC URL and some other information)

Once you complete all of these tasks, you may finally launch configTool and deploy your WebLogic Admin Server and your various WebLogic Managed Services.  If you overlooked these instructions from the config guide, unfortunately you will need to blow away the entire domains/EPMSystem folder structure and start over.

Eying the Christmas Dinner leftovers in my fridge.... I may be a little delayed on writing my next post!  👀

Installing EPM 11.2: Things Waiting to Trip You Up

Christmas 2019 came early this year, and just about every EPM gear-head I know is trying to install EPM 11.2.0.0 so it may be taken for a spin.

Not so fast!  There's a few things to take care of first!

Christmas Eve 2019 update: Stay tuned!  New information has come in from Oracle and I will update this post after I finish a few plates of Christmas Day ham.  First and foremost, Windows Server 2016  is NOT certified by Oracle for EPM 11.2 at this time.  Please upgrade your sandboxes to Windows Server 2019.  Use Oracle 12 or SQL Server 2016 as your RDBMS until further guidance.

January 2020 update: Oracle's feedback is posted here: EPM 11.2 Certification Matrix post
Look for the text in red font.

EPM 11.1.2.4 only worked up through MS Windows Server 2012 R2.  EPM 11.2 works on 2016 and 2019.  (My previous post about the 11.2 Certification Matrix mentions 2016 as a potential omission)

MS Windows Server 2012 R2 imposed some new changes that weren't a problem for us in 2008 R2.  Server 2016 and 2019 present the same challenges, plus newer ones.

Before doing anything, change this Windows Registry key from 1 to 0 and reboot.

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\policies\system]
"EnableLUA"=dword:00000000

You will thank me later.  If you find a few days later that the key's value reverted from 0 back to 1, contact your IT department and get an exception to Group Policy Update so your EPM servers are excluded from the change to this key.

Here's the classic symptoms this key is wrong:

• Pop-up message asking for permission to change something within a directory.
• You try to pick a folder for the Oracle eDelivery download helper, and it says your access is denied.
• You manage to try a deployment and notice a folder named "Oracle EPM System" in the root of your Hyperion/EPM drive.

 Now for the fun ones:  Windows Defender and Windows Firewall

Windows Firewall.  Don't stop the service.  Your Remote Desktop session will immediately be terminated and you'll need someone with console access to turn the service back on.  Instead, go into the Windows Firewall settings and turn off notifications and blocking for Domain and Local network.  Leave it turned on for Public network.  Otherwise you will have to deal with a mess of individual port exceptions.

Windows Defender.  Go into the Windows Defender panel and add on-access scanning exceptions for where you're downloading the software to, and where you're installing it to.  Otherwise, look forward to a 20-30% performance degradation.  Unlike 2012 and prior, you can't disable or stop this Windows service.  There are also firewall settings in this service and you'll need to inspect and modify them.

Finally, was the install/config so bad that you need to blow away either user_projects or EPMSystem11R1?  You may hit the "path too long" error Microsoft helpfully introduced.  The poor man's solution is to install freely available 7-zip.  Launch 7-zip, do File->Open and pick the parent folder.  Then left-click the child folder to be deleted, and click Shift-Delete.  This will blow it away without encountering the Microsoft error.

Thoughts On EPM 11.2 Certification Matrix

I updated my earlier post Important Oracle EPM 11.2 Links with a link to the EPM 11.2 Certification Matrix

I'd like to share a few thoughts about what's currently in this matrix.


Christmas Eve 2019 update: Stay tuned!  New information has come in from Oracle and I will update this post after I finish a few plates of Christmas Day ham.  First and foremost, Windows Server 2016  is NOT certified by Oracle for EPM 11.2 at this time.  Please upgrade your sandboxes to Windows Server 2019.  Use Oracle 12 or SQL Server 2016 as your RDBMS until further guidance.

New Years Day Update January 1 2020: Answers to my questions below have been addressed by a senior person in Oracle.  I posted the answers in red text immediately after the relevant question.  If you are color blind, fear not as I prefixed the text with "ORACLE'S ANSWER".  I offered to provide accreditation to the Oracle person who was kind enough to read through this post thoroughly and provided a thoughtful response, but the person asked not to be named.  Again, a big THANK YOU to the person at Oracle who provided these answers!  We now have more clarity, as you will see in the red text below!  As the Oracle person asked not to be identified, I paraphrased the answers in my own words.

Questions posed:

• Why isn't Windows Server 2016 listed?  ORACLE'S ANSWER: Windows Server 2019 was the focus of Q&A testing because it has the "longest runway" from a Microsoft Extended Support perspective.  It might work on 2016 and 2012, but your mileage will vary and is not guaranteed.
• Yes, Windows Server 2012 R2 is coming out of Microsoft Extended Support in a few years, but will it work there? ORACLE'S ANSWER: As above, not worth the R&D / Quality Assurance time as Server 2012 has a limited shelf-life from a Microsoft perspective.  DATAVAIL ANSWER:  Ping me in LinkedIn if you couldn't attend my ODTUG webinar on this topic and I can quickly debrief you.
• The only version of MS SQL Server listed is 2016.  (One person told me they tried it on 2019 and it didn't work, so he's downgrading to 2016).  ORACLE'S ANSWER: From a Microsoft perspective, SQL Server 2016 is the version certified for Oracle Fusion Middleware 12c.
• The Firefox ESR version numbers are not listed.  In EPM 11.1.2.4, certain EPM screens stopped working once you surpassed a certain Firefox version.  Has this been fixed in 11.2?  PERSONAL ANSWER: Once the 2020 New Year champagne wears off I need to go back and check if this was answered.
• Windows 10 is the only supported Windows OS listed.  What about 8.1?  Not everyone has finished their migration to 10 yet.  ORACLE'S ANSWER: 8.1 is coming out of Microsoft Extended Support soon, so testing/QA emphasis was spent on Windows 10 from a desktop perspective.  DATAVAIL'S ANSWER: Windows 7 is close to end-of-life and a desktop upgrade to Windows 10 is recommended.
• Why is Windows Server 2012 listed as a certified client OS, but not as a certified server OS?  ORACLE'S ANSWER: As above.  Server testing was focused on MS Server 2019 due to the predicted shelf life.  PERSONAL ANSWER: Server 2012 is essentially the same technology as Desktop 10, so this kind of makes sense.
• JRE 8 is the only client Java plug-in supported?  Is the EPM system not backward-compatible for organizations who haven't completed their client desktop updates yet?  ORACLE AND PERSONAL JOINT ANSWER: Java 6 came out of Oracle Extended Support on Jan 1 2019.  Java 7 has a limited shelf life and the security updates for it are only available to paying customers; the general public doesn't have access to those updates.  Java 8 has the longest shelf life until a subsequent patch is issued for EPM 11.2 that certifies newer versions.
• Certification Matrices for 11.1.2.4 and prior listed Microsoft Office versions.  Microsoft Office is conspicuously missing here. We know 365 Cloud won't be certified until a newer version of SmartView is released, but could we get some details in the meantime.  ORACLE'S ANSWER: Oracle SmartView for Office is the only desktop client remaining that cares about the version of MS Office.  A conscious decision was made to remove Office from the matrix so as to reduce confusion.

No mention of Microsoft Edge.  We know this is on the roadmap, but it would be nice to see a statement in the matrix to the effect of "not supported yet; to be certified in a subsequent patch" or "it is certified now".  ORACLE'S ANSWER: IE11 for now. Stay tuned.

Speaking for myself, I've got a secret underground bunker / mad scientist lab with a 6-month Trial Edition of MS Windows Server 2016 and MS SQL Server 2016 Developer Edition.  I've gotten through the EPM 11.2.0.0 install and config, and everything works fine except for Oracle HTTP Server 12.  I'll write a separate post on that once I complete the troubleshooting process and can actually login.

The certification matrix is widely considered to be a "one stop shop"on these types of questions, rather than having to wade through a variety of readmes, installation and admin guides.  I'd love to see the matrix tightened up and improved.

Did I miss any of your questions?  Post them in the comments below!

EPM 11.2: No More Financial Reporting Essbase Connections?

I'm starting to crawl through the various Oracle Hyperion / EPM 11.2 readmes, and spotted this nugget:

Hyperion BI Plus. You can’t migrate Release 11.1.2.4 reports that used Essbase as a data source to Release 11.2. In Release 11.2, Oracle Hyperion Financial Reporting, which is now a component of Hyperion Financial Applications, continues to support connections to a Planning data source using an Essbase connection, as well as Profitability and Cost Management.

Across my customer portfolio, there are a sizable number of reports that use the Essbase data source.  This will be a migration impact.

I'm worried how this will impact customers who DON'T have Hyperion Planning, and just have Essbase.  There's quite a few of these environments out there.  If a customer doesn't have Planning, what happens when they have HFR reports that hit Essbase?

In all prior releases, when you setup data connections for HFR, you picked the data source type.  If you picked Planning, it would automatically "know" about the Planning apps because that information was picked up from Shared Services.  In this brave new world of EPM 11.2, what happens if you pick a Planning data source type for a pure Essbase app?  Would you even see it?

I hope to know the answer later tonight.  If I learn anything material, I will update this post.

EPM 11.2 Important Oracle Links

I will update this post after I can find the updated Certification Matrix for EPM 11.2.

Here's Oracle's EPM 11.2 Release Acccouncement

EPM 11.2 Documentation Landing Page.  This is your one-stop-shop for all of the administration and installation guides.

You can download the software by signing in with your Oracle Support credentials to Oracle eDelivery
Once you login, type "EPM" into the search box and then click the Search button.
You will see multiple "11.2.0.0" download packages.  You don't need to grab them all, as there is overlap across many of them.

Bookmark this page and come back to it soon.  As of this writing, it doesn't have the Supported Platform / Certification Matrix for EPM 11.2, but I suspect it will be updated very soon:
Hyperion Supported Platforms

Dec 20, 2019 Update: The EPM 11.2 Certification Matrix is now available.


Generally speaking, you will want to download:

• Oracle HTTP Server 12
• ODI Studio 12 if you are doing anything with FDMEE
• EPM zips  Grab the full set and then you can pick and choose which modules to install when you run through the installation tool.
• Windows Client Installers
• Installation Readmes

Some of the packages include optional packages you might not need, such as Oracle SOA or Oracle Identity Manager.

Analysis to follow after I've had a chance to finish downloading and can spin it up in a sandbox.

EPM 11.2 Is Here

My shortest blog post ever.

Analysis to follow separately.

ODI Tables Missing in FDMEE 11.1.2.4.2xx? Here's why!

So you did a fresh install of FDMEE, applied the latest patch, and then ran the config tool to "Create Database" and "Deploy To Application Server".  This is how I do it, as I hate repeating the same activity twice (before and after patching).

But lo' and behold, the ODI tables (SNP* etc) are missing within the FDMEE repository.  What's going on here?

Answer:

On your FDMEE server, inspect this file.  If FDMEE is clustered across multiple servers, you need to do this on just one server only.  Let's assume your 1st FDMEE server is hosted on Windows and resides on the D drive:

D:\Oracle\Middleware\EPMSystem11R1\products\FinancialDataQuality\bin\createODIRepos.bat

Click the image if it doesn't render completely.  I'm an EPM Infrastructure guy and am thus graphically-challenged.  But the key nugget of information is this:  Do you see "11.1.2.3.00" or "11.1.2.4.00"?  Change that 3 to a 4!

The "FinancialDataQuality\odi\11.1.2.3.00" folder doesn't exist in this patch, so your ODI Master and Work repository tables never get created!  The config wizard doesn't bark at you that you hit this problem.

Oh, there's one other thing.  In the later FDMEE 11.1.2.4.2xx patches, you get only one shot at creating the ODI tables within FDMEE.  Look within the same createODIRepos.bat file mentioned above. You'll see a line remarked out that was NOT remarked out in the unpatched 11.1.2.4.0 version of the software.  Do you see it?  Post in the comments below and I'll give you a hint.  This remarked line is what makes the configurator wizard create the ODI Master and Work tables!

Fix the .bat file, re-run FDMEE's "Configure Database" task from the EPM Configurator on just one server.  No need to redeploy to WebLogic.  Restart FDMEE and you're good.

I can understand why Oracle decided to remark out that line; if you need to re-run "Configure Database" for FDMEE for any reason, maybe you don't want to lose your ODI customizations.  But what if the ODI tables never got created to start with?  Best to backup the original file, make your edits, run the configurator, and then when things are working OK you can restore your backup copy of the file to its original name.

Will this remain a bug in 11.2, or will it be fixed?  Time will tell!

Hot Off the Press: EPM 11.2 is Actually Coming in Dec 2019?

Oracle updated their EPM 11.2 Release Date blog post today, Dec 6, 2019, indicating Oracle is still projecting a release of EPM 11.2 sometime in December 2019 (*).  The precise day of the month isn't known yet.

(*) - Oracle Safe Harbor applies.  Date can be subject to change.

The blog post I've linked above contains a wealth of information, so I suggest you read it if you haven't checked it out lately.