Wednesday, March 31, 2021

EPM 11.2, Visual Studio, and SSIS Packages

This is a bit of a weird one, and it isn't Oracle's fault at all.  This is actually a Microsoft issue.

Suppose you are migrating from Hyperion / Oracle EPM 11.1.2.4 to EPM 11.2.x.  Further suppose you have custom data integrations in FDMEE/ODI or MaxL that rely upon SSIS (SQL Server Integration Services) automation.

Your EPM 11.2.x system would be using MS SQL Server 2016 SP2+ if you want to use SSIS.  I checked the certification matrix last week and Oracle still hasn't certified MSSQL 2019 for EPM 11.2.x.

(Disclaimer: the above paragraph may become be dated as Oracle might certified MSSQL 2019 in the future. The Internet is forever!)

Now here's the fun part.

To migrate your SSIS packages to MSSQL 2016, you will need to first import and upgrade them via MS Visual Studio.  The version available to download is Visual Studio 2019.

Once upgraded, you use MS SQL Management Console to import them into SSIS.  The version of this software as of this writing is 18.8.

This combination of software do not play well with each other, and you'll get an error message in MS SQL Console when you try to work with the upgraded packages.  This exact error and solution has been written about on many blogs.  I'll just jump straight to the solution.

You'll need to uninstall MS SQL Management Console 18.8, and then grab version 16.x from Microsoft's download page.  There's a link there that lets you download older versions, and 16.x is still available as of this writing.  Version 17.x does not solve our issue... we have to go all the way down to 16.x.

You may then complete your migration in MS SQL Console 16.x.

One final tip: if your custom integrations are using Microsoft's dtexec utility, you'll need to replace the reference in the script's path to dtexec from 110 (SQL Server 2012) to 130 (SQL Server 2016).

Saturday, March 13, 2021

Latest Windows Update Removes Abobe Flash

This is a follow-up about Adobe Flash Player EOL and On-Premises EPM 11.1.2.x.

As you may have heard, Adobe ended support for Flash Player on Jan 1, 2021.  This date has been known for some time, and as a result, Oracle released patches for EPM 11.1.2.4 in Summer 2020 that removes Flash dependency from Hyperion Planning, Hyperion Calculation Manager, and the Financial Close suite.

Some Oracle EPM / Hyperion customers may have deferred patching or upgrading.  Perhaps the thinking was "we can just leave it on end-user workstations as-is even though it is out of support."

Microsoft says "Not so fast!"


If you're on EPM 11.1.2.4 and are using any of the 3 modules I mentioned above, please apply the patches right away.  They're listed in my link above, although newer ones might be available by  now.

If you're on EPM 11.1.2.3.500+, you need to upgrade.

As mentioned in my earlier post, Adobe removed the Flash Player download from their website.  They further stated that downloading it from any 3rd party site is a license violation.

The good news is EPM 11.2.x has the fix built-in.  As does the EPM Cloud.

Thursday, March 11, 2021

FDMEE 11.2 - Java 6 reference can be buried within the database

Depending upon you migrate FDMEE from Hyperion / Oracle EPM 11.1.2.4 to EPM 11.2.x, you may have an issue when trying to run your load jobs.

In 11.2, Oracle included a handy migration package specifically to migrate the AIF* tables from 11.1.2.4 to 11.2.  You'll find it on your 11.2 FDMEE server in \Oracle\Middleware\EPMSystem11R1\products\FinancialDataQuality\database\migrate.  There's a package each for MSSQL and Oracle - check the subfolders for the packages and instructions.

As per Oracle's documentation, you want to run this in 11.2 after you've executed "Configure Database" for FDMEE.  This because the migration package only migrates the AIF* (FDMEE) tables, and does not migrate the ODI-specific tables (SMP* etc.).

Now let's say you're in 11.2's Data Management and you're trying to run an import job.  It fails.  You check the log and see a reference to JAVA_HOME=%EPM_ORACLE_HOME%\..\jdk160_35.

This simply won't do!  Don't be tempted to just copy jdk160_35 from your 11.1.2.4 server to your 11.2.  Java SE 6 is out of support and won't be patched anymore.

You can run a simple SQL statement to fix the problem.  Here's how it looks in MSSQL, and Oracle's version of this would be the same:

Bounce your 11.2 FDMEE service(s) just to be on the safe side, and now when you go back in to Data Management, you won't see that reference to jdk160_35 in your process log anymore.


EPM 11.2 - Your environment has been set

All releases of Hyperion / Oracle EPM 11.2.0.0 through 11.2.4.0 hosted on MS Windows Server do the following:

While this doesn't hurt anything, if you want to clean things do, do the following after you've finished deploying all of your WebLogic services.

Open regedit and export the Hyperion Solutions registry hive to your desktop.

Open this file in notepad/Notepad++ and do a Search->Replace.

Change Your environment has been set.; to nothing.  Save the modified .reg file.

Double-click the .reg file to import it back into your registry.

Do this on every server.


Friday, February 19, 2021

OHS 12c Startup Script for EPM 11.2

The following assumes your OHS is hosted in MS Windows.  An equivalent bash shell script may be easily created to do the same in Linux/Exalytics.  The below assumes you have already executed the OHS storeUserConfig process previously documented on this blog.


SET CWD=%~dp0

REM This script sets up our environment variables.

CALL %CWD%ScriptEnv.bat

sc start "Oracle Weblogic ohs NodeManager (F_Oracle_Middleware_ohs_wlserver)"

start "StartOHS" /wait /i cmd /c %EPM_INSTANCE_HOME%\httpConfig\ohs\bin\startComponent.cmd ohs_component


You're welcome.

Change the "F" in red above to be your EPM system's drive letter.  The variable EPM_INSTANCE_HOME is the path to the instance, such as F:\Oracle\Middleware\user_projects\epmsystem1 and I define it in my custom ScriptEnv.bat.

Tuesday, February 9, 2021

Oracle EPM 11.2.x and Essbase Thick Client

Here's an intellectual exercise for my followers.

Suppose you've constructed a brand new Oracle EPM / Hyperion 11.2.x system (11.2.0.0 through 11.2.4.0 as of this writing).  Now also suppose you have a need to automate Essbase MaxL scripts... let's say for example an Essbase nightly level-0 backup, or an FDMEE nightly data load automation.

You run the Essbase Client installer, and MaxL works just fine.

Except the Essbase Client is 11.1.2.4 technology and you've just corrupted one small element of your 11.2.x system.

Trying launching the configTool graphical client.  The javaw.exe process crashes immediately and you never see the Oracle EPM Config logo pop up.

What to do?

Oracle does have a fix for this documented within their Knowledge Base.  You just need to know where to look.

Or, search for files dated in the year 2014 A.D.  Open the Windows Explorer and put in a date range of 1/1/2014 through 12/31/2014 as you see in my screenshot below.

 

Don't bother with the html files.  What you care about are the EXE and JAR file extensions.  The act of running EssbaseClient.exe causes the 2020-2021 versions of these files to be blown away by the 2014 versions.

The fix: stop everything.  Replace the 2014 files from a server where you did NOT run EssbaseClient.exe to install Essbase MaxL.  Then launch configTool and confirm it works again.  Customers who have security policies mandating database & WebLogic password changes every 90 days will need a working configTool utility.

Try this out in a DEV environment.  I like Ahi Tuna sashimi with a bit of soy sauce and wasabi, and thus you can thank me with that.  ;)

Sunday, January 31, 2021

EPM 11.2.x - Why Did My Web Service Crash During Startup?

So you're working on your shiny new upgrade: Oracle EPM / Hyperion 11.2.2.0 through 11.2.4.0.  (You want to be on 11.2.2.0 at a minimum, as that's the only path to subsequently jump to higher releases as of this writing).

You attempt a first-time start up of web services, and one or more web services shuts itself down.  The problem I'm about to describe existed in 11.1.2.4 and prior, but it still rears its ugly head in 11.2.x from time to time.

Here is the symptom.  For ease of discussion, I'm talking about an MS Windows server, but the problem can happen in Linux as well.  Flip the slashes and change .cmd to .sh.

Inspect Oracle\Middleware\user_projects\EPM_ORACLE_INSTANCE\diagnostics\logs\services\*-sysout.log for the failed process in question, and discover this text:


(Note: This "services" folder does not exist in Linux and you'll need to crawl through the WebLogic domain's log folders)

What to do here?

Answer: the fix is very simple.

1. Start up your WebLogic Admin Server.  There's no service for it (yet! A future post I'll write will discuss the easy way to do it.  Don't use the Knowledge Base article to create "beasvc"!)

For now, double-click Oracle\Middleware\user_projects\domains\EPMSystem\bin\startWeblogic.cmd.  Leave the DOS command window that pops up alone, and wait until you see the text RUNNING in upper-case.  This takes 2-1/2 to 5 minutes, depending upon your server.


This process encrypts the master boot.properties file for your WebLogic Admin Server.

2. Copy the file Oracle\Middleware\user_projects\domains\EPMSystem\servers\AdminServer\security\boot.properties to the Oracle\Middleware\user_projects\domains\EPMSystem\servers\FailedServerName\security\ folder, overwriting the boot.properties file that might exist in that folder.  Do this while your AdminServer process is still running in that DOS command window.

3. Restart the failed service in question.  Now it sees the encrypted boot.properties file and is able to communicate with AdminServer.

4. Once you've been able to successfully start all web services (Planning Web, HFM Web, EAS Web, etc.) you may hit Ctrl-C in that AdminServer DOS command window to end the process, if you wish to.


Appendix:  I've stood up every 11.2.x release released thus far:  11.2.0.0, 11.2.1.0, 11.2.2.0, 11.2.3.0, and now 11.2.4.0 as of this writing.  I've hit the issue described herein multiple times, and most recently in 11.2.4.0.  If you hit this issue yourself - you did nothing wrong during your config.  It just happens sometimes.

Once you fix it, the solution tends to be permanent and I've yet to have to do it a 2nd time in the same WebLogic domain.

Friday, January 29, 2021

EPM 11.2.4 - Online and Here's Some Findings!

One day after my 11.2.4 Initial Thoughts post, I have a near-full-stack online consisting of everything except for Tax and Cost/Profitability.  (The customers I've worked with who used those modules were a small slice of my customer population, so elected to go for the "biggest bang for the buck" where my  "spare time" is concerned.

(Disclaimer - as of Jan 29, 2021 this post is a rough draft and may contain typos.  I'll fix them over the weekend if any are found)

For the most part, standing up Oracle EPM / Hyperion 11.2.4.0 is exactly the same as Releases 11.2.0.0 through 11.2.3.0.  Meaning:

  • Microsoft SQL Server 2016 SP2 only if you're going the Microsoft route, or Oracle RDBMS through 19c.
  • Oracle WebLogic 12c is still behind one "dot" release.  Oracle Fusion Middleware ("FMW") 11.2.1.3 versus 11.2.1.4.  The Certification Matrix suggests EPM 11.2.4 could work on FMW 11.2.1.4, as it says FMW "11.2.1.3+".  This may be a permutation I attempt later after all Unit Testing is complete.
  • Windows Server 2016 and 2019 and Linux 7 (RHEL and Oracle's only).  Essbase 21c is still limited to Linux 7-8 as of this writing.  
  • While Essbase 21c is certified for Linux 7 and 8, Essbase 11.2.4 (really 11.1.2.4 behind the scenes) is certified for Windows 2016-2019 and Linux 7 only.  Pro tip: if you're going to upgrade, don't use Windows 2012.... Essbase 11.2.4 (11.1.2.4) works on it, but 2012 is coming out of support before you will see a budget approved for another upgrade.
  • DRM and HFM still restricted to MS Windows servers only.
  • RCU hasn't gone away in 11.2.4 and we still need to deal with it.  Click the link to see my posts which mention RCU.
  • RCU does not support MS SQL Server when MSSQL is configured for SSL.
  • Certain Financial Reporting and EAS Server configurations require manual edits of WebLogic and/or Oracle HTTP Server ("OHS") files.  Instructions are contained within the 11.2.4 Release Notes.
  • OHS does not get setup as a Windows service.  The corresponding Linux pre-delivered startup scripts don't start OHS, either.  See my OHS StoreUserConfig post for a reminder on how OHS in 11.2.x (it is OHS 12c in this release) is different from all prior EPM releases going back to over a decade ago.
  • Installing OHS under the Foundation node in installTool is deselected by default.  You need to manually select it and then ignore the warning message in the bottom pane about a missing "null" folder.  It installs just fine.
  • Planning RMI still does not bind to its port and you have to apply my RMI Java solution.
  • The Java JDK that is bundled with EPM 11.2.4.0 is still over one year old: Java 8 update 181.  You need to grab a newer Java 8 from the latest quarterly Oracle Critical Patch Update before your IT department runs a scanner and starts hectoring you about running vulnerable software.
Here are a few ways EPM 11.2.4 is different from 11.2.0.0 through 11.2.1.3:
  • "Configure Database" step in configTool moved the checkbox that makes all DB entries the same.  Instead of being at the top left corner, it is now above the DB Name column.  In my testing, clicking that checkbox in an attempt to deselect all rows en-masse did not work.  I had to click each row by hand, so that I could make the database/schema names different.
  • "Deploy to Single Managed Server" is still the default.  But now if you deselect it, deploy Foundation only, and then go back to deploy more things, the option is no longer grayed out.  I have not explored this further.  I avoid the "Single Managed Server" architecture and someday I'll write a lengthy blog post explaining my rationale.  I understand why some folks like it.  I don't.  To each their own!
  • The pre-delivered validate batch file in EPM_INSTANCE/bin produces an HTML report that begins with "EPM Deployment Topology Report (Release 11.2.3.0.0)".  This is a typo.
  • The EPM Workspace home page is back!!!!


A final note before I dive back in and resume my Unit Test.

I said above that the JDK is at least 1 year behind.  A lot of other things are behind as well.  Oracle Middleware (WebLogic, ADF and OHS) and the Essbase Suite.  Here's a quick reference of the patches you get "out of the box" after you finish running installTool.

Oracle/Middleware/EPMSystem11R1 OPatch #s

29260139
30717472
31206874
31206851
31206864
(Look them up and you'll see they're mostly the Essbase Suite 11.1.2.4 and 6 months to a year behind - EAS, Essbase RunTime Client, Essbase Server, etc).  Grab the Essbase Suite patches 11.1.2.4.041 or a higher version you trust, and you may apply them to your 11.2.4.0 system.  Try it and you will see OPatch lets you do it.

Oracle/Middleware/ohs OPatch #s

30687404 
26550599 
26355633
26287183  
26261906 
26051289 
25549931
25360603
25115951

Oracle/Middleware OPatch #s
26626168
30447461
29338031 
27747932
27430060
26355633
26287183
26261906
26051289

"Now wait a minute, Dave?  No oracle_common patches?  No util\bsu WebLogic patches?  You slacker?!?"  Welcome to Oracle Middleware 12c, my friend!  Those things are gone and what I listed in bold above are the -oh parameters you pass to the Opatch utility (add your drive letter of course).

/Oracle/Middleware is the combination of WebLogic and ADF/JDeveloper for Oracle 12c.  OHS and EPM still get their own OPatch folders as before.

I'm not looking up the English descriptions of those patch #s for you, as I've already given away too much for free.  That lookup is very easy to do with an Oracle Support subscription and access to https://support.oracle.com.

Snowed in this week, so I figured I'd make myself useful to the community somehow!

Be well, upgrade to 11.2, and patch up!

January 31, 2021 Update
My earlier remark about being able to change the "Deploy to Single Managed Server" option may have been premature.  I went back into configTool tonight, and the option is now grayed out.  The earlier behavior I experienced, where it was not grayed out, may have been a one-time fluke?

OHS 12c for EPM 11.2.x - StoreUserConfig

Because I know I'll be asked this again many times, here's a reminder on how to avoid being prompted for the WebLogic password when starting Oracle HTTP Server ("OHS") for Oracle EPM / Hyperion 11.2.x.  This applies in EPM 11.2.0.0 through 11.2.4.0 and likely beyond.

One time only, open a command prompt, CD to your EPM Oracle Instance ("epmsystem1" by default), and then navigate to the folder where startcomponent.cmd / startcomponent.sh resides.  This is pictured below for my testing sandbox.

Type:

startcomponent.[cmd|sh] ohs_component StoreUserConfig

This will ask for the WebLogic password (the same password you setup for the WebLogic admin user "epm_admin"), but now it will be saved behind the scenes so you'll never be asked again under the user account you're logged in as.  The "StoreUserConfig" option at the end of the command is what does this for you.


Notice the command tells you the filename where the password was stored.  If you have multiple people managing the system, copy that \.wlst\nm-key-ohs.props file from your home directory structure to theirs.

Having accomplished this, you may now add the startcomponent ohs_component command to your custom start-up script (you do have one, right?) to include OHS in your EPM start-up sequence.

Thursday, January 28, 2021

EPM 11.2.4 - Initial Thoughts

It is time yet again for another "Initial Thoughts" post about on-premises Oracle EPM / Hyperion.  This time, the topic is EPM 11.2.4.0, which just came out.

I've completed the download and am about to fire off a clean install & config, but while I'm waiting for my VM to finish patching, I thought I'd share some thoughts from reading through the EPM 11.2.4 Release Notes.

My comments are by no means exhaustive.  I'm just mentioning the things that jump out at me as an EPM Infrastructure guy.  I encourage anyone interested in 11.2.4 to download the Installation document zip, which includes the Release Notes and Install+Config Guide.  The eDelivery product number for this is V1006299-01.

At a high-level view, this release fixes many bugs relating to HFM and FDMEE from 11.2.0.0 through 11.2.3.0.  Aside from bugfixes, I spotted these important features, improvements, and gotchas:

  • LCM for Public Sector Planning is now supported to migrate from 11.1.2.4 to 11.2.4.  The Release Notes state this caveat: "Note: You can’t upgrade Public Sector Planning and Budgeting applications with Decision Packages or Budget Requests enabled."
  • The Certification Matrix has been updated for 11.2.4.  It explicitly states that the "Core Server" option for MS Windows Server 2019 is not certified, and this makes complete sense to me.  The "Core Server" option means there's no graphical desktop for the server, which would make getting through installTool and configTool problematic.  I used the console version for UNIX back in the day, but the console version is painful to use.
  • FDMEE install issues for Linux observed in 11.2.2 and 11.2.3 have been fixed in 11.2.4, provided all required products are installed at the same time on the Linux server.
  • Numerous SSL fixes.  If you are using SSL within EPM itself, rather than doing it through an SSL-offloader such as F5, read the Release Notes carefully.
  • Numerous FDMEE and HFM bug fixes.  28 issues fixed for FDMEE, and 14 for HFM.  I suggest reading carefully through the Release Notes where these products are concerned.
  • SSL for MSSQL is still not supported where RCU is concerned.  No workaround has been published to my knowledge.
  • EPM's validate.bat does not recognize all modules in an Apply Update situation. No workaround listed in the Release Notes.
  • Tax Governance no longer depends upon Adobe Flash Player, which came out of Adobe support on Dec 31, 2020.  This is a very important update if you use Tax Governance.
  • A few fixes for Shared Services and Financial Reporting / Workspace were included.
  • HFML preview for Financial Reporting has been fixed, and "Related Content" links now work in Chrome & Edge.
  • Linux.  Still limited to RHEL 7 and Oracle Linux 7.  Essbase 21c, on the other hand, is certified for both Releases 7 and 8 for Linux.
  • There's a new command-line utility for Hyperion Planning: SetDBNEssbasePassword.  This utility allows you to update the Essbase and relational database passwords for multiple applications all in one shot.  
    • I think the database side will work well for MSSQL shops, but most Oracle RDMBS shops I've worked with frequently have DBAs who prefer a unique password for each schema.  So your mileage will vary where this utility is concerned.  You're prompted to enter 1 password for Essbase and 1 password for relational.
    • You can run the utility multiple times, specifying the application(s) for each run.
End-users will appreciate that the EPM Workspace home page has returned.  Favorites, recently opened documents, etc. should be visible again, just as we had in 11.1.2.4 and prior.

OK, and with all of the above, my VM's server patching is done and it is time to get to work on 11.2.4!

I intend to start with a pure Windows implementation first.  After this is working, I want to see if I can tack on a Linux instance running Essbase 21c and integrate it with the 11.2.4 system.