2021: EPM 11.2 Upgrade AND Data Center Change? Things to Consider

Many of my readers are looking at a situation where something needs to be done to their EPM system in Fiscal 2021:  Either upgrade, move to Oracle EPM Cloud, or move to a 3rd party solution such as OneStream.

The deciding factors are End Of Life for Hyperion / Oracle EPM 11.1.2.4 in DEC 2021, Microsoft IE11, Microsoft Server 2012 and prior, Microsoft SQL Server 2012 and prior, Linux 6, and so on.  I've written about these topics before and you can find details if you search through this blog's history.

One thing I would like to caution my readers about is conflating multiple significant changes in a single shot.  Such as:  Upgrading from EPM 11.1.2.4 or prior to EPM 11.2.x, AND doing a shift at the same time from an on-premises data center to a hosted cloud such as OCI, Azure, Amazon AWS, CenturyLink, etc.

Yes, many organizations combine these activities into a single project, often with success.  But please do consider one factor that goes beyond technology: your friends in Finance.

Finance MUST be involved in a significant project such as this, as they will need to validate the new system has apples-to-apples data results and performance (or better).

I mention this for the case where things don't quite match up right: data is wrong, or performance worsened.  Combining a significant version upgrade AND a shift in data center hosting leaves doubt as to where the problem stems from.

This being said, I'd like to offer general answers to questions I've received over the past few years along these lines.  Bear in mind these are general answers and every organization is different.  As I pose general questions & answers, if anything I say below triggers a further question, I highly suggest finding a Partner and locking them in now.  Don't wait until Spring/Summer 2021 as by then many Partners will already be booked for the year for their 2021 EPM 11.2 upgrade projects.

My favorite analogy along these lines relates back to my experience living in the US MidWest: try finding a competent roofer after a major storm passes through town.  The local experts are all booked, and you could be stuck with somebody from out of town - dubious credentials - who won't be around when you find problems later.

So without further ado, here we go....

Q: Any certification issues between Amazon AWS, Microsoft Azure, and Oracle Cloud Infrastructure (OCI) that I need to know about?

A:  No.  Certification revolves around the guest virtual operating system and relational database, not the physical host.  If you speak with Oracle Sales and if you are using Oracle RDBMS for Hyperion, you will be "guided" toward OCI because of database licensing.  This topic can get pretty detailed and is beyond the scope of this blog article.  Suffice to say, if you're using Oracle RDMBS and are looking at shifting from on-premises to hosted, make sure BOTH an IT Director AND your IT Procurement folks are involved.

Q: May I use my existing hardware to host EPM 11.2?

A: You can... but consider this upgrade project as an opportunity to leverage a hardware refresh for faster performance.  If your EPM platform is virtualized, the virtual guests can be moved to faster hardware as IT finds capacity.  If your EPM platform is physical and not virtual, you should either consider virtualizing (a topic for another blog post I could spend some time writing) or a hardware refresh.  Your physical EPM 11.1.2.4 servers are at least 5 years old and it is time for a speed upgrade.

Q: I want to change my database platform from X to Y.  Any issues?

A: If you're just running Essbase, no issues and go for it!  If you're running HFM or Financial Close, be careful.  By "change my database platform" I'm not talking about a version change -- say from Oracle 10 to 19, or MSSQL 2012 to 2016.  I'm talking about switching between Oracle & MSSQL in either direction.

The Hyperion Financial Close / Account Reconciliation suite for on-premises is a special animal.  You really should look at involving a Partner if you're on 11.1.2.3 or older.

HFM is very touchy when it comes to its database artifacts.  If you are switching between Oracle & MSSQ, just expect your project will see a 15-20% overhead due to working Service Requests (SRs) with Oracle.  Switching your database type for HFM isn't supported out-of-the-box and you will hit issues.

Q: How is EPM 11.2 different in terms of requirements for CPU cores, memory and disk?

A: The EPM Foundation 11.2 service takes a minute or two longer to start up, unless you have the WebLogic Admin Server (WLS) process running.  WLS is a bit hungerier for memory in this release.

Generally speaking, the various web processes (CalcMgr, EAS, Foundation, APS, etc) have about the same requirements in EPM 11.2 as in EPM 11.1.2.3.500 through EPM 11.1.2.4.x.  That is, you want to reserve 1 CPU core and 1.5GB-2GB of RAM for each web process.  Hyperion Planning and Hyperion Financial Reporting are exceptions due to a Java memory leak: set aside 8GB for each.

The above is just general advice.  Think of it as a rough rule of thumb.  When in a formal upgrade project, I'd want about an hour to speak with the IT Director/architect and DBA to dive into the details and develop an environment diagram appropriate to the situation at hand.

Q: Which is better?  Amazon AWS, Microsoft Azure, Oracle Cloud Infrastructure (OCI), or 3rd party host XYZ?

A: How much money do you have?  All kidding aside....

Spinning up EPM 11.2 in a hosted environment looks and feels exactly the same as on-premises.  Your users' experience ultimate depends upon several things:

1. Network connectivity between the users' location(s) and the hosted data center.
2. In the specific case of Essbase, is the Essbase server able to use disk equivalent to solid state or not?
3. In the specific cases of HFM, FCM and/or ARM, how fast is the database?
4. Backups and disaster recovery.  Can you control the backup retention, and is the backup / DR plan reliable?
5. Bad guys.  Assume 2021 will be worse than 2020 in terms of malware, data leaks, extortion, etc.  Couple this thought with what I just mentioned about backups and DR.  If you stay on-premises, you have your IT Security folks to yell at.  If you shift to a 3rd party host, now it is their problem to deal with, there must also be trust between you and them.
   

Q: IT wants to enact Secure Socket Layer (SSL) for everything.  What do I need to know about EPM 11.2?

A: This is a growing trend I've observed.  Be careful if you're using MSSQL Server and your DBA wants to turn on SSL.  EPM 11.2 requires Oracle's Repository Creation Utility (RCU), and it doesn't like MSSQL with SSL enabled.  Last I checked, no workaround has been published by Oracle for this specific configuration.  Be warned and be careful.

The good news is on the Oracle WebLogic and Oracle HTTP Server (OHS) side of the fence, EPM 11.2 allows more secure SSL/TLS protocols than EPM 11.1.2.4 and prior.  It is still messy as you have to deal with SSL certificates that eventually expire, but EPM 11.1.2.4 and prior use insecure SSL protocols that are essentially worthless.  EPM 11.2 leaps forward multiple generations on the back-end, and as a result TLS in particular is much better.

------

I've just scratched the surface here, but I hope you have a few things to consider now.  Again, as I said previously this blog article is general in nature, and may not be suited for your specific circumstance.  Safe Harbor, yada yada yada.

Hey Why Does My New SmartView Keep Timing Out?

Oracle SmartView for Office still needs the Windows Registry keys it has been wanting since... the very beginning really.  It does not matter if we're talking Hyperion / Oracle EPM on-premises or Oracle EPM Cloud.

If your user has upgraded to a new Windows 10 machine and is complaining about SmartView timeout issues, the culprit is almost always the Windows Registry.  This is what a non-tuned user's workstation looks like, from my point of view:

The registry key names you see in this example screenshot are shorthand for the full registry key name.

These registry keys and suggested values were taken directly out of a series of Oracle Knowledge Base articles that have been available for a very long time.

Here we're looking at two areas of focus: network and MS Office animations.

Network of course is the true silver bullet here.  Get the keys applied, reboot the user's workstation, and the problem is nearly always solved.

The animation keys help with weird flickering issues occasionally reported by SmartView users.  I haven't heard any of these complaints recently, as the flickering problem is very Office version-specific.  I've written about this issue here before.  If your user has the on-premises version of Office 365, they are likely OK and wouldn't have seen the issue.  It still wouldn't hurt to have the keys, though.

So here's the script that generates the above screenshot.  Within this very simple script -- nothing proprietary here! -- you can see the full key names and values expected.

 

echo ------------------------
echo KeepAliveTimeout should be set to dword:2bf20
echo Here is the current value:
reg query "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings" /v KeepAliveTimeout

echo ------------------------
echo ServerInfoTimeout should be set to dword:2bf20
echo Here is the current value:
reg query "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings" /v ServerInfoTimeout

echo ------------------------
echo ReceiveTimeout should be set to dword:75300
echo Here is the current value:
reg query "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Internet Settings" /v ReceiveTimeout

echo ------------------------
echo DisableWindowTransitionsOnAddinTaskPanes should be set to dword:1
echo Here is the current value:
reg query HKEY_CURRENT_USER\Software\Microsoft\Office\15.0\Common\Toolbars /v DisableWindowTransitionsOnAddinTaskPanes

echo ------------------------
echo DisableAnimations should be set to dword:1
echo Here is the current value:
reg query HKEY_CURRENT_USER\Software\Microsoft\Office\15.0\Common\Graphics /v DisableAnimations

echo All done checking!
pause


One final point...  #MISSING suppression is handled on the end-user's workstation, not on the Essbase server (on-premises or Cloud - it does not matter).  If user requests a massive grid in either ad-hoc mode or by opening a Planning form in SmartView, the Essbase server is going to process it.  Essbase ships the complete grid back to SmartView and lets the chips fall where they may.  (The communication flow is more complex than this - I'm just trying to keep this post simple)

I mention this because I've seen some users attempt to "data mine" Essbase.  The mindset here is "give me everything and I'll sift through it in a linked sheet or in MS Access".  In my view, Essbase & Planning were never intended to be used this way.

When a user requests a grid containing a half million or more possible cells, it does not matter what you do to either the user's Windows Registry, the essbase.cfg file (only if you're on-premises), or tweaks to the cube's dense/sparse settings.  SmartView will remain locked up until the entire grid is fetched and processed.  I've noticed grid retrieval performance starts to get ugly when we're looking at 300,000 possible cells and higher.

Where on-premises is concerned, massive grid retrievals can lead to "OutOfMemoryException" errors in either Planning or Analytic Provider Services, depending upon the type of SmartView connection used.   You can bet these errors also happen in Oracle EPM Cloud, since the underlying technology is the same, but the Oracle WebLogic logs aren't exposed to us in the Cloud.  Bumping up the on-premises "-Xmx" registry keys for Planning & APS will provide some relief (e.g. changing from the 4GB default to 8GB), but all we're really doing here is throwing more RAM at the problem; it won't speed anything up.

The underlying issue to address, unfortunately, is end-user behavior.  Usually it is only 1-3 people within the organization who are trying to datamine the entire cube, and you'll typically find them in FP&A.  If you're in Oracle EPM Cloud, use EPMAutomate to grab the daily audit logs and that should help you identify who is pulling down large grids.  On-premises has different ways EPM Detectives such as yourself can probe and find who is bringing the system to its knees. 

So apply the keys recommended above, reboot the workstation, and keep retrieval slice sizes within reason.