Friday, October 4, 2019
Sunday, September 29, 2019
Let's talk about one of our favorite subjects: Sarbanes-Oxley compliance combined with on-premises Oracle EPM / Hyperion.
Auditors and IT Risk Management departments tend to frown on running SOX-relevant financial applications on systems where a vendor's Extended Support has expired. Plain English: no ongoing defect remediation via patches, and no new security vulnerability patches.
As I wrote in a prior post, this ship has already sailed for:
- Oracle EPM 188.8.131.52 and prior versions
- Microsoft Windows Server 2008 R2
- Microsoft SQL Server 2008 (all Service Packs)
- Java 6 and prior versions
- JRockit 6
- Oracle EPM 184.108.40.206
- Microsoft Windows Server 2012 R2
- Microsoft SQL Server 2012 SP3
- Java 7
What readers need to consider is their timeline to either upgrade to EPM 11.2 (once released), or migrate to the Oracle EPM Cloud.
December 2021 seems like a long time away, but let's again re-visit SOX.
Let's say your fiscal year aligns with the calendar year: Jan:Dec. In this scenario, SOX-relevant applications only get 2 windows per year to complete upgrades and do a go-live cutover to a new system: May and September. Shoot for May, and use September as your fall-back position. Going live during either your fiscal 1st Quarter or 4th Quarter will trigger a red flag in your SOX audit.
So keep these dates in mind and then start counting backward. Don't wait until late in 2021 to either upgrade or move to the cloud. By then most EPM consulting partners, such as the firm I work for, will likely be slammed trying to hit that Sept 2021 SOX deadline. I'm reminded of when Microsoft revoked support for browsers older than IE11... we were insanely busy because many customers were still on EPM 220.127.116.11 or older, and IT Risk Management departments forced Finance to upgrade to remain compliant.
One final thought: I've recently been contacted by a competitor promising cheaper support rates than Oracle's. I want to discourage people from considering this, unless you intend to completely retire Hyperion and switch to a different platform on or before Q3 2021. A 3rd party partner/consultant will face legal problems if they are discovered installing patches or upgrades a former Oracle customer is no longer entitled to receive.
Friday, September 6, 2019
"What will it cost me?"
"How long will it take?"
My answer tends to be:
"I won't know for sure until I see the Release Notes and install it in a lab."
But, I do have some directional answers.
Before I begin, however, there is a very important point: You must remain current on your Oracle Support maintenance agreement. This is the only way you can legally obtain the EPM 11.2 installation media. A partner who uploads their own copy of the media will lose their partner status with Oracle and face legal action.
Cost and Time are related
Whether you use a partner or perform the upgrade in-house, cost is unavoidable.
If you do it in-house, you are essentially paying opportunity cost; the people working on the upgrade are not working on other things to support the pre-existing system, do development work, etc. If you use a partner, of course you are either paying Time & Materials or are paying Fixed Price.
Either way, there is cost involved from a labor perspective.
You will want to do what Oracle calls an Out of Place upgrade (or what I call Lift & Shift). See my post Why Upgrade to EPM 11.2 or Move to the Cloud on why you don't want to do an in-place upgrade. (Speaking from a personal note, I hate in-place upgrades. They're almost always messy)
Translated to English:
- Stand up new servers using a newer operating system, faster CPUs, etc. Your existing servers are likely 4+ years old. Size the CPU core count, RAM, and disk to be no less than what you're utilizing in current Production. Remember that on an Essbase server, the disk needs to be double in size so you don't run out of space during an Essbase dense restructure.
- Stand up a new database server running newer database software, such as SQL Server 2016 instead of SQL Server 2012.
- Size the database schemas (Oracle 12.2), or databases (MS SQL Server 2016) to be no smaller than what you're using in current Production. Think forward and increase the size to accommodate future growth for the next few years. This is especially the case for folks who use HFM and/or FDMEE.
- Plan ahead and ask IT to add an Anti-Virus "On Access" scanning exception for D:\Oracle
Somewhere between 40-50% of the EPM customer population I work with use EPMA. EPMA is gone in 11.2, and replaced by DRM.
Oracle has stated a utility will be made available that converts EPMA content into DRM. That being said, account for some extra testing time by your Subject Matter Experts (and your partner if you will use one).
The architecture has changed and "Reporting & Analysis Framework" has been deprecated. I'm a big fan of this; RA Framework is always messy to troubleshoot when it won't start back up. The way reports are stored within Hyperion LCM has changed as a result. Assume there will be additional time spent on:
- Migrating reports from the old system into the new (hopefully the migration utility handles this OK).
- Migrating objects that aren't reports, but are stored within Workspace. Such as PDFs, videos, MS Word documents, MS Excel documents, etc.
- Updating LCM backup scripts, if you have nightly LCM backups enabled. (You should!!!)
From prior experience, I can say the first 5-6 months of a new EPM release can be.... interesting.
Especially where HFM and Essbase are concerned.
Have you counted the number of patches released for Essbase alone in 18.104.22.168?? I've been using Essbase since the 3.2 days in the mid-90s, and have never seen so many patches for it since 22.214.171.124. Expect perhaps some bumps in the road, especially if you use ASO Essbase.
Critical Security Patches During Your Project
Congratulations, you are now back on fully-supported software, from an Operating System, Database, and Oracle EPM standpoint. (EPM 126.96.36.199 has about 1-2 years to go, depending upon your support contract).
Most EPM shops haven't moved their servers from Java 6 to Java 7. Now you will be on Java 8. Oracle issues security patches (actually full installs) for both Java 7 and Java 8 every three months. You will also be on a newer version of Oracle WebLogic: You're going to move from WebLogic 10 to 12. This also gets critical security patches issued every Quarter.
This means, you can anticipate at least 1 major outage during your upgrade project, multiplied by the number of EPM 11.2 environments you expect to have (e.g. DEV, UAT & PROD). You will want to account for this in your Project Plan.
OK, But How Long Does an Upgrade Normally Take?
My rule of thumb is 5-7 business days per environment to:
- Migrate apps
Add more time if you use SSL and/or SSO. Where SSL is concerned, you may need new SSL certificates. The new system is expected to comply with TLS 1.2, whereas EPM 188.8.131.52 and prior could only use TLS 1.0. For SSO, I expect no changes to Shared Services / EPM Foundation, but we'll have to wait and see!
Workforce, CapEx, Project Modules for Planning
OK, here's the fun one. These modules don't exist in Planning 11.2. If you currently use them, put some extra hours+cost in your Project Plan for them. At best, we could LCM them into 11.2 and they will work OK. At worst, they will have to be re-implemented from scratch. Yuck!
Hyperion Tax - Gone!
If you're one of the few who use the Tax module, add some hours+cost to consider how this case would be handled.
If memory serves, the same will be true for Profitability, Essbase Studio and Strategic Finance. Oracle has the final say and we won't know for sure until the Release Notes are issued.
Essbase Studio - If it is Gone and You Use Drill-Through, What Next?
Not everyone uses Essbase Studio for drill-through. But if you do, plan on spending time in Design, Build and Test for whatever alternate solution you choose to use.
Who doesn't love them?
If your system is SOX-relevant, you might face extra cost if you utilize external auditors. The auditors, and your own people who interact with them, may need to perform their activities twice during the fiscal year when your EPM 11.2 Production cut-over goes live; one audit for the old system, and one audit for the new system.
This extra cost can be avoided if you cut over so the first month of the new fiscal year is operated within the new system. Many customers I've interacted with have difficulty achieving this, due to a variety of factors.
Don't plan to go "all in" once the software is released. Check out my post Action Plan for Early EPM 11.2 Adopters and plan on running a test lab / sandbox to "kick the tires" before you spin up the servers for the "real" EPM 11.2 environment that you plan on using.
Now it is time for me to go back and keep refreshing the Oracle eDelivery page....
Thursday, September 5, 2019
As of this writing, both the blog post linked above and the Oracle Knowledge Base article which links to the above (last dated Aug 1, 2019) list an "Oracle Safe Harbor" projected release date of September 2019.
Late September 2019 Update: "Safe Harbor" has been bumped from Sept 2019 to Oct 2019. Internally, I'm hearing this might be bumped again to Nov 2019. As always, "wait and see"!
Be sure to bookmark the above and check back.
The primary Oracle Knowledge Base article to check would be "When will EPM 11.2 become Available? (Doc ID 2553915.1)"
October 2, 2019 Update: the article 2553915.1 mentioned above isn't available anymore!
For those of you unfamiliar with term "Oracle Safe Harbor", it essentially means the posted information is directional in nature, and could change. The full text of Oracle Safe Harbor can be read within the Knowledge Base. Just put "EPM 11.2" into the KB's search box and several articles on this topic will appear.
Friday, August 23, 2019
By now, most on-premises customers have moved to Hyperion / Oracle EPM 184.108.40.206, which is the latest release currently available today
Upgrading on-premises can be a bit of a "project", depending upon how complex the environment is, and if any re-work is needed. (For example: if you're on 220.127.116.11 or prior and using FDM Classic Edition, the FDM content has to be re-implemented into FDM Enterprise Edition as part of the upgrade to 18.104.22.168).
For this reason, many on-premises EPM shops tend to delay upgrades as long as possible. Upgrades are usually disruptive to key end-users, and costs are involved from IT. Consulting costs are also involved if you don't have deep EPM Infrastructure expertise in-house. The typical upgrade cycle for many EPM customers, therefore, tends to be no more frequent than every 5 years.
There was a mad rush to upgrade to 22.214.171.124 a few years ago when Microsoft ceased support for all browsers older than IE 11. I think we will again see a mad rush to either EPM 11.2 or the Cloud very soon. Read on to find out why!
The majority of on-premises EPM systems run on Microsoft platforms. EPM 126.96.36.199 and older runs on MS Windows Server 2008 R2, and EPM 188.8.131.52 runs on either 2008 or 2012.
2008 is already out of Microsoft Extended Support. 2012 will come out of Extended Support on October 10, 2023.
In the case of UNIX variants, there many "flavors" of UNIX. The most popular one where EPM is concerned is Red Hat Enterprise Linux (RHEL). Both EPM 184.108.40.206 and 220.127.116.11 are certified to run on RHEL 6. Where RHEL 6 is concerned, we refer to the chart below.
(Content cropped from https://access.redhat.com/support/policy/updates/errata)
So RHEL6 was released in 2010, and depending upon which support package your IT department purchased, it is either out of support now, or is coming out of support soon.
The two most popular databases for EPM 18.104.22.168 I've seen "out in the wild" are MS SQL Server 2012 R2 and Oracle Enterprise Database 12.x. EPM 22.214.171.124 runs on MS SQL Server 2008 (SP3 should have been applied by now) and an older version of Oracle 12 or lower.
Let's talk about MS SQL Server first, as that one is slightly more popular than Oracle, as far as I've observed. Again, let's refer to the chart.
(Source: Google search "MS SQL Server End of Life")
Both from an Operating System and Database perspective, Finance will be pressured by IT to upgrade, due to operating system support. IT highly frowns on running systems that are out of vendor support! (As a former IT guy myself, I know all too well how IT shops run internally, and how Compliance and/or Risk Management departments can tend to be a bit inflexible in this regard).
Hyperion / Oracle EPM
If you are running 126.96.36.199 or older, the clock has already run out. You are on what is called Lifetime Support. In Plain English(TM), this means:
- No new defect fixes
- No new security patches
- Access to the Knowledge Base
- Access to previously issued patches
"... the end of Premier Support moved from December 2018 to December 2020. The end of Extended Support remains at December 2021."
(Source: Oracle Corporation)
This means according to the Oracle Support package your company purchased, you either have until Dec 2020 or Dec 2021 to either upgrade or move to the Cloud.
EPM 188.8.131.52 through 184.108.40.206 ship with Java SE 6 and JRockit 6. Only EPM 220.127.116.11 is certified to replace Java 6 and JRockit 6 with Java SE 7 (see: http://www.epmonprem.pro/2018/07/upgrading-to-java-7-for-epm-11124-it.html)
EPM Customers who are still paying their Oracle maintenance are on the "Extended Support" column as shown above.
Java 6 and JRockit 6 came out of support at the end of 2018. As of this writing, Oracle is still issuing Quarterly security patches for Java 7.
EPM 11.2 is expected to ship with Java 8. This would extend Java's support lifetime through March 2025. It is my hope that Oracle issues a certification and upgrade instructions to let us upgrade the servers to Java 9 sometime before March 2025. Otherwise, we'd be looking at a potential security problem in the future.
Look at all of the support expiration dates above, and we see the shortest "lifetime" is the one for EPM 18.104.22.168.
If you don't think EPM 11.2 will be ready in time for a 2020 upgrade, or if your 2020 budget has already been established, then either switch over to Oracle's Extended Support plan or start looking at the Cloud. (Note: the Cloud isn't for everyone. There are some cases where it is a perfect fit, and other cases where the on-premises applications are too mature and integrated with each other.)
A big benefit of the Cloud is that once you've migrated into the Cloud, you will never again have to worry about future upgrades required by support expiration dates. It is a double-edged sword, however: you need to regression test the monthly automatic Cloud updates.
I hope this information has been helpful. Keeping track of all of these support expiration dates can be a real pain sometimes!
Thursday, August 22, 2019
So the purpose of this post is more about what to do in advance in order to prepare a Proof of Concept (POC) lab / sandbox.
At this time ("Safe Harbor"), Oracle has disclosed the software will be introduced for Windows Server, and UNIX will come later. Oracle has also indicated Windows Server 2016 and SQL Server 2016 will be certified for EPM 11.2. What is not clear is if higher versions of Microsoft will be certified.
Unless there are further delays ("Safe Harbor"), EPM 11.2 is now just 1-2 months away. Oracle's Knowledge Base and blog indicates Sept 2019 is a possible release date, but a Q&A in an Oracle webinar earlier this week suggested Oct 2019 is now the target date. As with every new on-premises EPM/Hyperion release, it is always a "wait and see" approach.
All of the above being said, here are some things we can do to prepare our POC labs / sandboxes in anticipation of the new release.
- Procure license keys for Microsoft Windows Server 2016, and your choice of either an Oracle database or Microsoft SQL Server 2016. Time-limited trials are typically available if your POC lab will be used for throwaway testing & prototyping.
- Procure a few virtual machines. I'd suggest starting with 3: 1 for the web tier, 1 for Essbase (if you use it, otherwise put HFM & FDMEE on server #2), and 1 for the relational database. On server #1, where the web tier will run, ideally you want 1 virtual CPU core and 2-4 GB of RAM for each web service. As we are talking about a POC lab / sandbox, we can skimp a little. Future UAT and Production environments would need to be beefier. The disk footprint is unknown at this time -- when in doubt, provision a standard C: drive for the operating system, and 150GB or more for the D: drive where EPM will reside.
- Install the operating system and database.
- Run Windows Update and apply the recent database patches / updates.
- Install your favorite 3rd party utilities on server #1 and #2. I tend to use Notepad++ and 7-zip. If the relational database is to be hosted on a UNIX variant, I additionally install PuTTY and FileZilla. All of these are legally free (not shareware).
Avoid creating schemas/databases until the software and installation guide are released. I expect the schemas/databases needed will be the same as 22.214.171.124 and prior, but it would be wiser to wait.
We don't yet know if DRM will require Microsoft IIS, ASP.Net, and/or .Net Framework. I'd wait to install these until the installation guide is published.
Finally, make sure your company is current on paying the Oracle Support maintenance fee. Upgrades are only available to companies who are current on maintenance.
A reminder this blog post relates to a Proof of Concept lab / sandbox, which would be used to validate that your applications can be successfully migrated from your old system into EPM 11.2. Once you're ready to build out your "real" EPM 11.2 environments, the number of servers and their specifications may be much different.
What else would you do to prepare? Leave a comment!
Downloadable white paper for this post is now available over at Datavail.com!
Sunday, July 7, 2019
- Oracle has moved the Essbase development work out of EPM and into the Oracle Database development team. Essbase will remain an OLAP database rather than relational – Oracle just shuffled some deck chairs.
- When EPM 11.2 comes out, Essbase will initially remain 126.96.36.199 technology under the covers: we won’t be getting the new Sandbox features introduced with Essbase 12 in on-premises OBI12 / cloud OAC.
- On-premises CapEx, Workforce and Project planning modules will be deprecated and not available in EPM 11.2. (We knew for some time that Project Planning will be going away; nobody seems to use it). But losing CapEx and Workforce was a bit of a shock. This means those cubes would need to be re-implemented as custom cubes in 11.2 Planning; no more pre-built content for them.
- Disclosure Management, Interactive Reporting, WebAnalysis (deprecated when 188.8.131.52.900 came out), and SQR Reporting are also sunset and will not be available post-184.108.40.206.0/220.127.116.11.7xx.
- It is clear Oracle wants to push CapEx, Workforce and HSF customers to the cloud.
- At some point, we’re not sure when, EPMA will be gone in 11.2 and will be replaced by a limited-use version of Enterprise DRM. This DRM will be bundled with 11.2. This is a complete reversal from previous Safe Harbor statements made by Oracle
- Out-of-place / lift-and-shift migration from 18.104.22.168 to 11.2 will apparently be certified. Some kind of migration tool will be provided for this.
- OAC will no longer have Essbase bundled with it, effective immediately for all new customer-managed OAC implementations. Essbase “12c” will have to be installed separately as a standalone instance, and then Essbase cubes would need to be migrated from the old Essbase instance into a new standalone Essbase 12c instance. The jury is out on this. Unsure if Oracle will soften their position and bend to the collective will of their customers.
- A new Essbase, Essbase 19c, is under development for 11.2. It is expected to come out sometime next year. Essbase 19c will be for on-premises only. So the confusion: 22.214.171.124 Essbase is for 11.2 in the short term and also EPM Cloud. Customer-managed OAC will remain on Essbase 12. Oracle-managed OAC might eventually get Essbase 19c.
- 126.96.36.199 on-premises Essbase patch development has apparently stopped and will not continue. This makes no sense as there are still unsolved bugs lingering out there, and EPM Cloud uses 188.8.131.52 technology behind the scenes. No word yet if EPM Cloud will be bumped up to 11.2. My gut feeling is EPM Cloud will get 11.2 first, as the cloud’s servers run Linux. Plan on opening SRs galore if Oracle plans on doing this within the first 6 months of 11.2’s release. They’ve pushed out 11.2’s release multiple times now, so they must be fighting bugs.