Sunday, August 8, 2021

11.2.6 Doesn't Fix My Bricked In-Place Upgrade

Those of you who've been following my blog for some time know that I love to experiment.

A while back, I had attempted an in-place upgrade within the Hyperion / Oracle EPM series after applying one of the quarterly Oracle Critical Patch Updates.  I have at least 2 posts here on that very topic: my original findings, and a 2nd post offering a solution shared by one of our frequent commenters here, Deezel.

The README for EPM 11.2.6.0, the newest EPM Release as of this writing, claims to automatically roll back Middleware patches are part of the in-place upgrade process.  I decided to roll the dice without first applying the solution linked above.  Mostly, I was just curious to see what would happen!

Well..... this happened:












(Trying to redeploy other web applications, such as Calculation Manager, yields a similar error)

Given this is happening in a throwaway sandbox that isn't customer-facing, I opted not to open a Service Request on this.

Other weird errors encountered during the in-place upgrade:

  • EPMSystem11R1\diagnostics\logs\install\ gobbled up 80GB of space due to a runaway 32-bit Oracle DB Client silent install.  It ate up all remaining disk space I had available, and I ended up having to forcibly terminate the javaw.exe process for installTool.  I also ended up renaming the dbclient32 folder and deleting its entry in the global Oracle Inventory.
  • The OHS upgrade failed at 50% completion status with a vague error.  I renamed the ohs folder and installTool completed during the 2nd attempt.
  • 11.2 still has the Known Issue with respect to cancelling out of configTool; it blew away both setEnv.bat and configTool.bat in the epmsystem1 instance folder.
I need to move on to other matters of investigation now, so I destroyed the VM to free up space for the next one.

It was an interesting exercise.  I'm encouraging all my peers to back up the Middleware folder (minus user_projects) prior to applying the quarterly Oracle CPU Middleware patches.  I think I still have some of my hair remaining....


Posted by at
Labels: , , ,

10 comments:

  1. RJAugust 10, 2021 at 4:14 AM

    My upgrade from a patched 11.2.4 to 11.2.6 was successful, but I never attempted to upgrade it to 11.2.5.
    The installer did a rollback of the CPU patches and OHS and deployment of the webapps went without issues.

    ReplyDelete
  2. EMAugust 10, 2021 at 6:04 PM

    I ran into the exact same OHS 50% failure trying the 11.2.5 to 11.2.6 in-place upgrade. Also ran my D: drive out of place eventually; at least now I know the reason :-)

    ReplyDelete
    Replies
    1. Dave ShayAugust 12, 2021 at 8:56 AM

      Glad to know I wasn't going crazy! These quarterly 11.2 in-place upgrades are really starting to concern me.

      Delete
    2. ShadowBoneAugust 25, 2021 at 2:00 AM

      Yes, similar log size issue. I also faced issue due to weblogic patches where components didn't deployed from 11.2.5. Ultimately the hack provided for 11.2.5 works. But certainly weblogic patches are still causing problems

      Delete
    3. VeeraAugust 26, 2021 at 4:05 PM

      Hello,

      I have 11.2.4 environment, never applied Critical Patch Updates, then updated fine without any issues to 11.2.5. I am now trying to update 11.2.6 ran into same issue with OHS after 50% File Copy failed error. I did not look for space issue but exited that command prompt. Rest of installation went fine without any issues.

      I opened ticket with Oracle, but did you continued with work arounds to fix that OHS issue or do I have to blew it away. I guess this problem persists for everyone as I have not applied any CPU's.

      Please post, how you addressed it. I will update as I hear from Oracle support.

      Thanks.

      Delete
    4. VeeraAugust 26, 2021 at 4:13 PM

      Hello,

      I heard from Support and this is the solution they provided. I asked question why this is different in 11.2.6 as this issue was not identified in previous updates like 11.2.4 and 11.2.5 etc., Hope this solution works.

      1. Enable Long Path support on every EPM Windows server, details available here: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd
      To enable long path support:
      The registry key Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled (Type: REG_DWORD) must exist and be set to 1.

      Registry script:
      ======
      Windows Registry Editor Version 5.00

      [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
      "LongPathsEnabled"=dword:00000001

      ======

      2. The EPM InstallTool should not be running when the above registry change is made and it will pick up the change without rebooting server.

      3. Run 11.2.6 Installer again to update server

      Thanks

      Delete
  3. Larry LappAugust 18, 2021 at 3:56 PM

    I think anyone that went to 11.2.5 is going to have real problems going to 11.2.6. Looks like 11.2.4 to 11.2.6 is definitely less troublesome. I'm going to attempt 11.2.4 to 11.2.6 on my Sandbox environment and see how it goes. (Had to rebuild back to 11.2.4 after attempting to update to 11.2.5 blew it up.)

    ReplyDelete
  4. VeeraAugust 27, 2021 at 1:06 PM

    Dave,

    "The OHS upgrade failed at 50% completion status with a vague error. I renamed the ohs folder and installTool completed during the 2nd attempt."
    Oracle support fix to enable LongPathEnabled in registry did not help to fix OHS failure at 50%.
    When you renamed ohs folder to fix OHS upgrade failure, did you have to update inventory file also or just renaming folder and re-install will do the job.
    Appreciate your response.

    Thanks

    ReplyDelete
  5. VeeraSeptember 3, 2021 at 7:07 AM

    Hello Friends,

    These are my findings with 11.2.6
    1) After enabling LongPathsEnabled setting in registry, I rebooted and also restored Middleware Home completely to successfully complete installation.

    2) In a distributed environment on Essbase server, install tool is selecting to update OHS, this is new and surprising. Oracle recommended to exclude OHS assemblies while installing on non OHS server.

    3) This is another surprising for me that there is a note in installation and configuration guide that there is no need to run configuration if you are updating from 11.2.5 to 11.2.6

    Configuration is not required if you are applying an update from Release 11.2.5 to Release 11.2.6.

    https://docs.oracle.com/en/applications/enterprise-performance-management/11.2/hitis/applying_an_update_configuring_102x0cf8e3ce.html

    Thanks

    ReplyDelete
    Replies
    1. Dave ShaySeptember 20, 2021 at 9:15 AM

      Thanks for sharing your findings, Veera!

      I agree with Oracle that on a non-OHS server, you would not want to include Oracle HTTP Server when running installTool. Unfortunately, if you're doing an in-place upgrade and OHS was previously installed on a non-OHS server, you might have no choice but to upgrade it as the option might be grayed out for you. If this is the case for you and the in-place upgrade errors out for OHS on a non-OHS node, I'd ignore the error and move on.

      My rule of thumb to avoid the path issue and particularly concerning OHS in 11.2.x, I map a drive to the parent folder where my 11.2.x installation media is stored. If you map you drive directly to the 11.2.x installation media folder, the OHS install will throw you fits. This workaround is documented as a Known Issue in the 11.2.x README.

      Dave

      Delete

Thank you very much for your interest in this blog! I hope you're finding it helpful.

Please keep comments relevant to the topic in the post, as this blog is not a free-for-all substitute for Oracle Support or traditional consulting. If you have many questions unrelated to the specific topic at hand, consider contacting me on LinkedIn (https://www.linkedin.com/in/daveshay) so we may discuss the possibility of consulting.

Commenting on posts older than 90 days unfortunately goes into moderation, thanks to spammers who've been hitting this blog. Please have patience, and thanks for your understanding!

Comments including URLs linking back to gambling or other things unrelated to Oracle EPM will be deleted on sight. If you're an EPM consultant and are offering me constructive criticism or a tip, go ahead and DO link back to your blog or firm's website if you so desire.

Thanks again for reading!

Subscribe to: Post Comments (Atom)