EUS stopped working, solution found

Some time ago I wrote that EUS stopped working. Today I was finally able to spend some time on this issue. Obviously I could have know that the solution was easy: the default password policy was blocking the account. Creating a new password policy with an enormous password expiry date for the cn=OracleContext,dc=acme,dc=com subtree solved all EUS problems.

P.S. if you happen to create your own custom password policy, please follow the documentation carefully. The right-click-create-like method does not work.

Cloning 11g Fusion Middleware on 64-bit OEL

I am discovering the mass deployment features of Fusion Middleware 11g. Oracle’s documentation sents you towards cloningclient.jar, which helps you creating a golden image and clone this image to new machines. What the documentation forgot to mention is that you only find cloningclient.jar on your filesystem after you installed an extra Oracle product in your middleware home (just WLS is not enough, you need SOA or some other component). That’s okay by me, since I am working on a SOA deployment anyway, but the docs could have been more clear on that one.

First of I copied the cloningclient.jar to some NFS share that is present on all my machines. This makes sure I will never have to look for this .jar again. Creating a golden image with cloningclient.jar is rather easy, just follow the documentation and you’ll be okay:

java -jar /software/oracle/32bit/FusionMiddleware/cloningclient.jar createclone -archiveLocation /software/oracle/32bit/FusionMiddleware/OFM_golden_image.jar -sourceMWHomeLoc /u01/app/oracle/mwhome  -excludePattern “*.log,*.bak” -invPtrLoc /u01/app/oracle/mwhome/Oracle_SOA1/oraInst.loc -silent true

Next you logon to a brand new installed machine and you call cloningclient.jar again, again all according to documentation:

java -jar /software/oracle/32bit/FusionMiddleware/cloningclient.jar applyClone -archiveLocation /software/oracle/32bit/FusionMiddleware/OFM_golden_image.jar -targetLocation /u01/app/oracle/mwhome -sourceID all -silent TRUE

Now this should give you a newly installed Fusion Middleware home, but instead you get this nice error:

Log: /u01/app/oracle/oraInventory/logs/install2009-10-21_05-07-57PM.log
contextFile:/tmp/PREREQ20091021_050730/Scripts/prereq/oui/agent_prereq_context.xml
java.lang.UnsatisfiedLinkError: /tmp/PREREQ20091021_050730/oui/lib/linux/liboraInstaller.so: /tmp/PREREQ20091021_050730/oui/lib/linux/liboraInstaller.so: cannot open shared object file: No such file or directory (Possible cause: architecture word width mismatch)
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(Unknown Source)
        at java.lang.ClassLoader.loadLibrary(Unknown Source)
        at java.lang.Runtime.load0(Unknown Source)
        at java.lang.System.load(Unknown Source)
        at oracle.sysman.oii.oiip.osd.unix.OiipuUnixOps.loadNativeLib(OiipuUnixOps.java:402)
        at oracle.sysman.oii.oiip.osd.unix.OiipuUnixOps.<clinit>(OiipuUnixOps.java:125)
        at oracle.sysman.oii.oiip.oiipg.OiipgFileSystem.changePermission(OiipgFileSystem.java:1995)
        at oracle.sysman.oii.oiix.OiixFileOps.changePermission(OiixFileOps.java:2110)
        at oracle.sysman.oii.oiix.OiixFileOps.changePermissions(OiixFileOps.java:2042)
        at oracle.sysman.oii.oiit.OiitLockHeartbeat.<init>(OiitLockHeartbeat.java:178)
        at oracle.sysman.oii.oiit.OiitTargetLocker.getReaderLock(OiitTargetLocker.java:473)
        at oracle.sysman.oii.oiii.OiiiTargetAreaControl.grabReaderLock(OiiiTargetAreaControl.java:361)
        at oracle.as.install.engine.modules.prereq.action.PrereqLauncher.initInstallInventory(PrereqLauncher.java:393)
        at oracle.as.install.engine.modules.prereq.action.PrereqLauncher.<init>(PrereqLauncher.java:128)
        at oracle.as.install.engine.modules.prereq.action.PrereqLauncher.getEnginePreReqLauncher(PrereqLauncher.java:119)
        at oracle.as.clone.util.PreReqUtil.prereqsetupAndLuncher(PreReqUtil.java:148)
        at oracle.as.clone.util.PreReqUtil.prereqcheck(PreReqUtil.java:433)
        at oracle.as.clone.mwhome.cloner.OracleHomeApplyCloner.preReqRun(OracleHomeApplyCloner.java:199)
        at oracle.as.clone.mwhome.cloner.OracleHomeApplyCloner.restoreOH(OracleHomeApplyCloner.java:154)
        at oracle.as.clone.mwhome.cloner.OracleHomeApplyCloner.restoreArchive(OracleHomeApplyCloner.java:122)
        at oracle.as.clone.cloner.ApplyCloner.doClone(ApplyCloner.java:50)
        at oracle.as.clone.request.ApplyCloneRequest.applyArchive(ApplyCloneRequest.java:70)
        at oracle.as.clone.request.ApplyCloneRequest._clone(ApplyCloneRequest.java:48)
        at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:110)
        at oracle.as.clone.process.CloningExecutionProcess.execute(CloningExecutionProcess.java:95)
        at oracle.as.clone.client.CloningClient.main(CloningClient.java:112)

Obviously the architecture word width mismatch is a dead giveaway: I need a 32-bit JRE to run the cloningclient, even though I created the golden image with a 64-bit JRE (the default one from OEL 4). If you happen to look for one, there’s one in the original middleware home. Just copy it to the new server and set the JAVA_HOME environment to the 32-bit JRE.  If you apply the golden image with the cloningclient.jar again, you will run into the next error:

2009-10-21_05-49-49PM : ERROR  : CLONE-20269   Setting Java Library path has failed.
2009-10-21_05-49-49PM : CAUSE  : CLONE-20269   There are following possible causes
 1. The oui library path /tmp/PREREQ20091021_054922/oui/lib was empty
 2. Unable to find oui lib directory name.
  3. The oraInstaller library path /tmp/PREREQ20091021_054922/oui/lib/linux64 was empty.

Again this is 64-bit trouble, only this time the fix is easier. Just call linux32 bash before you restart the clone and now the cloning process will run successfully.  The cloning process also creates an entry in inventory.xml (located in oraInventory/ContentsXML ). You might want to remove this entry if you replay the cloning process over and over again. 

Hope this helps.

Installing 11.1 SOA on 64-bit OEL

When you try to install the 11.1 SOA software, on 64-bit Linux, you might just forget to call linux32 before starting the installer. The actually error in the logging is this:

[2009-10-21T14:35:34.369+02:00] [as] [TRACE:16] [] [oracle.as.install.engine] [tid: 10] [ecid: 0000IHr64fm5mZWBdzR5D51Ark0J000001,0] [SRC_CLASS: oracle.as.install.engine.InstallEngine] [SRC_METHOD: startOperation] THROW[[
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at oracle.as.install.engine.InstallEngine.launchModule(InstallEngine.java:513)
        at oracle.as.install.engine.InstallEngine.processAndLaunchModules(InstallEngine.java:467)
        at oracle.as.install.engine.InstallEngine.startOperation(InstallEngine.java:420)
        at oracle.sysman.oio.oioc.OiocOneClickInstaller.main(OiocOneClickInstaller.java:449)
Caused by: java.lang.UnsatisfiedLinkError: no oraInstaller in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1709)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1030)
        at oracle.sysman.oii.oiip.osd.unix.OiipuUnixOps.loadNativeLib(OiipuUnixOps.java:420)
        at oracle.sysman.oii.oiip.osd.unix.OiipuUnixOps.<clinit>(OiipuUnixOps.java:125)
        at oracle.as.install.engine.modules.statistics.util.SysInfo.getAvailableMemory(SysInfo.java:162)
        at oracle.as.install.engine.modules.statistics.element.AvailableMemoryElement.<init>(AvailableMemoryElement.java:39)
        at oracle.as.install.engine.modules.statistics.standard.StandardElementLocator.loadElements(StandardElementLocator.java:63)
        at oracle.as.install.engine.modules.statistics.boot.ProfilerInitializer.initializeProfile(ProfilerInitializer.java:82)
        at oracle.as.install.engine.modules.statistics.Statistics.launchModule(Statistics.java:284)
        … 8 more

]]

This is a lengthly way of saying: hey! you’re running 32-bit software on a 64-bit OS.

OOW Day 5

A quick switch to day 5, yesterday was kind of hard on me (and it was not only the video thing 😉 ). Anyway, today is my last day at OOW. I skipped my schedule and moved to the unconference session of Doug Burns. I have no clue which sessions I will attend next, so this post might just be updated during the day.

OOW Day 3

When I arrived at the OTN Night last night it looked like the party just ended (even though it was supposed to go on for two more hours). Since I was tricked into a video interview by Alex earlier, I couldn’t convince myself to go “home”. I ended up watching a boxing match between a couple of Brits, which was kind of unsatisfactory (no blood).

Anyway, here’s my schedule for today. I was *almost* tricked into a Tom Kyte session *again* (11r2 new features), but likely I found out on time. The unconference stuff might actually be better than the “official” sessions, so there might just be some schedule changes.

Time	Title	Speaker/Company
11:30-12:30	Oracle Data Masking Pack: The Ultimate DBA Survival Tool in the Modern World	Jagan Athreya, Oracle; Sandra Cheevers, Oracle; Ravi Meda, Qualcomm
13:00-14:00	Oracle Fusion Middleware 11g Performance Shape-Up	Soumendu Chakraborty, Oracle; Stevan Malesevic, Oracle; David Rorke, Oracle
14:30-15:30	What to Expect from the Oracle Optimizer When Upgrading to Oracle Database 11g	Maria Colgan, Oracle; Mohamed Zait, Oracle
16:00-17:00	Authentication, SSO, and Authorization for Oracle WebLogic Server Applications	April Escamilla, Oracle; Christopher Johnson, Oracle; Rey Ong, Oracle

No more Tom Kyte

Please don’t hesitate to go and see Tom Kyte if you have the chance, but I’m done. I just watched another Tom Kyte show and I still think he’s brilliant, but it’s not for me. It wears me out. I just can’t handle someone going on for 50 minutes without hardly taking any breath. The information density is so high you have to pay attention every second, leaving you bedazzled after the show: what did I actually learn in detail? The level of the knowledge is suitable for DBA’s from junior to senior (whatever that may be), which draws a big crowd (and it should), but I’ll stick with the slides from now on.

OOW Day 2

Yesterday was a productive day, although the hands on lab wasn’t as solidly prepared as I expected from Oracle (would you have some Java developers start a RAC cluster on Linux?). After recovering from the ACE dinner (and boy I needed some recovery thanks to certain dba’s), I will attend some more sessions today. Tonight is the OTN Night at Howard st. (next to the conference center) and the happy hour of Oracle Benelux, but I might just pass the beers tonight.

Time	Title	Speaker/Company
10:15-11:15	Latency Is a Bug: Tuning Your Java Virtual Machine for Optimal Performance	Staffan Friberg, Oracle; David Simms, Oracle; Patrik Torstensson, Oracle; Mikael Vidstedt, Oracle
11:30-12:30	Why Telling the Database About Your Schema Matters	Thomas Kyte, Oracle
13:00-14:00	Bare-Bones Oracle Automatic Storage Management: What Every DBA Should Know	Jay Caviness, McKesson Provider Technologies; Sunil Wadhwa, Oracle
14:30-15:30	How to Hint	Jonathan Lewis, JL Computer Consultancy
16:00-17:00	Oracle JRockit: What’s New and What’s Coming	Daniel Kallander, Oracle; Tomas Nilsson, Oracle; Henrik Stahl, Oracle
17:30-18:30	DBA 2.0: Battle of the DBAs Revisited	Thomas Kyte, Oracle; Leng Tan, Oracle
16:00-17:00	Visualization for analysis and communication (unconference)	Chen Shapira
18:30-19:30	Happy Hour	Oracle Benelux
19:30-22:00	OTN Night	OTN

*Last minute changes applied to the schedule*