As soon as I logged into OIM and clicked on tasks the following error appeared in the oim_server1.out file:

< javax.naming.AuthenticationException [Root exception is java.lang.SecurityException: User 'principals=[weblogic, Administrators]' has administration role. All tasks by adminstrators must go through an Administration Port.]>


Now that is interesting. It seems that the OIM SOA integration stops working because of the administration port. So I started to read documentation, but found no clues here. Then I started looking some further and found this document that states:

“Connections that specify administrator credentials can use only the administration port”

Now there’s the answer for you, just as the logging states: you cannot use an administrator account to integrate OIM and SOA.

So how can I change this? First of all you need to setup a new account in weblogic. Navigate to your console and click on security realms –> myrealm –> User and Groups. Then click on new. Fill in the user details and click on ok

Do not assign any roles to the user. Next navigate to EM

First we will set the password for the soaadmin user in the credential map. Click on WebLogic Domain –> domain name. Then on WebLogic Domain –> Security –> credentials.

Select oim and then SOAAdminPassword. Click on edit and change the username from weblogic to soaadmin and the password to the password you set for the soaadmin user

Next up click on SOA –> soa-infra. Then click on SOA Infrastructure –> security –> application roles.

Now click on the button next to the the role name input box to find all roles.

Select the SOAAdmin role and click on “Add User” and select to soaadmin user.

Click on OK and you have completed the first step. Next you have to setup OIM to use this SOAAdmin user. This can be configured in EM to. Click on Identity and Access –> OIM –> oim (11.1.1.3.0). Then click on Oracle Identity Manager –> Sytem MBean Browser

Scroll al the way down and select oracle.iam –> Server: oim_server1 –> Application: oim –> XMLConfig –> Config –> XMLConfig.SOAConfig –> SOAConfig and change the username (SOA config username) from weblogic to soaadmin

Finally log into OIM and create a new user. Click on administration –> create user and fill in the form

click on save then on roles and assign the administrator role to the soaadmin user:

*presto*. Your OIM SOA integration is fully operational again.

Hope this helps.

Hope this helps.

If we add “HttpSevereErrorFirstLine=HTTP/1.1 400 Bad Request” in the config.cfg and restart the Content Server, the actual error message is seen instead of the bridge error. This undocumented parameter overrides the default 503 response sent by the Content Server in case of an error to 400.

Apache complains about the bridge error when a 503 response is sent and doesn’t when it’s something like HTTP/1.1 400 Bad Request.

This feature was tested on Universal Content Manager (UCM) 11.1.1.4.

hope this helps


[ERROR] Data Insufficient to start Install.
[ERROR] One and Only One of the following variables must be present

Variable Name:SKIP_SOFTWARE_UPDATES     Expected Value:true
Variable Name:SPECIFY_DOWNLOAD_LOCATION Expected Value:true
. Aborting Install


As the error shows, the response file for iam 11.1.1.1.5 is missing the SKIP_SOFTWARE_UPDATES directive.

Just add SKIP_SOFTWARE_UPDATES=true to the .rsp file and you’re set to go.

The new file would look like this:


[ENGINE]
#DO NOT CHANGE THIS.
Response File Version=1.0.0.0.0
[GENERIC]
#Provide the complete path of the Oracle Home. The Oracle Home directory name may only contain alphanumeric , hyphen (-) , dot (.) and underscore (_) characters, and it must begin with an alphanumeric character.
ORACLE_HOME=/u01/app/oracle/middleware/Oracle_IAM1
#Provide the complete path to a valid Middleware Home.
MIDDLEWARE_HOME=/u01/app/oracle/middleware
#Give the list of complete paths of all the valid Middleware Homes existing on the system.
MIDDLEWARE_HOME_LIST=/u01/app/oracle/middleware
SKIP_SOFTWARE_UPDATES=true
[SYSTEM]
[APPLICATIONS]
[RELATIONSHIPS]


The schedule builder can be found here.

• Jrockit (jrockit-jdk1.6.0_22-R28.1.1-4.0.1)
• WebLogic 10.3.4
• SOA Suite PS3 (11.1.1.4)

Now whenever I started WebLogic I ran into a rather lengthy core dump, which you can view: here (just rename to .txt). The interesting part of this core dump is this:


Loaded modules:
(* denotes the module where the exception occured)
0000000000400000-0000000000412703 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/bin/java
0000003353200000-0000003353201f93 /lib64/libdl.so.2
0000003353600000-0000003353615a6f /lib64/libpthread.so.0
0000003352a00000-0000003352b4d067 */lib64/libc.so.6
0000003351a00000-0000003351a1bb7f /lib64/ld-linux-x86-64.so.2
00002ba4453b2000-00002ba4456b734b /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/jrockit/libjvm.so
00002ba445991000-00002ba4459af9eb /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libosal.so
00002ba445bb2000-00002ba445bbe4fb /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libutil.so
0000003352e00000-0000003352e8122f /lib64/libm.so.6
0000003354600000-0000003354606f5b /lib64/librt.so.1
00002aaaaabb4000-00002aaaaabe9c17 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libjdwp.so
00002aaaaacf3000-00002aaaaacf5307 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libnpt.so
00002aaaae3d0000-00002aaaae3dbd03 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libjfr.so
00002aaaf1a92000-00002aaaf1a9e337 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libverify.so
00002aaaf1ba1000-00002aaaf1bc8c73 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libjava.so
0000003355200000-00000033552140c7 /lib64/libnsl.so.1
00002aaaf1d10000-00002aaaf1d165bf /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/native_threads/libhpi.so
00002aaaf22dc000-00002aaaf22e98cb /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libzip.so
00002aaaf2530000-00002aaaf2532e73 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libdt_socket.so
00002aaaf2643000-00002aaaf264c64b /lib64/libnss_files.so.2
00002aaaf2a4e000-00002aaaf2a6098f /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libnet.so
00002aaaf3b26000-00002aaaf3b2bf43 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/liborii.so
00002aaaf3ded000-00002aaaf3df38ab /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libnio.so
00002aaaf8e80000-00002aaaf8e818dd /u01/app/oracle/product/11.1.1.4/wlserver_10.3/server/native/linux/x86_64/libweblogicunix1.so

Now this gave me a hook to start querying google and metalink, ending up with only one advice: downgrade jrockit. Obviously I tried this, but this and it gave me the root cause:

<Apr 12, 2011 2:47:54 PM CEST> <Critical> <WebLogicServer> <BEA-000252> <Cannot switch to group "dba". java.lang.IllegalArgumentException: Cannot setegid("dba"), system error: ''
java.lang.IllegalArgumentException: Cannot setegid("dba"), system error: ''
at weblogic.platform.Unix.setEGroup0(Native Method)
at weblogic.platform.Unix.setEffectiveGroup(Unix.java:73)
at weblogic.t3.srvr.SetUIDRendezvous.setEGroup(SetUIDRendezvous.java:156)
at weblogic.t3.srvr.SetUIDRendezvous.makeUnPrivileged(SetUIDRendezvous.java:174)
at weblogic.t3.srvr.SetUIDRendezvous.initialize(SetUIDRendezvous.java:88)
at weblogic.t3.srvr.BootService.start(BootService.java:48)
at weblogic.t3.srvr.ServerServicesManager.startService(ServerServicesManager.java:461)
at weblogic.t3.srvr.ServerServicesManager.startInStandbyState(ServerServicesManager.java:166)
at weblogic.t3.srvr.T3Srvr.initializeStandby(T3Srvr.java:879)
at weblogic.t3.srvr.T3Srvr.startup(T3Srvr.java:566)
at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:467)
at weblogic.Server.main(Server.java:71)
>

Now this did help me. Suddenly I realized that I had been playing with machine configuration (uid and gid setting to be more specific)

<machine xsi:type="unix-machineType">
<name>testbox</name>
<node-manager>
<name>testbox</name>
<listen-address>localhost</listen-address>
</node-manager>
<post-bind-uid-enabled>true</post-bind-uid-enabled>
<post-bind-uid>testuser</post-bind-uid>
<post-bind-gid-enabled>true</post-bind-gid-enabled>
<post-bind-gid>dba</post-bind-gid>
</machine>

Now testuser doesn’t have dba as primary group:

$ id testuser
uid=10641(testuser) gid=100(users) groups=4006(dba),100(users)

So because of this jrockit crashes… (wrong group for the machine). The error is kind of unfriendly, downgrading seems to be a way of improving error messages.

Hope this helps.

Anyway, constructing rewrite rules is difficult (to me), chances are that you make a mistake while constructing the rule. So when I made a configuration error I noticed this:

$ /u01/app/oracle/admin/instance1/bin/opmnctl restartproc process-type=OHS
opmnctl restartproc: restarting opmn managed processes...
================================================================================
opmn id=testbox.area51.local:6701
0 of 1 processes restarted.

ias-instance id=instance1
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
——————————————————————————–
ias-component/process-type/process-set:
ohs1/OHS/OHS/

Error
–> Process (index=1,uid=893024996,pid=14351)
Process restart failed.
Process is still alive — check configuration.

Log:
/u01/app/oracle/admin/instance1/diagnostics/logs/OHS/ohs1/console~OHS~1.log

Now that’s interesting. Oracle added a new feature: check the configuration before restarting Apache. The “Process is still alive — check configuration.” is new to me. Didn’t read this is the docs, still nice to know.

Hope this helps.

Processes in Instance: asinst_1
--------------+--------------+------+---------
ias-component | process-type | pid | status
--------------+--------------+------+---------
oid1 | oidldapd | N/A | Down
oid1 | oidldapd | N/A | Down
oid1 | oidmon | N/A | Down
EMAGENT | EMAGENT | 3868 | Alive


Now when I called opmn to start all processes, I didn’t see an error, but somehow OID still didn’t start:

opmnctl startall
opmnctl startall: starting opmn and all managed processes...

Processes in Instance: asinst_1
--------------+--------------+------+---------
ias-component | process-type | pid | status
--------------+--------------+------+---------
oid1 | oidldapd | N/A | Down
oid1 | oidldapd | N/A | Down
oid1 | oidmon | N/A | Down
EMAGENT | EMAGENT | 3868 | Alive


When I checked the logfiles I noticed that the .oidmonstdout had a different filesize from normal. The contents was this:


ORA-24550: signal received: Unhandled exception: Code=c0000005 Flags=0

----- End of Call Stack Trace -----


Some googleing tought me that this has to do with some sqlnet settings for diagnostics. Funny how diagnostics can actually break your stuff on Windows… Anyway, placing a sqlnet.ora in C:\Oracle\asinst_1\config (yes, that’s the config directory of your instance, or any other location if you have modified your opmn.xml manually) with this contents fixed the issue:

DIAG_ADR_ENABLED=OFF
DIAG_SIGHANDLER_ENABLED=FALSE
DIAG_DDE_ENABLED=FALSE

Hope this helps.

If you happen to start your nodemanager from wlst, i.e. like this:

$ rlwrap java weblogic.WLST

Initializing WebLogic Scripting Tool (WLST) …

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

wls:/offline>
wls:/offline>
wls:/offline> startNodeManager(NodeManagerHome=’/u01/app/oracle/nodemanager’,PropertiesFile=’/u01/app/oracle/nodemanager/nodemanager.properties’)
Launching NodeManager …

<>

Node Manager starting in the background
wls:/offline>

your nodemanager gets started just like expected. If you check your process tree you might find something you won’t like:

wls_user 21785 19630 0 17:35 pts/0 00:00:00 rlwrap java weblogic.WLST
wls_user 21786 21785 4 17:35 pts/1 00:00:09 java weblogic.WLST
wls_user 22740 21786 15 17:38 pts/1 00:00:06 /u01/app/oracle/jrmc-4.0.0-1.6.0/jre/bin/java -classpath /u01/app/oracle/jrmc-4.0.0-1.6.0/jre/lib/rt.jar:/u01/app/oracle/jrmc-4.0.0-1.6.0/jre/lib/i18n.jar:/u01/app/oracle/middleware/wlserver_10.3/server/ext/jdbc/oracle/11g/ojdbc6dms.jar:/u01/app/oracle/middleware/patch_wls1033/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/u01/app/oracle/middleware/patch_ocp353/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/u01/app/oracle/jrmc-4.0.0-1.6.0/lib/tools.jar:/u01/app/oracle/middleware/wlserver_10.3/server/lib/weblogic_sp.jar:/u01/app/oracle/middleware/wlserver_10.3/server/lib/weblogic.jar:/u01/app/oracle/middleware/modules/features/weblogic.server.modules_10.3.3.0.jar:/u01/app/oracle/middleware/wlserver_10.3/server/lib/webservices.jar:/u01/app/oracle/middleware/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/u01/app/oracle/middleware/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:/u01/app/oracle/middleware/wlserver_10.3/server/lib/weblogic.jar:/u01/app/oracle/middleware/oracle_common/soa/modules/commons-cli-1.1.jar:/u01/app/oracle/middleware/oracle_common/soa/modules/oracle.soa.mgmt_11.1.1/soa-infra-mgmt.jar:/u01/app/oracle/middleware/oracle_common/modules/oracle.jrf_11.1.1/jrf.jar:/u01/app/oracle/middleware/wlserver_10.3/common/derby/lib/derbyclient.jar:/u01/app/oracle/middleware/wlserver_10.3/server/lib/xqrl.jar -DPropertiesFile=/u01/app/oracle/middleware/wlserver_10.3/common/nodemanager/nodemanager.properties -DNodeManagerHome=/u01/app/oracle/middleware/wlserver_10.3/common/nodemanager -DQuitEnabled=true weblogic.NodeManager -v


as you can see, the user wls_user owns the pid that started rlwrap. This pid is the parent of the wlst session, which is the parent of the nodemanager pid. Now guess what happens if you exit out of your wlst session?

To make this worse, guess what process is the parent of the startWebLogic.sh script that starts your managed server?

wls_user 23727 22740 0 17:41 ? 00:00:00 /bin/sh /u01/app/oracle/middleware/user_projects/domains/ooid_domain/bin/startWebLogic.sh

So, here’s my friendly advice: do not start the nodemanager with a rl-wrapped wlst session or you will be finding yourself banging your head against the wall.

Well, in short hotsos-tuesday was all about statistics. The sessions were in the wrong order for me though. For me personally it would have made much more sense to first let Doug Burns and Maria Colgan explain the theoretics and after that let Margaret Norman show a real world implementation. I still don’t get why Margaret wrote her own implementation of degree input parameter for dbms_stats, but that’s probably my ignorance. All three sessions were great and I can advise everyone to look for the presentations online.

Another highlight of Tuesday was Tim Gorman showing a perfect valid example of how you can use AWR and ASH for more than performance tuning. An entertaining presentation, exactly the right amount of content for an end-of-the-day presentation.

Hotsos wednesday was interesting. I went to see Karl Arao, Toon Koppelaars (tip: great presenter!), Mark Farnham, Tanel Põder and did something I promised I would not do again: I went to see Tom Kyte. Tom was enormous entertaining, a true keynote presenter: way to go mr. Kyte.

Next I had to sprint to a cab (which I shared with Alex G.) to take my flight to Detroit, a 50 minute transfer and then off to Amsterdam.

All in all I had a great conference. It’s been a while since I attended a conference with that much technical content. Hopefully I will attend again next year for the tenth anniversary.

Finally, to conclude this post: Disclosure; I’m attending this year’s Hotsos Symposium without the help of any other program. The time off work, travel & accommodation is at my own expense.