ORACLE Queryable patch inventory on HP-UX V3 : /usr/lib/hpux32/dld.so: Unable to find library ‘libjli.so’ ($ORIGIN)

This a quick note on recent problem i have faced when trying to access the Queryable patch inventory “OPATCH_XML_INV” and also when trying to use datapatch (which need to access the Queryable patch inventory) on ORACLE 12.1.0.2 installed on HP-UX V3 itanium 2 server (So i am unable to patch the database) :

UPDATE 26/02/2016 : This was finally marked as a bug and the workaround is the same as explained here (12.1 : Datapatch Fails with ERROR “KUP-04004,KUP-04017,KUP-04118,KUP-04095,ORA-29913″,” fatal: libjli.so: open failed” (Doc ID 2085653.1))

Extract from qopatch_log.log :
KUP-05004:   Warning: Intra source concurrency disabled because parallel select was not requested.

KUP-05007:   Warning: Intra source concurrency disabled because the preprocessor option is being used.

Field Definitions for table OPATCH_XML_INV
  Record format DELIMITED BY NEWLINE
  Data in file has same endianness as the platform
  Reject rows with all null fields

  Fields in Data Source:

    XML_INVENTORY                   CHAR (100000000)
      Terminated by “UIJSVTBOEIZBEFFQBL”
      Trim whitespace same as SQL Loader
KUP-04004: error while reading file /db/bin/oracle/product/12.1.0/dbhome_1/QOpatch/qopiprep.bat
KUP-04017: OS message: Error 0
KUP-04017: OS message: /usr/lib/hpux64/dld.so: Unable to find library ‘libjli.so’.
/db/bin/oracle/product/12.1.0/dbhome_1/OPatch/opatch[12]: 6545 Killed

So there is a problem when loading the shared library libjli.so.

To diagnose the problem i used “tusc”

/usr/local/bin/tusc -f -E  -e -p 27121

output extract :(failed session when running select * from OPATCH_XML_INV  )

.
.
env[2] @ 0x7b03b74e: “JAVA=/db/bin/oracle/product/12.1.0/dbhome_1/jdk/bin/java”
env[80] @ 0x15d88: “LD_LIBRARY_PATH=/db/bin/oracle/product/12.1.0/dbhome_1/jdk/jre/lib/IA64N:/db/bin/oracle/product/12.1.0/dbhome_1/jdk/jre/lib/IA64N/server:/db/bin/oracle/product/12.1.0/dbhome_1/jdk/jre/../lib/IA64N:/db/bin/oracle/product/12.1.0/dbhome_1/lib:/db/bin/oracle/product/12.1.0/dbhome_1/srvm/lib”
.
.
[19384] open(“$ORIGIN/../../jre/lib/IA64N/jli//libjli.so”, O_RDONLY, 0) ……………… ERR#2 ENOENT
.
.
.

When i run the preprocessing script directly “qopiprep.bat” everything is OK.

output extract : (running session when executing the script “qopiprep.bat” directly )

.
.
.
env[2] @ 0x7b03b60e: “JAVA=/db/bin/oracle/product/12.1.0/dbhome_1/jdk/bin/java”
env[17] @ 0x15d88: “LD_LIBRARY_PATH=/db/bin/oracle/product/12.1.0/dbhome_1/jdk/jre/lib/IA64N:/db/bin/oracle/product/12.1.0/dbhome_1/jdk/jre/lib/IA64N/server:/db/bin/oracle/product/12.1.0/dbhome_1/jdk/jre/../lib/IA64N:/db/bin/oracle/product/12.1.0/dbhome_1/lib:/db/bin/oracle/product/12.1.0/dbhome_1/srvm/lib”
.
.
.
open(“/db/bin/oracle/product/12.1.0/dbhome_1/jdk/bin/IA64N/../../jre/lib/IA64N/jli//libjli.so”, O_RDONLY, 0) …………………. = 3
.
.
.

Observation :

There is no apparent difference on the environment variables between the two cases (Using external tables or running the preprocessing script directly). The problem appear to be related to “$ORIGIN”  which the dynamic loader has not expanded in this case  to the path of the executable.It seems that the executable path is not considered a trusted directory and when using setuid (oracle exec) $ORIGIN will not be expanded.

Test case :

1- Create a shell script and set setuid  (setuid.sh):

#!/bin/sh

echo `id`
/db/bin/oracle/product/12.1.0/db_home1/QOpatch/qopiprep.bat

2- Change permission :

chmod 7755 setuid.sh

3- Set kernel parameter to allow setuid on shell scripts:

sudo /usr/sbin/kctune secure_sid_scripts=0

4- Run from another user.

$ ./setuid.sh

uid=116(hatem) gid=20(users) euid=400(oracle) egid=500(oinstall) groups=501(dba),107(devftp)
/usr/lib/hpux32/dld.so: Unable to find library ‘libjli.so’.
/db/bin/oracle/product/12.1.0/db_home1/OPatch/opatch[5]: 12657 Killed
/usr/lib/hpux64/dld.so: Unable to find library ‘libjli.so’.
/db/bin/oracle/product/12.1.0/db_home1/OPatch/opatch[12]: 12658 Killed

Workaround :

A quick and dirty fix is to create a link to the needed library on searched path :

  ln -s  /db/bin/oracle/product/12.1.0/db_home1/jdk/jre/lib/IA64N/jli/libjli.so /lib/hpux32/libjli.so

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s