2008

1/2/08  15705,499 – getting error starting JMS server on cluster member: MSGS0058E: Unable to start the

JMS Server as WebSphere Embedded Messaging has not been installed

– Emb msging is installed according to install program and install logs

– checked under server, app svrs, timsrvqa02, srv components, and JMS server is not there, contr to:

http://www-1.ibm.com/support/docview.wss?uid=swg21162965

– Ondrej Bizub (WAS) – run dspmq and mqver, file not found (mq was uninstalled by someone)

– removenode.sh, uninstalled WAS, reinstalled plus fp’s,

– addnode.sh timsrvqa01 8879 -includeapps, added server back to cluster, worked

1/9/08  In IDI log when running identity feed:

CTGIMD011E The Person profile cannot be found for the [EmpPerson] object class

– solution: check object classes in Config, Entities – make sure object class name matches

– changed to “Employee”, worked

1/11/08  16461 499 – In ibmditk, Adding new connector attrib to output map getting error

Exception in thread “AWT-EventQueue-0”

java.lang.ArrayIndexOutOfBoundsException: Array index out of range: 1

– tried dragging attrib over from work attribs, and adding directly

1/14/08 16634,499 – Getting the following error when provisioning accounts:

CTGIMO019E The following JNDI error occurred.Error: [LDAP: error code 1 – Operations Error].

– they tried to do an export/import using itim gui – problem

– exported ldap from dev using db2ldif

1/16/08 16922,499 – Installation of ITDI 6.1.1 hangs at 50% during

“Performing System Checks – Checking system status”

installs fine on 2 other servers.  Not working on production boxes

– had to install new version of SI to allow product to install to a network mounted drive

https://cs.opensource.ibm.com/projects/sispxenable/

– APAR # IO08028 already exists to fix this on the SI side

– asked for an IDI readme note

3/20/08  – getting error on WAS 6.1 fp9 install says prerequisite checking has failed…

A general exception has occurred during prerequisite checking

install_log.txt says “CWuPI0026E ”

– problem was actually update installer version

3/20/08 – 48099,180 – passport adv down, esr website down

3/24/08 – 69471 999 – After installing ITIM 5.0, When starting the app, get

EJB container caught com.ibm.ejs.container.DuplicateHomeNameException:

ITIM#mdb_ejb.jar#enroleejb.AdhocSyncMDB

– fix was to stop the node, run syncnode.sh, and restart the node

– the app started ok.  For some reason sync’ing from the WAS console, even

full sync was not working earlier.  Is this a bug with 6.0.0.0? ”

3/25/08            – 69580 999 – When trying to connect to the itim db, getting

SQL30081N: A Communication ERror has been detected…Location “10.14.129.8”

function “connect” Protocol specific error code “79, “*”, “*” SQLSTATE=08001

– when stopping and starting db2, then running ls, pwd, get

ksh: 0403-030 The fork function failed. Too many processes already exist

many db2sync processes exist

– changed maxuproc to 4096 in smitty, tried to connect to db, got

sql1224N – database manager is not able accept new requests, has terminated all requests in

progress, or has terminated your particular request due to a problem with your request.

sqlstate=55032

– tried setting num_poolagents to 100

– running “db2pd -agent|grep Current” showed up to 420 running when connecting to db,

then back to 100

– then after installing fixpack 4a, we’re getting this when running db2start

exec(): 0509-036 Cannot load program db2star2 because of the following errors:

0509-130 Symbol resolution failed for db2star2 because:

0509-136   Symbol sqloSetThreadScopeForDB2EDUs (number 115) is not exported from

dependent module /opt/IBM/db2/V9.1/lib64/libdb2e.a[shr_64.o].

0509-192 Examine .loader section symbols with the

‘dump -Tv’ command.

03/26/2008 16:20:53     0   0   SQL1042C  An unexpected system error occurred.

SQL1032N  No start database manager command was issued.  SQLSTATE=57019

– fix was to run db2iupdt ldapdb2 and postfix steps from:

http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp?                       topic=/com.ibm.db2.udb.uprun.doc/doc/t0006352.htm

3/26/08 48769 180 – using a password with an exclamation point for itim db user,

makes cfg_itim_mw_aix fail 2 cmds

– Lance Clinton – making APAR – # IZ18696

fixed 6/11/08  IF0006

3/26/08 – When runnig db2 9.1 fp 4a install..

Req’d minimum level of xlC runtime is 8.0.0.6

Actual version 8.0.0.4

– download IBM C++ Runtime Environment Component for AIX (Feb 2008)

– worked

4/1/08 49505 180 – when test connection using TAM adapter, getting ctgimM107W and ctgimT407E

– tried default adapter profile, getting CTGIMT401E – NoClassDefFoundError:

com.tivoli.pd.jutil.PDException

java com.tivoli.pd.jcfg.SvrSslCfg -action config -admin_id sec_master -admin_pwd <pwd> -appsvr_id sm5ts05 -policysvr sm5

ts02:7135:1 -port 7135 -authzsvr sm5ts02:7136:1 -mode remote -cfg_file /usr/java14/jre/PDPerm.properties -key_file /usr/java1

4/jre/PdPerm.ks -cfg_action create

– cd /opt/IBM/TDI/V6.1.1/jre/java/lib/ext

– cp /usr/java14/jre/lib/ext/PD.jar .

– Stop and start Adapter

– Test Connection

– OK

— Worked, closed

4/3 – When creating a person w/ a valid ID policy, getting error

“CTGIMU150E An error occurred while trying to locate the identity policy for Tiv Id Mgr user ID’s”

From trace.log: “Governing Policy couldn’t be found”

– on method “validatePassword”

7/22/08  21549,999 – What versions of ITDS are supported with WAS 5.1, 6.0?

8/22/08  18170,033

18179,033 – After federating node to the DM, creating a cluster, and adding an app

to the cluster, home page for app hangs and will not display.  App

(IDSWebApp.war) works fine with standalone server

9/3/08   18567 033 – WAS console not accessible, Getting authentication error when trying to restart the deployment mgr.  Don’t know that anyone enabled security

profiles/Dmgr/config/cells/cellname/security.xml

On the second line of the file you will see enabled=”true”. Change this to enabled=”false”.

– someone did enable security, got password and above instr, closed

9/8/08 – insert disk 3 comes up during itds 6.1 install by default – customers don’t have disk 3

9/8/08 – migrate doc doesn’t address upgrading WebSphere or in-place upgrades for middleware

9/9/08 – errors creating db and connecting to db in middleware config util when using a password with @ # and $

9/9/08  51686,227 – Queue: L2LDAP Center: 165 – Running idscfgdb get error GLPCTL036E Failed to update the database: ‘ldapdb2’

db2cli.log said

db2 9fp5, ids6.1 fp2

– mike cotociu

uninstalled v6.1, v6.0, reinstalled v6.1, config’d db

– got GLPCTL028E: Failed to create db ldapdb2. The failure might have occurred because the system was not set up correctly before using the tool

– created instance and db using db2admin instead of ldapdb2, worked

– db2 service was running with company\db2admin (domain acct) as the account, which was locked out

– probable cause

9/10/08 51809,227 – Queue WBETIM Center: 12H – getting ldap error code 52: serviceunavailableexception when trying to run ldapUpgrade.exe during upgrade to ITIM 5.0

DB2 v9.1 fp5 not recommended w/ itds 6.1

http://www-01.ibm.com/support/docview.wss?uid=swg21314966

The imported LDIF had a different server id than that in the ibmslapd.conf, changed, fixed

9/10/08 – IDIL2, 12H – when running itdi install for fixpack 3, get error when selecting location

9/16/08 29022,082 – When running ITDI data feed, recon hangs, and getting mult errors in trace.log:

com.ibm.ejs.container.SessionBeanTimeoutException: Stateful bean CMStatefulBeanO(BeanId(ITIM#mdb_ejb.jar#enroleejb.CompletionListenerRegistryHome, 10F1DD26-011C-4000-E000-F4820A521FB3), state = METHOD_READY) timed out

saw error in the SystemOut.log: DSRA7019W: Oracle10gDataStoreHelper or a subclass of it must

be used when configuring WebSphere DataSources to run using Oracle10g jdbc driver

– changed Oracle helper class in WAS console:resources, JDBC, Data Sources, (each one), picked 10g helper for each one

– restarted server, ran through 5000, trace.log showed success, but itim console hung

– changed max jdbc connections to 100, min 50 from 5/50

– changed entry cache to 30000, filter cache 100, acl cache 100

– ibmdefaultbp 98304, ldapbp 4096

– changed timeouts back in ap servers, itim01, container settings, transaction service

http://www.ibm.com/developerworks/tivoli/library/t-tds-perf/

– changed input file to original one (known good data)

– set timeouts to 120 sec

– changed entry cache to 100000, ldapbp 36864

– stopped WAS, deleted tranlogs, reran, no errors, success in trace.log, but hung as pending request

– sent logs with workflow and workflowextensions debug_max

– was working 6 out of 7 times

– found NullPointerException at com.ibm.itim.ui.impl.customform.SubFormLegacyFilter.doFilter

– deleted all persons and accounts, ran initial feed – 2hrs, 15 hung processes

– removed trace logging in WAS

– changed under was console app servers, srvnm, thread pools, web contn’r= 100min 200max

– ran full ititial load got nullptrs in trace.log

– 2761 activity, 7009 process, then 3114 act, 7262 proc, went up then down

– Upped all 3 jdbc connectors to 300min 500max

– Upped LDAP conn pool to 120min 150max from 50/100

– turned up wkflow & rmt resrcs logging, ran 1000 entries, worked fine

– turned off debug logging, ran 5000 entries, had 3 pending in activity, 5 in process

– upped entry cache to 600000, doubled bp’s

– turned up logging, ran 5000 entries

– error seen by L3 – IndexOutOfBoundsException in the trace log

– fixed by 5.0.0.2-TIV-TIM-IF0010 APAR: IZ24874

9/24/08 – PMR 47206,082,000 – Confusing display of status of running reconciliation request in TIM UI

– APAR IZ33245 has been created

9/24/08 – PMR 47311,082,000 – Opening workflow designer results in nullpointer exception in trace.log file               – APAR IZ33246 has been created

9/26/08 – When trying to create a person or an ITIM account, get this in completed requests:

CTGIME203E The following script interpreter error occurred. Error: Runtime error Java Array index 0 is out of range 0 detected at line 3 of function ‘createIdentity’ in string starting with: ‘function createIdentity()’… called at line 5 in string starting with: ‘function createIdentity()’…

– ITIM account is set to auto provision

– was because userID was blank

9/29/08 08403 7TD – When configuring TAm rvrs pwd sync, get this in pdweb msg log:

DPWCA0917E   Could not find ITIM  message header.

HPDIA0310W The password for user testuser2 was rejected due to policy violation.

-fixed in tam adapter 4.6.7, but using TAM combo adapter – latest appears to be 4.6.3

http://www-01.ibm.com/support/docview.wss?uid=swg1IY99323

10/3/08 15034 379 – When enabling auto provisioning policy for tam, got database connection errors in SystemOut.log

WSVR0605W: Thread “MessageListenerThreadPool : 238” (36d249ae) has been active for 602,812 milliseconds and may be hung.  There are 11 threads in total in the server that may be hung.

E J2CA0045E: Connection not available while invoking method queueRequest for resource enroleDataSource.

[10/3/08 14:35:56:945 EDT] 25f949ae ConnectionMan E J2CA0020E: The Connection Pool Manager could not allocate a Managed Connection: com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException: Connection not available, Timed out waiting for 180000

Also, out of memory errors in ibmdi.log

– set -Xms512m -Xmx1536m in ibmdisrv

– then when resubmitting the prov pol, got this in ibmdi.log

2008-10-06 19:11:19,

CTGDIK404E Could not log on to TAM

CTGDIK450E Unable to interface with the TAM server. The following unknown message type was retrieved by TAM while processing an exception thrown by TAM: The server could not locate the session for the client

– in msg__amj_error1.log, get

PROGRAM ERROR null null $com.tivoli.pd.jutil.AuthResponse <AuthResponse constructor> RMI TCP Connection(374)-146.47.243.125 [The server could not locate the session for the client.]

– David Hooks/Austin, Charles Schultz

– upgrade profile, dispatcher

– set -Xms1024m -Xmx2048m in ibmdisrv, 1536/2048 in WAS, resubmitted

– session error occurred at 15:37:12,903

– got error starting AL tam_add early on – this is normal and by design

– got out of memory error on WAS when checking completed failed request

MEM_ERROR in inflateInit2

– increased all ulimits for wasadmin, including hard fsize, core, data, rss

– got past that situation, but then got out of mem when aborting prov pol, still session errors

– changed session timeouts for TAM to 10sec, 30 sslv3, 30 inactv, maxThreads on ITDI to 50

– got session errors right away on first account

– changed session timeouts to 30sec, 90 inactv

– got OoM on ITDI

– changed max heap ITDI 1536m

– upgraded dispatcher to 5.0.11, fixed UID to be an array in prov policy script for DN

– changed AL cache to 0 in <idi_home>/itim_listener.properties

– changed Servers -> Application Servers -> server1 -> ORB Service -> Thread Pool = 20

– Servers -> Application Servers -> server1 -> Web Container -> Thread Pool = 20

– Servers -> Application Servers -> server1 -> Message Listener Service -> Thread Pool =20

– still same errors, few days later got hung threads (pmr 24707)

– set orb and web cont back to 50, mls to 25

– FITS  MR1022083847

10/3/08 – when starting WAS, getting in SystemOut.log

QueueManagerM E MSGS0153E: The Queue Manager process strmqm could not be started – error: com.ibm.ws.process.exception.InvalidExecutableException: PROC0004E: Executable: [strmqm] does not appear to be a valid executable.  Process could not be created.

JMSService    E MSGS0001E: Starting the JMS Server failed with exception: com.ibm.ws.process.exception.InvalidExecutableException: PROC0004E: Executable: [strmqm] does not appear to be a valid executable.  Process could not be created.

– solution was to start was as “wasadmin” user (owner of was app)

10/8/08 15044 379 – When submitting password change, getting in ibmdi.log:

java.lang.NoClassDefFoundError: com.tivoli.pd.jutil.bn (initialization failure)

– Brian Hemric, Charles Schultz

– upgrade profile, dispatcher to 4.6.4 of combo adapter

(had just copied in tamcomboutils.jar from 4.6.4)

if fails, log4j – root category=debug

– upgraded, made changes, change password hung, changed prov pol to manual, aborted process,

upped logging on remt svcs, stopped was, got err in sysout:

CNTR0019E: Non-application exception occurred while processing method “dummyTest”. Exception data: com.ibm.websphere.csi.CSIException: Begin global tx failed; nested exception is:

org.omg.CORBA.NO_PERMISSION: Transaction service is unavailable

– was starting WAS with root, started with wasadmin, worked, closed (fix was upgrade rest of aptr)

10/14/08 – java.lang.ClassCastException: java.lang.Object when creating a person

id policy seemed ok

10/14/08 24455 379 – LDAP Error code 32: No Such Object when creating person, and tries to create TAM acct

– OU and given name were null

– tamUpdate AL runs for some reason

10/14/08 24525 379 – after upgrading TDI to 6.1.1 and installing the TAM Combo adapter? Error when running LDAP recon after reading several records:

[status:fail, connectorname:conLDAPUser, operation:get, exception:javax.naming.CommunicationException: connection closed [Root exception is java.io.IOException: connection closed],

Can you use an existing LDAP profile/service after upgrading?

4.6.5 min – introduced supt for tdi 6.1.1, 4.6.7 latest

– imported 4.6.7 profile, same error

– turned AL caching off (com.ibm.di.dispatcher.disableConnectorCache=true)

– got out of mem err too – sent logs

– tried with ITDI on other machine (256/1024 mem), same error

– reinstalled TDI 6.1.1, fp3, dispatcher, same error

– checked /<tdi_home>/solutions/etc/reconnect.rules for com.ibm.di.connector.LDAPConnector::javax.naming.CommunicationException:reconnect

– checked Auto Reconnect on Connection Loss” checkbox option on the Connection Errors tab of the connector setup in the TDI assy line

– Fern – 3 issues: upgrade, assy line err, reconnect in ldap

– tried filtered recon – on sunone, got

com.ibm.itim.remoteservices.ejb.mediation.AccountEntryHandler.FAILURE_THRESHOLD_EXCEEDED; 516; 530; 15

– set enrole.reconciliation.failurethreshold=100% in enRole.props, filtered worked

– full Sun recon now gets “error unmarshalling return; nested exception is: java.io.EOFException”

– for IDS, added this to service.def under LDAPSearch operation

<parameter name=”ldapPageSize”><default>100</default></parameter>

– recon ran, but no accounts returned

– was using companyDlrLdap2 as ADT project name, changed back to companyDlrLdap, reran recon

– accounts were added, but under LDAPAccount object class

– deleted all accounts, prov pol, svc, profile pieces, obj cls for dealer ldap

– reimported profile and recreated svc, prov pol, reran recon, still erLDAPUserAccount oc

– custom still errors, but solution is probably upgrade profile and add paging

– still get EOF errors with default profile

– I changed the connectorpooltimeout, restarted the adapter, had not gotten to run a recon yet, changed it back, restarted the adapter again, and restarted TDS before running the recon.  It’s now working.

FITS #  MR0210094525

10/15/08 – getting error in sysout when multiple person modify requests are sent

CNTR0020E: Non-application exception occurred whi

le processing method “handleMessage” on bean “BeanId(enRole#mdb_ejb.jar#enroleejb.ContainerManagedMess

ageHandlerHome, null)”. Exception data: Process ID: 0

Activity ID: 0

com.ibm.itim.workflow.engine.RemoteServiceFailure: CTGIMO006E The following SQL error occurred.

Error: 1

SQL State: 23000

Caused by: com.ibm.websphere.ce.cm.DuplicateKeyException: ORA-00001: unique constraint (ENROLE.ACTIVIT

Y_PK) violated

10/15/08 – 24707 379 – getting hung threads in WAS when submitting auto prov pol on tam combo adapter.  processing just stops entirely after itim processing of changes is done (~20 min) and nothing gets to IDI

– set orb and web cont back to 50, mls to 25, worked, closed

10/23/08 35942 379 – after setting up TAM adapter on 2nd box (PMR 15034,379), get

CTGDIK403E Invalid TAM configuration file: /opt/PolicyDirector/etc/tamsslcfg.conf

– changed conf file in itim service form to /opt/IBM/IBMDirectoryIntegrator/solutions/PDCfgFile.conf

got HPDIA0200W Authentication failed. You have used an invalid user name, password or client certificate..

deleted conf & ks files, reran svrsslcfg, re-entered password in service form, worked.

10/24/08 36054 379 – from 24525 379, cannot install tdi 6.1.1, getting error

The following requirements have not been met: – Nothing selected to install, or the product is already installed at this location. Target (default) dir is empty.  Uninstall was successful.

– when choose cancel, get

Error in copying /tmp/ismp011/7050297/data/faddba0181b51d9316ef30cc8eb4700d/6.1.1.0/resource.map  there may not be enough space on device

java.io.IOException: File does not exist: /tmp/istemp131786298120653/_bundledJRE_/jre/lib/ext/svcdump.jar

– 600 mb free on tmp, 8 gb free on target dir

– deleted /usr/ibm/common/acsi, /var/common, AD data from /etc/inittab, etc/services

– freed up 2.5 mb free on /tmp

– install worked, closed

10/24/08  36116,379 – when turning on autoprov or manual for tam, getting

java.lang.ClassCastException: java.lang.Object

Chris Weber

– changed line in id policy to return currUID[0];

got error in TDI:  CTGDIK415E Unable to interface with the TAM server.

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0

– followed by out of memory

– changed /etc/security/limits – stack_hard = -1

– creating a user gave this for dn: uid=[object JavaArray],ou=dealerstaff,o=company.com

– changed dn script to subjUID = subject.getProperty(“UID”)[0];

– getting IndexOutOfBoundsException when running sslsvrcfg

– you have to run sslsvrcfg to interface with TAM on another box

– tried truss

– copied in different PD.jar

– tried running command w/o -cp option, specifying -cp /opt/PolicyDirector

– tried running java w/o fully qualifying, and setting JAVA_HOME to same dir before running svrsslcfg

– deleted PDCfgFile.conf and retried it without the -cfg_action replace, and it worked

– got error during recon “Reference Error : ‘TamLdapReconFactory’ not found”

– copied in correct TAMComboUtils.jar (4.6.4), worked

– getting ojbect class violation from companyActive during Add – should not be an attrib

– Chris sent me a jar with some more hooks and logging

–      did?

– set GloabalRunnALCount to 10 in itim_listener.properties

— closed b/c ssltimeout=1200 worked

12/29/08 65458 379 – getting “CTGDIK450E Unable to interface with the TAM server. The following unknown message type was retrieved by TAM while processing an exception thrown by TAM: The server could not locate the session for the client” when enabling autoprov for TAM Combo

Tried the latest 5.012 (rel 12/30/08), with GlobalRunALCount set to 5.

– still failed

– tried to upgrade to itdi 611 fp4, got errors

I get this when I first run it:

Error in copying /tmp/ismp001/1216208/data/a7390f9ecf6457bf3713f951c90b1bdb/1.0.0.0/resource.map  there may not be enough space on device

(Jan 2, 2009 11:46:48 AM), Install, com.ibm.wizard.platform.aix.AixRegistryServiceImpl, wrn, AixRegistryServiceImpl: Unable to initialize AIX  registry.

and this when I select Next, Install:

Exception in thread “Thread-19” java.lang.NullPointerException

at com.ibm.ci.gmi.issi.wizard.actions.OfferingQueryAction$OfferingQueryOperation.performOfferingQuery(OfferingQueryAction.java:168)

at com.ibm.ci.gmi.issi.wizard.actions.OfferingQueryAction$OfferingQueryOperation.execute(OfferingQueryAction.java:243)

at com.installshield.wizard.service.AsynchronousOperation.run(Unknown Source)

at java.lang.Thread.run(Thread.java:801)

and then it hangs at “Performing offering query, please wait…”

– make sure acsi process is running: /usr/ibm/common/acsi/bin/acsisrv.sh -start

– killed acsi, restarted

– tried GlobalRunALCnt = 5 (diff spelling), same error, uploaded logs

– TDI fp4 problem was conflict w/ 6.1 (article swg21255761)

– ran /usr/ibm/common/acsi/bin/de_instmaint -mfile TDI_MAIN_IU.jar

– back to main problem

http://www-01.ibm.com/support/docview.wss?uid=swg1IZ00350

– Edit the /opt/PolicyDirectory/etc/pdmgrd_routing and /opt/PolicyDirector/etc/pdacld_routing files.  The edit is uncommenting the last line in each file. Send /var/PolicyDirector/log/trace__pdacld_utf8.log /var/PolicyDirector/log/trace__pdmgrd_utf8.log

– Set ssl-v3-timeout = 1200 in /opt/PolicyDirector/etc/ivmgrd.conf

– hung for 3 days

– Set GlobalRunALCount to 30

– worked, closed – solution was ssl-v3-timeout = 1200 and GlobalRunALCount to 30