This document provides information on tuning middleware for IBM Tivoli Identity Manager 5.0 and 5.1. It includes tunings for WebSphere, database servers (IBM DB2, Oracle, and MS SQL), directory servers (IBM Tivoli Directory Server and Sun ONE Directory Server), IBM Tivoli Directory Integrator, the ITIM application, and ITIM adapters. This edition includes a troubleshooting, best practices, and regular maintenance sections as well. This is a working document and will be updated as more information becomes available.
Related Publications
Scripts to assist with tuning and performance analysis are available separately, as is a document with performance best practices for using the ITIM API:
Performance Tuning Guides for other versions of ITIM are also available:
See also the new IBM Tivoli Identity Manager wiki.
This fix resolves the APARs and defects listed in the "Problems Fixed"
section below
Copyright International Business Machines Corporation 2011.
All rights reserved.
Component: IBM(R) Tivoli(R) Composite Application Manager for Microsoft(R) Applications:
Microsoft(R) SQL Server Agent, Version 6.2.3
Component ID: 5724B96MO
Interim Fix 0003, (6.2.3-TIV-ITM_SQL-IF0003)
Date: NOVEMBER 16, 2011
Contents:
1.0 General description
2.0 Problems fixed
3.0 Architecture and prerequisites
4.0 Image directory contents
5.0 Installation instructions
6.0 Additional installation information
7.0 Known problems and workarounds
8.0 Additional product information
9.0 Notices
1.0 General description
=======================
This fix resolves the APARs and defects listed in the "Problems Fixed"
section below. This fix also includes the superseded fixes listed in
section 2.4.
2.0 Problems fixed
==================
The following problems are addressed by this fix.
2.1 APARS:
-------------------
APAR: IV07344
Abstract: ATTRIBUTE AGE OF LAST ERROR(MIN) SHOWS INCORRECT
VALUE ON PORTAL.
Additional Information: Attribute Age of Last Error(Min) from Alert
Summary attribute group shows the wrong value on Portal.
Detailed Recreation Procedure:
1) Go to the ErrorLog Alerts Navigator item.
2) Verify the Age of Last Error(min) attribute value from
Alert Summary attribute group on Portal.
APAR: IV08062
Abstract: Attributes Log Max Growth Size(MB) shows incorrect value
on Portal.
Additional Information: Attribute Log Max Growth Size(MB) from Database
Details attribute group shows the wrong value on Portal.
Detailed Recreation Procedure:
1) Go to the Databases Navigator item
2) Verify the Log Max Growth Size(MB) attribute value from
Database Details attribute group on Portal.
APAR: IV09025
Abstract: AGENT SHOWS NO DATA ON PORTAL WHEN ANY DATABASE IS LOCKED
Additional Information: Whenever agent try to get the data from
locked database, its keep on waiting & no data is seen
on portal (TEP).
2.2 Defects
-----------
None
2.3 Enhancements
----------------
None
2.4 Superseded fixes
--------------------
6.2.3-TIV-ITM_SQL-IF0002
6.2.3-TIV-ITM_SQL-IF0001
2.5 APARs and defects included from superseded fixes
----------------------------------------------------
6.2.3-TIV-ITM_SQL-IF0002
--------------------------
APAR: IV00459
Abstract: Agent doesn’t show all instances in server list while
configuring the agent locally.
Additional Information: Generally all the instances of SQL server
of same version should be displayed in server list.
But due to incorrect registry key path of the instance,
the problematic instance and next coming instances of same
version doesn't show in server list.
APAR: IV01570
Abstract: AGENT CONFIGURATIONS LOST.
Additional Information: When you close the dialog to configure the
templates, all the custom variables for the previously
defined instances get lost, and they disappear from both
the .ini files for the instances and the registry.
6.2.3-TIV-ITM_SQL-IF0001
--------------------------
APAR: IZ99911
Abstract: COLLECTOR CRASHES WHEN JOB RUNS MORE THAN 99 HOURS.
Additional Information: Create a job in SQL server and keep it
running for more than 99 hours.
APAR: IZ95858
Abstract: KOQOS.EXE CRASHES IF NOT ABLE TO COLLECT DATA FROM PERFMON.
Additional Information: koqos.exe crashes if it not able to collect
data from perfmon. Following popup message may appears:
"koqos.exe encountered a problem and needed to close."
APAR: IZ98919
Abstract: MEMORY LEAK OBSERVED IN KOQCOLL.EXE WHEN SQL QUERY FAILS.
Additional Information: Memory leak observed when one or more
databases are not accessible or any SQL query fails.
koqcoll.exe gets crash because of memory allocation error.
APAR: IZ98901
Abstract: SQL SERVER CRASHES AND PROVIDE STACK DUMPWHEN MS SQL AGENT
RUNS SP_REPLCOUNTERS.
Additional Information: Microsoft Event viewer for SQL Server crash and
stack dumps:
1) Install SQL server agent 6.2.3 on system
2) Configure agent
3) Start the agent
4) Make sure that few databases are not published
5) Observe the SQL Server ERRORLOG as well as Microsoft
Event viewer for SQL Server crash and stack dumps.
APAR: IV00339
Abstract: SQL SERVER AGENT ALERTS THE FALSE VALUE ON SITUATION
"MS_SQL_STATUS_INACTIVE".
Additional Information: The situation is triggering when the SQL
Server is busy and the value of the attribute which is
responsible to trigger the situation is showing value
as Inactive.
APAR: IV00420
Abstract: SOME ATTRIBUTES SHOWS WRONG DATA IN MS SQL DATABASE DETAIL
ATTRIBUTE GROUP FOR PUBLISHED DATABASES.
Additional Information:
1) Install SQL server agent 6.2.3 on system
2) Configure agent
3) Make more than 1 databases published
4) Start the agent
5) Observer value of replication related attributes in
Databases Information workspace
Related Files and Output:
Verify the values of replication related attributes for
published databases on Databases Information workspace.
3.0 Architecture and prerequisites
==================================
This fix is supported on all operating systems listed in Chapter 2 of
the IBM Tivoli Monitoring: Microsoft SQL Server Agent User's Guide,
version 6.2.3.
The following link is for the Tivoli operating system and application
support matrix. Please refer to this matrix for the latest
certification information.
http://www.ibm.com/software/sysmgmt/products/support/Tivoli_Supported_Platforms.html
3.1 Prerequisites for this fix
------------------------------
The prerequisite level for this fix is as follows:
- IBM Tivoli Monitoring, Version V6.22 Fix Pack 2.
- IBM Tivoli Monitoring for Microsoft Applications:
Microsoft SQL Server Agent, Version 6.23.0
As this fix is cumulative, it can be installed on any fix level
for this version, release and mod level above the prerequisite.
4.0 Image directory contents
============================
This fix image contains the following files:
- 6.2.3-TIV-ITM_SQL-IF0003.README - This README file
- 6.2.3-TIV-ITM_SQL-IF0003.tar - Fix archive .tar format
- 6.2.3-TIV-ITM_SQL-IF0003.zip - Fix archive .zip format
Note: The .tar and .zip files are identical in content. Use the .tar
file if you are working in a UNIX(R) environment; use the .zip file
if you are working in a Windows(R) environment.
The fix archive file contains the following files:
6.2.3-TIV-ITM_SQL-IF0003/koq_tems_teps_tepd_IF0003.tar
6.2.3-TIV-ITM_SQL-IF0003/koq_tems_teps_tepd_IF0003.zip
6.2.3-TIV-ITM_SQL-IF0003/koq_winnt_tema_IF0003.cab
6.2.3-TIV-ITM_SQL-IF0003/koq_wix64_tema_if0003.cab
6.2.3-TIV-ITM_SQL-IF0003/KOQWINNT.dsc
6.2.3-TIV-ITM_SQL-IF0003/KOQWIX64.dsc
6.2.3-TIV-ITM_SQL-IF0003/oq_dd.properties
6.2.3-TIV-ITM_SQL-IF0003/oq_dd_062300003.xml
5.0 Installation instructions
=============================
This fix can only be installed over an existing installation. Use
the following steps to install this fix.
5.1 Before installing the fix
-----------------------------
- The prerequisites listed under section 3.1 entitled 'Prerequisites
for this fix' must be installed before this fix can be installed.
- For the purpose of this README, the symbol <CANDLEHOME> is the
IBM Tivoli Monitoring installation directory. The default value
for CANDLEHOME is '/opt/IBM/ITM' on UNIX systems and 'C:\IBM\ITM'
on Windows systems.
Before installing this fix on UNIX systems, set the environment
variable CANDLEHOME to the IBM Tivoli Monitoring installation
directory.
For example:
> CANDLEHOME=/opt/IBM/ITM
> export CANDLEHOME
- Refer to section 7.0 entitled "Known problems and workarounds"
before installation.
- Because there is no uninstall utility for this fix, make sure to
perform a backup of your environment before installing this fix.
5.2 Local agent update
----------------------
1. Transfer the appropriate archive file
(6.2.3-TIV-ITM_SQL-IF0003.tar or .zip) to a temporary directory
on the system that contains the agent code to be updated. For
the purpose of this README, the symbol <TEMP> represents the
fully qualified path to this directory.
Note: On Windows systems, this path includes the drive letter.
2. Expand the archive file using the "tar" command on UNIX systems
or an extract utility on Windows systems. This step creates a
directory structure that contains fixes for all of the
supported platforms.
3. Use the "itmpatch" command to install the fix for that agent
platform. For more information on the "itmpatch" command, see
section 6.2.
On Windows systems, if the fix was expanded to
<TEMP>\6.2.3-TIV-ITM_SQL-IF0003, the install command is:
For 32 bit Agent:
> itmpatch -h <CANDLEHOME>
-i <TEMP>\6.2.3-TIV-ITM_SQL-IF0003\
koq_winnt_tema_IF0003.cab
For 64 bit Agent:
> itmpatch -h <CANDLEHOME>
-i <TEMP>\6.2.3-TIV-ITM_SQL-IF0003\
koq_wix64_tema_if0003.cab
5.3 Remote agent update
-----------------------
1. Transfer the appropriate archive file
(6.2.3-TIV-ITM_SQL-IF0003.tar or .zip) to a temporary directory
on the IBM Tivoli Enterprise Monitoring Server system. For the
purpose of this README, the symbol <TEMP> represents the fully
qualified path to this directory.
Note: On Windows systems, this path includes the drive letter.
2. Expand the archive file using the "tar" command on UNIX systems
or an extract utility on Windows systems. This step creates a
directory structure that contains fixes for all of the
supported platforms.
3. To add the agent fix bundles into the remote deploy depot, use
the "tacmd addBundles" command found in $CANDLEHOME/bin on UNIX
systems or in %CANDLE_HOME%\bin on Windows systems. For more
information on the "tacmd addBundles" command, see the IBM Tivoli
Monitoring Administrator's Guide.
On UNIX systems,
if the fix was expanded to <TEMP>/6.2.3-TIV-ITM_SQL-IF0003:
> $CANDLEHOME/bin/tacmd addBundles
-n -i <TEMP>/6.2.3-TIV-ITM_SQL-IF0003
On Windows systems,
if the fix was expanded to <TEMP>\6.2.3-TIV-ITM_SQL-IF0003:
> %CANDLE_HOME%\bin\tacmd addBundles
-n -i <TEMP>\6.2.3-TIV-ITM_SQL-IF0003
where:
-n indicates that prerequisite bundles are not automatically
added. The -n parameter must be used because the fix
directory does not contain any prerequisites that the fix
might require. Please see Section 3.1 for the prerequisites
for this fix.
-i is the directory that contains the deployment bundles to
be added to the depot.
4. To log in to the Tivoli Enterprise Monitoring server, and deploy
the fix to the appropriate nodes where the agent is running, use
the following "tacmd" commands. For more information on the
"tacmd login" and "tacmd updateAgent" commands, see Appendix A.
Commands reference of the IBM Tivoli Monitoring Administrator's
Guide.
On UNIX systems:
> $CANDLEHOME/bin/tacmd login -s <server>
-u <itmuser>
-p <password>
> $CANDLEHOME/bin/tacmd listSystems
The output shows the Managed System Name for the OS agent on the
remote system to be updated. Use this value as the target of the
"tacmd updateAgent" command.
> $CANDLEHOME/bin/tacmd updateAgent -t OQ
-n <Managed system name>
-v 062300003
On Windows systems:
> %CANDLE_HOME%\bin\tacmd login -s <server>
-u <itmuser>
-p <password>
> %CANDLE_HOME%\bin\tacmd listSystems
The output shows the Managed System Name for the OS agent on the
remote system to be updated. Use this value as the target of the
"tacmd updateAgent" command.
> %CANDLE_HOME%\bin\tacmd updateAgent -t OQ
-n <Managed system name>
-v 062300003
Note:
- The component (-t) for the "tacmd updateAgent" command is
specified as two characters (oq), not three characters (koq).
- The node (-n) for the "tacmd updateAgent" command is the
managed system name of the operating system (OS) agent to be
updated.
The target node for the "tacmd updateAgent" command is always
an OS agent.
5.4 Agent support update
------------------------
Use the following steps to update the Tivoli Enterprise Monitoring
Server, Tivoli Enterprise Portal Server, or Tivoli Enterprise
Portal Desktop.
1. Transfer the appropriate archive file
6.2.3-TIV-ITM_SQL-IF0003.tar or .zip) to the Tivoli
Enterprise Monitoring Servers, Tivoli Enterprise Portal
Servers, or Tivoli Enterprise Portal Desktops.
2. Expand the archive file using the "tar" command on UNIX systems
or an extract utility on Windows systems. This step creates a
directory structure that contains fixes for all of the supported
platforms.
3. Expand the archive file (koq_tems_teps_tepd_IF0003.tar or .zip)
that contains the updates for Tivoli Enterprise Monitoring
Server, Tivoli Enterprise Portal Server, and Tivoli Enterprise
Portal Desktop using the "tar" command on UNIX systems or an
extract utility on Windows systems. This step creates a
directory structure that includes a subdirectory called CD-ROM,
with the necessary updates.
4. Use the Application Support Installer(ASI) GUI or the silent
installation method to install the application support files.
GUI installation option
------------------------
The GUI can be started by using one of the following commands
from within the CD-ROM directory where setup.jar is located.
On UNIX systems:
> ./itmasi.sh [-h <CANDLEHOME>] [-j <JAVAHOME>]
On Windows systems:
> itmasi [-h <CANDLEHOME>] [-j <JAVAHOME>]
When prompted by the Application Support Installer for the
installable media directory, select the CD-ROM directory, not the
component directory. The installer can install updates for
multiple components at the same time.
Silent installation option
--------------------------
To use the silent installation option, use one of the following
commands from within the CD-ROM directory where setup.jar is
located.
Note: Using the silent installation option for Linux and zLinux
operating systems requires the following files contained in
interim fix 6.2.3-TIV-ITM_ASI-IF0003:
- Linux: setupLinux.bin
- zLinux: setupLinux390.bin
Follow the installation instructions contained in the
"6.2-TIV-ITM_ASI-IF0004.README ".
On UNIX systems:
> ./itmasi.sh -h <CANDLEHOME>[-silent]
[-r <response file>]
[-j <JAVAHOME>]
On Windows systems:
> itmasi -h <CANDLEHOME> [-silent]
[-r <response file>]
[-j <JAVAHOME>]
where:
-h <CANDLEHOME> - Required. The path to the IBM Tivoli
Monitoring installation directory.
-r <response file> - Optional. Customized response file name.
The default response file "response.txt" in
the CD-ROM directory can be copied and modified
for your environment. Do not update "response.txt".
-j <JAVAHOME> - Optional. The fully-qualified path to the "bin"
directory where Java(R) is installed. The
location of Java on your system might vary.
5. The next panel presented by the Application Support Installer
asks for the selection of which Tivoli Monitoring components
you want to add application support to.
For this fix, the check boxes should be set as follows:
unchecked - Tivoli Enterprise Monitoring Server(TEMS)
checked - Tivoli Enterprise Portal Server(TEPS)
unchecked - Tivoli Enterprise Desktop Client(TEPD)
Continue through the remaining GUI panels selecting the
Microsoft SQL Server 06.23.00.03 support to complete the
installation.
6. If the Tivoli Enterprise Portal Desktop or Tivoli Enterprise
Portal Browser was running when the update was installed, it
must be restarted.
6.0 Additional installation information
=======================================
Any existing Microsoft SQL Server Agent instances should be restarted
after the installation of this fix.
6.1 Installation instructions for agent baroc file
--------------------------------------------------
There are no updates to the baroc files included in this fix or
any of the superseded fixes. No additional installation steps are
required.
6.2 Additional information on using "itmpatch" command
----------------------------------------------------
The "itmpatch" command has the following syntax.
Usage: itmpatch -h <installation home> [OPTIONS]
itmpatch -h <installation home>
-t { <patch_file_directory> | <patch_file> }
itmpatch -h <installation home>
-i { <patch_file_directory> | <patch_file> }
where:
-h Specifies the IBM Tivoli Monitoring installation directory
-i Specifies the path to the directory or patch file to be
installed
-t Generates a report of the actions to be taken by the patch
For example, on UNIX systems:
- To preview the fix installation, use the "-t" option:
> <CANDLEHOME>/bin/itmpatch -h <CANDLEHOME> -t <TEMP>
- To install the fix, use the "-i" option:
> <CANDLEHOME>/bin/itmpatch -h <CANDLEHOME> -i <TEMP>
where:
<CANDLEHOME> is the fully qualified IBM Tivoli Monitoring
installation directory. On Windows systems, this path must include
the drive letter.
<TEMP> represents the fully qualified directory specification where
the fix is located. On Windows systems, this must include the drive
letter.
6.3 Verifying the update
------------------------
1. To verify the agent was updated correctly, use the "tacmd" command
to view the agent's current version after the agent is restarted.
You are required to log in to a Tivoli Enterprise Monitoring
Server prior to viewing the agent version.
For example:
On UNIX systems, where $CANDLEHOME is the IBM Tivoli
Monitoring installation directory, the default location
is '/opt/IBM/ITM'.
> $CANDLEHOME/bin/tacmd login -s <server>
-u <itmuser>
-p <password>
> $CANDLEHOME/bin/tacmd listSystems -t OQ
On Windows systems, where %CANDLE_HOME% is the IBM Tivoli
Monitoring installation directory, the default location
is 'C:\IBM\ITM'.
> %CANDLE_HOME%\bin\tacmd login -s <server>
-u <itmuser>
-p <password>
> %CANDLE_HOME%\bin\tacmd listSystems -t OQ
Note:
- The component (-t) for the tacmd "listSystems" command is
specified as two characters (OQ), not three characters (KOQ).
When the agent update is successful, the agent version is:
06.23.00.03.
After the agent is restarted, you can also use the GUI to verify
the agent was successfully updated.
For the agent on Windows systems, the version number is
06.23.00.03.
2. To verify the agent support files were updated correctly, use
the "kincinfo" command on Windows systems or the "cinfo" command
on Linux or UNIX systems. The sample output below shows the
versions of the Tivoli Enterprise Portal Server, Tivoli
Enterprise Monitoring Server, or Tivoli Enterprise Portal
Desktop systems after this fix has been successfully applied.
Note: The displayed date of the build might not be accurate.
This is a known problem.
On UNIX or Linux systems:
-------------------------
To validate that all components have been installed, run the
following command:
./cinfo -i
On Windows systems:
-------------------
To validate that all components have been installed, run the
following command:
For example:
> %CANDLE_HOME%\InstallITM\kincinfo -i
>kincinfo -i
OQ Microsoft SQL Server Support
WINNT Version: 06.23.00.00 Build: 10811
OQ Microsoft SQL Server Support
WINNT Version: 06.23.00.03 Build: 201111151809
OQ Microsoft SQL Server Support
WINNT Version: 06.23.00.00 Build: 10811
OQ Microsoft SQL Server Support
WINNT Version: 06.23.00.00 Build: 10811
7.0 Known problems and workarounds
==================================
Abstract: Remote agent update fails for OS agent version
06.22.02.00 (IZ73759 : ADD 32-BIT WINDOWS APP
SUPPORT ON WIX64 TO ITMPATCH)
Problem: In case of Remote agent update using OS agent (version
06.22.02.00), the update of agent fails with the error:
KDY1031E: The agent installation command
<CANDLEHOME>\bin\itmpatch.exe -h <CANDLEHOME> -i
<CANDLEHOME>\tmaitm6\agentdepot\XX\062300003\
XXX_winnt_tema_if0003.cab
returned an error with the return code 34 . The agent
was not installed because there is not enough disk
space on the server or you do not have permission to
install the agent.
Workaround: This is the problem with OS agent version 06.22.02.00
and has been fixed in version 06.22.03.00.
For successful remote agent update use OS agent version
06.22.01.00 or 06.22.03.00.
Abstract: Upgrading from an Interim Fix to an Interim Fix or
Provisional does not work (IZ84841)
Problem: Upgrading from an Interim Fix to an Interim Fix or
Provisional does not work and the following message is
displayed:
KDY0041E: The agent bundle XX is already updated to the
highest version 062101040 found in the depot and deployed
to <hostname>.
Workaround: The fix for this APAR is contained in the following
maintenance packages:
6.2.2-TIV-ITM-FP0004
or update the patch locally
or apply the workaround provided in APAR:IZ84841 (not
applicable for 64bit patch upgrade with 32bit OS agent and
a PMR:35628,999,744 is opened to track the issue)
Abstract: Applying 32-bit program updates with itmpatch on 64-bit
Windows systems fails (IZ73759)
Problem: Using the itmpatch utility to update 32-bit programs
fails to update machines running IBM Tivoli Monitoring
6.2.2 Fix Pack 1 on 64-bit Windows platforms. The
returned user message states that the update is not
applicable to the machines architecture.
Workaround: The fix for this APAR is contained in the following
maintenance packages:
6.2.2-TIV-ITM-FP0003
Abstract: Remote 64-bit patch upgradation fails with 32 bit
OS agent
Problem: Remote patch upgradation fails on already
installed patch of (i.e. IF1 and IF2).If we are
trying to upgrade SQL agent patch IF0003 remotely
from patch IF1/IF2 then it is showing the following
error message: Error Message : KDY0003E: The agent
bundle OQ with version 0623000XX (XX : 10/20) is
already deployed to XXXX (Manage instace name).
An agent bundle cannot be deployed if it is
already deployed.
Workaround: upgrade the patch locally. A PMR:35628,999,744 is opened to track
the issue and we can get any other workaround if the PMR get updated.
8.0 Additional product information
==================================
1. To get effect of TEMS support defects/APARs,
To update support at TEMS for defect 109023.2, First run
Application Support Installer (ASI) and follow steps given below:
1. Go to Manage Tivoli Enterprise Monitoring Service(MTEMS) window
2. Select Tivoli Enterprise Monitoring Server (TEMS)->right click->
Advanced->Add TEMS application support...
3. After selecting TEMS location click on OK
4. On next dialog select koq_upg.sql among list of 'Component'
column and press OK
5. After few moment a message box will appear to ask about
recycling of TEMS, press Yes
6. Now the TEMS database updated
2. APAR:IZ73254
A new environment variable has been added named 'QUERY_TIMEOUT'
which can be set by user. The variable has added with reference to
APAR:IZ73254.
The variable has introduce to set max time (in seconds) to execute
a query from agent to SQL Server. If query does not return within
specified time, the agent will skip the query (Hence there will
not be any data displayed on portal for this query result) and
will proceed further execution/operation. After setting the query
timeout, the agent will not wait for infinite time to execute
a query and user will be able to see data of other databases,
workspace etc.
Following instructions should be followed while setting
'QUERY_TIMEOUT'
1.The variable can be set by following menu selection:
“Action->Advanced -> Edit Variables”.
2.The QUERY_TIMEOUT value should be less than 45.
3.If number of locked database (or the databases which are not
accessible to user at that time and causes long time to execute
query) are greater the value of 'QUERY_TIMEOUT' should be less.
4.If 'QUERY_TIMEOUT' is not set or set to zero, there will not be
any effect on query execution.
3. APAR:IV09025
A new environment variable has been added named 'COLL_DBCC_NO_LOCK'
which can be set by user. The variable has added with reference to
APAR:IV09025.
If the tables of the databases are locked then queries from the
agent to SQL Server goes to infinite loop (hang).If we set the
variable then agent will execute the query with "no lock" option and
the agent will not wait for infinite time to execute but it might not
be return the recent data.
Following instructions should be followed while setting the
'COLL_DBCC_NO_LOCK'.
1.The variable can be set by following menu selection:
“Action->Advanced -> Edit Variables”.
2.The COLL_DBCC_NO_LOCK value should be 0 or 1.
3.If number is 1 then Query will execute with "no lock” option.
4.If 'COLL_DBCC_NO_LOCK' is set to 0, then there will not be
any effect on query execution i.e. not set the "no lock" option.
9.0 Notices
===========
This information was developed for products and services offered in
the United States. IBM may not offer the products, services, or
features discussed in this document in other countries. Consult your
local IBM representative for information on the products and services
currently available in your area. Any reference to an IBM product,
program, or service is not intended to state or imply that only that
IBM product, program, or service may be used. Any functionally
equivalent product, program, or service that does not infringe any
IBM intellectual property right may be used instead. However, it is
the user's responsibility to evaluate and verify the operation of any
non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject
matter described in this document. The furnishing of this document
does not grant you any license to these patents. You can send license
inquiries, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
U.S.A.
The following paragraph does not apply to the United Kingdom
or any other country where such provisions are inconsistent
with local law:
INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION
"AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Some states do not allow disclaimer of express or implied warranties
in certain transactions, therefore, this statement may not apply
to you.
Trademarks and service marks
----------------------------
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks
of International Business Machines Corp., registered in many
jurisdictions worldwide. Other product and service names might be
trademarks of IBM or other companies. A current list of IBM trademarks
is available on the Web at "Copyright and trademark information" at
www.ibm.com/legal/copytrade.shtml.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks
of Microsoft Corporation in the United States, other countries, or
both.
Java and all Java-based trademarks are trademarks of Sun Microsystems,
Inc. in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States
and other countries.
Linux is a registered trademark of Linus Torvalds in the United States,
other countries, or both.
Other company, product, or service names may be trademarks or service
marks of others.
-IBM Tivoli Monitoring, Version V6.22 Fix Pack 2.
-IBM Tivoli Monitoring for Microsoft Applications:
Microsoft SQL Server Agent, Version 6.23.0
Download | RELEASE DATE | LANGUAGE | SIZE(Bytes) | Download Options |
---|---|---|---|---|
6.2.3-TIV-ITM_SQL-IF0003.tar | 16-11-2011 | English | 15411200 | FTP |
6.2.3-TIV-ITM_SQL-IF0003.zip | 16-11-2011 | English | 14936338 | FTP |
6.2.3-TIV-ITM_SQL-IF0003.README | 16-11-2011 | English | 30334 | FTP |
Problems (APARS) fixed
IZ95858, IZ98901, IZ98919, IZ99911, IV00339, IV00420, IV00459, IV01570, IV07344, IV08062, IV09025
README documentation for IBM Tivoli Monitoring for Version 6.2.2 Fix Pack 7 including installation-related instructions, prerequisites and corequisites, and list of fixes.
README file for: IBM Tivoli Monitoring
Product/Component Release: 6.2.2
Update Name: Fix Pack 7
Fix ID: 6.2.2-TIV-ITM-FP0007
Publication date:December 15 2011
Last modified date:December 14 2011
Download location
Prerequisites and corequisites
Known issues
Known limitations
Installation information
Prior to installation
Installing
Performing the necessary tasks after installation
Troubleshooting installation problems from the Support site
Additional Information
List of fixes
Copyright and trademark information
Document change history
Download IBM Tivoli Monitoring Fix Pack 7 from the following location:
http://www.ibm.com/support/fixcentral/swg/quickorder?parent=ibm~Tivoli&product=ibm/Tivoli/IBM+Tivoli+Monitoring&release=All&platform=All&function=fixId&fixids=6.2.2-TIV-ITM-FP0007&source=fc
Alternatively, access the fix from the table below.
Below is a list of components, platforms, and file names that apply to this README file.
Product/Component Name | Platform | File Name |
---|---|---|
IBM Tivoli Monitoring Base | Windows | 6.2.2-TIV-ITM_TMV-Windows-FP0007/ |
IBM Tivoli Monitoring Base | UNIX | 6.2.2-TIV-ITM_TMV-Unix-FP0007/ |
IBM Tivoli Monitoring Base | Linux | 6.2.2-TIV-ITM_TMV-Linux-FP0007/ |
IBM Tivoli Monitoring Base Agents | All | 6.2.2-TIV-ITM_TMV-Agents-FP0007/ |
IBM Tivoli Monitoring Tools | All | 6.2.2-TIV-ITM_TMV-Tools-FP0007/ |
IBM Tivoli Monitoring Agent Reports | All | 6.2.2-TIV-ITM_TMV-Agent-Reports-FP0007/ |
For a complete set of prerequisites, refer to the "Hardware and software requirements" section in the IBM Tivoli Monitoring Installation and Setup Guide Version 6.2.2 Fix Pack 2 available at the following URL:
http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/topic/com.ibm.itm.doc_6.2.2fp2/welcome.htm
The following issues have been found in a version of this release, either in previous releases, or in this release. The symptom and the workaround has been listed for each issue.
The following issues are unique to this fix pack.
NA
The following issues affect prior IBM Tivoli Monitoring Monitoring 6.2.2 fix packs and are still applicable to this fix pack.
Symptom:
When upgrading the IBM Tivoli Enterprise Portal Server to this fixpack you may see a message that indicates eWas failed to upgrade. This may occur if you have less than 1.2GB of free space on the drive where the portal server is being upgraded, or less than 800MB on the drive where the TEMP directory is located.
Workaround:
Make sure that there is at least 1.2GB of free space on the drive where the portal server is being upgraded and 800MB of free space on the drive where the TEMP directory is located. If the portal server and TEMP share the same drive, then 2GB is space is required to upgrade eWas.
Symptom:
When Help is loaded from the Tivoli Portal client interface, the resulting help web pages displayed may have slightly different graphical banners display. The banner may display "Eclipse", "IBM", or "Tivoli". This is only a visual difference and there is no effect on how help works.
Workaround:
There is no work around, this is a visual difference only.
Symptom:
You may see the following messages when starting up a remote management server when the hub management server is down from Primary/Secondary (FTO) systems:
Starting TEMS...
It can take up to ten minutes.
KCIIN0084E Timeout waiting for TEMS to become ready...
KCIIN2514E TEMS service is still pending start. Check ITM documentation for more details.
Workaround:
There may be a slight delay from seeing this message and the remote management server going on-line. Please run "cinfo -r" to see if your remote management server is on-line.
Symptom:
You may see the following messages when executing itmcmd to configure a remote management server when the hub management server is down from Primary/Secondary (FTO) systems:
TEMS configuration completed...
Would you like to restart the component to allow new configuration to take effect? [1=Yes, 2=No] (Default is: 2): 1
Stopping Tivoli Enterprise Monitoring Server
Tivoli Enterprise Monitoring Server stopped
Starting Tivoli Enterprise Monitoring Server
Unable to start component, please check logs for more details.
Workaround:
There may be a slight delay from seeing this message and the remote management server going on-line. Please run "cinfo -r" to see if your remote management server is on-line.
Symptom:
Running the "migrate-export" or "migrate-import" command on Linux systems generates the following unexpected messages:
/products/ITM/ls3263/cq/bin/KfwServices: /products/ITM/ls3263/iw/java/jre/bin/libjsig.so: no version information available (required by /products/ITM/ls3263/cq/bin/KfwServices)
/products/ITM/ls3263/cq/bin/KfwServices: /products/ITM/ls3263/iw/java/jre/bin/libjsig.so: no version information available (required by /products/ITM/ls3263/cq/lib/libKfwORBInit.so)
Workaround:
The messages are introduced by a change in JRE 1.5 and can be ignored.
Symptom:
The "Tivoli Enterprise Build-level listing" output shows a backlevel version for the product codes "IW" and "UI".
Workaround:
This occurs on Windows only. Validate the versions for "IW" and "UI", if applicable, using the "KinCInfo" command.
Symptom:
Beginning with IBM Tivoli Monitoring Version 6.2.1 Fix Pack 3 and IBM Tivoli Monitoring Version 6.2.2 Fix Pack 3, the default port number for the Warehouse Proxy Agent was changed to 63358 for IP.PIPE/IP.UDP and 65100 for IP.SPIPE. This change was made to minimize the chance of conflict between the Warehouse Proxy agent and other IBM Tivoli Monitoring components. Agents that export historical data to the Warehouse Proxy agent will switch ports and may fail to export because the new port being used on the Warehouse Proxy agent machine is not open for connection.
Workaround:
Ensure the new default port number 63358 for IP.PIPE/IP.UDP or 65100 for IP.SPIPE is open for connection. The following Technote contains more information on this change:
http://www-01.ibm.com/support/docview.wss?uid=swg21469499
You can verify that port 63358 for IP.PIPE/IP.UDP or 65100 for IP.SPIPE was established by the Warehouse Proxy agent by following the instructions found in the following Technote below "Confirming the Warehouse Proxy Agent Listener port from the Command Line":
http://www-01.ibm.com/support/docview.wss?uid=swg21315085
Symptom:
During the installation of an agent, administrators may get a pop up warning that the certificate the software has been signed with has expired. IBM Tivoli Monitoring 6.2.2 Fix Pack 2 packaged a new certificate with the infrastructure and agents. The pop up can occur when installing agents that have not been packaged with the new certificate.
Workaround:
Accepting the certificate will allow the installation to continue.
Symptom:
When IBM Tivoli Monitoring and OMNIbus event synchronization is installed, status changes to IBM Tivoli Monitoring events originating from either IBM Tivoli Monitoring or OMNIbus result in the Severity and Summary of the events in OMNIbus being incorrect.
Workaround:
The following Technote contains more information on resolving the issue: http://www-01.ibm.com/support/docview.wss?uid=swg21428227
Symptom:
On UNIX and Linux platforms for the Warehouse Proxy Agent, when the Warehouse Proxy Agent is stopped it may create a core file. This core file is located in the directory where the agent was started. The core is a failure of the Java Virtual Machine (JVM) SR11 version shutdown process. This problem was introduced in SR11. The stopping of the Warehouse Proxy agent completes correctly and no side affect from the core causes any problems other than the space required for core file. This problem has been reported to the IBM JVM group and APAR IZ84541 has been opened. This APAR is scheduled to ship in SR13.
Workaround:
The use of the java argument: -XXallowvmshutdown:false can be used in the hd.ini file to revert the shutdown process to the old method prior to SR11. Set the following parameter in the hd.ini file:
KHD_JAVA_ARGS=-XXallowvmshutdown:false
Refer to JVM APAR IZ59734 available at the following URL for additional information:
http://www.ibm.com/support/docview.wss?uid=swg1IZ59734
Symptom
On a Windows 64-bit endpoint, ITM will support the coexistence of both 32-bit and 64-bit agents. However, in order to support this coexistence, a new bundle which is the Agent Compatibility Package (AC), has been created. This Agent Compatibility Package contains both the 32-bit and 64-bit Agent Framework libraries. The issue may be diagnosed by examining the ITM Installation Log for the associated agent deployment. You will find a failure like the following: RC_INFO: The Agent Compatibility Package in version 06220100 is required but unavailable. Affected platforms are Windows 64-bit at V6.2.2 Fix Pack 1 or later level.
Workaround:
The AC package (which can be found in the base ITM agents CD image) will need to be added to the depot and the agent redeployed. Refer to this Technote for information and guidelines:
http://www.ibm.com/support/docview.wss?uid=swg21427756
General information about installing the Agent Compatability component can be found in the Installation and Setup Guide:
http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/index.jsp?topic=/com.ibm.itm.doc_6.2.2fp2/win_instma.htm
Symptom:
When using the Agentless Monitor for Windows, and the instance is configured to use Windows API Connection to monitor a domain controller running Windows Server 2003, excessive CPU on the domain controller is used. This problem can occur only when the Windows API Connection (WMI) is configured and does not apply when the SNMP Connection is configured for the instance.
Workaround:
Microsoft has a hotfix available for this problem which should be installed on the domain controller:
http://support.microsoft.com/kb/933593
Refer to this Technote for information and guidelines:
http://www.ibm.com/support/docview.wss?uid=swg21430280
Symptom:
On zLinux, an upgrade of the OS Agent to the Fix Pack 2 or later level will result in the Agent Management Services watchdog utility being turned off regardless of whether it was active or inactive prior to the upgrade. On Intel Linux and other supported platforms, the Agent Management Services watchdog utility defaults to "enabled".
Workaround:
To enable the watchdog utility, edit the lz.ini file and comment out the line "KCA_CAP_DIR=" and uncomment the line that starts with "KCA_CAP_DIR=$CANDLEHOME$". For example:
# PAS Off
# KCA_CAP_DIR=
# PAS On
KCA_CAP_DIR=$CANDLEHOME$/$BINARCH$/$PRODUCTCODE$/bin/CAP:$CANDLEHOME$/config/CAP:/opt/IBM/CAP
Symptom:
When uninstalling a multi-instance agent from a managed system that also has an IBM Tivoli Monitoring version 6.2.2 Fix Pack 2 or later agent installed, instances are not uninstalled, leaving an entry in the management server. This only occurs on the Windows 64-bit platform. Other IBM Tivoli Monitoring components running on the system continue to operate normally.
Workaround:
Remove the agent instances first and then uninstall the agent. If the agent was already uninstalled, you can re-install the agent again, remove the instances, and then uninstall the agent again.
Symptom:
Some versions of the Korn Shell (ksh) that are newer than the Public Domain Korn Shell (pdksh) have changed the behavior of typeset -i. Current versions of the IBM Tivoli Monitoring installer for all IBM Tivoli Monitoring release families; 6.1, 6.2, 6.2.1, 6.2.2, have all been modified to function correctly with this change. However, there are still some IBM Tivoli Monitoring application agents in the field that are still based on a version of the installer that has not been updated to function correctly with these newer ksh shells. Installing these older application agent releases can potentially back-level the installer and itmcmd tools.
Workaround:
Before installing, see the Flash note #1408506 titled, "Newer ksh shell may allow regression of ITM installation" available at the following URL for more information on diagnosing, preventing, and (if necessary) correcting this problem.
http://www.ibm.com/support/docview.wss?uid=swg21408506
Symptom:
IBM Tivoli Monitoring JAR files are not downloaded to the browser portal client due to JAR version differences before and after the year 2010. The user does not get the expected updated JARs and the function is not updated.
Workaround:
After applying this fix pack, you can validate your portal browser client is at the correct fix pack and maintenance level as follows:
Symptom:
During an installation of the IBM Tivoli Monitoring OS Agents Cognos Reports TCR 2.1, the following error may occur: C:\Reports>setup.bat "Usage: setup.bat " C:\Reports>setup.bat "C:\IBM\tivoli\tipv2" Unable to determine Java Runtime Environment in Tivoli Common Reporting directory. Enter valid Tivoli Common Reporting installation location.
Workaround:
Since the IBM Tivoli Monitoring OS Agents Reports installer does not work with TCR 2.1, in order to install on TCR 2.1 the following procedure should be used:
1. In the ITM reports installation media, identify the reports zip file (ex. IBM Tivoli Monitoring for Virtual Servers Reports.zip)
2. Copy the report zip file in the TCR deployment directory: /products/tcr/Cognos/c8/deployment
3. In the ITM Reports installation media, identify the images directory (ex. C:\IBM_Tivoli_Monitoring_Virtual_Servers_Reports\packages\VMwareVIAgents\images)
4. Copy the content of the images directory in the TCR images directory /profiles/TIPProfile/installedApps/TIPCell/IBM Cognos8.ear/p2pd.war/tivoli/ITM/images
5. Launch the IBM Cognos Administration from the TCR portal
6. From the Configuration tab create a TDW data source
7. Enter the data source name that you wish (i.e. TDW)
8. Enter the data source type
9. Enter the ITM Tivoli Data Warehouse database name, and provide user and password to access it
10. From the Content Administration tab choice New Import Button and the report package that needs to be imported
11. Click Next, choose a name for your report package, and then click Next again.
12. Select the report package and click on the Next button
13. Click Next twice, and on this panel choose the Save and run once option and click Finish.
14. Finally, click Run and the report package will appear.
The following limitation is unique to this fix pack.
Symptom:
A security risk has been fixed (in 622 Fixpack 7) that allows a user to gain access to system files (any with files with read access, ie:/etc/passwd or c:\windows\system.ini), using the Agent Service Interface Client. The problem will exist until this fixpack is applied to each agent. Remote deploy or local install can be used to install the agent framework component to their machines.
Additional information:
NA
The following limitations were documented in the README for prior IBM Tivoli Monitoring 6.2.2 fix packs and are still applicable to this fix pack.
Symptom:
Situation does not fire when using the attribute "Process Filter" from the UNIX and Linux OS agents when the regular expression begins with a parenthesis ('(') or space.
Additional information:
The UNIX and Linux OS agents previously added support for long process names, requirement 1012. This included adding a new attribute "Process Filter" to both agents to define a regular expression to filter processes based on their full command line (Process Command(Unicode) on UNIX or Command Line on Linux) and return only a portion of the command line. When creating a situation for the "Process Filter", the regular expression cannot begin with parenthesis ('(') or space or the situation will not trigger correctly.
Symptom:
Executing the tacmd login -stdin option with missing standard input may cause looping or will not prompt user for the missing inputs.
Additional information:
For tacmd login and tepslogin, if -stdin option is used, and an echo is used to pass the parameters, then all the mandatory parameters need to be passed in the echo.
The following example is valid:
echo "-s localhost -u userid -p password" | tacmd login -stdin
The following example is not allowed:
echo "-s localhost" | tacmd login -stdin
Symptom:
In rare cases, user custom settings are can be lost after applying an IBM Tivoli Monitoring fix pack. Please refer to the Technote below to see if you might be impacted by this and instructions on how to resolve it.
Additional information:
Refer to this Technote for information and guidelines:
http://www.ibm.com/support/docview.wss?uid=swg21307768
Symptom:
When running the KinCinfo command with –x as an option, an error appears in the OS agent log when remote deploying agents in ITM V6.2.2 Fix Pack 1 environment.
Additional information:
No corrective action needs to be taken. The error message in this log is harmless and may be ignored. Refer to this Technote for information and guidelines::
http://www.ibm.com/support/docview.wss?uid=swg21427753
Symptom:
The GUI of an executable file launched from executecommand can not be seen in Remote Desktop, but can be seen in VNC.
Additional information:
This command is intended for executing short CLI based actions/commands on the remote target. It should not be used to execute long running, GUI, or interactive executables.
Before proceeding with the installation process, please make note of these important points:
$ITMHOME | 1200 MB |
/tmp/ | 800 MB |
Run the pdcollect utility to gather all of the current logs and environment files. These backups can be used for reference, if needed, after the upgrade. The "tacmd pdcollect -help" command can be issued to obtain the syntax for the pdcollect utility.
If using the ITMSuper tool, run the tool to take a snapshot of the entire ITM infrastructure prior to the install of this fix pack.
Run the pdcollect utility to gather all of the current logs and environment files. These backups can be used for reference, if needed, after the upgrade. The "tacmd pdcollect -help" command can be issued to obtain the syntax for the pdcollect utility.
Run the migrate-export.bat (Windows) or migrate-export.sh (UNIX/Linux) to backup the contents of the portal server database. This snapshot will allow for recovery of the database back to the point prior to installing this fix pack maintenance.
If using the ITMSuper tool, run the tool to take a snapshot of the entire ITM infrastructure prior to the install of this fix pack.
If using the ITMSuper tool, it is recommended to run the tool after applying the fix pack to create snapshot of the entire ITM environment that can be compared to the baseline snapshot taken with ITMSuper before the installation. The baseline snapshot can be done by selecting the Topology, Connectivity, and TEMS Servers tabs and saving the session. Information about the ITMSuper tool can be found at the following URL:
http://www-01.ibm.com/software/brandcatalog/ismlibrary/details?catalog.label=1TW10TM6LYou can validate your installation by running the KinCInfo command on Windows systems or the cinfo command on Linux or UNIX systems. If you run these commands before and after your upgrade, you can clearly see the component versions that have changed.
For interim fixes, only components that have been updated during this release have the latest listed level. If a component has not been updated, the package will contain the most recent GA copy. There is no need to upgrade this component if your system is at the listed level, the install programs should automatically handle this for you.
Note: The KinCInfo and cinfo command output examples in the sections that follow show all components that can be updated. If you do not have all of these components installed, then components not installed will not be displayed in the output of these commands.
Note: The "Monitoring agent for UNIX Logs" agent name has been deprecated. Going forward, this agent will be named "UNIX Logs."
When applying IBM Tivoli Monitoring maintenance updates it is necessary to know if the corresponding baroc definition contained in the maintenance needs to be applied to the event integration server such as a Tivoli Enterprise Console Server or Netcool/OMNIbus Probe for Tivoli Event Integration Facilty (EIF).
This fix pack contains baroc files at the level documented in the list below. If you have an earlier level installed you will need to reinstall them.
IBM Tivoli Monitoring Data Warehouse Agents | ||
---|---|---|
Warehouse Proxy | khd.baroc | 06.21.00.00 |
Summarization and Pruning | ksy.baroc | 06.21.00.00 |
IBM Tivoli Monitoring 6.x for OS Agents | ||
Windows | knt.baroc | 06.22.04.00 |
UNIX | kux.baroc | 06.22.04.00 |
Linux | klz.baroc | 06.22.04.00 |
i5/OS | ka4.baroc | 06.22.02.00 |
UNIX Log Alert | kul.baroc | 06.21.00.00 |
IBM Tivoli Monitoring 6.x for Agentless OS Agents | ||
Windows | kr2.baroc | 06.22.02.00 |
AIX | kr3.baroc | 06.21.00.00 |
Linux | kr4.baroc | 06.21.00.00 |
HP | kr5.baroc | 06.21.00.00 |
Solaris | kr6.baroc | 06.22.01.00 |
For additional troubleshooting help not covered in this README document, please visit the IBM Tivoli Monitoring support page at the following URL:
http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Monitoring_V6
This is a cumulative fix pack for IBM Tivoli Monitoring, Version 6.2.2. This fix pack supersedes all previous IBM Tivoli Monitoring versions, including GA Version 6.2.2 and above. This fix pack can be installed in just one step on top of any level of IBM Tivoli Monitoring version 6.1, 6.2, 6.2.1 or 6.2.2.
The IBM Tivoli approach for fix packs and interim fixes is to deliver regular, incremental quality improvements.The main areas of quality improvements made in this maintenance deliverable relative to the most recently shipped IBM Tivoli Monitoring maintenance (IBM Tivoli Monitoring 6.2.2 Fix Pack 6) are shown below.
The following new features are included in IBM Tivoli Monitoring 6.2.2 Fix Pack 7:
Agentless OS Monitor APARs | |
---|---|
IV09353 | When the Watchdog code which is part of the OS agent is monitoring the Agentless agents (kr2 - kr6), if the Agentless Agent has an "Agent Management Status" of "Managed" from running the Take Action "AMS Start Management", then when installing an upgrade (fix pack) for an Agentless agent, the resulting "Agent Management Status" will be reset to "Unmanaged" after upgrade completes its installation. |
IV09814 | In the WMI Event Log workspace, the "Recent Event Log Entries" view displays a garbled value for the Message attribute for Event IDs 4624 and 4625 on a Japanese system. |
Basic Services APARs | |
IV07861 | IBM Tivoli Monitoring Service Index entries are removed at random intervals. All service points on the IBM Tivoli Monitoring Service Index are subject to this defect and may be removed in error. This leads to loss of the portal server, or IBM Tivoli Monitoring Service Console, or Agent Service Interface connectivity. This defect is more likely to occur on systems with many portal client web browser-initiated connects and disconnects. Defective code matches on port and not the entire socket address. In use, this defect results in the owner of a service with the same port as a portal client being removed in error from the IBM Tivoli Monitoring Service Index. |
Command Line Interface APARs | |
IV05147 | When a name is empty in the importNavigator XML, the import fails. |
IV07952 | The tacmd suggestbaseline command incorrectly gets error KUICAC019E. The tacmd listsitattributes command and the portal client Add Expression Overrides dialog do not display the correct key/condition attributes for some situations. |
Event Synchronization APARs | |
IV02038 | If a UTF-8 column is part of the situation formula, this may cause the situation formula to get a parsing error. As a result, the event does not get sent from the hub monitoring server to the Omnibus/TEC Event server. |
IV04696 | The script and external procedure combination used to log events in the SUF cache file removes spaces due to the way it handles arguments. |
IV07994 | If a slot mapping contains DBCS characters and the hub monitoring server locale is not en_US or UTF8 but is the local locale, the value may not get added to the event. |
IBM Monitoring Agent for Linux OS APARs | |
IV04679 | Linux OS Agent does not register the hash sum of the changed file so that the comparison is done ever with the original hash sum. |
IV06844 | On Linux, the monitoring agent for Linux OS does not execute the Take Action related to the FILEINFO attribute group when there are multiple situations that monitors the same file. |
IV07267 | The monitoring agent for Linux OS reports "Collisions per minute" > 0, but the systems commands at the same time show all collisions equal to zero. |
IV08056 | If the Linux OS agent is running for a while, some filesystems gets displayed as "Not Collected" in "Disk Usage". |
IBM Monitoring Agent for Unix Logs APARs | |
IV06583 | On AIX, the Unix Log Agent does not use the external time zone (TZ) environmental variable when displaying the Log Entries for errlog file. This forces the log entries to be retrieved with a GMT timestamp rather than local time on the portal client. |
IV07600 | UNIX Log agent fails to recognize the new log file when it is rotated using mv and touch commands and it continues to monitor the old log file (also if its name has been changed). |
IBM Monitoring Agent for Unix OS APARs | |
IV03684 | On Solaris, the Monitoring Agent for UNIX OS kuxagent process uses prstat command to collect zones CPU and memory information, but CPU values are lower than values collected by other commands because micro-state process information is not accounted for. |
IV03687 | Provide an environment variable to allow for overriding location where kcawd.pid and .kcawd_disable_stop files are written. These files are currently written to //<pc>/bin directory. Customers who want to have these files written to a different location, in order to allow just read-only files in the bin directory, can use the new environment variable KCAWD_WRITE_DIR. The new environment variable KCAWD_WRITE_DIR can be set in ux.ini or lz.ini file and must be set to an existing directory with read-write permissions. For example: KCAWD_WRITE_DIR=$/tmp NOTE: This new feature is not supported by Windows OS Agent using Watchdog. |
IV06191 | When the OS agent is started, the HOME environment variable is part of the environment. After the agent has been restarted by watchdog, the HOME variable becomes undefined. This causes TADDM discovery using the IBM Tivoli Monitoring pipe to fail. |
IV06842 | On UNIX, the monitoring agent for UNIX OS does not execute the Take Action related to the FILEINFO attribute group when there are multiple situations that monitor the same file. |
IV06905 | The monitoring agent for UNIX OS is not able to manage situations with the FILEINFO attribute formatted as path+file. |
IV06922 | The monitoring agent for UNIX OS kuxagent process generates high CPU usage when it copies a very large number of strings. This is related to the process attribute group using the strncpy API. |
IV06977 | The UNIX OS Agent opens a password file in read-write mode on Solaris, but it requests this file in read-only mode. |
IV07319 | When the Watchdog code which is part of the OS agent is monitoring the KMQ and KQI agents on Solaris, the status is displayed as Stopped and the Process ID=0, even though the agents are running. The monitoring of the agents continues to work correctly, so this is a display-only problem. |
IV07361 | On AIX, the monitoring agent for UNIX OS kuxagent process can collect wrong values for the "CPU Pct" attribute in the Process attribute group. For example, it may always report the same value, like 12.5 or 25%. |
IV07415 | On Solaris, the monitoring agent for UNIX OS stat_daemon and kuxdstat processes can generate high CPU usage if they monitors a Veritas Dynamic Multipathing with thousands of disks. In order to avoid this issue, a configuration variable to disable collecting disk information has been created. |
IBM Monitoring Agent for Windows OS APARs | |
IV03507 | Monitoring the Event Log when the description is in Japanese characters and the string is greater than 377 characters causes the field, description_u to be empty |
IV06485 | In the file KNTWICMA.ver for Windows Itanium OS Agent, the platform type is reported incorrectly. |
IV07486 | When displaying the Logical Disks view from the portal client, the rows for Mount points with names longer than 64 characters are not displayed. Also, situations for Logical Disks will not detect such entries. |
IBM Tivoli Enterprise Monitoring Agent APARs | |
IV03676 | For historical workspaces showing data from several z/OS subnodes, where each subnode is connected to a different subnode manager (agent), Persistent Data Store (PDS) indexes are not utilized for this type of historical requests. This results in slow PDS data retrieval and delay in displaying data in the portal client historical workspaces. This applies only to historical SQL requests with two or more ORIGINNODEs in the WHERE clause connected to the OR operator, where ORIGINNODEs are subnodes reporting to different subnode managers. |
IV03943 | The # character in a node name prevents the agent from registering with the management server. The agent will continue to run and will be connected to the management server with the invalid name unless CTIRA_BLOCK_INVALID_NODE=Y is set when configuring the agent. When CTIRA_BLOCK_INVALID_NODE=Y is set and an invalid node name is detected, the agent will continue to run, but it will not be registered with the management server. |
IV04585 | Agent fails to discard events collected in autonomous mode for a stopped or modified situation. Situation events remain stored in persistent files on the agent if a situation is stopped while an agent is running in autonomous mode. Symptom: Presence of persistent files <hostname><pc>_autoattr.asa in the agent directory when agent is connected to the monitoring server. |
IV04683 | Agent in autonomous mode crashes due to unserialized access to the autonomous agent data depository file. |
IV06261 | Incorrectly calculated print format precision specifier causes the monitoring server to crash if KPX trace is enabled: |
IBM Tivoli Enterprise Monitoring Installation APARs | |
IV02310 | The ITMhostName command returns incomplete information when the local system name is different from the local node name. |
IV03537 | On an AIX operating system booted as 32 bit running on 64 bit hardware, IBM Tivoli Monitoring cannot monitor 64 bit DB2 instances. Even after specifically selecting and installing the 64 bit DB2 monitoring agent, IBM Tivoli Monitoring attempts to start the 32 bit DB2 monitoring agent even though the DB2 instance to monitor is 64 bit. |
IV05819 | In some cases, the Re-type prompt for passwords during configuration is rendered such that the leading R is presented at the end of the previous line: Enter TSM Administrator Password (default is: ****): R e-type : TSM Administrator Password (default is: ****): When the standard input, output, and error streams from the configure step are captured via the script command, we can see the cause is an out-of-place backspace character immediately preceding the R: hex: 08 20 08 20 0D 0A 08 52 65 2D 74 79 70 65 20 3A char: BS SP BS SP CR LF BS R e - t y p e SP : |
IV06886 | The encryption key specified in a silent install response file on UNIX/Linux is ignored and the internally defined default value is used when performing a silent install. This can be seen with the following message in logs/candle_installation.log runGSkit: Setting encryption key - ThisIsMyTestForTestEncryptionKey using gskSetkey class |
IV07595 | Agents using Secure Socket Layer (SSL) communications protocol setting IP.SPIPE are unable to establish a connection to the monitoring server after running the secureMain script to lock down permissions on the directory tree. The required SSL libraries are not loaded because the execute permission has been removed from the library files under /*/gs/icc/icclib and /*/gs/icc/osslib |
IV07735 | If an agent using .xml file instead of .cfg file to store the configuration information, the command itmcmd config tries to make a symbolic link to a non-existant .cfg file, which causes the configuration to fail. |
IV07968 | If a multi-instance defined Agent Builder or Agent Factory agent was configured without -o instance, and the configuration file name does not contain *_instance.config, then starting the agent without specifying -o instance will fail with the following message: KCIIN0201E Specified product is not configured... |
IV10341 | Changes were made to ITMISUtil.dll in FP5 but the internal Windows version of the file was not updated. This caused the file to be replaced even though the file versions were the same. This replacement of the DLL should not have effected it, but since an older DLL was present on the ITMfVE CD at the same level the DLL was replaced and back leveled. The back leveled DLL caused the configuration of the newly installed agent to fail with the popup error. Popup message box with text "KCICF8038E Agent configuration aborted due to user action or errors encountered." appears during GUI install. |
IZ95802 | After upgrading the MSSQL agent, the existing instances are unconfigured then re-configured. This fails, thus causing the instances to be deleted. |
IZ96919 | "tacmd updateagent" fails under these conditions: - The UNIX OS agent is version 6.22.02.00 or older. - The UNIX OS agent was installed via non-root user. - "SetPerm -a" was run on the agent endpoint. - The UNIX OS agent was started by a user other than root. - The UNIX OS agent is being upgraded to 6.22.03.00 or above. When the above conditions are met, the update fails with these symptoms: - tacmd getdeploystatus on the management server reports that the request did not succeed. - tacmd listsystems on the management server reports that the agent is still running at the prior level. - cinfo in the agent endpoint reports that the agent is still running at the prior level. - cinfo in the agent endpoint reports that Java RunTime Environment (JRE) is newer than before the failed upgrade attempt. Specifically, the JRE level shows 05.11.01.00 or higher in cinfo output. - The file CANDLEHOME/tmaitm6/agentdepot/stdout.txt on the agent endpoint contains this error message: KCIIN2374E ERROR - could not access CANDLEHOME |
IBM Tivoli Enterprise Monitoring Server APARs | |
IV00681 | When monitoring log events from an agent, and the event rate is very high, multiple events can be merged into a single buffer by proxy before they can be sent to the data server. If the event rate is high enough, the buffer can grow large enough to cause a monitoring server crash. |
IV03425 | The remote monitoring server crashes after a situation used by a policy is restarted. The restart is issued after the remote monitoring server re-establishes its lost connection to the HUB. The situation had been updated since the remote monitoring server's connection had been lost and, upon reconnecting, needed to update its tables to reflect these changes. When the situation is updated at the remote monitoring server, its current rules are deleted making it impossible to restart. When the policy's thread wakes up to check the status of the situation, the monitoring server crashes. |
IV04070 | The monitoring server fails during startup when a malformed situation is defined and "autostart = yes" is specified. |
IV05547 | SOAP queries to agents connected to the remote monitoring server do not return data if <target> is not specified in the query. A query like this one: <CT_Get> <userid>sysadmin</userid> <password>xxxxxx</password> <object>NT_Logical_Disk</object> </CT_Get> will return data only for agents connected to the Hub monitoring server and not for the agents connected to the remote monitoring server. |
IV07358 | Backslashes in Take Action commands are not displayed or handled correctly when the command is executed at the monitoring server. This problem does not occur when the Take Action command is executed at agents. This problem occurs when the non-Windows system is set to use codepage 943. This problem does not occur on Windows. |
IV07524 | An abend can occur on either the primary or secondary management servers during FTO stage II when a record in the TEMPLATE table contains a blank value for the NAME column. |
IV07550 | The hub monitoring server RAS1 log contains thousands of messages stating "Sequence number overflow, reusing 999". Hundreds or thousands of these messages occur every second. These messages appear just after the monitoring server start. The problem is caused by the monitoring server heartbeat being issued as quickly. This occurs when the QOMEGAMON_ONLINE situation has an interval of less than one minute. The situation does not have to be started for this problem to occur. |
IV09873 | SQL requests to the management server can result in a race condition during the freeing of stale connection handles. Thus, two threads can be using a connnection handle that has already been released. |
IZ87546 | When displaying "Situation Event Results" at the portal client, the initial situation values are empty. The situation event was replaced in the history table with another unrelated situation event. |
IZ94783 | Need ability to update agent framework components remotely without updating the OS agent already installed. |
IZ99471 | Warehousing of large short term history files is slow. Performance may be impacted if there is a large backlog after an outage. The short term history file upload is slow for large files because the file is read from the start rather than where it was in the file prior to the outage. This avoids reading the same exported rows multiple times in the short term history file. |
IBM Tivoli Enterprise Portal Client APARs | |
IV02306 | In the portal client situation editor, the distribution tab upon creation will display *HUB. When the situation editor is closed and then reopened, the situation distribution tab will not only show *HUB but the HUB managed system name as well. This occurs because the HUB sends two events to the portal server indicating the situation has been distributed to both *HUB and the HUB managed system name. |
IV03626 | Changing the position of an attribute column in the portal client's situation editor does not persist if a column is moved and the situation editor is closed and reopened. |
IV03954 | It is possible to specify a threshold based on a value being equal to an enumerated constant. If this is done for a scaled attribute the matching values will not be highlighted. Post filter expressions of this kind will not work either. |
IBM Tivoli Enterprise Portal Server APARs | |
IV06458 | itmcmd execute cq tmsdla.sh script failed with following error itmcmd execute cq tmsdla.sh Starting user script... .... No errors during run. Results in: /opt/IBM/ITM/aix536/cq/bin/tmsdla/TMSDISC620-xxxxxxx.xml Error: Failed tcom.ibm.cdb.bulk.exceptions.CdbBulkParseExceptiono parse input idml book at com.ibm.cdb.bulk.CdbIdmlObjects.parse(CdbIdmlObjects.java:195) at com.ibm.cdb.bulk.CdbIdmlObjects.parse(CdbIdmlObjects.java:225) at com.ibm.tivoli.monitoring.tmsdla.CleanRelationshipsInIDMLBook.clean(CleanRelationshipsInIDMLBook.java:119) at com.ibm.tivoli.monitoring.tmsdla.TmsDla.run(TmsDla.java:395) at com.ibm.tivoli.monitoring.tmsdla.TmsDla.main(TmsDla.java:164) Ending tmsdla.sh at .... This problem could happen on any platform after Fix Pack 4. |
IV06921 | To connect to a user defined portal server interface, the interface reference (IOR) file name must be specified as part of the URL used to connect to the portal server, for example, http://hostname:1920///cnp/client/?ior=external.ior where <external> is the name you gave for your KFW_INTERFACE. When using this URL, the portal client login dialog will be displayed, but after entering the user ID and password, the dialog displays an error: "KFWITM392E Internal error occurred during logon." |
IBM Tivoli Universal Agent APARs | |
IV04685 | Universal Agent subnodes may fail to come online during startup due to the internal data structure not being thread safe. This exposure only exists during the startup phase of Universal Agent when the internal data structure is being populated. |
IV07671 | The Universal Agent's API Data Provider was not checking for the receipt of a SetSourceName request. Instead, the API Data Provider is checking for a BeginInput request. The data provider returns an 'out of sequence' error condition to API client program since it receives SetSourceName instead of BeginInput. |
Remote Deploy APARs | |
IV06661 | The OS Agent's configuration is contained within a file whose name is constructed using the machine's hostname. When the OS Agent starts or when the OS Agent's configuration is updated, this file is read to obtain the current configuration data. If the name of the machine is changed after the OS Agent is installed without changing the OS Agent's configuration file name to match, the file cannot be located when an attempt is made to read the file. |
IV06856 | The cleardeploystatus command is issued to all remote monitoring servers, including ones that are on z/OS. However, because z/OS does not support remote reploy, the command request received by the z/OS remote monitoring server is rejected. The rejected command causes the hub monitoring server to interpret the remote monitoring server response as failure to locate matching entries rather than a connection failure. |
IV07417 | Specifying an illegal syntax for addSystem or ConfigureSystem configuration parameter will result in a memory access violation, abnormally terminating the monitoring server. |
Summarization and Pruning Agent APARs | |
IV02111 | Shift and vacation setting data aggregates with duplicate rows for shift 1 and shift 2 entries. |
itmpatch APARs | |
IV02302 | Patch checking is done at the release level, and not modification level, since the original design assumption was that all Fix Packs and Interim Fixes were cumulative. |
Date | Description of change |
---|---|
December 14, 2011 | Original version |
Problems (APARS) fixed
IV00112 IV00113 IV00124 IV00146 IV00167 IV00279 IV00340 IV00440 IV00522 IV00655 IV00697 IV00722 IV00749 IV00769 IV00783 IV00799 IV00828 IV01047 IV01050 IV01053 IV01055 IV01156 IV01391 IV01532 IV01588 IV01608 IV01632 IV01719 IV01794 IV02101 IV02109 IV02200 IV02222 IV02271 IV02280 IV02284 IV02285 IV02305 IV02711 IV02713 IV02726 IV02781 IV02818 IV02866 IV02871 IV03075 IV03104 IV03216 IV03341 IV03365 IV03379 IV03405 IV03411 IV03416 IV03417 IV03430 IV03687 IV03695 IV04110 IV04679 IV06268 IV06583 IV06829 IV06842 IV06844 IV06896 IV06922 IV06977 IV07415 IV07486 IV07600 IZ18659 IZ23064 IZ36593 IZ42673 IZ42859 IZ45226 IZ45235 IZ45531 IZ45666 IZ47904 IZ47946 IZ48289 IZ48347 IZ50036 IZ50147 IZ50390 IZ50476 IZ50902 IZ50995 IZ51078 IZ51232 IZ51285 IZ51485 IZ52298 IZ52380 IZ52588 IZ52599 IZ52776 IZ52777 IZ52960 IZ53199 IZ53277 IZ53689 IZ53834 IZ53960 IZ53975 IZ53978 IZ53981 IZ54250 IZ54598 IZ54659 IZ54673 IZ54715 IZ54737 IZ54795 IZ54823 IZ55246 IZ55248 IZ55249 IZ55292 IZ55386 IZ55508 IZ55590 IZ55754 IZ55766 IZ55790 IZ56002 IZ56036 IZ56142 IZ56255 IZ56407 IZ56420 IZ56669 IZ56818 IZ57078 IZ57602 IZ57703 IZ57725 IZ57975 IZ58055 IZ58398 IZ58733 IZ58742 IZ58877 IZ58988 IZ59242 IZ59291 IZ59326 IZ59327 IZ59449 IZ59540 IZ59983 IZ60114 IZ60115 IZ60581 IZ60739 IZ61158 IZ61631 IZ62255 IZ62494 IZ62801 IZ63115 IZ63130 IZ64132 IZ64618 IZ64708 IZ64732 IZ64799 IZ64835 IZ64836 IZ64872 IZ64873 IZ64879 IZ64888 IZ64923 IZ65395 IZ65561 IZ66531 IZ66666 IZ66713 IZ67267 IZ67343 IZ67394 IZ67429 IZ67439 IZ67443 IZ67733 IZ67857 IZ67955 IZ68031 IZ68104 IZ68162 IZ68312 IZ68368 IZ68369 IZ68376 IZ68466 IZ68467 IZ68471 IZ68492 IZ68560 IZ68584 IZ68610 IZ68665 IZ68708 IZ68748 IZ69015 IZ69056 IZ69060 IZ69074 IZ69101 IZ69119 IZ69455 IZ69621 IZ69740 IZ69796 IZ69970 IZ70095 IZ70119 IZ70183 IZ70282 IZ70401 IZ70448 IZ70789 IZ70814 IZ70818 IZ70928 IZ71021 IZ71067 IZ71077 IZ71079 IZ71085 IZ71110 IZ71222 IZ71237 IZ71695 IZ71739 IZ71871 IZ71873 IZ72074 IZ72139 IZ72140 IZ72224 IZ72275 IZ72283 IZ72313 IZ72417 IZ72491 IZ72542 IZ72574 IZ72679 IZ72914 IZ72916 IZ73032 IZ73050 IZ73071 IZ73074 IZ73157 IZ73172 IZ73219 IZ73237 IZ73248 IZ73268 IZ73365 IZ73414 IZ73490 IZ73542 IZ73633 IZ73700 IZ73759 IZ73996 IZ73999 IZ74000 IZ74023 IZ74118 IZ74120 IZ74138 IZ74140 IZ74143 IZ74158 IZ74289 IZ74386 IZ74480 IZ74608 IZ74698 IZ74752 IZ74811 IZ74812 IZ74848 IZ74892 IZ74912 IZ75033 IZ75037 IZ75046 IZ75055 IZ75071 IZ75134 IZ75138 IZ75146 IZ75147 IZ75148 IZ75153 IZ75174 IZ75211 IZ75222 IZ75223 IZ75225 IZ75226 IZ75227 IZ75228 IZ75229 IZ75244 IZ75245 IZ75246 IZ75285 IZ75286 IZ75287 IZ75288 IZ75289 IZ75290 IZ75297 IZ75298 IZ75365 IZ75475 IZ75508 IZ75529 IZ75534 IZ75538 IZ75552 IZ75554 IZ75576 IZ75643 IZ75702 IZ75719 IZ75728 IZ75750 IZ75775 IZ75805 IZ75850 IZ75855 IZ75871 IZ75912 IZ75991 IZ75994 IZ76004 IZ76026 IZ76027 IZ76031 IZ76032 IZ76059 IZ76089 IZ76102 IZ76188 IZ76258 IZ76288 IZ76336 IZ76353 IZ76364 IZ76367 IZ76375 IZ76377 IZ76393 IZ76410 IZ76429 IZ76431 IZ76521 IZ76559 IZ76616 IZ76635 IZ76761 IZ76819 IZ76838 IZ76907 IZ76984 IZ77107 IZ77114 IZ77234 IZ77365 IZ77378 IZ77499 IZ77537 IZ77554 IZ77565 IZ77637 IZ77639 IZ77715 IZ77849 IZ77903 IZ77964 IZ77981 IZ78022 IZ78243 IZ78288 IZ78406 IZ78424 IZ78512 IZ78623 IZ78695 IZ78723 IZ78795 IZ79300 IZ79625 IZ79678 IZ79755 IZ79757 IZ79758 IZ79772 IZ79817 IZ79899 IZ80078 IZ80108 IZ80142 IZ80146 IZ80179 IZ80239 IZ80295 IZ80418 IZ80454 IZ80773 IZ80797 IZ80869 IZ80916 IZ80918 IZ80933 IZ80991 IZ81161 IZ81182 IZ81202 IZ81212 IZ81260 IZ81303 IZ81334 IZ81337 IZ81405 IZ81433 IZ81461 IZ81476 IZ81489 IZ81520 IZ81878 IZ82307 IZ82690 IZ82799 IZ82833 IZ82847 IZ82944 IZ83008 IZ83043 IZ83183 IZ83418 IZ83496 IZ83502 IZ83534 IZ83545 IZ83558 IZ83569 IZ83576 IZ83578 IZ83587 IZ83646 IZ83651 IZ83731 IZ84120 IZ84130 IZ84133 IZ84209 IZ84215 IZ84324 IZ84372 IZ84377 IZ84397 IZ84428 IZ84642 IZ84756 IZ84839 IZ84840 IZ84841 IZ84879 IZ84910 IZ84939 IZ84982 IZ85001 IZ85034 IZ85086 IZ85102 IZ85241 IZ85255 IZ85264 IZ85281 IZ85351 IZ85604 IZ85611 IZ85668 IZ85763 IZ85796 IZ85924 IZ85959 IZ86002 IZ86035 IZ86133 IZ86190 IZ86191 IZ86195 IZ86245 IZ86248 IZ86269 IZ86454 IZ86487 IZ86596 IZ86819 IZ86997 IZ87367 IZ87447 IZ87528 IZ87541 IZ87563 IZ87749 IZ87789 IZ87796 IZ87817 IZ87820 IZ87893 IZ88088 IZ88094 IZ88159 IZ88194 IZ88507 IZ88719 IZ88737 IZ88832 IZ88839 IZ88861 IZ88921 IZ88934 IZ88973 IZ89187 IZ89282 IZ89314 IZ89470 IZ89527 IZ89532 IZ89545 IZ89551 IZ89559 IZ89636 IZ89664 IZ89693 IZ89717 IZ89747 IZ89815 IZ89835 IZ89970 IZ90041 IZ90042 IZ90181 IZ90193 IZ90489 IZ90894 IZ91032 IZ91082 IZ91167 IZ91182 IZ91340 IZ91446 IZ91512 IZ91516 IZ91637 IZ91659 IZ91917 IZ92000 IZ92007 IZ92036 IZ92124 IZ92210 IZ92223 IZ92307 IZ92469 IZ92559 IZ92564 IZ92566 IZ92570 IZ92574 IZ92797 IZ92854 IZ93258 IZ93286 IZ93329 IZ93346 IZ93548 IZ93715 IZ93904 IZ94065 IZ94071 IZ94075 IZ94084 IZ94091 IZ94095 IZ94248 IZ94257 IZ94515 IZ94559 IZ94561 IZ94586 IZ94626 IZ94729 IZ94732 IZ94770 IZ95043 IZ95044 IZ95074 IZ95091 IZ95119 IZ95330 IZ95342 IZ95349 IZ95357 IZ95382 IZ95466 IZ95504 IZ95540 IZ95541 IZ95549 IZ95627 IZ95675 IZ95680 IZ95681 IZ95737 IZ95752 IZ95815 IZ95823 IZ95881 IZ95883 IZ95923 IZ96038 IZ96100 IZ96148 IZ96222 IZ96316 IZ96334 IZ96338 IZ96437 IZ96496 IZ96528 IZ96545 IZ96568 IZ96588 IZ96597 IZ96600 IZ96602 IZ96646 IZ96653 IZ96656 IZ96683 IZ96836 IZ96957 IZ96995 IZ97074 IZ97078 IZ97197 IZ97235 IZ97331 IZ97359 IZ97732 IZ97878 IZ97890 IZ97984 IZ97987 IZ97998 IZ98187 IZ98538 IZ98586 IZ98638 IZ98649 IZ98868 IZ98991 IZ99005 IZ99020 IZ99025 IZ99255 IZ99304 IZ99326 IZ99327 IZ99339 IZ99388 IZ99688 IZ99714 IZ99773 IZ99777 OA33356 OA33989 OA34102 OA34366 OA35495 OA35496 OA35882 OA35920 OA36000 OA36117 OA36203 OA36291 OA36402 OA36544 OA36593 OA36660