Data Collector Log Files
Before resorting to examining Data Collector logs, use the following System Administration reports to check collection status:
• Collection Message Summary
• Data Collection Schedule Summary
• Data Collector Status Summary
• File Analytics Collection Status
You can also view live collection status using the Collection Status page on the Admin tab.
The data collection process logs activity to provide additional processing details that support troubleshooting. Use the Admin > Advanced > Support Tools feature to request a support package that contains Data Collector Server files. To identify specific files, it may be helpful to have some knowledge of the logging structure and naming conventions.
Beginning with APTARE Release Version 10, the data collection logging structure has been reorganized to make it easier for support and developers to find relevant information. Refer to the following sections to understand this structure.
Data Collector Log File Organization
Data Collector logs are organized into two directory hierarchies:
Checkinstall and Validation Probes
• /opt/aptare/mbs/logs/validation/
• C:\Program Files\Aptare\mbs\logs\validation\
Scheduled Probes (such as running as a service)
• /opt/aptare/mbs/logs/scheduled/
• C:\Program Files\Aptare\mbs\logs\scheduled\
Within this directory structure, there is a
framework sub-directory. The
framework sub-directory is used in the beginning of the collection process, before the Data Collector type has been identified. Once IT Analytics knows the type of collector (for example, EMC Isilon), logging is recorded in the main
scheduled directory using the naming convention described in
Data Collector Log File Naming Conventions.
Data Collector Log File Naming Conventions
Within the directory structure described in
Data Collector Log File Organization, log files have the following naming convention:
<vendor.product>/<subsystem>#META_<ID>/Probe.log
For example, an EMC Isilon probe from checkinstall would result in a file name similar to:
/opt/aptare/mbs/logs/validation/emc.isilon/alphpeifr023#META_EA1BA380E95F73C72A72B3B0792111E5/IsilonClusterDetailProbe.log
Some collectors may have a period of time when they are not processing a specific subsystem. For those periods, logging will occur in an aggregate log file similar to:
/opt/aptare/mbs/logs/validation/emc.isilon/#META_EA1BA380E95F73C72A72B3B0792111E5/IsilonClusterDetailProbe.log
Sample Vendor.Product Naming Convention
Examples of vendor.product folder names within this directory structure include:
cisco.cisco
commvault.simpana
dell.compellent
emc.avamar
emc.clariion
generic.host (Valid for a host resources discovery policy)
hp.3par
symantec.bue
Additionally, each Java Virtual Machine (JVM) creates its own logging file(s) when starting up. This is necessary because multiple processes logging to the same file could overwrite each other’s log messages. These log files can be found in the framework sub-directory, as described in
Data Collector Log File Organization. See also,
Checkinstall Log.
Log File Names Based on Data Collector Generation
Throughout the APTARE IT Analytics life cycle, some collectors have been upgraded to include new functionality and new Data Collectors have been designed with an improved architecture. The log file naming convention depends on the generation of the Data Collector that you are using. You do not need to know the generation of the Data Collector to find relevant log files.
When running a scheduled data collection, the following log files are created, depending on the generation of the Data Collector. For technical reasons, the following naming conventions are used for data collection logs:
• older-generation collectors follow the convention, {scheduled,validation}/vendor.product/#META_ or {scheduled,validation}/vendor.product/EVENT_, where EVENT_ or META_ is prepended to the collector policy ID; for example, META_CA6EC7685A9E6330EC3BBFC0DD4811E4.
• newer-generation collectors share a single file named with the main collector ID, for example,
{scheduled,validation}/vendor.product/#<Collector_PolicyID>
Several log file names include a specific ID. This ID can be found in a System Administration report,
Data Collection Schedule Summary. See also,
Find the Event/Meta Collector ID.
Examples
• /opt/aptare/mbs/logs/scheduled/dell.compellent/#META_EA1BA380E95F73C72A72B3B0792111E5/META_EA1BA380E95F73C72A72B3B0792111E5.log
• /opt/aptare/mbs/logs/scheduled/emc.avamar/#HQBackupCollector/HQBackupCollector.log
Checkinstall Log
The checkinstall process produces its own log file, but in most cases, there is very little to report in this log.
For example, the checkinstall creates:
/opt/aptare/mbs/logs/validation/framework/#checkinstall/checkinstall.log
Test Connection Log
When you initiate a Test Connection action from within a Data Collector policy, a TestConnection.log file captures the steps and their status.
• /opt/aptare/mbs/logs/validation/<vendor.product>/#TestConnection/TestConnection.log
• C:\Program Files\Aptare\mbs\logs\validation\<vendor.product>\#TestConnection\TestConnection.log
Log File Naming Convention by Collected System
The log file names will have one of the following prefixes substituted for the <policyID>:
• #EVENT_<policyID>
• #META_<policyID>
• #<policyID>
Example: scheduled\legato.nw\#META_D922ACBCCFFA2933A301A530A0E011E4
Note: Some collectors may have both a #META_ and an #EVENT_ log file.
IT Analytics Product | Collected System | Where to find the logs (Linux syntax) |
Backup Manager | Commvault Simpana | scheduled/commvault.simpana/#<policyID> validation/commvault.simpana/#<policyID> |
| EMC Avamar | scheduled/emc.avamar/#<policyID> validation/emc.avamar/#<policyID> |
| EMC Data Domain Backup | scheduled/emc.datadomain/#<policyID> validation/emc.datadomain/#<policyID> |
| EMC NetWorker | scheduled/legato.nw/#<policyID> validation/legato.nw/#<policyID> |
| HP Data Protector | scheduled/hp.dp/#<policyID> validation/hp.dp/#<policyID> |
| IBM Spectrum Protect (TSM) | scheduled/ibm.tsm/#<policyID> validation/ibm.tsm/#<policyID> |
| Oracle Recovery Manager (RMAN) | scheduled/oracle.rman/#<policyID> validation/oracle.rman/#<policyID> Note: In addition to the hostname#<productId>/ directories, RMAN also creates directories for each INSTANCE.SCHEMA from which it collects. |
| Veeam Backup & Replication | scheduled/veeam.backupandreplication/#<policyID> validation/veeam.backupandreplication/#<policyID> |
| Veritas Backup Exec | scheduled/symantec.bue/#<policyID> validation/symantec.bue/#<policyID> |
| Veritas NetBackup | scheduled/symantec.netbackup/#<policyID> validation/symantec.netbackup/#<policyID> |
| | |
Capacity Manager | Dell Compellent | scheduled/dell.compellent/#<policyID> validation/dell.compellent/#<policyID> |
| EMC Data Domain Storage | scheduled/emc.datadomain/#<policyID> validation/emc.datadomain/#<policyID> |
| EMC Isilon | scheduled/emc.isilon/#<policyID> validation/emc.isilon/#<policyID> |
| EMC Symmetrix | scheduled/emc.symmetrix/#<policyID> validation/emc.symmetrix/#<policyID> |
| EMC VNX (Celerra) | scheduled/emc.celerra/#<policyID> validation/emc.celerra/#<policyID> |
| EMC VNX (CLARiiON) | scheduled/emc.clariion/#<policyID> validation/emc.clariion/#<policyID> |
| EMC VPLEX | scheduled/emc.vplex/#<policyID> validation/emc.vplex/#<policyID> |
| EMC XtremIO | scheduled/emc.xtremio/#<policyID> validation/emc.xtremio/#<policyID> |
| HP 3PAR | scheduled/hp.3par/#<policyID> validation/hp.3par/#<policyID> |
| HP EVA | scheduled/hp.eva/#<policyID> validation/hp.eva/#<policyID> |
| Hitachi Block | scheduled/hds.hds/#<policyID> validation/hds.hds/#<policyID> |
| Hitachi HCP | scheduled/hds.hcp/#<policyID> validation/hds.hcp/#<policyID> |
| Hitachi NAS | scheduled/hitachi.hnas/#<policyID> validation/hitachi.hnas/#<policyID> |
| Huawei Ocean Stor | scheduled/huawei.oceanstor/#<policyID> validation/huawei.oceanstor/#<policyID> |
| IBM Enterprise | scheduled/ibm.ent/#<policyID> validation/ibm.ent/#<policyID> |
| IBM SVC | scheduled/ibm.svc/#<policyID> validation/ibm.svc/#<policyID> |
| IBM XIV | scheduled/ibm.xiv/#<policyID> validation/ibm.xiv/#<policyID> |
| INFINIDAT InfiniBox | scheduled/infinidat.infinibox/#<policyID> validation/infinidat.infinibox/#<policyID> |
| NetApp 7-Mode | scheduled/netapp.netapp/#<policyID> validation/netapp.netapp/#<policyID> |
| NetApp Cluster | scheduled/netapp.netapp/#<policyID> validation/netapp.netapp/#<policyID> |
| NetApp E-Series | scheduled/netapp.netapp/#<policyID> validation/netapp.netapp/#<policyID> |
| Pure Storage FlashArray | scheduled/purestorage.flasharray/#<policyID> validation/purestorage.flasharray/#<policyID> |
| | |
Cloud | Amazon Web Services | scheduled/amazon.webservices/#<policyID> validation/amazon.webservices/#<policyID> |
| Microsoft Azure | scheduled/microsoft.azure/#<policyID> validation/microsoft.azure/#<policyID> |
| OpenStack Ceilometer | scheduled/openstack.ceilometer/#<policyID> validation/openstack.ceilometer/#<policyID> |
| OpenStack Swift | scheduled/openstack.swift/#<policyID> validation/openstack.swift/#<policyID> |
| | |
Fabric Manager | Brocade | scheduled/brocade.brocadeswitch/#<policyID> validation/brocade.brocadeswitch/#<policyID> |
| Brocade Zone Alias | scheduled/brocade.brocade/#<policyID> validation/brocade.brocade/#<policyID> |
| Cisco | scheduled/cisco.ciscoswitch/#<policyID> validation/cisco.ciscoswitch/#<policyID> |
| Cisco Zone Alias | scheduled/cisco.cisco/#<policyID> validation/cisco.cisco/#<policyID> |
| | |
File Analytics | Hosts | scheduled/generic.fa/#<policyID> validation/generic.fa/#<policyID> |
| | |
Host Collection | Hosts | scheduled/generic.host/#<policyID> validation/generic.host/#<policyID> |
| | |
Virtualization Manager | IBM VIO | scheduled/cisco.cisco/#<policyID> validation/cisco.cisco/#<policyID> |
| VMware | scheduled/vmware.esx/ validation/vmware.esx/ |
General Data Collector Log Files
Locations in this table represent the default locations, but these may have been modified for your environment.
Table 1 General Data Collector Logs
Log File Name | Default Location | Description | Component |
start_watchdog.log | /opt/aptare/mbs/logs | Logging for the high-level management of the Watchdog component. Management includes startup, shutdown, and initialization. | Watchdog |
wrapper.log | C:\Program Files\Aptare\mbs\logs |
watchdog.log | C:\Program Files\Aptare\mbs\logs /opt/aptare/mbs/logs | Detailed logging for the Watchdog component. Logs all the functions Watchdog performs including downloading new configuration files. |
upgradeMgr.log upgrade_<version>.log | C:\Program Files\Aptare\upgrade\upgradeManager\logs /opt/aptare/upgrade/upgradeManager/logs | Detailed logging for the Data Collector Upgrade Manager. <version> refers to the version of the aptare.jar to which the Data Collector is being upgraded. Note: The logs directory will not exist for a new installation of the Data Collector. The logs directory will only be created once the collector goes through an aptare.jar upgrade cycle. | Upgrade Manager |
Find the Event/Meta Collector ID
This ID is displayed in the Data Collection Schedule Summary, but you can also use the following script to list the IDs.
1. List the Data Collectors to get the Event/Metadata Collector ID.
Windows:
C:\Program Files\APTARE\mbs\bin\listcollectors.bat
Linux:
/opt/aptare/mbs/bin/listcollectors.sh
In the output, look for the Event Collectors section associated with the Software Home—the path that was specified when the Data Collector Policy was created.
==== Event Collectors ===
Event Collector Id: EF14CEE486DF781F312E5D40411C11E5
Active: true
Active: true
Software Home: C:\Program Files\EMC NetWorker\nsr\bin
Server Address: networker3
Domain: 100000
Sub-system/Server Instance/Device Manager Id: 110050
Schedule: 10