DBCollect - Oracle Database Info Collector

Wiki Manuals

Detailed manuals for dbcollect are available here: Dirty Cache Wiki - DBCollect/Manuals

Description

dbcollect is a metadata collection tool for Oracle databases, providing workload and config data from database hosts for:

• Dell LiveOptics
• Dell's internal Splunk-based Workload Analyzer for Oracle (used by Oracle specialists at Dell)
• LoadMaster (The advanced DB workload analyzer tool I have developed - more info TBD)

It is written in Python, and collects various OS configuration files and the output of some system commands, as well as AWR or Statspack reports for each database instance, and other database information.

The results are collected in a ZIP file named (default): /tmp/dbcollect-<hostname>.zip

For LiveOptics and Dell's Workload Analyzer, the per-database directories in the ZIP file contain all files you need to upload to the reporting tools.

For the advanced analyzer (LoadMaster), send the entire ZIP file (via secure FTP or other means).

Quick howto

• Download dbcollect: latest dbcollect version
• Move it to /usr/local/bin (if you are root) or $HOME/bin (if you are not root)
• Make it executable: chmod 755 /usr/local/bin/dbcollect
• Test if it works (run with help option): dbcollect -h
• Collect the data: dbcollect -o
• Get and upload the dbcollect ZIP datafile (/tmp/dbcollect-<hostname>.zip)

Info collected by dbcollect:

• AWR or Statspack reports for a default period of about 10 days
• Database configuration (such as version, dbname, database and system settings)
• Database file info (sizes and settings of datafiles, tempfiles, redo logs, control files etc)
• Additional database file info (backup, archive logs, flashback logs)
• Tablespace details, Segment sizes, AWR/Statspack details
• ASM disk group and disk info
• Init parameters
• Feature usage
• Linux: CPU, memory, disks, network interfaces etc
• AIX, Solaris (SPARC): CPU, memory, limited other info
• SAR: Binary SAR files (Linux) or SAR reports (UNIX) (Linux SAR provides much more detail)
• HP/UX: Experimental support
• Windows: Not yet supported (TBD)

More collectors may be added in the future.

Important note

Ideally, dbcollect collects Oracle AWR reports in HTML format as HTML can be processed (parsed) reliably. It can also generate Statspack reports. This is how dbcollect decides which reports to produce:

Oracle AWR reports require Diagnostics Pack (see Oracle doc id 1490798.1). dbcollect can not directly determine if you have this license or not - it checks the dba_feature_usage_statistics table whether "AWR Reports" has been used previously or not, and falls back to Statspack otherwise, unless the --force-awr flag has been specified. The decision flow is as follows:

1. Check if 'AWR Reports' is a feature that has been used before. If so, use AWR reports
2. If AWR usage is not detected but the --force-awr flag has been specified, use AWR reports
3. If AWR usage is not detected and no --force-awr flag is specified, check if STATSPACK is available. If so, use Statspack
4. If AWR usage is not detected, no --force-awr has been specified and no statspack reports are available, but --ignore is specified, ignore and continue to the next
5. If AWR usage is not detected, no --force-awr has been specified and no statspack reports are available, abort with an error

So if you ARE licensed for Diagnostics Pack but dbcollect complains, you can run dbcollect with the --force-awr option. DBcollect will only warn about the AWR detection and just generate the reports anyway.

** Use the --force-awr flag ONLY if you are sure you are correctly licensed! **

If you are NOT licensed for diagnostics pack then you can use Statspack instead (see section below)

More info on Oracle feature usage: Oracle Feature Usage

Non-Linux (UNIX) systems

DBCollect now supports AIX and Solaris systems, and HP-UX has experimental support. You need to have Python installed (at least version 2.6).

Windows is not (yet) supported.

Requirements

See Dirty Cache Wiki - DBCollect/Install for more details.

Known limitations and caveats

• Backup files, logs etc may no longer exist but still be reported, depending on backup catalog accuracy. Best effort.
• Very long names for files, tablespaces, disk groups etc may be truncated/wrapped
• Very large sized elements or very large amounts of objects may result in #### notation and no longer be useful. Limits have been increased to insane values so this should not be a problem
• Newer Oracle versions (20c and up) may cause unreliable numbers, not yet tested
• Oracle RAC sometimes is very slow with generating AWR reports. Known issue. Be patient. See Dirty Cache Wiki - DBCollect Troubleshooting

Safety

A lot of safety guards have been built into dbcollect. It is designed to not require root access and only writes files to the /tmp directory by default. Even if it fails for some reason, no harm can be done.

More info can be found in Dirty Cache Wiki - DBCollect Security

Source code

dbcollect is packaged as a Python "zipapp" package which is a specially prepared ZIP file. You can unzip it with a normal unzip tool. For example, listing the files in the package:

unzip -l /usr/local/bin/dbcollect

Q&A

See Dirty Cache Wiki - DBCollect FAQ

License

dbcollect is licensed under GPLv3. See "COPYING" for more info or go to GPLv3+ License Info

Author

Bart Sjerps (bart <at> dirty-cache <dot> com) - with great contributions from others