9 Database engine

The Database engine obtains data by running SQL statements against a database. It is the default engine: a monitor without General.Engine uses it.

9.1 Configuration: the [Database] section#

Parameter Description
DBDriver Mandatory. Driver: Oracle, PGsql or ODBC.
ConnectionString Mandatory. Connection string. Its format depends on the driver (section 9.2).
UserName Mandatory. Database user.
Password Mandatory. Password.
ConnectOptions Special options. The only one supported is SYSDBA (required for ASM).

The database user only needs read permission on the views queried by the XML files.

9.2 Drivers#

The DBDriver parameter selects the driver the monitor connects with. Each driver interprets the connection string in its own way, supports specific contexts and offers its own runtime data.

DBDriver Databases Contexts Connection string
Oracle Oracle (OCI) Container tnsnames.ora alias, Easy Connect or full descriptor.
PGsql PostgreSQL (libpq) Database libpq key=value pairs.
ODBC Any with an ODBC driver (SQL Server, DB2/400…) Database SQLDriverConnect string: DSN=name or DRIVER={…};…

Note.  PostgreSQL and Oracle can also be accessed through the ODBC driver, using an ODBC driver from the vendor or a third party. Even so, the native driver (PGsql or Oracle) is recommended whenever possible: it is the one that knows each database’s contexts (Database in PostgreSQL, Container in Oracle) and provides its specific runtime data, such as ORACLE.CONTAINER.

In all drivers, the bind variables of BIND queries are always written as :name. The driver converts them to each database’s syntax ($1, $2… in PostgreSQL, ? in ODBC).

9.2.1 Oracle driver#

Connection string.

Passed as is to the Oracle client, so it accepts any form the client accepts:

ConnectionString = oraprod                          # tnsnames.ora alias
ConnectionString = dbhost.example.com:1521/ORAPROD  # Easy Connect
ConnectionString = (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=dbhost)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORAPROD)))

To use aliases, the service’s TNS_ADMIN variable must point to the tnsnames.ora directory.

ConnectOptions.

SYSDBA connects with administrator privileges (required for ASM). Empty for a normal connection.

Contexts.

Container: the current container (PDB), changed with ALTER SESSION SET CONTAINER. It is only available on Oracle 12c or later, on a multitenant database and when connected to the root container (CDB$ROOT). In any other case the driver supports no contexts and ORACLE.CONTAINER is 0.

Bind variables.

Native Oracle (:name).

9.2.2 PGsql driver#

Connection string.

libpq key=value pairs separated by spaces (host, port, dbname, sslmode, connect_timeout…). The user and password are added from UserName and Password; do not put them in the string.

ConnectionString = host=pg01.example.com port=5432 dbname=postgres sslmode=require
ConnectOptions.

Not used.

Contexts.

Database: the current database. Since PostgreSQL cannot switch databases within a connection, the driver opens one connection per database used (replacing the string’s dbname) and keeps them open.

Bind variables.

Converted to $1, $2…

Warning.  Always include dbname in the connection string: the driver uses it as the default database and replaces it when switching context. The password is appended to the string without quotes, so it cannot contain spaces.

9.2.3 ODBC driver#

Connection string.

Passed to SQLDriverConnect, appending ;UID=…;PWD=… with the user and password. It can refer to a data source in odbc.ini or describe the full connection:

ConnectionString = DSN=MSSQL_PROD
ConnectionString = DRIVER={FreeTDS};SERVER=mssql.example.com;PORT=1433;DATABASE=master

To use DSN, the service’s ODBCINI variable must point to the odbc.ini file.

ConnectOptions.

Not used.

Contexts.

Database: the current database (catalogue), changed within the same connection with the ODBC attribute SQL_ATTR_CURRENT_CATALOG. It works with drivers that support it, such as the SQL Server ones.

Bind variables.

Converted to ?.

Note.  The password is appended to the string without quotes, so it cannot contain the ; character.

9.2.4 Runtime data#

Runtime data is read in <when> conditions with runtime(’NAME’) (section 9.6) and in queryexplorer with getRuntimeValue.

Name Driver Value
DATABBIX.PROGRAM_NAME all databbix, or queryexplorer in that utility.
CONN.TYPE all Database.
DB.DRIVER Oracle Oracle.
PGsql PostgreSQL.
ODBC ODBC.
DB.DATABASE Oracle Oracle.
PGsql PostgreSQL.
ODBC Product name reported by the ODBC driver (SQL_DBMS_NAME): for example, Microsoft SQL Server or DB2/400 SQL.
DB.VERSION Oracle Server version, in the form major.minor.update.
PGsql The server’s server_version setting, for example 15.4.
ODBC Version reported by the ODBC driver (SQL_DBMS_VER).
ORACLE.CONTAINER Oracle 1 if the Container context is available; 0 otherwise.

This way, the same XML file can choose the SQL by product (runtime(’DB.DATABASE’) = ’Microsoft SQL Server’), by version (major(runtime(’DB.VERSION’)) >= 12) or by whether the database is multitenant (runtime(’ORACLE.CONTAINER’) = ’1’).

9.3 SQL queries: engine-specific elements#

Besides the common elements (chapter 7), Database engine queries use:

Element / attribute Description
<sql> Statement text. Mandatory except in constant queries. Accepts ${…} variables and alternatives with <multiple> (section 9.6).
<sql context="C"> Discoveries only: statement returning the values of context C to be iterated (section 9.4.4).
select <query> attribute: FILTER or BIND. Without it, the query is basic.
<item column="…"> Result column the value comes from. Case-insensitive.
<parameter column> In FILTER queries: column compared with the parameter.
<parameter bind> In BIND queries: SQL bind variable.
<parameter context> Context the query runs in.
<default> Fixed context of the query (section 9.5).
<field column|context> In discoveries: where the macro value comes from.
<context> At <Queries> level: declaration of a context.

Note.  Always give computed columns an explicit alias (as name) and use that alias in column.

9.4 Query types#

The examples in this section show, for each type, the result of the SQL statement at a given moment and the value each Zabbix key obtains from it.

9.4.1 Basic query#

The statement returns exactly one row. Each <item> takes the value of one column.

<query name="PGWriter" module="PG">
    <sql><![CDATA[
        select checkpoints_timed, checkpoints_req from pg_stat_bgwriter
        ]]>
    </sql>
    <item name="tez.pg.checkpointtimed" column="checkpoints_timed"/>
    <item name="tez.pg.checkpointreq"   column="checkpoints_req"/>
</query>

Statement result

checkpoints_timed checkpoints_req
1842 37

Values obtained

Key Value
tez.pg.checkpointtimed 1842
tez.pg.checkpointreq 37

If the statement returns zero rows or more than one, all the query’s items become not supported.

9.4.2 Filter query (select="FILTER")#

The statement returns many rows. Each <parameter> with column links a key parameter with a column, and each item takes the row whose columns match its key parameters. The statement runs only once for all items, so this is the most efficient way to watch collections of objects (tablespaces, databases, jobs…).

<query name="PGSessions" module="PG" select="FILTER">
    <sql><![CDATA[
        select datname,
               count(st.pid) as n,
               sum(case st.state when 'active' then 1 else 0 end) as active
        from pg_database db
             left outer join pg_stat_activity st using (datname)
        where datname not like 'template%'
        group by datname
        ]]>
    </sql>
    <parameter column="datname" order="1" type="STRING"/>
    <item name="tez.pg.sessions"       column="n"/>
    <item name="tez.pg.sessionsactive" column="active" default="0"/>
</query>

Statement result

datname n active
postgres 3 1
sales 12 4
hr 0 0

Values obtained

Key Selected row Value
tez.pg.sessions[sales] sales 12
tez.pg.sessionsactive[sales] sales 4
tez.pg.sessions[hr] hr 0
tez.pg.sessions[purchasing] none not supported
tez.pg.sessionsactive[purchasing] none 0 (default attribute)
tez.pg.sessions[Sales] none not supported

The first four keys are resolved with a single execution of the statement. The comparison is exact: it is case-sensitive (Sales does not match sales) and only trims spaces at the ends of the parameter. If a column is space-padded (char type), trim it in the SQL with trim().

Several filter columns#

With several filter parameters, the key has one parameter for each, in the order given by order, and the row must match all of them:

<query name="MSStateRepl" module="MSSQL" select="FILTER">
    <sql><![CDATA[ exec sp_replmonitorhelppublication ]]></sql>
    <default context="Database" value="distribution"/>
    <parameter column="publisher_db" order="1" type="STRING"/>
    <parameter column="publication"  order="2" type="STRING"/>
    <item name="tez.mssql.repl.status" column="status"/>
</query>

Statement result

publisher_db publication status
sales pub_orders 3
sales pub_customers 2
purchasing pub_orders 6

Values obtained

Key Value
tez.mssql.repl.status[sales,pub_orders] 3
tez.mssql.repl.status[purchasing,pub_orders] 6
tez.mssql.repl.status[sales] not supported (missing parameter)

Numeric parameters#

With type="INT", the parameter and the column are compared as numbers. This way, the key does not have to reproduce the exact format of the column:

<parameter column="job_id" order="1" type="INT"/>
<item name="local.job.status" column="status"/>

Statement result and values obtained

job_id status Key Value
7 OK local.job.status[7] OK
42 ERROR local.job.status[042] ERROR

With type="STRING", the key local.job.status[042] would not find the row, because the text 042 differs from 42.

9.4.3 Bind variable query (select="BIND")#

The key parameter is passed to the statement as a bind variable. The statement runs once for each distinct key and must return exactly one row.

<query name="PGParameter" module="PG" select="BIND">
    <sql><![CDATA[
        select setting from pg_settings where name = :name
        ]]>
    </sql>
    <parameter bind=":name" order="1" type="STRING"/>
    <item name="tez.pg.param" column="setting"/>
</query>

Executions and values obtained

Key Value of :name Value
tez.pg.param[max_connections] ’max_connections’ 200
tez.pg.param[work_mem] ’work_mem’ 4096
tez.pg.param[no_such] ’no_such’ not supported (0 rows)

The variable is always written as :name, whatever the driver (section 9.2). With type="INT", the value is bound as a number.

Warning.  Since it runs once per key, a BIND query is much more expensive than a FILTER query when there are many keys. Use it only when the set of possible values is very large and only a few are queried.

9.4.4 Discovery query#

Each result row is a discovered entity, and each <field> takes the value of a column (column) or of a context (context).

<query name="TSDiscover" module="DB" discovery="yes">
    <sql><![CDATA[
        select tablespace_name, contents
        from dba_tablespaces where status = 'ONLINE'
        ]]>
    </sql>
    <item name="tez.ora.tspace.discovery"/>
    <field name="TSNAME"   column="tablespace_name"/>
    <field name="CONTENTS" column="contents"/>
</query>

Statement result

tablespace_name contents
SYSTEM PERMANENT
USERS PERMANENT
TEMP TEMPORARY

Value of the item tez.ora.tspace.discovery:

{"data":[
  {"{#TSNAME}":"SYSTEM", "{#CONTENTS}":"PERMANENT"},
  {"{#TSNAME}":"USERS",  "{#CONTENTS}":"PERMANENT"},
  {"{#TSNAME}":"TEMP",   "{#CONTENTS}":"TEMPORARY"}
]}

In Zabbix, the prototype tez.ora.tspace.pctfree[{#TSNAME}] creates the items tez.ora.tspace.pctfree[SYSTEM], [USERS] and [TEMP], resolved by a FILTER query with parameter column="tablespace_name".

Discoveries only accept context parameters.

Discovery across contexts#

A discovery can iterate over all the values of a context: for example, list the filegroups of all the databases of an SQL Server. To do so, add an <sql> with a context attribute that returns, in its first column, the values to iterate. The main statement runs once in each of them:

<query name="MSDiscoverFG" module="MSSQL" discovery="yes">
    <sql context="Database"><![CDATA[
        select name from sys.databases where name not in ('model')
        ]]>
    </sql>
    <sql><![CDATA[
        select name from sys.filegroups
        ]]>
    </sql>
    <item name="tez.mssql.fg.discovery"/>
    <field name="DBNAME" context="Database"/>
    <field name="FGNAME" column="name"/>
</query>

Statement results

Context statement Main statement in each database
name database name
master master PRIMARY
sales sales PRIMARY
sales HISTORY

Value of the item tez.mssql.fg.discovery:

{"data":[
  {"{#DBNAME}":"master", "{#FGNAME}":"PRIMARY"},
  {"{#DBNAME}":"sales",  "{#FGNAME}":"PRIMARY"},
  {"{#DBNAME}":"sales",  "{#FGNAME}":"HISTORY"}
]}

Context values that cannot be entered (for example, an offline database) are skipped and logged.

9.5 Contexts#

A context is a connection state that can be changed between queries, such as the current database in SQL Server or PostgreSQL or the container (PDB) in Oracle multitenant. This way, a single monitor with a single connection can query several databases on the same server. Contexts only exist in the Database engine.

9.5.1 Declaration#

Contexts are declared at <Queries> level:

<environment name="PGAdminDB" default="postgres"/>
<context name="Database" module="PG" default="${PGAdminDB}"/>
Attribute Description
name Mandatory. Context name. The driver must know how to change it; the usual names are Database (SQL Server, PostgreSQL) and Container (Oracle multitenant).
module Mandatory. Module it belongs to.
default Mandatory. Default value. Accepts ${…} variables.

9.5.2 Use in queries#

There are three ways to choose the context of a query:

  1. Default. If the query says nothing, it runs with the context’s default value.

  2. Fixed for the query, with <default>:

    <default context="Database" value="msdb"/>

    If value is __IGNORE__ (for example, through an unset environment variable), the element is ignored and the default value is kept.

  3. Taken from the item key, with a context parameter:

    <parameter context="Database" order="1"/>

9.5.3 Example: a filter inside each database#

With a context parameter, the first key parameter selects the database and the following ones select the row:

<context name="Database" module="MSSQL" default="master"/>

<query name="MSFGSpace" module="MSSQL" select="FILTER">
    <sql><![CDATA[
        select g.name as filegroup_name,
               sum(cast(fileproperty(f.name, 'SPACEUSED') as bigint) * 8192) as used
        from sys.database_files f
             inner join sys.filegroups g on g.data_space_id = f.data_space_id
        group by g.name
        ]]>
    </sql>
    <parameter context="Database" order="1"/>
    <parameter column="filegroup_name" order="2" type="STRING"/>
    <item name="tez.mssql.fgspace.used" column="used"/>
</query>

Statement result in each database

database filegroup_name used
master PRIMARY 5242880
sales PRIMARY 73400320
sales HISTORY 524288000

Values obtained

Key Context Value
tez.mssql.fgspace.used[sales,HISTORY] sales 524288000
tez.mssql.fgspace.used[sales,PRIMARY] sales 73400320
tez.mssql.fgspace.used[master,PRIMARY] master 5242880

Databbix groups executions by context: it enters sales only once, runs the statement and resolves the first two keys with the same result. If it cannot enter a context, it marks the items of that context as not supported. If the failing context is the default one and ConnLostOnDefaultCtxError = yes, the connection is considered lost and re-established.

9.6 Alternative SQL depending on the database#

The same XML file can serve different database products or versions if the SQL text is chosen at run time, with <multiple>:

<sql>
  <multiple>
    <when condition="runtime('DB.DATABASE') = 'Oracle'"><![CDATA[
        select count(*) as n from ${SvmSchema}.f986110
        where trim(jcfndfuf2) = :jobname
        ]]>
    </when>
    <when condition="runtime('DB.DATABASE') = 'Microsoft SQL Server'"><![CDATA[
        select count(*) as n from ${SvmSchema}.f986110
        where rtrim(jcfndfuf2) = :jobname
        ]]>
    </when>
    <when><![CDATA[
        -- no condition: used if none of the above applies
        select count(*) as n from ${SvmSchema}.f986110
        where jcfndfuf2 = :jobname
        ]]>
    </when>
  </multiple>
</sql>

The first <when> whose condition holds is used. A <when> without a condition always holds, so it must be the last one. If no condition holds, the query fails.

Conditions are expressions that can use these functions:

Function Description
runtime(’X’) Runtime data of the active connection, for example DB.DATABASE or DB.VERSION (section 9.2.4).
environment(’X’) Value of an environment variable.
major(’X.Y.Z’) First number of a version, as a number.
toscalar(’X’) Converts a text into a number.

They accept comparison operators (=, !=, <, >…) and logical operators (and, or, not). The runtime data available with each driver can be listed with the getRuntimeNames command of queryexplorer.

9.7 Configuration examples#

9.7.1 Example: Oracle#

[General]
Name = ORAPROD

[Zabbix]
Server = zabbix.example.com
Port = 10051
PassivePort = 10153

[Data]
Module = DB
Module = Instance

[Database]
DBDriver = Oracle
ConnectionString = oraprod          # tnsnames.ora alias
UserName = zabbix
Password = secret

9.7.2 Example: Oracle multitenant#

[General]
Name = ORACDB
QueryFile = /etc/databbix/query-mt.xml

[Zabbix]
Server = zabbix.example.com
Port = 10051

[Data]
Module = MT

[Database]
DBDriver = Oracle
ConnectionString = oracdb
UserName = c\043\043zabbix          # c##zabbix
Password = secret

9.7.3 Example: ASM#

[General]
Name = asm

[Zabbix]
Server = zabbix.example.com
Port = 10051

[Data]
Module = ASM

[Database]
DBDriver = Oracle
ConnectionString = asm
ConnectOptions = SYSDBA
UserName = sys
Password = secret

9.7.4 Example: PostgreSQL#

[General]
Name = PGPROD
QueryFile = /etc/databbix/query-pg.xml
QueryFile = /etc/databbix/query-pg-10.xml
Environment = PGAdminDB=postgres

[Zabbix]
Server = zabbix.example.com
Port = 10051

[Data]
Module = PG

[Database]
DBDriver = PGsql
ConnectionString = host=pg01.example.com port=5432 dbname=postgres
UserName = zabbix
Password = secret

9.7.5 Example: SQL Server through ODBC#

First, define the data source in /etc/databbix/odbc.ini:

[MSSQL_PROD]
Driver=FreeTDS
Description=Production SQL Server
Server=mssql.example.com
Port=1433
Database=master

Then the monitor refers to it by name:

[General]
Name = MSSQLPROD
QueryFile = /etc/databbix/query-mssql.xml

[Zabbix]
Server = zabbix.example.com
Port = 10051

[Data]
Module = MSSQL

[Database]
DBDriver = ODBC
ConnectionString = DSN=MSSQL_PROD
UserName = zabbix
Password = secret