9 Database engine
The Database engine obtains data by running SQL statements against a database. It is the default engine: a monitor without General.Engine uses it.
9.1 Configuration: the [Database] section#
| Parameter | Description |
|---|---|
DBDriver |
Mandatory. Driver: Oracle, PGsql or ODBC. |
ConnectionString |
Mandatory. Connection string. Its format depends on the driver (section 9.2). |
UserName |
Mandatory. Database user. |
Password |
Mandatory. Password. |
ConnectOptions |
Special options. The only one supported is SYSDBA (required for ASM). |
The database user only needs read permission on the views queried by the XML files.
9.2 Drivers#
The DBDriver parameter selects the driver the monitor connects with. Each driver interprets the connection string in its own way, supports specific contexts and offers its own runtime data.
| DBDriver | Databases | Contexts | Connection string |
|---|---|---|---|
Oracle |
Oracle (OCI) | Container |
tnsnames.ora alias, Easy Connect or full descriptor. |
PGsql |
PostgreSQL (libpq) | Database |
libpq key=value pairs. |
ODBC |
Any with an ODBC driver (SQL Server, DB2/400…) | Database |
SQLDriverConnect string: DSN=name or DRIVER={…};… |
Note. PostgreSQL and Oracle can also be accessed through the ODBC driver, using an ODBC driver from the vendor or a third party. Even so, the native driver (PGsql or Oracle) is recommended whenever possible: it is the one that knows each database’s contexts (Database in PostgreSQL, Container in Oracle) and provides its specific runtime data, such as ORACLE.CONTAINER.
In all drivers, the bind variables of BIND queries are always written as :name. The driver converts them to each database’s syntax ($1, $2… in PostgreSQL, ? in ODBC).
9.2.1 Oracle driver#
- Connection string.
-
Passed as is to the Oracle client, so it accepts any form the client accepts:
ConnectionString = oraprod # tnsnames.ora alias ConnectionString = dbhost.example.com:1521/ORAPROD # Easy Connect ConnectionString = (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=dbhost)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORAPROD)))To use aliases, the service’s
TNS_ADMINvariable must point to thetnsnames.oradirectory. - ConnectOptions.
-
SYSDBAconnects with administrator privileges (required for ASM). Empty for a normal connection. - Contexts.
-
Container: the current container (PDB), changed withALTER SESSION SET CONTAINER. It is only available on Oracle 12c or later, on a multitenant database and when connected to the root container (CDB$ROOT). In any other case the driver supports no contexts andORACLE.CONTAINERis0. - Bind variables.
-
Native Oracle (
:name).
9.2.2 PGsql driver#
- Connection string.
-
libpq
key=valuepairs separated by spaces (host,port,dbname,sslmode,connect_timeout…). The user and password are added fromUserNameandPassword; do not put them in the string.ConnectionString = host=pg01.example.com port=5432 dbname=postgres sslmode=require - ConnectOptions.
-
Not used.
- Contexts.
-
Database: the current database. Since PostgreSQL cannot switch databases within a connection, the driver opens one connection per database used (replacing the string’sdbname) and keeps them open. - Bind variables.
-
Converted to
$1,$2…
Warning. Always include dbname in the connection string: the driver uses it as the default database and replaces it when switching context. The password is appended to the string without quotes, so it cannot contain spaces.
9.2.3 ODBC driver#
- Connection string.
-
Passed to
SQLDriverConnect, appending;UID=…;PWD=…with the user and password. It can refer to a data source inodbc.inior describe the full connection:ConnectionString = DSN=MSSQL_PROD ConnectionString = DRIVER={FreeTDS};SERVER=mssql.example.com;PORT=1433;DATABASE=masterTo use
DSN, the service’sODBCINIvariable must point to theodbc.inifile. - ConnectOptions.
-
Not used.
- Contexts.
-
Database: the current database (catalogue), changed within the same connection with the ODBC attributeSQL_ATTR_CURRENT_CATALOG. It works with drivers that support it, such as the SQL Server ones. - Bind variables.
-
Converted to
?.
Note. The password is appended to the string without quotes, so it cannot contain the ; character.
9.2.4 Runtime data#
Runtime data is read in <when> conditions with runtime(’NAME’) (section 9.6) and in queryexplorer with getRuntimeValue.
| Name | Driver | Value |
|---|---|---|
DATABBIX.PROGRAM_NAME |
all | databbix, or queryexplorer in that utility. |
CONN.TYPE |
all | Database. |
DB.DRIVER |
Oracle | Oracle. |
| PGsql | PostgreSQL. |
|
| ODBC | ODBC. |
|
DB.DATABASE |
Oracle | Oracle. |
| PGsql | PostgreSQL. |
|
| ODBC | Product name reported by the ODBC driver (SQL_DBMS_NAME): for example, Microsoft SQL Server or DB2/400 SQL. |
|
DB.VERSION |
Oracle | Server version, in the form major.minor.update. |
| PGsql | The server’s server_version setting, for example 15.4. |
|
| ODBC | Version reported by the ODBC driver (SQL_DBMS_VER). |
|
ORACLE.CONTAINER |
Oracle | 1 if the Container context is available; 0 otherwise. |
This way, the same XML file can choose the SQL by product (runtime(’DB.DATABASE’) = ’Microsoft SQL Server’), by version (major(runtime(’DB.VERSION’)) >= 12) or by whether the database is multitenant (runtime(’ORACLE.CONTAINER’) = ’1’).
9.3 SQL queries: engine-specific elements#
Besides the common elements (chapter 7), Database engine queries use:
| Element / attribute | Description |
|---|---|
<sql> |
Statement text. Mandatory except in constant queries. Accepts ${…} variables and alternatives with <multiple> (section 9.6). |
<sql context="C"> |
Discoveries only: statement returning the values of context C to be iterated (section 9.4.4). |
select |
<query> attribute: FILTER or BIND. Without it, the query is basic. |
<item column="…"> |
Result column the value comes from. Case-insensitive. |
<parameter column> |
In FILTER queries: column compared with the parameter. |
<parameter bind> |
In BIND queries: SQL bind variable. |
<parameter context> |
Context the query runs in. |
<default> |
Fixed context of the query (section 9.5). |
<field column|context> |
In discoveries: where the macro value comes from. |
<context> |
At <Queries> level: declaration of a context. |
Note. Always give computed columns an explicit alias (as name) and use that alias in column.
9.4 Query types#
The examples in this section show, for each type, the result of the SQL statement at a given moment and the value each Zabbix key obtains from it.
9.4.1 Basic query#
The statement returns exactly one row. Each <item> takes the value of one column.
<query name="PGWriter" module="PG">
<sql><![CDATA[
select checkpoints_timed, checkpoints_req from pg_stat_bgwriter
]]>
</sql>
<item name="tez.pg.checkpointtimed" column="checkpoints_timed"/>
<item name="tez.pg.checkpointreq" column="checkpoints_req"/>
</query>Statement result
| checkpoints_timed | checkpoints_req |
|---|---|
| 1842 | 37 |
Values obtained
| Key | Value |
|---|---|
| tez.pg.checkpointtimed | 1842 |
| tez.pg.checkpointreq | 37 |
If the statement returns zero rows or more than one, all the query’s items become not supported.
9.4.2 Filter query (select="FILTER")#
The statement returns many rows. Each <parameter> with column links a key parameter with a column, and each item takes the row whose columns match its key parameters. The statement runs only once for all items, so this is the most efficient way to watch collections of objects (tablespaces, databases, jobs…).
<query name="PGSessions" module="PG" select="FILTER">
<sql><![CDATA[
select datname,
count(st.pid) as n,
sum(case st.state when 'active' then 1 else 0 end) as active
from pg_database db
left outer join pg_stat_activity st using (datname)
where datname not like 'template%'
group by datname
]]>
</sql>
<parameter column="datname" order="1" type="STRING"/>
<item name="tez.pg.sessions" column="n"/>
<item name="tez.pg.sessionsactive" column="active" default="0"/>
</query>Statement result
| datname | n | active |
|---|---|---|
| postgres | 3 | 1 |
| sales | 12 | 4 |
| hr | 0 | 0 |
Values obtained
| Key | Selected row | Value |
|---|---|---|
| tez.pg.sessions[sales] | sales | 12 |
| tez.pg.sessionsactive[sales] | sales | 4 |
| tez.pg.sessions[hr] | hr | 0 |
| tez.pg.sessions[purchasing] | none | not supported |
| tez.pg.sessionsactive[purchasing] | none | 0 (default attribute) |
| tez.pg.sessions[Sales] | none | not supported |
The first four keys are resolved with a single execution of the statement. The comparison is exact: it is case-sensitive (Sales does not match sales) and only trims spaces at the ends of the parameter. If a column is space-padded (char type), trim it in the SQL with trim().
Several filter columns#
With several filter parameters, the key has one parameter for each, in the order given by order, and the row must match all of them:
<query name="MSStateRepl" module="MSSQL" select="FILTER">
<sql><![CDATA[ exec sp_replmonitorhelppublication ]]></sql>
<default context="Database" value="distribution"/>
<parameter column="publisher_db" order="1" type="STRING"/>
<parameter column="publication" order="2" type="STRING"/>
<item name="tez.mssql.repl.status" column="status"/>
</query>Statement result
| publisher_db | publication | status |
|---|---|---|
| sales | pub_orders | 3 |
| sales | pub_customers | 2 |
| purchasing | pub_orders | 6 |
Values obtained
| Key | Value |
|---|---|
| tez.mssql.repl.status[sales,pub_orders] | 3 |
| tez.mssql.repl.status[purchasing,pub_orders] | 6 |
| tez.mssql.repl.status[sales] | not supported (missing parameter) |
Numeric parameters#
With type="INT", the parameter and the column are compared as numbers. This way, the key does not have to reproduce the exact format of the column:
<parameter column="job_id" order="1" type="INT"/>
<item name="local.job.status" column="status"/>Statement result and values obtained
| job_id | status | Key | Value |
|---|---|---|---|
| 7 | OK | local.job.status[7] | OK |
| 42 | ERROR | local.job.status[042] | ERROR |
With type="STRING", the key local.job.status[042] would not find the row, because the text 042 differs from 42.
9.4.3 Bind variable query (select="BIND")#
The key parameter is passed to the statement as a bind variable. The statement runs once for each distinct key and must return exactly one row.
<query name="PGParameter" module="PG" select="BIND">
<sql><![CDATA[
select setting from pg_settings where name = :name
]]>
</sql>
<parameter bind=":name" order="1" type="STRING"/>
<item name="tez.pg.param" column="setting"/>
</query>Executions and values obtained
| Key | Value of :name | Value |
|---|---|---|
| tez.pg.param[max_connections] | ’max_connections’ | 200 |
| tez.pg.param[work_mem] | ’work_mem’ | 4096 |
| tez.pg.param[no_such] | ’no_such’ | not supported (0 rows) |
The variable is always written as :name, whatever the driver (section 9.2). With type="INT", the value is bound as a number.
Warning. Since it runs once per key, a BIND query is much more expensive than a FILTER query when there are many keys. Use it only when the set of possible values is very large and only a few are queried.
9.4.4 Discovery query#
Each result row is a discovered entity, and each <field> takes the value of a column (column) or of a context (context).
<query name="TSDiscover" module="DB" discovery="yes">
<sql><![CDATA[
select tablespace_name, contents
from dba_tablespaces where status = 'ONLINE'
]]>
</sql>
<item name="tez.ora.tspace.discovery"/>
<field name="TSNAME" column="tablespace_name"/>
<field name="CONTENTS" column="contents"/>
</query>Statement result
| tablespace_name | contents |
|---|---|
| SYSTEM | PERMANENT |
| USERS | PERMANENT |
| TEMP | TEMPORARY |
Value of the item tez.ora.tspace.discovery:
{"data":[
{"{#TSNAME}":"SYSTEM", "{#CONTENTS}":"PERMANENT"},
{"{#TSNAME}":"USERS", "{#CONTENTS}":"PERMANENT"},
{"{#TSNAME}":"TEMP", "{#CONTENTS}":"TEMPORARY"}
]}
In Zabbix, the prototype tez.ora.tspace.pctfree[{#TSNAME}] creates the items tez.ora.tspace.pctfree[SYSTEM], [USERS] and [TEMP], resolved by a FILTER query with parameter column="tablespace_name".
Discoveries only accept context parameters.
Discovery across contexts#
A discovery can iterate over all the values of a context: for example, list the filegroups of all the databases of an SQL Server. To do so, add an <sql> with a context attribute that returns, in its first column, the values to iterate. The main statement runs once in each of them:
<query name="MSDiscoverFG" module="MSSQL" discovery="yes">
<sql context="Database"><![CDATA[
select name from sys.databases where name not in ('model')
]]>
</sql>
<sql><![CDATA[
select name from sys.filegroups
]]>
</sql>
<item name="tez.mssql.fg.discovery"/>
<field name="DBNAME" context="Database"/>
<field name="FGNAME" column="name"/>
</query>Statement results
| Context statement | Main statement in each database | |
| name | database | name |
| master | master | PRIMARY |
| sales | sales | PRIMARY |
| sales | HISTORY | |
Value of the item tez.mssql.fg.discovery:
{"data":[
{"{#DBNAME}":"master", "{#FGNAME}":"PRIMARY"},
{"{#DBNAME}":"sales", "{#FGNAME}":"PRIMARY"},
{"{#DBNAME}":"sales", "{#FGNAME}":"HISTORY"}
]}
Context values that cannot be entered (for example, an offline database) are skipped and logged.
9.5 Contexts#
A context is a connection state that can be changed between queries, such as the current database in SQL Server or PostgreSQL or the container (PDB) in Oracle multitenant. This way, a single monitor with a single connection can query several databases on the same server. Contexts only exist in the Database engine.
9.5.1 Declaration#
Contexts are declared at <Queries> level:
<environment name="PGAdminDB" default="postgres"/>
<context name="Database" module="PG" default="${PGAdminDB}"/>| Attribute | Description |
|---|---|
name |
Mandatory. Context name. The driver must know how to change it; the usual names are Database (SQL Server, PostgreSQL) and Container (Oracle multitenant). |
module |
Mandatory. Module it belongs to. |
default |
Mandatory. Default value. Accepts ${…} variables. |
9.5.2 Use in queries#
There are three ways to choose the context of a query:
Default. If the query says nothing, it runs with the context’s
defaultvalue.Fixed for the query, with
<default>:<default context="Database" value="msdb"/>If
valueis__IGNORE__(for example, through an unset environment variable), the element is ignored and the default value is kept.Taken from the item key, with a context parameter:
<parameter context="Database" order="1"/>
9.5.3 Example: a filter inside each database#
With a context parameter, the first key parameter selects the database and the following ones select the row:
<context name="Database" module="MSSQL" default="master"/>
<query name="MSFGSpace" module="MSSQL" select="FILTER">
<sql><![CDATA[
select g.name as filegroup_name,
sum(cast(fileproperty(f.name, 'SPACEUSED') as bigint) * 8192) as used
from sys.database_files f
inner join sys.filegroups g on g.data_space_id = f.data_space_id
group by g.name
]]>
</sql>
<parameter context="Database" order="1"/>
<parameter column="filegroup_name" order="2" type="STRING"/>
<item name="tez.mssql.fgspace.used" column="used"/>
</query>Statement result in each database
| database | filegroup_name | used |
|---|---|---|
| master | PRIMARY | 5242880 |
| sales | PRIMARY | 73400320 |
| sales | HISTORY | 524288000 |
Values obtained
| Key | Context | Value |
|---|---|---|
| tez.mssql.fgspace.used[sales,HISTORY] | sales | 524288000 |
| tez.mssql.fgspace.used[sales,PRIMARY] | sales | 73400320 |
| tez.mssql.fgspace.used[master,PRIMARY] | master | 5242880 |
Databbix groups executions by context: it enters sales only once, runs the statement and resolves the first two keys with the same result. If it cannot enter a context, it marks the items of that context as not supported. If the failing context is the default one and ConnLostOnDefaultCtxError = yes, the connection is considered lost and re-established.
9.6 Alternative SQL depending on the database#
The same XML file can serve different database products or versions if the SQL text is chosen at run time, with <multiple>:
<sql>
<multiple>
<when condition="runtime('DB.DATABASE') = 'Oracle'"><![CDATA[
select count(*) as n from ${SvmSchema}.f986110
where trim(jcfndfuf2) = :jobname
]]>
</when>
<when condition="runtime('DB.DATABASE') = 'Microsoft SQL Server'"><![CDATA[
select count(*) as n from ${SvmSchema}.f986110
where rtrim(jcfndfuf2) = :jobname
]]>
</when>
<when><![CDATA[
-- no condition: used if none of the above applies
select count(*) as n from ${SvmSchema}.f986110
where jcfndfuf2 = :jobname
]]>
</when>
</multiple>
</sql>The first <when> whose condition holds is used. A <when> without a condition always holds, so it must be the last one. If no condition holds, the query fails.
Conditions are expressions that can use these functions:
| Function | Description |
|---|---|
runtime(’X’) |
Runtime data of the active connection, for example DB.DATABASE or DB.VERSION (section 9.2.4). |
environment(’X’) |
Value of an environment variable. |
major(’X.Y.Z’) |
First number of a version, as a number. |
toscalar(’X’) |
Converts a text into a number. |
They accept comparison operators (=, !=, <, >…) and logical operators (and, or, not). The runtime data available with each driver can be listed with the getRuntimeNames command of queryexplorer.
9.7 Configuration examples#
9.7.1 Example: Oracle#
[General]
Name = ORAPROD
[Zabbix]
Server = zabbix.example.com
Port = 10051
PassivePort = 10153
[Data]
Module = DB
Module = Instance
[Database]
DBDriver = Oracle
ConnectionString = oraprod # tnsnames.ora alias
UserName = zabbix
Password = secret9.7.2 Example: Oracle multitenant#
[General]
Name = ORACDB
QueryFile = /etc/databbix/query-mt.xml
[Zabbix]
Server = zabbix.example.com
Port = 10051
[Data]
Module = MT
[Database]
DBDriver = Oracle
ConnectionString = oracdb
UserName = c\043\043zabbix # c##zabbix
Password = secret9.7.3 Example: ASM#
[General]
Name = asm
[Zabbix]
Server = zabbix.example.com
Port = 10051
[Data]
Module = ASM
[Database]
DBDriver = Oracle
ConnectionString = asm
ConnectOptions = SYSDBA
UserName = sys
Password = secret9.7.4 Example: PostgreSQL#
[General]
Name = PGPROD
QueryFile = /etc/databbix/query-pg.xml
QueryFile = /etc/databbix/query-pg-10.xml
Environment = PGAdminDB=postgres
[Zabbix]
Server = zabbix.example.com
Port = 10051
[Data]
Module = PG
[Database]
DBDriver = PGsql
ConnectionString = host=pg01.example.com port=5432 dbname=postgres
UserName = zabbix
Password = secret9.7.5 Example: SQL Server through ODBC#
First, define the data source in /etc/databbix/odbc.ini:
[MSSQL_PROD]
Driver=FreeTDS
Description=Production SQL Server
Server=mssql.example.com
Port=1433
Database=masterThen the monitor refers to it by name:
[General]
Name = MSSQLPROD
QueryFile = /etc/databbix/query-mssql.xml
[Zabbix]
Server = zabbix.example.com
Port = 10051
[Data]
Module = MSSQL
[Database]
DBDriver = ODBC
ConnectionString = DSN=MSSQL_PROD
UserName = zabbix
Password = secret