F.52. pg_query_id#
F.52. pg_query_id #
Extension to manipulate aspects of query id computation.
F.52.1. About pg_query_id #
Version: 1.0.0
F.52.2. Getting started #
Caution
Extension must be last in
shared_preload_libraries
or at least last
extension that uses query id. There is not check for that, so be
careful.
Install extension by adding it to
shared_preload_libraries
and set GUC to ignore
arrays of constants. And also add
pg_stat_statements
for further checking.
shared_preload_libraries = 'pg_query_id' pg_query_id.ignore_array_of_constants = on
Start DB and run these 2 queries:
SELECT * FROM generate_series(1, 100) x WHERE x IN (101, 102, 103); SELECT * FROM generate_series(1, 100) x WHERE x IN (101, 102, 103, 104, 105);
pg_stat_statements
must contain single query
with only 3 constants (other queries omitted):
SELECT query FROM pg_stat_statements; query ----------------------------------------------------------------- SELECT * FROM generate_series($1, $2) x WHERE x IN ($3, $4, $5)
F.52.3. Functionality #
Note
For checking query id we use
pg_stat_statements
in examples
To change configuration below, change configuration files and then send SIGHUP.
F.52.3.1. Enabling/disabling #
GUC | Default |
---|---|
pg_query_id.enabled
|
on
|
This setting controls, whether extension is up and running. Using this you can disable extension at runtime, but note that computed query ids will be different.
F.52.3.2. Temporary table names ignoring #
GUC | Default |
---|---|
pg_query_id.ignore_temp_tables
|
off
|
If this setting is on
, then OIDs of temporary
tables are not used during query id computation. This leads to
same query ids for queries, where only temporary tables differs,
but rest of the query is the same.
Example:
CREATE TEMP TABLE temp_tbl1(id int, str text); CREATE TEMP TABLE temp_tbl2(id int, str text); SELECT * FROM temp_tbl1 WHERE id = 1; SELECT * FROM temp_tbl2 WHERE id = 1; SELECT query FROM pg_stat_statements; query ----------------------------------------------- CREATE TEMP TABLE temp_tbl1(id int, str text) SELECT * FROM temp_tbl1 WHERE id = $1
As you can see only first executed query was recorded and second executed query (with only different table name) has same query id so go into same tuple.
F.52.3.3. Ignore arrays of constants #
GUC | Default |
---|---|
pg_query_id.ignore_constant_arrays
|
off
|
If this setting is on
, then arrays of only
constants (in different queries with different length of array)
will not contribute to query id computation. This includes:
ARRAY[]
VALUES ()
(if every row has only constants)IN ()
If such construction contains CONSTANTs or IMPLICIT CASTs OF
CONSTANT then it will be ignored. I.e. if you have
SELECT 1
inside array it will NOT be ignored
and computed by default.
Example:
SELECT * FROM generate_series(1, 10) x WHERE x IN (11, 12, 13); SELECT * FROM generate_series(1, 10) x WHERE x IN (11, 12, 13, 14, 15); SELECT * FROM generate_series(1, 10) x WHERE x = ANY(VALUES (11), (12), (13)); SELECT * FROM generate_series(1, 10) x WHERE x = ANY(VALUES (11), (12), (13), (14), (15)); SELECT * FROM generate_series(1, 10) x WHERE x = ALL(ARRAY[11, 12, 13]); SELECT * FROM generate_series(1, 10) x WHERE x = ALL(ARRAY[11, 12, 13, 14, 15]); SELECT query FROM pg_stat_statements; query -------------------------------------------------------------------------------------------- SELECT * FROM generate_series($1, $2) x WHERE x = ALL(ARRAY[$3, $4, $5]) SELECT * FROM generate_series($1, $2) x WHERE x = ANY(VALUES ($3), ($4), ($5)) SELECT * FROM generate_series($1, $2) x WHERE x IN ($3, $4, $5)
But not only constants, but implicit cast also work:
CREATE TABLE tbl(id int, str char(20)); SELECT * FROM tbl WHERE str IN ('a', 'b', 'c'); SELECT * FROM tbl WHERE str IN ('a', 'b', 'c', 'd', 'e'); SELECT query FROM pg_stat_statements; query --------------------------------------------- SELECT * FROM tbl WHERE str IN ($1, $2, $3) CREATE TABLE tbl(id int, str char(20))
But this will NOT work:
SELECT * FROM generate_series(1, 10) x WHERE x IN (11, 12, (SELECT 13)); SELECT * FROM generate_series(1, 10) x WHERE x IN (11, 12, 13, (SELECT 14)); SELECT * FROM generate_series(1, 10) x WHERE x = ANY(VALUES (11), (12), ((SELECT 13))); SELECT * FROM generate_series(1, 10) x WHERE x = ANY(VALUES (11), (12), (13), ((SELECT 14))); SELECT * FROM generate_series(1, 10) x WHERE x = ALL(ARRAY[11, 12, (SELECT 13)]); SELECT * FROM generate_series(1, 10) x WHERE x = ALL(ARRAY[11, 12, 13, (SELECT 14)]); SELECT query FROM pg_stat_statements; query ----------------------------------------------------------------------------------------------- SELECT * FROM generate_series($1, $2) x WHERE x IN ($3, $4, (SELECT $5)) SELECT * FROM generate_series($1, $2) x WHERE x IN ($3, $4, $5, (SELECT $6)) SELECT * FROM generate_series($1, $2) x WHERE x = ALL(ARRAY[$3, $4, (SELECT $5)]) SELECT * FROM generate_series($1, $2) x WHERE x = ALL(ARRAY[$3, $4, $5, (SELECT $6)]) SELECT * FROM generate_series($1, $2) x WHERE x = ANY(VALUES ($3), ($4), ((SELECT $5))) SELECT * FROM generate_series($1, $2) x WHERE x = ANY(VALUES ($3), ($4), ($5), ((SELECT $6)))
F.52.3.4. Constant query id for utility statements #
GUC | Default |
---|---|
pg_query_id.utility_constant_query_id
|
off
|
If this setting is on
, then query id for some
types of utility statement types assigned with predefined
constant. Query id assignment table:
Utility statement | Query Id |
---|---|
CREATE TABLE
| 1 |
CREATE TEMP TABLE
| 2 |
CREATE TABLE ... AS
| 3 |
CREATE TEMP TABLE ... AS
| 4 |
CREATE INDEX
| 5 |
CREATE INDEX (temp table)
| 6 |
VACUUM
| 7 |
TRUNCATE
| 8 |
DROP TABLE
| 9 |
DROP INDEX
| 10 |
COPY FROM ... (temp table)
| 11 |
COPY FROM ...
| 12 |
COPY TO ... (temp table)
| 13 |
COPY TO ...
| 14 |
This functionality may be useful for coarse-grained statistics collection when we want to measure overhead of multiple DMLs.
Example:
CREATE TABLE tbl1(id int); CREATE TABLE tbl2(id int, str text); CREATE TEMP TABLE temp_tbl1(id int); CREATE TEMP TABLE temp_tbl2(id int, str text); CREATE TABLE tbl3 AS SELECT x FROM generate_series(1, 10) x; CREATE TABLE tbl4 AS SELECT x, x::text y FROM generate_series(1, 10) x; CREATE TEMP TABLE temp_tbl3 AS SELECT x FROM generate_series(1, 10) x; CREATE TEMP TABLE temp_tbl4 AS SELECT x, x::text y FROM generate_series(1, 10) x; CREATE INDEX tbl1_id_idx ON tbl1(id); CREATE INDEX temp_tbl1_id_idx ON temp_tbl1(id); VACUUM tbl1; VACUUM temp_tbl1; COPY tbl1 TO STDOUT; COPY tbl2 TO STDOUT; COPY tbl3 TO STDOUT; COPY tbl4 TO STDOUT; COPY temp_tbl1 TO STDOUT; COPY temp_tbl2 TO STDOUT; COPY temp_tbl3 TO STDOUT; COPY temp_tbl4 TO STDOUT; COPY tbl1 FROM STDIN; COPY tbl2 FROM STDIN; COPY tbl3 FROM STDIN; COPY tbl4 FROM STDIN; COPY temp_tbl1 FROM STDIN; COPY temp_tbl2 FROM STDIN; COPY temp_tbl3 FROM STDIN; COPY temp_tbl4 FROM STDIN; TRUNCATE tbl1; TRUNCATE temp_tbl1; DROP INDEX tbl1_id_idx; DROP TABLE tbl1; DROP TABLE tbl2; DROP TABLE tbl3; DROP TABLE tbl4; DROP TABLE temp_tbl1; DROP TABLE temp_tbl2; DROP TABLE temp_tbl3; DROP TABLE temp_tbl4; SELECT queryid, query FROM pg_stat_statements ORDER BY queryid; queryid | query ---------------------+------------------------------------------------------------------------ 1 | CREATE TABLE tbl1(id int) 2 | CREATE TEMP TABLE temp_tbl1(id int) 3 | CREATE TABLE tbl3 AS SELECT x FROM generate_series($1, $2) x 4 | CREATE TEMP TABLE temp_tbl3 AS SELECT x FROM generate_series($1, $2) x 5 | CREATE INDEX tbl1_id_idx ON tbl1(id) 6 | CREATE INDEX temp_tbl1_id_idx ON temp_tbl1(id) 7 | VACUUM tbl1 8 | TRUNCATE tbl1 9 | DROP TABLE tbl1 10 | DROP INDEX tbl1_id_idx 11 | COPY temp_tbl1 FROM STDIN 12 | COPY tbl1 FROM STDIN 13 | COPY temp_tbl1 TO STDOUT 14 | COPY tbl1 TO STDOUT
F.52.4. Known issues #
F.52.4.1. Single-element array #
If pg_query_id.ignore_constant_arrays
is set
and array contains single element it automatically converts to
binary operator or another form of expression tree node. For
example: WHERE a IN (1)
during parsing
converts to WHERE a = 1
.
Currently, we can not track such situations, so for such queries
we will have 2 query ids: for single element and for multiple
elements. I.e. for WHERE a IN (1)
and for
WHERE a IN (1, 2)
,
WHERE a IN (1, 2, 3)
etc…