pgq
Generic queue for PostgreSQL
pgq : Generic queue for PostgreSQL
Overview
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
| --s-d-- | No | Yes | No | Yes | no | no |
| Relationships | |
|---|---|
| Schemas | pg_catalog |
| See Also | pgmq pgmb ulak pgmqtt redis kafka_fdw wal2json decoderbufs tcn pg_durable |
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PGDG | 3.5.1 |
18 17 16 15 14 | pgq |
- |
| RPM | PGDG | 3.5.1 |
18 17 16 15 14 | pgq_$v |
- |
| DEB | PGDG | 3.5.1 |
18 17 16 15 14 | postgresql-$v-pgq3 |
- |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
| el8.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| el8.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| el9.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| el9.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| el10.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| el10.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| d12.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| d12.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| d13.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| d13.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| u22.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| u22.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| u24.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| u24.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| u26.x86_64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
| u26.aarch64 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 | PGDG 3.5.1 |
Source
github.com/pgq/pgq
Install
Make sure PGDG repo available:
Install this extension with pig:
Create this extension with:
Usage
PgQ is a PostgreSQL extension that provides a generic, high-performance lockless queue with a simple SQL function API. It uses a producer-consumer model with batch-based event processing.
Core Concepts
- Queue: A named event stream. Events are inserted by producers and consumed in batches.
- Consumer: A named subscriber registered to a queue. Each consumer tracks its own position.
- Batch: A group of events retrieved together. Consumers process events batch by batch.
- Ticker: A background process that creates batch boundaries (ticks) at regular intervals.
Queue Management
Consumer Registration
Producing Events
Consuming Events
Typical Consumer Loop
Maintenance
PgQ requires a ticker daemon (pgqd) to run in the background for creating batch boundaries and performing maintenance tasks like table rotation and retry event processing.
Key Functions
| Function | Description |
|---|---|
pgq.create_queue(name) |
Create a new queue |
pgq.drop_queue(name) |
Remove a queue |
pgq.register_consumer(queue, consumer) |
Register a consumer |
pgq.unregister_consumer(queue, consumer) |
Unregister a consumer |
pgq.insert_event(queue, type, data, ...) |
Insert an event |
pgq.next_batch(queue, consumer) |
Get next batch ID |
pgq.get_batch_events(batch_id) |
Get events from a batch |
pgq.event_retry(batch_id, event_id, seconds) |
Schedule event retry |
pgq.finish_batch(batch_id) |
Mark batch as processed |
pgq.get_queue_info([name]) |
Get queue statistics |
pgq.get_consumer_info(queue) |
Get consumer statistics |