pgpdf
pgpdf : PDF type with meta admin & Full-Text Search
Overview
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
| --sLdtr | No | Yes | Yes | Yes | yes | yes |
| Relationships | |
|---|---|
| See Also | pg_search pg_textsearch byteamagic external_file pg_render pg_readme pg_bestmatch vchord_bm25 pg_fts |
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PIGSTY | 0.1.0 |
18 17 16 15 14 | pgpdf |
- |
| RPM | PIGSTY | 0.1.0 |
18 17 16 15 14 | pgpdf_$v |
- |
| DEB | PIGSTY | 0.1.0 |
18 17 16 15 14 | postgresql-$v-pgpdf |
- |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
| el8.x86_64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| el8.aarch64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| el9.x86_64 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 |
| el9.aarch64 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 |
| el10.x86_64 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 |
| el10.aarch64 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 | PGDG 0.1.0 |
| d12.x86_64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| d12.aarch64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| d13.x86_64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| d13.aarch64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| u22.x86_64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| u22.aarch64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| u24.x86_64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| u24.aarch64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| u26.x86_64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
| u26.aarch64 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 | PIGSTY 0.1.0 |
Source
github.com/Florents-Tselai/pgpdf
pgpdf-0.1.0.tar.gz
Install
Make sure PGDG and PIGSTY repo available:
Install this extension with pig:
Config this extension to shared_preload_libraries:
Create this extension with:
Usage
The actual PDF parsing is done by poppler.
This allows you to work with PDFs in an ACID-compliant way. The usual alternative relies on external scripts or services which can easily make your data ingestion pipeline brittle and leave your raw data out-of-sync.
Download some PDFs.
You can create a pdf type, by casting either a text filepath or bytea column.
If you don’t have the PDF file in your filesystem, but have already stored its content in a bytea column, you can just cast it to pdf.
Examples
Create a table with a pdf column:
Parsing and validation should happen automatically. The files will be read from the disk only once!
The filepath should be accessible by the postgres process / user!
That’s different than the user running psql.
If you don’t understand what this means, as your DBA!
String Functions and Operators
Standard Postgres String Functions and Operators should work as usual:
Full-Text Search (FTS)
You can also perform full-text search (FTS), since you can work on a pdf file like normal text.
Document similarity with pg_trgm
You can use pg_trgm to get the similarity between two documents:
Metadata
The following functions are available:
-
pdf_title(pdf) → text -
pdf_author(pdf) → text -
pdf_num_pages(pdf) → integerTotal number of pages in the document
-
pdf_page(pdf, integer) → textGet the i-th page as text
-
pdf_creator(pdf) → text -
pdf_keywords(pdf) → text -
pdf_metadata(pdf) → text -
pdf_version(pdf) → text -
pdf_subject(pdf) → text -
pdf_creation(pdf) → timestamp -
pdf_modification(pdf) → timestamp
Getting a subset of pages
Installation
Install poppler dependencies
Linux
Homebrew/MacOS
After the installation, in a session:
Docker
Get the Docker image with:
This adds pgpdf to the Postgres image (replace 17 with your Postgres server version, and run it the same way).
Run the image in a container.
Through another terminal, connect to the running server (container).
Reading arbitrary binary data (PDF) into your database can pose security risks. Only use this for files you trust.