zhparser
a parser for full-text search of Chinese
zhparser : a parser for full-text search of Chinese
Overview
| ID | Extension | Package | Version | Category | License | Language |
|---|---|---|---|---|---|---|
| 2130 | zhparser | zhparser | 2.3 |
FTS | PostgreSQL | C |
| Attribute | Has Binary | Has Library | Need Load | Has DDL | Relocatable | Trusted |
|---|---|---|---|---|---|---|
| --s-d-r | No | Yes | No | Yes | yes | no |
| Relationships | |
|---|---|
| See Also | pgroonga pg_cjk_parser pg_jieba pg_bigm pg_tokenizer pg_pinyin unaccent dict_xsyn icu_ext pg_xenophile |
Packages
| Type | Repo | Version | PG Major Compatibility | Package Pattern | Dependencies |
|---|---|---|---|---|---|
| EXT | PIGSTY | 2.3 |
18 17 16 15 14 | zhparser |
- |
| RPM | PIGSTY | 2.3 |
18 17 16 15 14 | zhparser_$v |
- |
| DEB | PIGSTY | 2.3 |
18 17 16 15 14 | postgresql-$v-zhparser |
- |
| Linux / PG | PG18 | PG17 | PG16 | PG15 | PG14 |
|---|---|---|---|---|---|
| el8.x86_64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| el8.aarch64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| el9.x86_64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| el9.aarch64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| el10.x86_64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| el10.aarch64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| d12.x86_64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| d12.aarch64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| d13.x86_64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| d13.aarch64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| u22.x86_64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| u22.aarch64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| u24.x86_64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| u24.aarch64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| u26.x86_64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
| u26.aarch64 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 | PIGSTY 2.3 |
Source
github.com/amutu/zhparser
zhparser-2.3.tar.gz
Install
Make sure PGDG and PIGSTY repo available:
Install this extension with pig:
Create this extension with:
Usage
zhparser is a PostgreSQL extension for full-text search of Chinese, based on the Simple Chinese Word Segmentation (SCWS) library.
Features
- Chinese text segmentation for PostgreSQL full-text search
- Built on the SCWS (Simple Chinese Word Segmentation) library
- Supports custom dictionaries (TXT and XDB formats)
- Database-level custom word tables (since v2.1)
- Multiple tunable parameters for segmentation behavior
Quick Start
Configuration Parameters
zhparser provides several GUC parameters to control segmentation behavior:
| Parameter | Default | Description |
|---|---|---|
zhparser.punctuation_ignore |
off |
Ignore all punctuation |
zhparser.seg_with_duality |
off |
Perform duality segmentation on long words |
zhparser.dict_in_memory |
off |
Load the whole dictionary into memory |
zhparser.multi_short |
off |
Short word compound segmentation |
zhparser.multi_duality |
off |
Duality compound segmentation |
zhparser.multi_zmain |
off |
Key word in first compound segmentation |
zhparser.multi_zall |
off |
Use all compound segmentation |
Token Types
zhparser supports the following token types from SCWS:
| Code | Description |
|---|---|
a |
Adjective |
b |
Distinguishing word |
c |
Conjunction |
d |
Adverb |
e |
Exclamation |
f |
Position word |
g |
Root word |
h |
Prefix |
i |
Idiom |
j |
Abbreviation |
k |
Suffix |
l |
Temporary idiom |
m |
Numeral |
n |
Noun |
o |
Onomatopoeia |
p |
Preposition |
q |
Classifier |
r |
Pronoun |
s |
Locative word |
t |
Time word |
u |
Auxiliary |
v |
Verb |
w |
Punctuation |
x |
Unknown |
y |
Modal particle |
z |
State word |
Custom Dictionaries
File-based Dictionaries
Place custom dictionary files in the share directory (typically $SHAREDIR/tsearch_data/):
- TXT format: one word per line
- XDB format: compiled SCWS dictionary format
Custom dictionaries take precedence over built-in dictionaries.