7.9.1. Summary¶
Groonga has token filter module that some processes tokenized token.
Token filter module can be added as a plugin.
You can customize tokenized token by registering your token filters plugins to Groonga.
A table can have zero or more token filters. You can attach token filters to a table by token_filters option in table_create.
Here is an example table_create that uses TokenFilterStopWord
token filter module:
Execution example:
plugin_register token_filters/stop_word
# [[0, 1337566253.89858, 0.000355720520019531], true]
table_create Terms TABLE_PAT_KEY ShortText \
--default_tokenizer TokenBigram \
--normalizer NormalizerAuto \
--token_filters TokenFilterStopWord
# [[0, 1337566253.89858, 0.000355720520019531], true]