Skip to content

An adapter that bridges between tantivy and jieba-rs.

License

Notifications You must be signed in to change notification settings

jiegec/tantivy-jieba

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

44 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

tantivy-jieba

Crates.io version docs.rs Changelog FOSSA Status

An adapter that bridges between tantivy and jieba-rs.

Usage

Add dependency tantivy-jieba to your Cargo.toml.

Example

use tantivy::tokenizer::*;
let mut tokenizer = tantivy_jieba::JiebaTokenizer {};
let mut token_stream = tokenizer.token_stream("测试");
assert_eq!(token_stream.next().unwrap().text, "测试");
assert!(token_stream.next().is_none());

Register tantivy tokenizer

use tantivy::schema::Schema;
use tantivy::tokenizer::*;
use tantivy::Index;
let tokenizer = tantivy_jieba::JiebaTokenizer {};
let index = Index::create_in_ram(schema);
index.tokenizers()
     .register("jieba", tokenizer);

See examples/mod.rs for detailed example.

License

FOSSA Status

About

An adapter that bridges between tantivy and jieba-rs.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages