Go

Tokenizers (12)

  • Overall Score
  • Popularity
  • Trending
  • Activity
  • Maturity
348
P A M T

go-ego gse

Go efficient multilingual NLP and text segmentation; support English, Chinese, Japanese and others.
GoApache-2.0Star magnet • 2.7k stars
314
P A M T

yanyiwu gojieba

"结巴"中文分词的Golang版本
GoMITStar magnet • 2.6k stars
278
P A M T

gosimple slug

URL-friendly slugify with multiple languages support.
GoMPL-2.0Star magnet • 1.3k stars
230
P A M T

neurosnap sentences

A multilingual command line sentence tokenizer in Golang
GoMIT458 stars
196
P A M T

pebbe textcat

A Go package for n-gram based text categorization, with support for utf-8 and raw text
GoBSD-2-Clause72 stars
181
P A M T

blevesearch segment

A Go library for performing Unicode Text Segmentation as described in Unicode Standard Annex #29
GoApache-2.088 stars
170
P A M T

awsong MMSEGO

Chinese word splitting algorithm MMSEG in GO
Go62 stars
166
P A M T

mozillazg go-slugify

Pretty Slug.
GoMIT97 stars
163
P A M T

dchest stemmer

Stemmer packages for Go programming language. Includes English, German and Dutch stemmers.
GoBSD-2-Clause53 stars
141
P A M T

avelino slugify

A Go slugify application that handles string
GoMIT34 stars
136
P A M T

xujiajun gotokenizer

A tokenizer based on the dictionary and Bigram language models for Go. (Now only support chinese segmentation)
GoApache-2.021 stars
129
P A M T

osamingo shamoji

The shamoji (杓文字) is a word filtering package
GoMIT13 stars