ChineseWordTokenizer

java.lang.Object

org.languagetool.tokenizers.zh.ChineseWordTokenizer

All Implemented Interfaces:: Tokenizer

public class ChineseWordTokenizer extends Object implements Tokenizer

Field Summary

Fields

Modifier and Type

Field

Description

private static final com.hankcs.hanlp.seg.Segment

SEGMENT
Constructor Summary

Constructors

Constructor

Description

ChineseWordTokenizer()
Method Summary

Modifier and Type

Method

Description

List<String>

tokenize(String text)

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details
- SEGMENT
  
  private static final com.hankcs.hanlp.seg.Segment SEGMENT
Constructor Details
- ChineseWordTokenizer
  
  public ChineseWordTokenizer()
Method Details
- tokenize
  
  public List<String> tokenize(String text)
  
  Specified by:
  
  tokenize in interface Tokenizer