我使用BreakIterator.getWordInstance将中文文本拆分成单词。下面是我的例子
import java.text.BreakIterator;
import java.util.Locale;
public class Sample {
public static void main(String[] args) {
String stringToExamine = "I like to eat apples. 我喜欢吃苹果。";
//print each word in order
BreakIterator boundary = BreakIterator.getWordInstance(new Locale("zh", "CN"));
boundary.setText(stringToExamine);
printEachForward(boundary, stringToExamine);
}
public static void printEachForward(BreakIterator boundary, String source) {
int start = boundary.first();
for (int end = boundary.next(); end != BreakIterator.DONE; start = end, end = boundary.next()) {
System.out.println(start + ": " + source.substring(start, end));
}
}
}我的示例文本取自https://stackoverflow.com/a/42219474/954439
我得到的输出是
0: I
1:
2: like
6:
7: to
9:
10: eat
13:
14: apples
20: .
21:
22: 我喜欢吃苹果
28: 。然而,预期输出是
0 I
1
2 like
6
7 to
9
10 eat
13
14 apples
20 .
21
22 我
23 喜欢
25 吃
26 苹果
28 。我甚至尝试了纯中文文本,但单词在空格和标点符号字符上断开。
我正在为服务器编程,所以jar文件大小并不是一个大问题。我正在尝试找出与使用最小共同子序列(但在单词上)的样本内容相比,给定内容中不同的单词数量。
我做错了什么?
https://stackoverflow.com/questions/44507838
复制相似问题