NotFoundError Tensorflow Text Tokenization不工作

Question

环境: Google

!pip install -q tensorflow-text

我正在用TensorFlow text进行文本令牌化

from tensorflow_text import text
docs = tf.data.Dataset.from_tensor_slices([['Never tell me the odds.'], ["It's a trap!"]])
tokenizer = text.WhitespaceTokenizer()

tokenized_docs = docs.map(lambda x: tokenizer.tokenize(x))
iterator = iter(tokenized_docs)

print(next(iterator).to_list())
print(next(iterator).to_list())

我得到了的错误

NotFoundError                             Traceback (most recent call last)
<ipython-input-28-ca7b908c09b1> in <module>()
----> 1 from tensorflow_text import text
      2 docs = tf.data.Dataset.from_tensor_slices([['Never tell me the odds.'], ["It's a trap!"]])
      3 tokenizer = text.WhitespaceTokenizer()
      4 tokenized_docs = docs.map(lambda x: tokenizer.tokenize(x))
      5 iterator = iter(tokenized_docs)

4 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/load_library.py in load_op_library(library_filename)

NotFoundError: /usr/local/lib/python3.6/dist-packages/tensorflow_text/python/ops/_wordpiece_tokenizer.so: undefined symbol: _ZNK10tensorflow11ResourceMgr8DoLookupERKSsSt10type_indexS2_PPNS_12ResourceBaseE

Answer 1

您是否检查了安装Tensorflow的Python版本与正在执行的版本？都在执行3.6版吗？

您可以在以下链接中尝试其中一些建议：https://github.com/google/sentencepiece/issues/293