为什么texts_to_sequences()输出为空数组？

Question

我试着用预先训练过的模型来预测。

texts_to_sequences(twt)返回空数组。因此，预测总是否定的。所有的投入。

from keras.preprocessing.sequence import pad_sequences
twt=['happy']
#vectorizing the tweet by the pre-fitted tokenizer instance
twt = tokenizer.texts_to_sequences(twt)
print(twt)
#padding the tweet to have exactly the same shape as `embedding_2` input
twt = pad_sequences(twt, maxlen=50, dtype='int32', value=0)
print(twt)
sentiment = model.predict(twt,batch_size=1,verbose = 2)[0]
print(sentiment)
if(np.argmax(sentiment) == 0):
    print("negative")
elif (np.argmax(sentiment) == 1):
    print("positive")

产出：

[[]]
[0.89889544 0.10110457]
negative

怎么解决这个问题？

Answer 1

我遇到了同样的问题，您需要做的就是在这两个函数tokenizer.fit_on_texts和tokenizer.texts_to_sequences.中传递列表。

示例: tokenizer.fit_on_texts(test_word)

model = ks.models.load_model('trained')
tokenizer = Tokenizer(num_words=5000)
test_word ="This is soo cool"
tokenizer.fit_on_texts([test_word])
tw = tokenizer.texts_to_sequences([test_word])
print('tw: ', tw)