nltk k-means集群或使用纯python的k-means

Question

虽然我已经看到了相当多的与此相关的问题，但我没有真正得到答案，可能是因为我是使用nltk集群的新手。我真的需要一个关于聚类的新手的基本解释，特别是关于NLTK K均值聚类的向量表示以及如何使用它。我有一个像猫，狗，猫，小狗等单词的列表和另外两个单词列表，如食肉动物，草食动物，宠物等和哺乳动物，家养等。我想能够集群后两个单词列表基于第一个使用第一个作为手段或质心。我试过了，我收到的AssertionError是这样的：

clusterer = cluster.KMeansClusterer(2, euclidean_distance, initial_means=means)
  File "C:\Python27\lib\site-packages\nltk\cluster\kmeans.py", line 64, in __init__
    assert not initial_means or len(initial_means) == num_means

AND
    print clusterer.cluster(vectors, True)
  File "C:\Python27\lib\site-packages\nltk\cluster\util.py", line 55, in cluster
    self.cluster_vectorspace(vectors, trace)
  File "C:\Python27\lib\site-packages\nltk\cluster\kmeans.py", line 82, in cluster_vectorspace
    self._cluster_vectorspace(vectors, trace)
  File "C:\Python27\lib\site-packages\nltk\cluster\kmeans.py", line 113, in _cluster_vectorspace
    index = self.classify_vectorspace(vector)
  File "C:\Python27\lib\site-packages\nltk\cluster\kmeans.py", line 137, in classify_vectorspace
    dist = self._distance(vector, mean)
  File "C:\Python27\lib\site-packages\nltk\cluster\util.py", line 118, in euclidean_distance
    diff = u - v
TypeError: unsupported operand type(s) for -: 'numpy.ndarray' and 'numpy.ndarray'

我想在向量表示中有我的意思。矢量表示的基本示例和示例代码将受到高度赞赏。任何使用nltk或纯python的解决方案都将受到欢迎。提前感谢您的好意回复

Answer 1

如果我没理解错你的问题，像这样的东西应该可以用。kmeans最难的部分是找到集群中心，如果你已经找到了集群中心或者知道你想要什么中心，你可以:对于每个点，找到到每个集群中心的距离，并将该点分配给最近的集群中心。

(顺便说一句，sklearn是一个用于集群和机器学习的很好的包。)

在您的示例中，它应该如下所示：

Levenstein

# levenstein function is not my implementation; I copied it from the 
# link above 
def levenshtein(s1, s2):
    if len(s1) < len(s2):
        return levenshtein(s2, s1)

    # len(s1) >= len(s2)
    if len(s2) == 0:
        return len(s1)

    previous_row = xrange(len(s2) + 1)
    for i, c1 in enumerate(s1):
        current_row = [i + 1]
        for j, c2 in enumerate(s2):
            insertions = previous_row[j + 1] + 1 # j+1 instead of j since previous_row and current_row are one character longer
            deletions = current_row[j] + 1       # than s2
            substitutions = previous_row[j] + (c1 != c2)
            current_row.append(min(insertions, deletions, substitutions))
        previous_row = current_row

    return previous_row[-1]

def get_closest_lev(cluster_center_words, my_word):
    closest_center = None
    smallest_distance = float('inf')
    for word in cluster_center_words:
        ld = levenshtein(word, my_word)
        if ld < smallest_distance:
            smallest_distance = ld
            closest_center = word
    return closest_center

def get_clusters(cluster_center_words, other_words):
    cluster_dict = {}
    for word in cluster_center_words:
        cluster_dict[word] = []
    for my_word in other_words:
        closest_center = get_closest_lev(cluster_center_words, my_word)
        cluster_dict[closest_center].append(my_word)
    return cluster_dict

示例：

cluster_center_words = ['dog', 'cat']
other_words = ['dogg', 'kat', 'frog', 'car']

结果：

>>> get_clusters(cluster_center_words, other_words)
{'dog': ['dogg', 'frog'], 'cat': ['kat', 'car']}