尝试使用pop从pd.Series列表中删除项，但不能正常工作

Question

我试图使用统计方法(如分位数范围和zscore )来创建一个类来查找数据集中的异常值。我想知道为什么有些离群值会从我的pd.Series列表中删除，而有些没有给出空条件。

class OutliersDetector:
  def __init__(self, X):
    self.outliers = []
    self.X = X

  def detect_range(self):
    self.reset_outliers()
    # to implement
    self.remove_empty_items()

  def detect_zscore(self):
    self.reset_outliers()
    zscore = np.abs(stats.zscore(self.X))
    threshold_std = 3
    for index, col_name in enumerate(self.X.columns): # X and zscore always have the same shape
      col = zscore[:, index]
      self.outliers.append( pd.Series(col[col >= threshold_std], name=col_name) )
    self.remove_empty_items()

  # none of the if statements i tried worked
  def remove_empty_items(self):
    for index, item in enumerate(self.outliers):
      #if item.size == 0:
      #if len(item.index) == 0:
      if item.empty:
        print("[no outliers] {}".format(item.name))
        self.outliers.pop(index)

  def reset_outliers(self):
    self.outliers = []

  def show_outliers(self):
    for item in self.outliers:
      print("[name]: {}\n[outliers]: {}\n".format(item.name, item.size))

outliers_detector = OutliersDetector(X_train_transformed)
outliers_detector.detect_zscore()
print("\noutliers found: ")
outliers_detector.show_outliers()

输出:降雨量，月份，位置，WindDir9a不应打印在下面的“发现的离群点”，因为有0大小，但.

[no outliers] RainToday
[no outliers] Year
[no outliers] Day
[no outliers] WindGustDir
[no outliers] WindDir3pm
[no outliers] Sunshine
[no outliers] Humidity3pm
[no outliers] Cloud9am

outliers found:
[name]: Rainfall
[outliers]: 0

[name]: Evaporation
[outliers]: 289

[name]: Month
[outliers]: 0

[name]: Location
[outliers]: 0

[name]: WindDir9am
[outliers]: 0

我怎么才能解决这个问题？

Answer 1

在remove_empty_items中，您是在迭代self.outliers列表时修改它。这会导致不明确的行为。您的代码应该创建一个新列表，而不是修改当前的列表：

  def remove_empty_items(self):
    non_empty_outliers = []
    for item in self.outliers:
      if item.empty:
        print("[no outliers] {}".format(item.name))
      else:
        non_empty_outliers.append(item)
    self.outliers = non_empty_outliers