熊猫df.at() AttributeError：'BlockManager‘对象没有属性'T’

Question

我有一个比较大的数据。我试图迭代每一行，并根据特定的列值更新列(基本上是试图循环查找，直到无法更新进一步的列)。

我有以下几点：

df = the huge dataframe (1K to 10K+ rows x 51 cols)

has_update = True
while has_update:
   has_update = False

   for_procdf = df.loc[df['Incident Group ID'] == '-']

   for i, row in for_procdf.iterrows():
       #Check if the row's parent ticket id is an existing ticket id in the bigger df
       resultRow = df.loc[df['Ticket ID'] == row['Parent Ticket ID']]
       resultCount = len(resultRow.index)
       if resultCount == 1:
           IncidentGroupID = resultRow.iloc[0]['Incident Group ID']
           if IncidentGroupID != '-':
               df.at[i, "Incident Group ID"] = IncidentGroupID
               has_update = True

当我执行脚本时，下面的回溯会发生错误：

Traceback (most recent call last):
  File "./sdm.etl.py", line 76, in <module>
    main()
  File "./sdm.etl.py", line 28, in main
    fillIncidentGroupID(sdmdf.df)
  File "./sdm.etl.py", line 47, in fillIncidentGroupID
    df.at[i, "Incident Group ID"] = IncidentGroupID
  File "/usr/local/lib/python3.6/site-packages/pandas/core/indexing.py", line 2159, in __setitem__
    self.obj._set_value(*key, takeable=self._takeable)
  File "/usr/local/lib/python3.6/site-packages/pandas/core/frame.py", line 2580, in _set_value
    series = self._get_item_cache(col)
  File "/usr/local/lib/python3.6/site-packages/pandas/core/generic.py", line 2490, in _get_item_cache
    res = self._box_item_values(item, values)
  File "/usr/local/lib/python3.6/site-packages/pandas/core/frame.py", line 3096, in _box_item_values
    return self._constructor(values.T, columns=items, index=self.index)
AttributeError: 'BlockManager' object has no attribute 'T'

但是，创建类似的场景不会返回错误。

>>> qdf = pd.DataFrame([[0, 2, 3], [0, 4, 1], [10, 20, 30], [10, 13, 17]], index=[0,1,2,3], columns=['Ab 1', 'Bc 2', 'Cd 3'])
>>> qdf
   Ab 1  Bc 2  Cd 3
0     0     2     3
1     0     4     1
2    10    20    30
3    10    13    17
>>>
>>> qdf1 = qdf.loc[qdf['Ab 1'] == 0]
>>> qdf1
   Ab 1  Bc 2  Cd 3
0     0     2     3
1     0     4     1
>>>
>>> for i, row in qdf1.iterrows():
...     qdf.at[i, 'Ab 1'] = 10
...
>>>
>>> qdf
   Ab 1  Bc 2  Cd 3
0    10     2     3
1    10     4     1
2    10    20    30
3    10    13    17

我的执行似乎有什么问题？

Answer 1

发现，尼哈尔是对的，错误是由重复的列名引起的。我的数据太大了，所以我不小心有了一个重复的列名。现在一切都很好。离开代码一小段时间，休息和吃饭让我看到了重复的列。干杯!

下面是我的dataframe列。"RCA“在结尾处有重复。

['Incident Group ID', 'RCA Group ID', 'Parent Ticket ID', 'Ticket ID', ..., 'RCA Group ID', 'Is Sector Down', 'Relationship Type']

Answer 2

此错误是由重复的列名引起的。

对我来说是这样。

您可以使用以下函数快速确定哪些列名是重复的。

def get_duplicate_cols(df: pd.DataFrame) -> pd.Series:
    return pd.Series(df.columns).value_counts()[lambda x: x>1]

来源