熊猫群问题

Question

使用dataframe groupby()函数有一个快速的问题。我有一个BS4 html表要解析为Pandas，然后目标是使用Groupby()为一个“字段”嵌套多个行。不知道为什么我会得到

'AttributeError：'list‘对象没有属性’groupby‘

错误。我的str(table)不正确吗？

谢谢。

import requests
from bs4 import BeautifulSoup
import json
from datetime import datetime
import pandas as pd


starttime = datetime.now()

#Agent detail to prevent scraping bot detection
user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'
header = {'User-Agent' : user_agent }



# Webpage connection
html = "http://factpages.npd.no/ReportServer?/FactPages/TableView/field
_production_monthly&rs:Command=Render&rc:Toolbar=false&rc:Parameters
=f&Top100=True&IpAddress=108.171.128.174&CultureCode=en"

r=requests.get(html, headers=header)
c=r.content
soup=BeautifulSoup(c,"html.parser")

table = soup.find('table', attrs={'class':'a133'})


#Pandas dataframe 
df = pd.read_html(str(table))


j = (df.groupby(["Field (Discovery)","NPDID information carrier"], 
    as_index=False)
      .apply(lambda x: x[[ 'Year','Month','Oil - saleable [mill Sm3]','Gas - 
      saleable [bill Sm3]','NGL - saleable [mill Sm3]','Condensate - 
      saleable [mill Sm3]','Oil equivalents - saleable [mill Sm3]','Water - 
      wellbores [mill Sm3]' ]].to_dict('r'))
     .reset_index(drop=True)
     .rename(columns={0: 'MonthlyProduction'})
     .to_json(orient='records'))

print(j)

print(json.dumps(json.loads(j), indent=2, sort_keys=True))

Answer 1

根据熊猫文档，html.html pd.read_html返回数据列表(每个表一个数据)。所以，你需要选择你想要分组的。比如：

# for the first one:
df = pd.read_html(str(table))[0]