从变量Python (urllib2.urlopen) + Beautifulsoup4打开链接

Question

我正在使用Python2.7+ urllib2 + Beautifulsoup4

当我有绳子时：

soup = BeautifulSoup(urllib2.urlopen('http://www.some-website.com', 'html'))

它工作得很好，但是当我将URl移动到变量时，它就不能工作了。

variable = 'http://www.some-website.com'
soup = BeautifulSoup(urllib2.urlopen(variable, 'html'))

错误：

edit: errcode is: File "C:\Python27\lib\urllib2.py", line 285, in get_type
  raise ValueError, "unknown url type: %s" % self.__original
    ValueError: unknown url type: api/Abc-Abc/def/7/179 –

解出

问题是其中一个链接只是对服务器数据库的引用。

Answer 1

# Note: Make sure you add live website like http://vaibhavmule.com not http://some-website.com
variable = 'http://www.some-website.com' # Do not forget 'http' prefix here

# As you used 'html' which is not parser library.
soup = BeautifulSoup(urllib2.urlopen(variable))  

这应该能行。

用于使用解析器库的参考文献。

Answer 2

下列措施应能发挥作用：

var='http://www.example.com'
variable = urllib2.urlopen(var).read()
from BeautifulSoup import BeautifulSoup
Soup = BeautifulSoup()
import BeautifulSoup
soup = Soup(variable)