BeautifulSoup网络抓取

Question

我正在使用BeautifulSoup来刮一个网站。当它返回链接时，它以/watch/411012/1/test.html的形式出现--我如何使它能够返回http://website.co.uk/watch/411012/1/test.html呢？

from bs4 import BeautifulSoup
import urllib2
import re

resp = urllib2.urlopen("http://website.co.uk") #add more sites
soup = BeautifulSoup(resp, from_encoding=resp.info().getparam('charset'))

for link in soup.find_all('a', href=re.compile('sports')): 
    print link['href']

Answer 1

使用urlparse.urljoin()

from urlparse import urljoin

base_url = "http://website.co.uk"
for link in soup.find_all('a', href=re.compile('sports')): 
    print urljoin(base_url, link['href'])