scrapy 在不同的抓取级别的Request之间传递参数的办法,下面的范例中,parse_item通过meat传递给了parse_details参数item,这样就可以再parse_details抓取完成所有的数据后一次返回 12345678910111213141516171819202122 class (BaseSpider): name = 'myspider' start_urls = ( 'http://example.com/page1', 'http://example.com/page2', ) def parse(self, response): for item_url in item_urls: yield Request(url=item_url, callback=self.parse_item) def parse_item(self, response): item = MyItem() # populate `item` fields yield Request(url=item_details_url, meta={'item': item}, callback=self.parse_details) def parse_details(self, response): item = response.meta['item'] # populate more `item` fields return item 赞微海报分享
近期评论