使用scrapy爬虫,响应结果状态码为460是什么鬼?
发布于 6 年前 作者 lovegnep 2658 次浏览 来自 问答

代码如下:

  def parse_err(self, failure):
	  # log all failures
	  self.logger.error(repr(failure))

	  # in case you want to do something special for some errors,
	  # you may need the failure's type:

	  if failure.check(HttpError):
		  # these exceptions come from HttpError spider middleware
		  # you can get the non-200 response
		  response = failure.value.response
		  self.logger.error('HttpError on %s, statuscode:%d', response.url, response.status)

	  elif failure.check(DNSLookupError):
		  # this is the original request
		  request = failure.request
		  self.logger.error('DNSLookupError on %s', request.url)

	  elif failure.check(TimeoutError, TCPTimedOutError):
		  request = failure.request
		  self.logger.error('TimeoutError on %s', request.url)

打印如下

1609:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1057242, statuscode:460
1686:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1071318, statuscode:460
1688:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1076998, statuscode:460
1690:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1077870, statuscode:460
1692:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1058858, statuscode:460
1697:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1066989, statuscode:460
1699:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1060215, statuscode:460
1707:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1078484, statuscode:460
1709:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1078482, statuscode:460
1770:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1078621, statuscode:460
1772:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1078623, statuscode:460
1774:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1078625, statuscode:460
1777:2018-05-22 14:34:14 [zhihu] ERROR: HttpError on https://www.weixinqun.com/group?id=1077919, statuscode:460
回到顶部