link_1999
/
crawler-MiniProgram
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

上传文件至 'dagongPro/spiders' 

新增：将新闻的发布者与发布时间分离，使用不同的item项保存到mysql中。
This commit is contained in:
link_1999 2022-04-07 18:49:20 +08:00
parent 7230145007
commit 576dc30851
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
dagongPro/spiders/dagong.py
View File

@ -54,12 +54,13 @@ class DagongSpider(scrapy.Spider):
content = response.xpath('//div[@class="wrap_left"]/div[3]//text()').extract() content = response.xpath('//div[@class="wrap_left"]/div[3]//text()').extract()
content = ''.join(content) content = ''.join(content)
print(content) print(content)
date = response.xpath('//div[@class="wrap_left"]/div[1]/div[1]//text()').extract() date = response.xpath('//div[@class="wrap_left"]/div[1]/div[1]/span[1]/text()').extract_first()
date = ''.join(date) author = response.xpath('//div[@class="wrap_left"]/div[1]/div[1]/span[2]/text()').extract_first()
item = response.meta['item'] item = response.meta['item']
item['content'] = content item['content'] = content
item['date'] = date item['date'] = date
item['author'] = author
yield item yield item