今天试着安装了scrapy,结果scrapy version –verbose出错:
(python37) PS E:\code\python\spyder> scrapy version --verbose
Traceback (most recent call last):
File "D:\soft\anaconda3\envs\python37\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "D:\soft\anaconda3\envs\python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\soft\anaconda3\envs\python37\Scripts\scrapy.exe\__main__.py", line 4, in <module>
File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\__init__.py", line 12, in <module>
from scrapy.http import FormRequest, Request
File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\http\__init__.py", line 10, in <module>
from scrapy.http.request.form import FormRequest
File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\http\request\form.py", line 19, in <module>
from parsel.selector import create_root_node
File "D:\soft\anaconda3\envs\python37\lib\site-packages\parsel\__init__.py", line 16, in <module>
from parsel.selector import Selector, SelectorList # NOQA
File "D:\soft\anaconda3\envs\python37\lib\site-packages\parsel\selector.py", line 31, in <module>
from .csstranslator import GenericTranslator, HTMLTranslator
File "D:\soft\anaconda3\envs\python37\lib\site-packages\parsel\csstranslator.py", line 8, in <module>
from cssselect.parser import Element, FunctionalPseudoElement, PseudoElement
ImportError: cannot import name 'PseudoElement' from 'cssselect.parser' (D:\soft\anaconda3\envs\python37\lib\site-packages\cssselect\parser.py)
这个错误需要升级cssselect,我本地的版本是cssselect 1.1.0,升级到cssselect 1.2.0后这个错误就没有 了。
pip install --upgrade cssselect
然而继续运行scrapy version –verbose依然出错:
(python37) PS E:\code\python\spyder> scrapy version --verbose
Traceback (most recent call last):
File "D:\soft\anaconda3\envs\python37\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "D:\soft\anaconda3\envs\python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\soft\anaconda3\envs\python37\Scripts\scrapy.exe\__main__.py", line 7, in <module>
File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\cmdline.py", line 144, in execute
cmd.crawler_process = CrawlerProcess(settings)
File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\crawler.py", line 281, in __init__
install_shutdown_handlers(self._signal_shutdown)
File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\utils\ossignal.py", line 19, in install_shutdown_handlers
reactor._handleSignals()
AttributeError: 'SelectReactor' object has no attribute '_handleSignals'
这次需要给twisted降级,我本地是twisted 23.8.0,这个版本与scrapy 存在一些兼容性问题:
pip install twisted==22.10.0
至此,问题解决:
(python37) PS E:\code\python\spyder> scrapy version --verbose
Scrapy : 2.5.1
lxml : 4.9.1.0
libxml2 : 2.9.14
cssselect : 1.2.0
parsel : 1.8.1
w3lib : 2.1.2
Twisted : 22.10.0
Python : 3.7.16 (default, Jan 17 2023, 16:06:28) [MSC v.1916 64 bit (AMD64)]
pyOpenSSL : 23.0.0 (OpenSSL 1.1.1h 22 Sep 2020)
cryptography : 39.0.1
Platform : Windows-10-10.0.22621-SP0