scrapy version –verbose运行出错的解决

scrapy version –verbose运行出错的解决

今天试着安装了scrapy,结果scrapy version –verbose出错:

(python37) PS E:\code\python\spyder> scrapy version --verbose
Traceback (most recent call last):
  File "D:\soft\anaconda3\envs\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "D:\soft\anaconda3\envs\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\soft\anaconda3\envs\python37\Scripts\scrapy.exe\__main__.py", line 4, in <module>
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\__init__.py", line 12, in <module>
    from scrapy.http import FormRequest, Request
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\http\__init__.py", line 10, in <module>
    from scrapy.http.request.form import FormRequest
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\http\request\form.py", line 19, in <module>        
    from parsel.selector import create_root_node
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\parsel\__init__.py", line 16, in <module>
    from parsel.selector import Selector, SelectorList  # NOQA
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\parsel\selector.py", line 31, in <module>
    from .csstranslator import GenericTranslator, HTMLTranslator
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\parsel\csstranslator.py", line 8, in <module>
    from cssselect.parser import Element, FunctionalPseudoElement, PseudoElement
ImportError: cannot import name 'PseudoElement' from 'cssselect.parser' (D:\soft\anaconda3\envs\python37\lib\site-packages\cssselect\parser.py)

这个错误需要升级cssselect,我本地的版本是cssselect 1.1.0,升级到cssselect 1.2.0后这个错误就没有 了。

pip install --upgrade cssselect

然而继续运行scrapy version –verbose依然出错:

(python37) PS E:\code\python\spyder> scrapy version --verbose       
Traceback (most recent call last):
  File "D:\soft\anaconda3\envs\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "D:\soft\anaconda3\envs\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\soft\anaconda3\envs\python37\Scripts\scrapy.exe\__main__.py", line 7, in <module>
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\cmdline.py", line 144, in execute
    cmd.crawler_process = CrawlerProcess(settings)
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\crawler.py", line 281, in __init__
    install_shutdown_handlers(self._signal_shutdown)
  File "D:\soft\anaconda3\envs\python37\lib\site-packages\scrapy\utils\ossignal.py", line 19, in install_shutdown_handlers
    reactor._handleSignals()
AttributeError: 'SelectReactor' object has no attribute '_handleSignals'

这次需要给twisted降级,我本地是twisted 23.8.0,这个版本与scrapy 存在一些兼容性问题:

pip install twisted==22.10.0

至此,问题解决:
(python37) PS E:\code\python\spyder> scrapy version --verbose    
Scrapy       : 2.5.1
lxml         : 4.9.1.0
libxml2      : 2.9.14
cssselect    : 1.2.0
parsel       : 1.8.1
w3lib        : 2.1.2
Twisted      : 22.10.0
Python       : 3.7.16 (default, Jan 17 2023, 16:06:28) [MSC v.1916 64 bit (AMD64)]
pyOpenSSL    : 23.0.0 (OpenSSL 1.1.1h  22 Sep 2020)
cryptography : 39.0.1
Platform     : Windows-10-10.0.22621-SP0