一、起因
最近在教务系统发现可以通过更改学号部分的url获取所有学生的学籍照片,这大概是没做好鉴权操作产生的后果,想着用python爬一下,但是用了网上现成的代码出现了如下报错
网上的源代码地址:批量下载文件并重命名 - 简书 (jianshu.com)
源代码如下:
import os
import requests
num = 1
while num < 170:
url = 'http://example.com/' + str(num) + '.zip'
d = 'D:\\B\\'
path = d + url.split('/')[-1]
try:
if not os.path.exists(d):
os.mkdir(d)
if not os.path.exists(path):
r = requests.get(url)
r.raise_for_status()
with open(path, 'wb') as f:
f.write(r.content)
f.close()
print("保存成功" + str(num))
else:
print("已存在" + str(num))
except:
print("获取失败" + str(num))
num += 1
path = input(d)
fileList = os.listdir(path)
n = 0
for i in fileList:
# 设置旧文件名(就是路径+文件名)
oldName = path + os.sep + fileList[n] # os.sep添加系统分隔符
# 设置新文件名
newName = path + os.sep + 'a' + str(n + 1) + '.JPG'
os.rename(oldName, newName) # 用os模块中的rename方法对文件改名
print(oldName, '======>', newName)
n += 1
运行后出现如下报错:
Traceback (most recent call last):
File "D:\PycharmProjects\download_photo\photo.py", line 2, in <module>
import requests
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\__init__.py", line 43, in <module>
import urllib3
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\__init__.py", line 8, in <module>
from .connectionpool import (
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 29, in <module>
from .connection import (
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py", line 39, in <module>
from .util.ssl_ import (
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\__init__.py", line 3, in <module>
from .connection import is_connection_dropped
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\connection.py", line 3, in <module>
from .wait import wait_for_read
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\wait.py", line 1, in <module>
from .selectors import (
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\selectors.py", line 14, in <module>
from collections import namedtuple, Mapping
ImportError: cannot import name 'Mapping' from 'collections' (C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\collections\__init__.py)
二、原因分析
在网上查找资料后发现,由于python3.10更改了collections的命名,因此在collections类中调用不到几个方法,于是需要更改对应的文件名
三、解决问题
1.修改报错对应的selectors.py文件
在pycharm中点击路径即可修改
路径为
C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\selectors.py
将第14行的from collections import namedtuple, Mapping
更改为
from collections import namedtuple
from collections.abc import Mapping
再次运行仍然报错
2.修改报错对应的_collections.py文件
报错如下
Traceback (most recent call last):
File "D:\PycharmProjects\download_photo\photo.py", line 2, in <module>
import requests
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\__init__.py", line 43, in <module>
import urllib3
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\__init__.py", line 8, in <module>
from .connectionpool import (
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 29, in <module>
from .connection import (
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py", line 50, in <module>
from ._collections import HTTPHeaderDict
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\_collections.py", line 2, in <module>
from collections import Mapping, MutableMapping
ImportError: cannot import name 'Mapping' from 'collections' (C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\collections\__init__.py)
在pycharm中点击路径即可修改
路径为
C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\_collections.py
将第2行的from collections import Mapping, MutableMapping
更改为
from collections.abc import Mapping
from _collections_abc import MutableMapping
再次运行仍然报错
3.修改报错对应的cookies.py
报错如下
Traceback (most recent call last):
File "D:\PycharmProjects\download_photo\photo.py", line 2, in <module>
import requests
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\__init__.py", line 97, in <module>
from . import utils
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\utils.py", line 32, in <module>
from .cookies import cookiejar_from_dict
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\cookies.py", line 172, in <module>
class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'
将第172行的class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
更改为
class RequestsCookieJar(cookielib.CookieJar, collections.abc.MutableMapping):
再次运行仍然报错
4.修改报错对应的structers.py
报错如下
Traceback (most recent call last):
File "D:\PycharmProjects\download_photo\photo.py", line 2, in <module>
import requests
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\__init__.py", line 97, in <module>
from . import utils
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\utils.py", line 33, in <module>
from .structures import CaseInsensitiveDict
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\structures.py", line 15, in <module>
class CaseInsensitiveDict(collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'
将第15行的class CaseInsensitiveDict(collections.MutableMapping):
更改为
class CaseInsensitiveDict(collections.abc.MutableMapping):
再次运行仍然报错
5.修改报错对应的sessions.py
报错如下
Traceback (most recent call last):
File "D:\PycharmProjects\download_photo\photo.py", line 2, in <module>
import requests
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\__init__.py", line 100, in <module>
from .api import request, get, head, post, patch, put, delete, options
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py", line 13, in <module>
from . import sessions
File "C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 13, in <module>
from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (C:\Users\Avicel\AppData\Local\Programs\Python\Python310\lib\collections\__init__.py)
将第13行的from collections import Mapping
更改为
from collections.abc import Mapping
再次运行终于不报错了!!!
太感谢了!!