生成API鏈接,提取IP
開(kāi)始爬取
代碼樣例。先把代碼貼出來(lái),再一步一步按要求來(lái)。大家先copy過(guò)去,代碼里需要注意以下三項(xiàng)皆為實(shí)際在快代理官網(wǎng)獲得的參數(shù)。api_url api鏈接page_url 默認(rèn)是快代理驗(yàn)證IP的測(cè)試網(wǎng)址,自己也可以改成其他的網(wǎng)址
以下代碼為python3所寫(xiě)
"""快代理代理IP使用教程"""
import requests
import random
# 要訪問(wèn)的目標(biāo)網(wǎng)頁(yè)
page_url = "//dev.kdlapi.com/testproxy"
# 添加header,模擬用戶請(qǐng)求
headers = {
"Accept-Encoding": "Gzip", # 使用gzip壓縮傳輸數(shù)據(jù)讓訪問(wèn)更快
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"}
# API接口,建議返回格式用json
api_url = ""
try:
api_res = requests.get(api_url).json()['data']['proxy_list']
except Exception:
api_res = requests.get(api_url).text.split("\r\n")
proxy = random.choice(api_res)
# 代理IP的格式
proxies = {
"http": "http://%(proxy)s/" % {'proxy': proxy},
"https": "https://%(proxy)s/" % {'proxy': proxy}}
# 添加header,模擬用戶請(qǐng)求
headers = {
"Accept-Encoding": "Gzip", # 使用gzip壓縮傳輸數(shù)據(jù)讓訪問(wèn)更快
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.117 Safari/537.36"}
# 發(fā)送request請(qǐng)求,打印響應(yīng)