🔬 淘宝反爬体系深度研究报告

Taobao Anti-Crawl System — Reverse Engineering Analysis

📅 2026-10-27
🔧 REA MCP Tools
🌐 taobao.com
🏷️ Chromium 155.0.8059.85

📑 目录

架构总览
核心组件逆向分析
检测维度分析
反爬触发流程
绕过策略研究
关键发现与风险点
调试与监控建议

一架构总览

淘宝的反爬系统由 5 大核心组件 构成，形成多层防御体系：

┌─────────────────────────────────────────────────┐
│ baxiaCommon.js (入口层) │
│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ AWSC │ │ Fireye │ │ securitySDK │ │
│ │ (调度器) │ │(行为分析) │ │ (设备指纹) │ │
│ └──────────┘ └──────────┘ └──────────────────┘ │
│ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ UAB │ │ UMID │ │ ET (Event Track) │ │
│ │(用户行为) │ │ (设备ID) │ │ (事件追踪) │ │
│ └──────────┘ └──────────┘ └──────────────────┘ │
│ ┌──────────┐ ┌──────────┐ │
│ │ NC │ │ sufei │ │
│ │ (验证码) │ │(数据采集) │ │
│ └──────────┘ └──────────┘ │
└─────────────────────────────────────────────────┘
↓ 签名/Token
┌─────────────────────┐
│ MTOP API Gateway │
│ (h5api.m.taobao.com) │
└─────────────────────┘

二核心组件逆向分析

1️⃣ baxiaCommon.js — 百夏反爬引擎 (入口层)

文件: g.alicdn.com/sd/baxia-entry/baxiaCommon.js (30KB)
版本: 2.5.37

🔐 AES 加密通信

baxia 内置完整的 AES 加密实现，用于与服务端的安全通信：

<code><span class="cm">// 固定密钥 (16字节)</span>
<span class="kw">var</span> ca = <span class="fn">ea</span>(<span class="str">"2l3yMAStgOegI7xP"</span>);

<span class="cm">// 加密请求体</span>
<span class="fn">aa</span>(JSON.<span class="fn">stringify</span>({cfg:{jsLoad:<span class="num">0</span>}}), ca, ca)

<span class="cm">// 解密服务端响应</span>
<span class="fn">oa</span>(a.dt, ca, ca)  <span class="cm">// → 得到配置 JSON</span></code>

📡 配置拉取流程

<code><span class="fn">POST</span> https://umdcv4.taobao.com/repTw.json?e=2075&sv=1.0&pv=1.0&pn=www.taobao.com&biz=0
  Body: AES加密({cfg:{jsLoad:0}})
  Response: AES加密的配置 → 解密后获取:
    {cfg: {jsLoad: {fy: bool, gap: number}}}
    <span class="cm">// gap: 配置刷新间隔（分钟）</span>
    <span class="cm">// fy: 是否加载 FireyeJS</span></code>

⚠️ 关键发现：

加密配置缓存到 localStorage:
- baxia_entry_config — 加密配置
- baxia_entry_config_time — 下次刷新时间
域名白名单包含 500+ 阿里系域名
敏感路径特殊处理: login, passport, cart, buy_now

2️⃣ AWSC.js — 阿里 Web 安全客户端 (调度层)

文件: g.alicdn.com/AWSC/AWSC/awsc.js (17KB)

📦 模块化架构

模块	文件	功能
umidPCModule	`AWSC/WebUMID/1.93.0/um.js`	设备唯一 ID 生成
uabModule	`AWSC/uab/1.140.0/collina.js`	用户行为采集
fyModule	`AWSC/fireyejs/1.234.24/fireyejs.js`	FireyeJS 核心检测
etModule	`AWSC/et/1.84.16/et_f.js`	事件追踪
ncModule	`AWSC/nc/1.97.0/nc.js`	滑块验证码
nsModule	`js/nc/60.js`	无感验证

🔀 灰度发布机制

<code><span class="cm">// 10000 分之一的灰度比例</span>
ratio: <span class="num">1e4</span>  <span class="cm">// Math.ceil(Math.random()*10000) &lt;= ratio → 灰度</span>

<span class="cm">// 灰度版 vs 稳定版</span>
grey: [<span class="str">"AWSC/fireyejs/1.234.37/fireyejs.js"</span>]
stable: [<span class="str">"AWSC/fireyejs/1.234.24/fireyejs.js"</span>]</code>

🌍 域名定制版本

平台	FireyeJS 版本	备注
taobao / tmall	1.234.24 (stable) / 1.234.37 (grey)	标准版
aliexpress	混淆路径 `a227ca/61F4nKXFqoL/...`	代码混淆
lazada	独立 UAB + UMID	东南亚定制
WindVane (App)	1.234.37	WebView 专用

3️⃣ FireyeJS — 核心检测引擎

⚙️ 初始化参数

<code>window.__fyModule.<span class="fn">UBInit</span>({
    AsynSwitch: <span class="kw">true</span>,     <span class="cm">// 异步采集开关</span>
    SyncSwitch: <span class="kw">true</span>,     <span class="cm">// 同步采集开关</span>
    interval: <span class="num">600</span>,        <span class="cm">// 采集间隔 (ms)</span>
    TraceInterval: <span class="num">10</span>,    <span class="cm">// 追踪间隔 (ms)</span>
    TraceMax: <span class="num">300</span>,        <span class="cm">// 最大追踪次数</span>
    validTime: <span class="num">3600</span>       <span class="cm">// Token 有效期 (秒)</span>
})</code>

🎫 Token 生成

<code><span class="cm">// FireyeJS Token (设备+行为指纹)</span>
fyObj.<span class="fn">getFYToken</span>({reqUrl: <span class="str">"目标API"</span>})
<span class="cm">// → 返回加密的设备行为 Token</span>

<span class="cm">// UID Token (用户标识)</span>
fyObj.<span class="fn">getUidToken</span>()</code>

4️⃣ MTOP 签名算法

从捕获的 API 请求中分析出的签名参数结构：

<code>https://h5api.m.taobao.com/h5/{api}/1.0/?
    jsv=2.7.2
    &amp;appKey=<span class="num">12574478</span>           <span class="cm">← 固定 AppKey</span>
    &amp;t=<span class="num">1793100957200</span>           <span class="cm">← 时间戳 (ms)</span>
    &amp;sign=<span class="str">1366a570b77af9040ccbe27c0691b2b4</span>  <span class="cm">← HMAC-MD5 签名</span>
    &amp;api={apiName}
    &amp;v=1.0                     <span class="cm">← API 版本</span>
    &amp;dataType=jsonp            <span class="cm">← 数据格式</span>
    &amp;type=originaljsonp
    &amp;callback=mtopjsonp{N}     <span class="cm">← JSONP 回调函数名（递增）</span>
    &amp;data={urlEncodedJSON}     <span class="cm">← 业务参数</span>
    &amp;ttid=1@tbwang_windows_1.0.0#pc  <span class="cm">← 渠道标识</span></code>

✅ 签名算法推测：

<code>sign = <span class="fn">MD5</span>(token + <span class="str">"&amp;"</span> + t + <span class="str">"&amp;"</span> + appKey + <span class="str">"&amp;"</span> + data)
<span class="cm">// token: 从 cookie 中获取（_m_h5_tk / _m_h5_tk_enc）</span>
<span class="cm">// t: 当前时间戳</span>
<span class="cm">// appKey: 12574478</span>
<span class="cm">// data: JSON 参数</span></code>

三检测维度分析

🔍 环境检测

检测项	检测方式	权重
Headless 浏览器	`navigator.webdriver`, `Chrome` 特征缺失	🔴 高
自动化工具	`window.__nightmare`, `window.callPhantom`, Puppeteer 特征	🔴 高
浏览器指纹	WebGL, Canvas, AudioContext, Font 枚举	🟡 中
代理检测	IP 信誉库 + 数据中心 IP 段	🔴 高
TLS 指纹	JA3/JA3S 指纹匹配	🟡 中
HTTP/2 指纹	帧顺序、窗口大小、优先级树	🟡 中

🖱️ 行为检测

检测项	检测方式
鼠标轨迹	移动速度、加速度、贝塞尔曲线拟合度
点击模式	点击间隔分布、精确像素偏移
键盘事件	按键间隔、输入速度突变检测
滚动行为	滚动速度、惯性模拟真实性
页面停留	各区块可见时间分布
导航模式	Referer 链、跳转路径合理性

📊 数据采集 (ET/sufei)

<code><span class="cm">// sufei_data SDK 采集项</span>
{
    deviceFingerprint: {
        canvas: canvasHash,
        webgl: webglHash,
        audioContext: audioHash,
        fonts: fontList,
        plugins: pluginList,
        screen: {width, height, colorDepth, pixelDepth},
        timezone: timezoneOffset,
        language: navigatorLanguage
    },
    behaviorData: {
        mouseEvents: [...],
        keyEvents: [...],
        scrollEvents: [...],
        touchEvents: [...]
    },
    networkInfo: {
        connectionType: <span class="str">'wifi|4g|ethernet'</span>,
        rtt: roundTripTime,
        downlink: bandwidth
    }
}</code>

四反爬触发流程

用户请求 → baxia 入口检测
│
├─ 正常用户 → 放行，注入 _bx_et token
│
├─ 可疑用户 → 增强监控
│ ├─ 加载 FireyeJS (1.234.24)
│ ├─ 启动行为采集 (interval: 600ms)
│ ├─ 设备指纹采集 (sufei_data)
│ └─ 返回 wcfg.json 配置
│
└─ 确认机器人 → 触发挑战
├─ 跳转 _____tmd_____/punish
├─ x5secdata 加密参数
├─ action=captcha (滑块/点选)
├─ 多步验证 (x5step=1→2→3)
└─ 验证失败 → 临时封禁 IP

五绕过策略研究

策略 1：浏览器环境伪装

<code><span class="cm">// 1. 消除 webdriver 标记</span>
Object.<span class="fn">defineProperty</span>(navigator, <span class="str">'webdriver'</span>, {get: () => <span class="kw">false</span>});

<span class="cm">// 2. 补全 Chrome 特征</span>
window.chrome = {
    runtime: {},
    loadTimes: <span class="kw">function</span>(){},
    csi: <span class="kw">function</span>(){},
    app: {isInstalled: <span class="kw">false</span>}
};

<span class="cm">// 3. 修复 permissions API</span>
<span class="kw">const</span> originalQuery = window.navigator.permissions.query;
window.navigator.permissions.query = (parameters) => (
    parameters.name === <span class="str">'notifications'</span> ?
    Promise.<span class="fn">resolve</span>({state: Notification.permission}) :
    <span class="fn">originalQuery</span>(parameters)
);

<span class="cm">// 4. 补全 plugins (至少3个)</span>
Object.<span class="fn">defineProperty</span>(navigator, <span class="str">'plugins'</span>, {
    get: () => [
        {name: <span class="str">'Chrome PDF Plugin'</span>, filename: <span class="str">'internal-pdf-viewer'</span>},
        {name: <span class="str">'Chrome PDF Viewer'</span>, filename: <span class="str">'mhjfbmdgcfjbbpaeojofohoefgiehjai'</span>},
        {name: <span class="str">'Native Client'</span>, filename: <span class="str">'internal-nacl-plugin'</span>}
    ]
});

<span class="cm">// 5. 修复 languages</span>
Object.<span class="fn">defineProperty</span>(navigator, <span class="str">'languages'</span>, {
    get: () => [<span class="str">'zh-CN'</span>, <span class="str">'zh'</span>, <span class="str">'en-US'</span>, <span class="str">'en'</span>]
});</code>

策略 2：MTOP 签名复现

<code><span class="kw">import</span> hashlib
<span class="kw">import</span> time
<span class="kw">import</span> json
<span class="kw">import</span> urllib.parse

<span class="kw">def</span> <span class="fn">generate_mtop_sign</span>(token, api, data, app_key=<span class="str">"12574478"</span>):
    <span class="str">"""
    MTOP 签名算法
    token: 从 cookie _m_h5_tk 获取
    """</span>
    t = <span class="fn">str</span>(<span class="fn">int</span>(time.<span class="fn">time</span>() * <span class="num">1000</span>))
    data_str = json.<span class="fn">dumps</span>(data, separators=(<span class="str">','</span>, <span class="str">':'</span>))

    <span class="cm"># 签名: MD5(token + "&amp;" + t + "&amp;" + appKey + "&amp;" + data)</span>
    sign_str = <span class="str">f"{token}&amp;{t}&amp;{app_key}&amp;{data_str}"</span>
    sign = hashlib.<span class="fn">md5</span>(sign_str.<span class="fn">encode</span>()).<span class="fn">hexdigest</span>()

    <span class="kw">return</span> {
        <span class="str">"sign"</span>: sign,
        <span class="str">"t"</span>: t,
        <span class="str">"appKey"</span>: app_key,
        <span class="str">"data"</span>: urllib.parse.<span class="fn">quote</span>(data_str)
    }

<span class="cm"># 获取 token 的方法：</span>
<span class="cm"># 1. 先请求任意 MTOP 接口（会返回 Set-Cookie: _m_h5_tk）</span>
<span class="cm"># 2. 从 _m_h5_tk 中提取 token 值（格式: {token}_{timestamp}）</span>
<span class="cm"># 3. 使用该 token 签名后续请求</span></code>

策略 3：Cookie 管理

<code><span class="cm"># 淘宝关键 Cookie 及用途</span>
COOKIES = {
    <span class="str">"cna"</span>:          <span class="str">"设备标识（长期有效）"</span>,
    <span class="str">"cookie2"</span>:      <span class="str">"会话标识"</span>,
    <span class="str">"_m_h5_tk"</span>:     <span class="str">"MTOP 签名 Token"</span>,
    <span class="str">"_m_h5_tk_enc"</span>: <span class="str">"MTOP Token 加密版"</span>,
    <span class="str">"sgcookie"</span>:     <span class="str">"安全网关 Cookie"</span>,
    <span class="str">"tfstk"</span>:        <span class="str">"Token 签名密钥"</span>,
    <span class="str">"unb"</span>:          <span class="str">"用户ID"</span>,
    <span class="str">"xlly_s"</span>:       <span class="str">"反爬相关"</span>,
}

<span class="cm"># Cookie 获取流程:</span>
<span class="cm"># 1. GET https://www.taobao.com → 获得 cna, cookie2</span>
<span class="cm"># 2. GET https://h5api.m.taobao.com/h5/任意接口 → 获得 _m_h5_tk</span>
<span class="cm"># 3. 使用 _m_h5_tk 签名后续 API 请求</span></code>

策略 4：请求头伪装

<code>HEADERS = {
    <span class="str">"User-Agent"</span>: <span class="str">"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"</span>,
    <span class="str">"Accept"</span>: <span class="str">"*/*"</span>,
    <span class="str">"Accept-Language"</span>: <span class="str">"zh-CN,zh;q=0.9,en;q=0.8"</span>,
    <span class="str">"Accept-Encoding"</span>: <span class="str">"gzip, deflate, br"</span>,
    <span class="str">"Referer"</span>: <span class="str">"https://www.taobao.com/"</span>,
    <span class="str">"Origin"</span>: <span class="str">"https://www.taobao.com"</span>,
    <span class="str">"Sec-Ch-Ua"</span>: <span class="str">'"Chromium";v="131", "Not_A Brand";v="24"'</span>,
    <span class="str">"Sec-Ch-Ua-Mobile"</span>: <span class="str">"?0"</span>,
    <span class="str">"Sec-Ch-Ua-Platform"</span>: <span class="str">'"Windows"'</span>,
    <span class="str">"Sec-Fetch-Dest"</span>: <span class="str">"script"</span>,
    <span class="str">"Sec-Fetch-Mode"</span>: <span class="str">"no-cors"</span>,
    <span class="str">"Sec-Fetch-Site"</span>: <span class="str">"cross-site"</span>,
}</code>

策略 5：请求频率控制

<code><span class="kw">import</span> random
<span class="kw">import</span> time

<span class="kw">class</span> <span class="fn">RateLimiter</span>:
    <span class="kw">def</span> <span class="fn">__init__</span>(self):
        self.last_request = <span class="num">0</span>

    <span class="kw">def</span> <span class="fn">wait</span>(self):
        <span class="str">"""模拟人类请求间隔"""</span>
        elapsed = time.<span class="fn">time</span>() - self.last_request

        <span class="cm"># 正态分布延迟：均值2s，标准差0.5s</span>
        delay = <span class="fn">max</span>(<span class="num">0.5</span>, random.<span class="fn">gauss</span>(<span class="num">2.0</span>, <span class="num">0.5</span>))

        <span class="cm"># 偶尔长停顿（模拟用户思考）</span>
        <span class="kw">if</span> random.<span class="fn">random</span>() &lt; <span class="num">0.1</span>:
            delay = random.<span class="fn">uniform</span>(<span class="num">5</span>, <span class="num">15</span>)

        remaining = delay - elapsed
        <span class="kw">if</span> remaining &gt; <span class="num">0</span>:
            time.<span class="fn">sleep</span>(remaining)

        self.last_request = time.<span class="fn">time</span>()</code>

策略 6：TLS/HTTP2 指纹

<code><span class="cm"># 使用 curl_cffi 模拟浏览器 TLS 指纹</span>
<span class="kw">from</span> curl_cffi <span class="kw">import</span> requests

session = requests.<span class="fn">Session</span>(impersonate=<span class="str">"chrome131"</span>)

<span class="cm"># 自动处理:</span>
<span class="cm"># - TLS 扩展顺序</span>
<span class="cm"># - Cipher Suite 列表</span>
<span class="cm"># - HTTP/2 SETTINGS 帧</span>
<span class="cm"># - Header 压缩字典</span>
<span class="cm"># - ALPN 协商</span>

response = session.<span class="fn">get</span>(
    <span class="str">"https://h5api.m.taobao.com/h5/mtop.trade.querybagcount/1.0/"</span>,
    params={...},
    headers={...}
)</code>

六关键发现与风险点

发现	风险等级	说明
AES 固定密钥 `2l3yMAStgOegI7xP`	🔴	可解密所有配置通信
MTOP appKey 硬编码 `12574478`	🟡	所有 PC 端共用同一 appKey
Token 有效期 3600s	🟡	FireyeJS Token 1小时有效
JSONP 回调递增	🟢	`mtopjsonp{N}` 可预测
灰度比例 1/10000	🟢	灰度版本可通过随机数触发
localStorage 缓存配置	🟡	清除 localStorage 可强制刷新配置

七调试与监控建议

Hook baxia 初始化

<code><span class="kw">const</span> origInit = window.baxiaCommon?.init;
<span class="kw">if</span> (origInit) {
    window.baxiaCommon.init = <span class="kw">function</span>(opts) {
        console.<span class="fn">log</span>(<span class="str">'[BX Init]'</span>, JSON.<span class="fn">stringify</span>(opts));
        <span class="cm">// 修改检测配置</span>
        opts.checkApiPath = []; <span class="cm">// 禁用 API 拦截</span>
        <span class="kw">return</span> origInit.<span class="fn">call</span>(<span class="kw">this</span>, opts);
    };
}</code>

监控 FireyeJS 加载

<code><span class="kw">const</span> observer = <span class="kw">new</span> <span class="fn">MutationObserver</span>((mutations) => {
    mutations.<span class="fn">forEach</span>((m) => {
        m.addedNodes.<span class="fn">forEach</span>((node) => {
            <span class="kw">if</span> (node.src &amp;&amp; node.src.<span class="fn">includes</span>(<span class="str">'fireyejs'</span>)) {
                console.<span class="fn">log</span>(<span class="str">'[FY Load]'</span>, node.src);
            }
        });
    });
});
observer.<span class="fn">observe</span>(document.head, {childList: <span class="kw">true</span>});</code>

Hook MTOP 请求

<code><span class="kw">const</span> origFetch = window.fetch;
window.fetch = <span class="kw">function</span>(url, opts) {
    <span class="kw">if</span> (url.<span class="fn">includes</span>(<span class="str">'h5api.m.taobao.com'</span>)) {
        console.<span class="fn">log</span>(<span class="str">'[MTOP]'</span>, url, opts);
    }
    <span class="kw">return</span> origFetch.<span class="fn">apply</span>(<span class="kw">this</span>, arguments);
};</code>

监控 localStorage 写入

<code><span class="kw">const</span> origSetItem = Storage.prototype.setItem;
Storage.prototype.setItem = <span class="kw">function</span>(key, val) {
    <span class="kw">if</span> (key.<span class="fn">includes</span>(<span class="str">'baxia'</span>)) {
        console.<span class="fn">log</span>(<span class="str">'[Storage]'</span>, key, val.<span class="fn">substring</span>(<span class="num">0</span>, <span class="num">100</span>));
    }
    <span class="kw">return</span> origSetItem.<span class="fn">apply</span>(<span class="kw">this</span>, arguments);
};</code>

铁血汉子 5 天前

0 31

淘宝反爬机制