收录各大搜索引擎与商业爬虫的 User-Agent、抓取行为、渲染能力与来路 IP 数据,为站长提供「识别 - 分析 - 拦截」全流程参考。取名源自镇守门户的瑞兽,为网站守住爬虫访问的大门。
| User-Agent | 爬虫名称 | 类型 | 抓取频率 | 抓取深度 | 遵守 Robots | 是否活跃 |
|---|---|---|---|---|---|---|
| iAskBot | iAskBot | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| ICC-Crawler | ICC-Crawler | AI数据 | No information. | — | 遵守 | 停用 |
| ImagesiftBot | ImagesiftBot | AI数据 | No information. | — | 遵守 | 停用 |
| imageSpider | imageSpider | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| img2dataset | img2dataset | AI数据 | At the discretion of img2dataset users. | — | 无视 | 停用 |
| Instinct | Instinct | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| ISSCyberRiskCrawler | ISSCyberRiskCrawler | AI数据 | No information. | — | 无视 | 停用 |
| kagi-fetcher | Kagi that fetches web content to answer user queries through | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| Kimi-Agent | Moonshot AI that browses websites while carrying out user-di | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| Kimi-User | Moonshot AI that fetches web content on behalf of users inte | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| KimiBot | KimiBot | AI数据 | No information provided. | — | 遵守 | 停用 |
| KlaviyoAIBot | KlaviyoAIBot | AI数据 | Indexes based on 'change signals' and us | — | 遵守 | 停用 |
| KunatoCrawler | KunatoCrawler | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| laion-huggingface-processor | laion-huggingface-processor | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| LCC | LCC | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| Lightpanda | Anyone who downloads the Lightpanda client. Possibly being u | AI数据 | Defined per-user. | — | 无视 | 停用 |
| LinerBot | LinerBot | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| Linguee Bot | Linguee Bot | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| Manus-User | Butterfly Effect, a company based in China | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| MistralAI-Training | MistralAI-Training | AI数据 | No information provided. | — | 遵守 | 停用 |
| MoonshotBot | MoonshotBot | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| Mozilla-Tabstack | Mozilla that performs programmatic, AI-driven interactions w | AI数据 | On demand via API. | — | 遵守 | 停用 |
| Muse | Muse | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| MyCentralAIScraperBot | MyCentralAIScraperBot | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| NagetBot | Naget Inc (founded by Chris Samarinas, headquarter in Amhers | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| netEstate Imprint Crawler | netEstate Imprint Crawler | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| newsai | newsai | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| NotebookLM | NotebookLM | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| NovaAct | NovaAct | AI数据 | Unclear at this time. | — | 无视 | 停用 |
| OAI-AdsBot | OAI-AdsBot | AI数据 | Only when a page is submitted as an ad. | — | 无视 | 停用 |