keywords:

spider

找到 465 个包，花费 6 毫秒

A specification compliant robots.txt parser with wildcard (*) matching support.

samclarkeover 2 years 前robots.txt, parser, user-agent, scraper

simplecrawler214.8kBSD-2-Clause1.1.9

Very straightforward, event driven web crawler. Features a flexible queue interface and a basic cache mechanism with extensible backend.

simplecrawleralmost 3 years 前simple, crawler, spider, cache

shift-parser135.4kApache-2.08.0.0

ECMAScript parser that produces a Shift format AST

shapesecurityalmost 3 years 前Shift, AST, node, parser

This is an ES6 adaptation of the original PHP library CrawlerDetect, this library will help you detect bots/crawlers/spiders vie the useragent.

JefferyHus2 months 前es6-javascript, crawler, bots, spider

isbot-fast78.3kMIT1.2.0

JavaScript module detecting bots/crawlers/spiders via user-agent

mahovichabout 3 years 前bot, spider, robot, crawler

包

Lightweight debugging utility for Node.js and the browser

Lodash modular utilities.

React is a JavaScript library for building user interfaces.

A querystring parser that supports nesting and arrays, with a depth limit

Higher-order functions and common patterns for asynchronous code

Babel compiler core.