ipproxypool

Version En | 中文

Collect and maintain IP proxy data, and provide a RESTful API for access.

Crawlers Status

Name	Status	Desc
89ip
ip3366
kuaidaili
ProxyListPlus		maybe github workflow's IP is blocked
zdaye		It seems that non-Chinese IP addresses will be blocked

Features

1. Crawl IP proxies from the web and store them in a SQLite database.
2. Periodically check if the proxies in the database are still valid and remove them if they are invalid.
3. Provide RESTful APIs for access.

API

GET /

Query proxies.

param	desc	sample
type	proxy type 1: transparent 2: annoymous 3: high annoymous	3
protocol	protocol type 1: HTTP 2: HTTPS 3: Both	3
limit	limit count	10

GET /count

Query total count of proxies.

POST /delete

Delete proxy from database.

param	desc	sample
ip	ip to delete	10.10.10.10
port	port	8080

Config

type Config struct {
	Mode string // "debug", "release"
	Host string // default "0.0.0.0"
	Port int    // default 9002

	CountThreshold int64 // if less than threshold, start crawler
	CrawlSleepSec  int64 // sleep seconds between crawls
	DetectSleepSec int64 // sleep seconds between detects proxy from db
}

Run

go run cmd/main.go