Octoparse Proxies

Because that's how Octoparse is built: “Use my own proxies” works for local runs only, while Cloud runs always go out through Octoparse's own cloud nodes. No proxy provider can change that — the run simply never touches your list. If your workflow depends on Cloud scheduling and specific exit IPs, the realistic options are: run locally on an always-on machine or a small VPS (whitelist its IP once and the whole list works with no credentials), or accept the built-in proxy service for cloud tasks and reserve your own addresses for the local ones where the geo and IP history actually matter.

You can't — the Configure box takes host:port and nothing else, and Octoparse's documentation states plainly that proxies requiring a username and password are not supported. Stripping the credentials out of the string doesn't help: the proxy server still expects them and answers with a 407. The working route is IP whitelisting. In our Dashboard, add the public IP of the machine that runs Octoparse (find it with any “what is my IP” page from that machine); from then on, the proxy authenticates the connection by its origin, and plain host:port entries work as-is. One caveat for home connections: if your ISP reassigns your address after a router restart, update the whitelist entry to match — sudden 407-style failures after a reconnect are almost always a stale whitelist, not dead proxies.

More than people budget for, because Octoparse is a browser, not an HTTP client. Every page load pulls the HTML plus images, fonts, scripts, and third-party tags — a product page that's 80 KB of markup routinely weighs 2–4 MB fully loaded. At 10,000 pages a day, that's 20–40 GB of daily transfer; on any per-gigabyte plan you're paying for every byte of it, and images your extraction never reads are most of the bill. That multiplication is the argument for unlimited-bandwidth packages: per-IP pricing doesn't care whether pages are heavy. If you want to shrink transfer anyway (for speed, not cost), Octoparse's task settings can disable image loading — extraction rules that only read text keep working, and page loads get noticeably faster.

Octoparse runs a task as one browser session working through the list sequentially, so the pace is bounded by the tool, not the pool — this isn't Scrapy firing dozens of concurrent requests. In practice 10–30 addresses per task is a sensible band for most targets: enough that each IP faces the site only every Nth page, few enough that every address builds a plausible browsing history. Set the rotation interval so that one IP handles a handful of page loads before switching — for a task averaging 15 seconds per page, rotating every 1–2 minutes lands in that range. Rotate too fast (every page) and you look like a proxy list; too slow (hours) and the active IP absorbs the full request rate alone, which defeats the pool. Strict targets respond better to slower crawling than to more addresses.