Proxies for Price Tracking

Size against your strictest retailer, not your average one — that's the rule that keeps the math honest. Suppose your most defensive target tolerates one product-page request per address per minute before friction: 10,000 SKUs refreshed twice a day at that retailer is ~14 requests a minute sustained, so 15–20 addresses cover it with margin, and the more permissive retailers ride the same pool at higher per-IP rates without adding cost. Then apply the two multipliers trackers forget: markets (each country you price in polls through its own addresses, so a five-market program multiplies the base by five, not by adding a few IPs) and reserve (keep 20–25% of the pool idle to absorb retries and addresses benched after a CAPTCHA wave). For most mid-size programs that lands in the low hundreds of IPs — comfortably inside a $19-tier package with room to grow — and the calculator above prices any width instantly. If your grid is hundreds of thousands of SKU-market pairs, talk to sales instead: bulk mixes are the segment we specialize in.

Almost always geography, in one of three flavors — and it's worth diagnosing precisely because each has a different fix. First and most common: your requests exit from the wrong country, and the retailer serves its geo-fallback — a default international price, or a quiet redirect to the storefront matching your IP rather than the URL you asked for. Check which country your egress address actually sits in, not which domain you requested. Second: the currency trap — cross-border listings sometimes render a non-local currency even on a localized domain, so store the rendered currency string with every price instead of assuming it from the market; a EUR figure logged as GBP corrupts the history silently. Third: you're comparing against a personalized view — the shopper reporting the discrepancy may be seeing a logged-in promo, a coupon price, or an app-only offer your logged-out collector correctly doesn't see; that's not an error, it's a different data layer, and deciding whether to track it is a scope question. Rule of thumb: when a price parses cleanly but looks wrong, suspect the vantage point before the parser.

Datacenter for the fleet, ISP for the exceptions — and the split follows the retailers, not fashion. The bulk of price polling is high-volume, logged-out, and tolerant of occasional retries: datacenter packages fit that profile economically, with unlimited bandwidth so a twice-daily sweep of your whole catalog costs the same as a weekly one, and per-IP pricing low enough that widening the pool is the answer to most friction. The exceptions are the handful of targets whose anti-bot scoring declines datacenter origins outright — some marketplaces and fashion retailers are notorious for it — where no amount of pacing helps because the classification happens before your first request finishes. For those, route just that slice through static ISP addresses: consumer-range registration, same static workflow, higher per-IP price justified by being the only thing that works there. The architecture most teams end up with is 90/10 — and resisting the temptation to run everything through the expensive tier is what keeps the monthly bill proportional to the problem.

Public prices are among the safest data on the internet to collect — a price is a public offer to any visitor, contains no personal information, and competitive price intelligence is so established that entire SaaS categories exist to sell it. US case law has generally favored collection of publicly accessible data, and price-match guarantees at major retailers are themselves powered by exactly this kind of monitoring. The real constraints live one layer down. Retailer terms of service typically prohibit automated access — a contract question, not a criminal one, whose practical enforcement is the blocking this page teaches you to engineer around politely. Rate discipline matters ethically as well as tactically: a collector that hammers a small shop's server is a problem regardless of legality, so pace your polling to what the target won't notice. And keep your collection logged-out — scraping prices behind a login changes the legal posture, since you've accepted terms to get there. Public pages, polite cadence, no personal data: that combination is how the whole industry operates, ours included. For a commercial product built on the feed, have counsel do one review; that's cheaper than retrofitting compliance later.