78 cities. 5 competitors. Pricing decisions in minutes, not weeks.
A tiered scraping pipeline + Postgres warehouse + Slack alerts that turn competitor pricing into actionable signal — the ops team sees under-priced inventory the morning it appears, not the quarter after.
Pricing decisions ran quarterly because the data took weeks to assemble.
Northfield Storage's pricing was set by hand, market by market, quarter by quarter. The data the ops team needed — competitor median price, occupancy band, premium/budget tier split — required a contractor to spend 2-3 weeks scraping competitor sites, building spreadsheets, and presenting findings. By the time the report landed, half the markets had already moved.
Competitor sites varied wildly. Some were JavaScript-heavy SPAs requiring browser rendering. Some had aggressive bot protection. Some changed their DOM structure every few weeks. A one-shot scrape didn't work; a sustained pipeline needed tiered tooling — datacenter proxies for the lightweight requests, residential proxies + Playwright for the hostile ones, manual fallback for the most defended.
Even with the data assembled, ops couldn't tell what mattered. A median price change of 4% in Lyon was noise; a 15% drop across budget-tier inventory in Manchester was a market signal. Without thresholds, alerts, and historical context, every number looked like every other number.
A tiered scraping pipeline → Postgres warehouse → MCP query layer → Slack alerts when markets move.
The scraper runs on a tier system. Tier 1: lightweight HTTP fetches via datacenter proxies for the simplest competitor sites — cheap and fast, handles ~60% of providers. Tier 2: Playwright + residential proxies for the JS-heavy and protected ones — handles another ~35%. Tier 3: manual collection for the 5% that resist both tiers. Webshare provides the proxy pool; rotation + retry logic absorbs the inevitable failures.
Every scrape lands in a Postgres warehouse with full historical state. Schema is tight: provider × city × tier × scraped_at × price_band × inventory_count. 35 new expansion cities were seeded in April; baseline coverage is 43. Total 78 cities across 5 providers, refreshing on tier-appropriate cadences (daily for active markets, weekly for stable ones).
An MCP server exposes the warehouse to Claude.ai — same pattern as the four-MCP suite for ops/marketing. The pricing team queries 'which cities moved more than 10% in the last week?' or 'what's the gap to budget tier for our Berlin locations?' in plain English. No SQL.
Slack alerts fire automatically when a city moves outside its defined band. The ops team gets a morning message: 'Lyon budget tier dropped 18% week-over-week; review.' The signal-to-noise is high because thresholds are per-city and learn from historical volatility.
The dynamic pricing engine downstream consumes the same warehouse. Recommendations are generated nightly; partners review and approve before they're pushed to the live marketplace.
T1 datacenter proxies, T2 Playwright + residential, T3 manual fallback — Webshare proxy pool
Full historical price+inventory state by provider × city × tier × time
43 baseline UK/EU + 35 expansion markets (seeded April 2026)
Pricing team queries warehouse in plain English via Claude.ai connector
Per-city threshold bands; morning alerts only when markets actually move
Nightly recommendation engine; partner review + approve before publish
Statistical outlier flags — surfaces 'this provider just changed their algorithm' before humans notice
Per-city competitive position, gap-to-floor, historical chart — single screen
Pricing review cadence moved from quarterly to weekly. Specific decisions — 'budget tier in Manchester needs a 12% adjustment by Friday' — happen in minutes once the alert fires, not after a quarter's data has staled.
The ops team stopped flying blind on new market entry. The 35 expansion cities seeded in April were chosen and priced against real competitor data from week one rather than 'we'll see how Q3 numbers look.'
The pricing system is now the central infrastructure for the marketplace's competitive intelligence — feeding the dynamic pricing engine, the partner review dashboard, the strategic planning cycle, and (newly) the market expansion algorithm covered in the next case study.
More systems like this
How fast can your firm react to a competitor pricing change?
Book a 30-minute call to talk through where competitive intelligence + fast feedback loops would change your firm's pricing economics.