Staff engineer building web-scale crawling infrastructure for AI-native search platform.
Design and operate distributed systems for discovering, fetching, and refreshing web content at internet scale. You'll architect crawl scheduling, URL deduplication, and ingestion workflows while managing billions of URLs and high-throughput data pipelines. Requires 8+ years of backend/systems experience with strong Go or C++ skills and proven expertise in large-scale distributed infrastructure.
Membership is €29/month, cancel anytime: every rate, every original listing link, and a daily alert for roles matching your filters.
Found at a specialist agency · listed 17 October 2026 · InsideJobs links you to the original posting.