Overview
The Indexing & Backfilling service lets you sync Polymarket data directly into your own infrastructure. Whether you need to backfill historical data or stream updates in realtime, Struct handles the heavy lifting so you can focus on building.Historical Backfilling
Bulk-load historical markets, trades, positions, and events into your database with a single configuration.
Realtime Streams
Subscribe to Kafka topics for live updates — new markets, price changes, trades, and settlements as they happen.
Database Sync
Keep a fully mirrored copy of Polymarket data in your own Postgres, ClickHouse, or other supported databases.
Managed Infrastructure
Struct manages the indexing pipeline end-to-end — schema migrations, data validation, and delivery guarantees included.
Key Capabilities
- Kafka Streams — Connect to managed Kafka topics for low-latency, ordered event delivery. Consume market updates, trades, and order book changes as they occur.
- Historical Backfilling — Load the full history of Polymarket data into your database. Backfills run incrementally so you can restart without re-processing.
- Database Sync — Struct writes directly to your database, handling schema creation, upserts, and deduplication automatically.
- Custom Filters — Scope your data pipeline to specific markets, tags, or event types to reduce noise and storage costs.
Getting Started
The Indexing & Backfilling service is currently set up on a per-customer basis. To get started, reach out to our team and we’ll work with you to configure a pipeline tailored to your needs.Contact us
Get in touch at struct.to/contact to discuss your data requirements and infrastructure setup.
Define your data scope
We’ll work with you to determine which data types (markets, trades, events, etc.) and filters match your use case.
We configure your pipeline
Struct provisions and configures the indexing pipeline — including your database destination, Kafka topics, and backfill schedule.