Customer story
Expedock uses PeerDB to deliver Real Time AI driven Supply Chain Automation
Expedock empowers supply chain businesses by automating the collection and movement of data from their semi-structured internal and external sources, and by providing valuable insights through data.
Expedock needed real-time and strongly consistent replication from Postgres to Snowflake
Expedock uses RDS Postgres as its primary database, storing data related to customer invoices, reports, and documents. They utilize Snowflake to power BI/analytics, including invoice reconciliation, revenue metrics, operational trackers, and more. They were looking a tool capable of replicating data from Postgres to Snowflake with 3 below requirements, as their current ETL tool was not able to meet these needs.
Replicate data from Postgres to Snowflake a with sub 1-min latency
Our customers want to see if their invoices are reconciled as soon as possible. So a replication tool that provides sub-minute latency was critical to us
Guarantee Strong Consistency
Data accuracy is of utmost priority to us. Our customers want to see accurate data. It is not acceptable for our customers to see invoices that don’t have line items
A tool that is cost effective
We are looking for a tool that is reasonably priced and keeps a check on Data Warehouse costs.
Expedock chose PeerDB because it was fast, reliable and cost effective - Under 1 min latency and 4X cost savings
Throughput per minute (click on the image to enlarge)
At PeerDB, we are building a fast and a cost effective way to replicate data from Postgres to Data Warehouses such as Snowflake, BigQuery and ClickHouse. PeerDB was a perfect match for Expedock, as PeerDB’s value proposition aligned precisely with what they were looking for.
- Real-time replication from Postgres to Snowflake with sub 1-min latency: PeerDB implements multiple optimizations such as parallel snapshotting, binary/compressed data-transfer, continuous reading and flushing of the replication slot to provide best possible performance. Expedock was able to reduce latency from 1 hr to 1-min while replicating 700M rows per month from Postgres to Snowflake.
- Save 5x in costs - With laser focus on Postgres to Snowflake replication, PeerDB implements multiple infrastructural and data-store native optimizations to cut costs. With this, PeerDB was able to reduce data-movement cost by 5x compared to the previous ETL tool.
- Strongly Consistent Data Replication - One hidden feature of PeerDB is its guarantee of transactional integrity when committing data to the target. PeerDB achieves this by preserving the order of transactions and the sequence of operations across tables on the target, mirroring that of the source. This came in very handy to solve Expedocks requirements.
While searching through multiple data movement tools, we found PeerDB to be the most cost-effective and to offer the lowest latency (< 1 min) when syncing data from Postgres to Snowflake at scales of 700 million active rows per month. The PeerDB team made our onboarding process as smooth as possible and was very competent in handling issues that the previous product had not yet supported. If Postgres is a major source of data to Snowflake, I definitely recommend PeerDB!
Enterprise-grade support that matters
At PeerDB, customer empathy is a core philosophy of our company. We are committed to providing unparalleled support to our customers to ensure their success. Furthermore, our goal is to become an expert in each data store we support. We believe that building a high-quality data movement tool that our customers can trust is essential. Below is a testimonial from Tim Dumol, Chief of Infrastructure of Expedock.
Loved working with PeerDB's team -- they showed a clear understanding of their problem domain & solution that exceeded that of competitors we've worked in, and were very responsive while we were setting up our integration.
Technology Stack and Architecture
Expedock replicates data from Postgres to Snowflake using PeerDB. Over 90% of the data in Snowflake comes from Postgres. Hence PeerDB is a critical component of their Data Stack. On Snowflake, they perform periodic transformations/aggregations using dbt and expose processed data to their customers in the form of dashboards.
About
Expedock empowers supply chain businesses by automating the collection and movement of data from their semi-structured internal and external sources, and by providing valuable insights through data.
Use case
Logistics, AI, Freight
Solutions
Postgres to Snowflake CDC