TLDR: https://github.com/xarg/pghoard-k8s

‍

This is a small tutorial on how to do incremental backups using pghoard for your PostgreSQL (I assume you’re running everything in Kubernetes). This is intended to help people to get started faster and not waste time finding the right dependencies, etc..

‍

pghoard is a PostgreSQL backup daemon that incrementally backups your files on a object storage (S3, Google Cloud Storage, etc..).
For this tutorial what we’re trying to achieve is to upload our PostgreSQL to S3.

‍

First, let’s create our docker image (we’re using the alpine:3.4 image cause it’s small):

FROM alpine:3.4

ENV REPLICA_USER "replica"
ENV REPLICA_PASSWORD "replica"

RUN apk add --no-cache \
   bash \
   build-base \        
   python3 \
   python3-dev \
   ca-certificates \
   postgresql \
   postgresql-dev \
   libffi-dev \
   snappy-dev
RUN python3 -m ensurepip && \
   rm -r /usr/lib/python*/ensurepip && \
   pip3 install --upgrade pip setuptools && \
   rm -r /root/.cache && \
   pip3 install boto pghoard


COPY pghoard.json /pghoard.json.template
COPY pghoard.sh /

CMD /pghoard.sh

‍

‍

REPLICA_USER and REPLICA_PASSWORD env vars will be replaced later in your Kubernetes conf by whatever your config is in production, I use those values to test locally using docker-compose.

‍

The config pghoard.json which tells where to get your data from and where to upload it and how:

‍

{
   "backup_location": "/data",
   "backup_sites": {
       "default": {
           "active_backup_mode": "pg_receivexlog",
           "basebackup_count": 2,
           "basebackup_interval_hours": 24,
           "nodes": [
               {
                   "host": "YOUR-PG-HOST",
                   "port": 5432,
                   "user": "replica",
                   "password": "replica",
                   "application_name": "pghoard"
               }
           ],
           "object_storage": {
               "aws_access_key_id": "REPLACE",
               "aws_secret_access_key": "REPLACE",
               "bucket_name": "REPLACE",
               "region": "us-east-1",
               "storage_type": "s3"
           },
           "pg_bin_directory": "/usr/bin"
       }
   },
   "http_address": "127.0.0.1",
   "http_port": 16000,
   "log_level": "INFO",
   "syslog": false,
   "syslog_address": "/dev/log",
   "syslog_facility": "local2"
}

‍

Obviously replace the values above with your own. And read pghoard docs for more config explanation.

‍

Note: Make sure you have enough space in your /data; use a Google Persistent Volume if you DB is very big.

‍

Launch script which does 2 things:

1. Replaces our ENV variables with the right username and password for our replication (make sure you have enough connections for your replica user)
2. Launches the pghoard daemon.

‍

‍

#!/usr/bin/env bash

set -e

if [ -n "$TESTING" ]; then
   echo "Not running backup when testing"
   exit 0
fi

cat /pghoard.json.template | sed "s/\"password\": \"replica\"/\"password\": \"${REPLICA_PASSWORD}\"/" | sed "s/\"user\": \"replica\"/\"password\": \"${REPLICA_USER}\"/" > /pghoard.json
pghoard --config /pghoard.json

‍

Once you build and upload your image to gcr.io you’ll need a replication controller to start your pghoard daemon pod:

‍

apiVersion: v1
kind: ReplicationController
metadata:
 name: pghoard
spec:
 replicas: 1
 selector:
   app: pghoard
 template:
   metadata:
     labels:
       app: pghoard
   spec:
       containers:
       - name: pghoard
         env:
           - name: REPLICA_USER
             value: "replicant"
           - name: REPLICA_PASSWORD
             value: "The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over. But it can't. Not with out your help. But you're not helping."
         image: gcr.io/your-project/pghoard:latest

‍

The reason I use a replication controller is because I want the pod to restart if it fails, if a simple pod is used it will stay dead and you’ll not have backups.

‍

Future to do:

• Monitoring (are you backups actually done? if not, do you receive a notification?)
• Stats collection.
• Encryption of backups locally and then uploaded to the cloud (this is supported by pghoard).

Hope it helps, stay safe and sleep well at night.

‍

Again, repo with the above: https://github.com/xarg/pghoard-k8s

At Gorgias we recently switched our flask & celery apps from Google Cloud VMs provisioned with Fabric to using docker with kubernetes (k8s). This is a post about our experience doing this.

‍

Note: I'm assuming that you're somewhat familiar with Docker.

‍

Docker structure

The killer feature of Docker for us is that it allows us to make layered binary images of our app. What this means is that you can start with a minimal base image, then make a python image on top of that, then an app image on top of the python one, etc..

‍

Here's the hierarchy of our docker images:

• gorgias/base - we're using phusion/baseimage as a starting base image.
• gorgias/pgbouncer
• gorgias/rabbitmq
• gorgias/nginx - extends gorgias/base and installs NGINX
• gorgias/python - Installs pip, python3.5 - yes, using it in production.
• gorgias/app - This installs all the system dependencies: libpq, libxml, etc.. and then does pip install -r requirements.txt
• gorgias/web - this sets up uWSGI and runs our flask app
• gorgias/worker - Celery worker

‍

Piece of advice: If you used to run your app using supervisord before I would advise to avoid the temptation to do the same with docker, just let your container crash and let k8s handle it.

‍

Now we can run the above images using: docker-compose, docker-swarm, k8s, Mesos, etc...

‍

We chose Kubernetes too

There is an excellent post about the differences between container deployments which also settles for k8s.

‍

I'll also just assume that you already did your homework and you plan to use k8s. But just to put more data out there:

‍

Main reason: We are using Google Cloud already and it provides a ready to use Kubernetes cluster on their cloud.

‍

This is huge as we don't have to manage the k8s cluster and can focus on deploying our apps to production instead.

‍

Let's begin by making a list of what we need to run our app in production:

• Database (Postgres)
• Message queue (RabbitMQ)
• App servers (uWSGI running Flask)
• Web servers (NGINX proxies uWSGI and serves static files)
• Workers (celery)

‍

Why Kubernetes again?

We ran the above in a normal VM environment, why would we need k8s? To understand this, let's dig a bit into what k8s offers:

• A pod is a group of containers (docker, rtk, lxc...) that runs on a Node. It's a group because sometimes you want to run a few containers next to each other. For example we are running uWSGI and NGINX on the same pod (on the same VM and they share the same ip, ports, etc..).
• A Node is a machine (VM or metal) that runs a k8s daemon (minion) that runs the Pods.
• The nodes are managed by the k8s master (which in our case is managed by the container engine from Google).
• Replication Controller or for short rc tells k8s how many pods of a certain type to run. Note that you don't tell k8s where to run them, it's master's job to schedule them. They are also used to do rolling updates, and autoscaling. Pure awesome.
• Services take the exposed ports of your Pods and publishes them (usually to the Public). Now what's cool about a service that it can load-balance the connections to your pods, so you don't need to manage your HAProxy or NGINX. It uses labels to figure out what pods to include in it's pool.
• Labels: The CSS selectors of k8s - use them everywhere!
• ‍

There are more concepts like volumes, claims, secrets, but let's not worry about them for now.

‍

Postgres

We're using Postgres as our main storage and we are not running it using Kubernetes.

‍

Now we are running postgres in k8s (1 hot standby + pghoard), you can ignore the rest of this paragaph.

The reason here is that we wanted to run Postgres using provisioned SSD + high memory instances. We could have created a cluster just for postgres with these types of machines, but it seemed like an overkill.

The philosophy of k8s is that you should design your cluster with the thought that pods/nodes of your cluster are just gonna die randomly. I haven't figured our how to setup Postgres with this constraint in mind. So we're just running it replicated with a hot-standby and doing backups with wall-e for now. If you want to try it with k8s there is a guide here. And make sure you tell us about it.

‍

RabbitMQ

RabbitMQ (used as message broker for Celery) is running on k8s as it's easier (than Postgres) to make a cluster. Not gonna dive into the details. It's using a replication controller to run 3 pods containing rabbitmq instances. This guide helped: https://www.rabbitmq.com/clustering.html

‍

uWSGI & NGINX

As I mentioned before, we're using a replication controller to run 3 pods, each containing uWSGI & NGINX containers duo: gorgias/web & gorgias/nginx. Here's our replication controller web-rc.yaml config:

‍

apiVersion: v1
kind: ReplicationController
metadata:
 name: web
spec:
 replicas: 3 # how many copies of the template below we need to run
 selector:
   app: web
 template:
   metadata:
     labels:
       app: web
   spec:
     containers:
     - name: web
       image: gcr.io/your-project/web:latest # the image that you pushed to Google Container Registry using gcloud docker push
       ports: # these are the exposed ports of your Pods that are later used by the k8s Service
         - containerPort: 3033
           name: "uwsgi"
         - containerPort: 9099
           name: "stats"
     - name: nginx
       image: gcr.io/your-project/nginx:latest
       ports:
         - containerPort: 8000
           name: "http"
         - containerPort: 4430
           name: "https"
       volumeMounts: # this holds our SSL keys to be used with nginx. I haven't found a way to use the http load balancer of google with k8s.  
         - name: "secrets"
           mountPath: "/path/to/secrets"
           readOnly: true
     volumes:
       - name: "secrets"
         secret:
           secretName: "ssl-secret"
And now the web-service.yaml:apiVersion: v1
kind: Service
metadata:
 name: web
spec:
 ports:
 - port: 80
   targetPort: 8000
   name: "http"
   protocol: TCP
 - port: 443
   targetPort: 4430
   name: "https"
   protocol: TCP
 selector:
   app: web
 type: LoadBalancer

‍

That type: LoadBalancer at the end is super important because it tells k8s to request a public IP and route the network to the Pods with the selector=app:web.
If you're doing a rolling-update or just restarting your pods, you don't have to change the service. It will look for pods matching those labels.

‍

Celery

Also a replication controller that runs 4 pods containing a single container: gorgias/worker, but doesn't need a service as it only consumes stuff. Here's our worker-rc.yaml:

‍

apiVersion: v1
kind: ReplicationController
metadata:
 name: worker
spec:
 replicas: 2
 selector:
   app: worker
 template:
   metadata:
     labels:
       app: worker
   spec:
     containers:
     - name: worker
       image: gcr.io/your-project/worker:latest

‍

‍

Some tips

• Installing some python deps take a long time, for stuff like numpy, scipy, etc.. try to install them in your namespace/app container using pip and then do another pip install in the container that extends it, ex: namespace/web, this way you don't have to rebuild all the deps every time you update one package or just update your app.
• Spend some time playing with gcloud and kubectl. This will be the fastest way to learn of google cloud and k8s.
• Base image choice is important. I tried phusion/baseimage and ubuntu/core. Settled for phusion/baseimage because it seems to handle the init part better than ubuntu core. They still feel too heavy. phusion/baseimage is 188MB.

‍

‍

Conclusion

With Kubernetes, docker finally started to make sense to me. It's great because it provides great tools out of the box for doing web app deployment. Replication controllers, Services (with LoadBalancer included), Persistent Volumes, internal DNS. It should have all you need to make a resilient web app fast.

At Gorgias we're building a next generation helpdesk that allows responding 2x faster to common customer requests and having a fast and reliable infrastructure is crucial to achieve our goals.

If you're interested in working with this kind of stuff (especially to improve it): we're hiring!

We've released a new version of the Chrome Extension, with sharing features and a new navigation bar. We hope you'll love it!

‍

Share templates inside the extension

Before, the only way to share templates with your teammates was to login on Gorgias.io.

If you're on the startup plan, when you create a template, you can choose who has access to it: either only you, specific people, or your entire team.

‍

‍

The account management section is now available in the extension, under settings.

‍

New navigation

Tags are now available on the left. It's easier to manage hundreds of templates with them.
You can also navigate through your private & shared templates. Shared templates include templates shared with specific people or with everyone.

‍

‍

We hope you'll enjoy this new version of our Chrome Extension. As usual, your feedback & questions are welcome!

Today, we’re thrilled to announce that we’ve raised a $1.5 million Seed round led by Charles River Ventures and Amplify Partners, to help build our new helpdesk.

‍

We’re incredibly grateful to early users, customers, mentors we’ve met both at and Techstars.

‍

We started the journey with Alex at the beginning of 2015 with our Chrome extension, which helps write email faster using templates. We’ve been pleased all along with customers telling us about how helpful it was, especially for customer support.

‍

While building the extension, we’ve realized that a big inefficiency in support lies in the lack of integration between the helpdesk, the payment system, CRM and other tools support is using. As a result, agents need to do a lot of repetitive work to respond to customer requests, especially when the company is big.

‍

That’s why we’ve decided to build a new kind of helpdesk to enable customer support agents to respond 2x faster to customers. You can find out more and sign up for our private beta here.

‍

When a company has a lot of customers, support becomes repetitive. We want to provide support teams with tools to automate the way they treat simple repetitive requests. This way, they have more time for complex customer issues.

‍

We'll now focus on this helpdesk and on growing the team, oh, and if you'd like to join, we're hiring! We're super excited about this new helpdesk product. If you’re using the extension, don’t worry.

‍

Romain & Alex

We've been busy, but not deaf!

Last few months we got lots of feedback about our extension and found to our delight that most people are satisfied, but still a few recurrent issues came up:

• The HTML/WYSIWYG editor sucks.
• No support for Outlook.com.

‍

We listened and now we're presenting:

• A brand new editor
• Support for outlook.com
• More on the Rich-Text editor

‍

WYSIWYG editors for the web are notoriously buggy and are just difficult to develop.

‍

I have yet to see one that is bug free. There are few venerable editors that do a good job like TinyMCE, FKEditor or CKEditor.. but they are big and all have edge cases that break the intended formatting and add a lot of garbage html.

‍

There are newer good quality editors in town such as Redactor. The one that got my attention and finally landed in Gorgias is this wonderful editor called which is super lightweight, uses modern content-editable (no i-frames) and 'just works' most of the time. That's not to say it's perfect, but it's good enough and I'm satisfied with it's direction in terms of development.

‍

Enjoy it and as always send us bug-reports or feedback on: support@gorgias.com

TL;DR:

• Most chargebacks occur due to poor merchant communication rather than fraud. Customers choose this path when they feel ignored or frustrated.
• 80% of customers report never being contacted by merchants after filing a chargeback. 23% file immediately after an issue and 38% file within 1-3 days if unresolved.
• The most common chargeback reason is "product not received" (35%). 79% of all chargebacks are actually "friendly fraud" filed for invalid claims.‍
• Prevention requires fast customer support and automated chargeback management. Combining Gorgias for AI-powered support with Chargeflow for automated dispute management provides a comprehensive solution with faster resolutions and higher win rates.

Chargebacks are more than a thorn in a merchant’s side — they’re a growing financial and operational threat. According to Ethoca, chargebacks are projected to more than double, from $7.2 billion in 2019 to $15.3 billion by 2026 in the U.S. alone. And while fraud plays a role, the primary reason customers file chargebacks is simpler: they feel ignored. 

At Chargeflow, we recently published a comprehensive report analyzing why customers dispute chargebacks. The findings were eye-opening. While it’s true that fraud is a real concern, most chargebacks happen for a different reason: a lack of communication between merchants and customers.  

Top stats from Chargeflow’s report:

• 23% of customers file a chargeback immediately after an issue.
• 38% file a chargeback within 1-3 days if unresolved.
• 80% report never being contacted by the merchant.
• 52% are likely to dispute if the response is too slow.

When customers feel ignored or frustrated, they often turn to their bank for a solution instead of reaching out to the merchant first. Understanding these behaviors is key to preventing disputes before they escalate and cause chaos. 

So, what actually drives customers to dispute charges? Here’s what the data says.

Why customers file chargebacks

While chargebacks are often the cost of doing business, the truth is that many disputes are preventable — but only if merchants understand the root causes. We identified five key drivers behind chargebacks.

1. Customers take immediate action

According to our research, most customers file a dispute right away after encountering an issue, leaving no opportunity to resolve the problem. Another 38% file within one to three days if they don’t receive a timely response. 

Why? Customers assume the fastest way to get their money back is by filing a chargeback, especially if they receive no response from the merchant.

2. Lack of communication leads to disputes

We found that 80% of customers never receive a follow-up after filing a chargeback. Additionally, 64% of customers state immediate communication is crucial, yet many businesses fail to reach out.

• 90% of customers tried to reach out to the merchant first.
• If they don’t receive a response, they quickly file a dispute. 

Why? Customers expect businesses to be proactive. When they don’t hear back quickly, they assume the merchant won’t help, making a chargeback seem like the best option.

3. Chargebacks are too easy for customers

98% of customers report a neutral to highly satisfactory experience when filing chargebacks, and only 12% are denied. 

Why? Many customers believe chargebacks are faster and easier than dealing with merchants directly, especially if return policies are unclear. 

4. Transaction issues drive chargebacks

The most common reason for filing a chargeback is “product not received” (35% of the cases). Other common reasons included:

• Fraudulent transaction claims - 16%
• Product significantly not as described - 15%
• Unauthorized transaction - 15%

Why? When customers don’t receive clear shipping updates or experience delivery delays, they assume their order won’t arrive and file a chargeback rather than waiting.

5. Friendly fraud is a major problem

Friendly fraud occurs when a cardholder makes a legitimate purchase but later disputes the charge as fraudulent or unauthorized, leading their card issuer to reverse the payment. 

Our research found that:

• 21% of customers admitted to not fully understanding the chargeback process. 
• Another 20% aren’t even aware of what a chargeback is. 
• 97% of consumers believe they’ve never filed a chargeback incorrectly, while only 3% admit they have.

According to our State of Chargebacks report, 79% of chargebacks are actually friendly fraud, meaning they were filed for invalid reasons.

Why? Many customers mistakenly believe that a chargeback is just another way to request a refund, rather than a process intended for fraud or merchant failure. 

📌 The takeaway: Most chargebacks aren’t actual fraud, but rather a result of customer confusion, impatience, or poor communication from merchants.

The solution: how to stop chargebacks before they happen

Merchants who want to stop chargebacks before they happen need a two-part strategy:

• Fast, customer-focused support to resolve issues before customers dispute charges. 
• Automated chargeback management to detect and fight disputes efficiently, so merchants don’t lose revenue to invalid claims.

Chargebacks result from slow response times, poor communication, and unresolved issues, not fraud. Adopting AI-driven customer support and chargeback automation allows businesses to significantly reduce disputes and retain more revenue. 

How AI-powered support & chargeback automation work together

Instant responses prevent frustration-driven chargebacks

Many chargebacks happen because customers don’t receive a fast enough response. In fact, 52% say they will dispute a charge if the response time is too slow. AI-powered chatbots provide real-time support, resolving issues before they escalate. 

Proactive communication reduces uncertainty

Customers expect updates regarding orders and refunds, but often don’t receive them. 80% of customers report never hearing from a merchant after filing a chargeback. 

Automated order updates, refund confirmations, and proactive notifications keep customers informed, reducing unnecessary disputes.

24/7 availability ensures no issues go unanswered

Customers expect round-the-clock support, but most businesses can’t provide live assistance. AI-powered ticketing and automation ensure every customer receives help, regardless of the time zone or urgency.

The result? Fewer chargebacks, faster resolutions, and increased customer satisfaction.

Actionable strategies for improving response times

Prioritize long-term clients

It’s impossible to please every customer. On average, chargebacks take 50 days to resolve successfully. Focus your energy on retaining high-value, long-term customers.

Prioritize high-risk inquiries

Lost inquiries take on average 15 days to resolve, and lost chargebacks take 38 days. Prioritize cases based on impact. 

Build efficient escalation systems

Advanced automated ticketing systems can route inquiries and prioritize urgent cases.

Use pre-approved resolution templates

Ensure customer service teams have quick-response templates to speed their resolutions.

Work closely with shipping carriers

“Product not received” was the most cited reason for delivery-related chargebacks. Work closely with carriers and third-party suppliers to improve fulfillment and reduce disputes.

Leverage chargeback management tools

Use automated tools for real-time analytics, enhanced communication, and proactive alerts, which will reduce response times. 

Gorgias & Chargeflow: A fully automated chargeback prevention system

Successfully tackling chargebacks requires both proactive customer support and automated dispute management. That’s why Gorgias and Chargeflow work so well together to give merchants a comprehensive defense against disputes.

Post-purchase automation isn’t just about reducing customer support workload or quick replies. It's about finding the most effective ways to increase customer loyalty and prevent disputes.

Learn more about how AI-driven automation enhances post-purchase experiences here.

How Gorgias prevents chargebacks with conversational AI

• Automated real-time responses engage customers before they decide to dispute charges.
• Proactive customer communication ensures customers receive updates on their orders, refunds, and transactions.
• 24/7 availability ensures customers receive the support they need without increasing overhead. 

How Chargeflow automates chargeback prevention & recovery

• Pre-dispute alerts notify merchants before a chargeback is finalized and provide proactive intervention.
• AI-powered chargeback responses to automate evidence collection and improve win rates. 
• Smart analytics to help merchants understand why disputes happen and how they can prevent them. 

Final thoughts: Stop chargebacks before they start

As you know, chargebacks are costly, frustrating, but most importantly, preventable. Our research shows that most chargebacks don’t stem from fraud, but from poor communication, slow response times, and customer uncertainty.

By prioritizing fast, AI-driven customer support and automated chargeback management, merchants can resolve issues before they escalate, improve customer experience, and protect their revenue. 

With Gorgias handling proactive customer support and Chargeflow managing chargeback disputes, merchants get a powerful, end-to-end prevention system that ensures fewer chargebacks, higher dispute win rates, and, at the end of the day, happier customers. 

Don’t let chargebacks drain your revenue. Take control today with faster, smarter automation.

Download Chargeflow’s full Psychology of Chargebacks Report to dive deeper into the data and start preventing disputes before they happen.

TL;DR:

• Gorgias eliminated helpdesk outages by implementing multiple database connection pools with PgBouncer, achieving over 99.99% uptime.
• We accelerated data retrieval by organizing 40TB of data into 128 partitions, reducing query times to less than 4ms.
• Our streamlined incident response process with dedicated Slack channels and clear roles now resolves almost all incidents in under an hour.
• Looking ahead, we're strengthening security, doubling our SRE team, implementing production readiness reviews, and enhancing monitoring tools.

When customer service teams are at their busiest, they need a helpdesk that keeps up. That’s exactly why our Site Reliability Engineering (SRE) team has been working behind the scenes to make the Gorgias platform faster than ever.

Over the past year, we've made remarkable improvements to our platform to eliminate bottlenecks, speed up data retrieval, and reduce incidents. For you, this means fewer disruptions, faster load times, and a more reliable helpdesk experience.

Here's how we did it.

Eliminating helpdesk outages by increasing our connection pools

The challenge

Our platform relied on a single, shared database connection pool to manage all queries. Think of it as having just one pipe handling all the water flowing through your house — when too much water rushes in at once, the whole system backs up.

In practice, this meant a single surge in database requests could clog the entire system. When lower-priority background tasks got stuck, they could prevent high-priority operations (like loading tickets or running automations) from working properly. This would cause the entire helpdesk to slow down or, worse, become completely unresponsive.

The solution

Using PgBouncer, a tool that manages database connections and reduces the load on a server, we implemented multiple connection pools. Instead of relying on a single pipeline to stream all requests, we created separate "pipes" for different requests.

Like how road traffic picks up again after an exit, routing our database traffic into separate connection pools makes sure high-priority customer interactions don’t lag behind automated background tasks.

This solution is future-proof. In the event that a lower-priority task is delayed in one connection pool, other functionalities of the helpdesk will continue working because of the remaining connection pools.

The benefits

The results speak for themselves:

• Complete elimination of helpdesk-wide outages caused by connection pooling issues
• Faster response times — 99% of automated rules tasks take less than 800 milliseconds to complete from inception
• Partial degradation instead of full outages if issues do occur — at worst, only a single feature might be affected instead of your entire helpdesk

We've eliminated incidents caused by connection pool issues in the helpdesk completely. This reduced major helpdesk outage incidents by around four per year and maintained an average uptime of over 99.99%.

Speeding up the helpdesk by organizing 40TB of data into 128 partitions

The challenge

As Gorgias grew to over 15,000 customers, so did the volume of data. We’re talking data from tickets, integrations, automations, and many more. The combination of more users and data meant slower searches within the helpdesk. 

However, the amount of data was not the problem — it was how our data was organized. 

Imagine this: An enormous storage room full of file cabinets containing every piece of data. Sure, those file cabinets kept data organized, but you would still need to spend time searching through the entire room, running up and down aisles of cabinets, to find your desired file. This method was cumbersome. 

We needed a more efficient way to keep our data easy to find, especially as more customers used our platform.

The solution

The answer was database partitioning — breaking our large datasets into smaller, more manageable segments. Using Debezium, Kafka, and Kafka-connect JDBC, all managed by Terraform, we migrated over 40TB of data, including 3.5 billion tickets, without a moment of downtime for our merchants.

Instead of a giant room with thousands of file cabinets, we divided that giant room into 128 smaller rooms. So now, instead of looking for a file in one room, you know you just need to go into room number 102, which has a much smaller area to search.

This approach allows our system to quickly pinpoint the location of data, significantly reducing the time it takes to find and deliver information to users. 

Additionally, database maintenance has become more efficient. Some of the partitions can probably sit without needing to be changed at all. We just have to maintain the partitions that are getting new files, which cuts down on maintenance time.

The benefits

Better database partitioning provides several benefits:

• Faster queries — We have an average of 600 lookups or updates per second across these databases, each taking less than 4ms
• More efficient database maintenance — We halved the number of automated maintenance runs and cut each run’s duration in half
• Better scalability as our infrastructure is now equipped to handle continued customer growth

Faster resolutions with a streamlined incident response process

The challenge

When incidents occurred in the past, our response process was inconsistent, leading to delays in resolution. It was sometimes unclear who should take the lead, what immediate actions were required, and how to effectively communicate with affected customers.

Additionally, post-incident reviews varied in quality, making it difficult to prevent similar issues from happening again. We needed a standardized framework to address incidents in a timely fashion.

The solution

To streamline incident management, we introduced a replicable, automated process:

1. Dedicated Slack channels — Every incident gets its own Slack channel, ensuring our team is immediately notified.
2. Clear roles & responsibilities — We defined specific roles so every engineer knows what next steps to take.
3. Retrospectives — After each incident, we conduct thorough post-mortems to analyze root causes, identify improvements, and share learnings across teams.
4. Proactive prevention — By improving our monitoring tools, we catch potential issues earlier, reducing the likelihood of major disruptions.

The benefits

With our improved incident management process:

• Response times have decreased significantly — almost all incidents are mitigated in under an hour
• Customers receive clearer communication during incidents, including our regularly updated Gorgias Status page
• 100+ smarter preventative measures to reduce the overall incident frequency or permanently fix recurring problems

What's next: Four ways we're improving the platform experience

With more brands catching on to how essential a solid CX platform is, our team's got our work cut out for us. Here's what's on the way:

1. Enhanced security measures — We've hired a dedicated security engineer to strengthen our security infrastructure.
2. A bigger SRE team — Our Site Reliability Engineering team has doubled, allowing us to address performance issues rapidly.
3. Production readiness reviews — We're formalizing a process to audit new and existing services, ensuring they meet our reliability standards before deployment.
4. Improved monitoring — We're investing in better monitoring tools to detect and resolve potential issues before they impact customers.

Count on a reliable future with Gorgias

Gorgias will inevitably face new challenges in performance — no system is completely immune to downtime.

But we've built our architecture with the future in mind, and it’s more resilient than ever as more and more brands realize the power of conversational AI CX platforms. 

The result? A platform you can count on to help you deliver exceptional customer service, without technical issues getting in the way.

{{lead-magnet-1}}

TL;DR:

• Use AI across both support and sales. Ecommerce brands are using AI to drive revenue and efficiency by combining automation in chat, email, and customer data with personalized product guidance and upsells.
• Analyze post-purchase surveys with AI to uncover customer insights. AI quickly identifies themes, sentiment, and trends from open-ended feedback to inform product, shipping, and support decisions.
• Predict customer intent with AI before they take action. By analyzing behavior like cart activity or page views, AI can engage high-intent shoppers with personalized nudges in real time.
• Automate QA and proactive support with AI. AI reviews 100% of conversations, flags quality issues, and triggers outreach for known problems — all before customers even ask.

Shoppers aren’t just open to AI — they’re starting to expect it.

According to IBM, 3 in 5 consumers want to use AI as they shop. And a McKinsey study found that 71% expect personalized experiences from the brands they buy from. When they don’t get that? Two-thirds say they’re frustrated.

But while most brands associate AI with support automation, its real power lies in something bigger: scaling personalization across the entire customer journey. 

We’ll show you how to do that in this article.

AI for customer data 

Before AI can personalize emails, recommend products, or answer support tickets, it needs one thing: good data.

That’s why one of the best places to start using AI isn’t in sales or support — but in enriching your customer data. With a deeper understanding of who your customers are, what they want, and how they behave, AI becomes a personalization engine across your entire business.

Enriching surveys with AI

Post-purchase surveys are gold mines for understanding customers — but digging through the data manually? Not so fun.

AI can help by analyzing survey responses at scale, identifying trends, and categorizing open-ended customer feedback into clear, actionable insights. Instead of skimming thousands of answers to spot what customers are saying about your shipping times, AI can surface those insights instantly — along with sentiment and behavior signals you might’ve missed.

Try this prompt when doing this: "Analyze 500 open-ended post-purchase survey responses. Identify the top 5 recurring themes, categorize customer sentiment (positive, neutral, negative), and surface any trends related to product quality, delivery experience, or customer support."

Predicting customer intent before they even say a word

One of AI’s biggest strengths? Spotting intent.

By analyzing things like page views, cart activity, scroll behavior, and previous purchases, AI can identify which shoppers are ready to buy, which ones are likely to churn, and which just need a little nudge to move forward.

This doesn’t just apply to email and retargeting. It also works on live chat, in real time.

Take TUSHY, for example.

To eliminate friction in the buying journey, TUSHY introduced AI Agent for Sales — a virtual assistant designed to guide shoppers toward the right product before they drop off. 

Instead of letting potential customers bounce with unanswered questions, the AI Agent steps in to offer:

• Personalized product recommendations based on shopper questions
• Compatibility guidance (especially for customers unsure which bidet works with their toilet)
• Real-time installation tips and links to helpful how-to articles

With a growing product catalog, TUSHY realized first-time buyers were overwhelmed with options — and needed help choosing what would work best for their home and hygiene preferences.

“What amazed us most is that the AI Agent doesn’t just help customers choose the perfect bidet for their booty — it also provides measurement and fit guidance, high-level installation support, and even recommends all the necessary spare parts for skirted toilet installations. It’s ushering in a new era of customer service — one that’s immediate, informative, and confidence-boosting as people rethink their bathroom habits.”

—Ren Fuller-Wasserman, Sr. Director of Customer Experience at TUSHY

Forecasting revenue by segment

AI also helps you see the road ahead.

Instead of looking at retention and loyalty metrics in isolation, AI can help you forecast what’s likely to happen next and where to focus your attention.

By segmenting customers based on behaviors like average order value, order frequency, and churn risk, AI can identify revenue opportunities and weak spots before they impact your bottom line.

All you need is the right prompt. Here’s an example you can run using your own data in any AI tool:

Prompt: “Analyze my customer data to forecast revenue by segment. Break customers into at least three groups based on behavior patterns like average order value, purchase frequency, and churn risk. 

For each segment, provide:

1. A projected revenue trend for the next quarter
2. A key insight about their behavior
3. One actionable recommendation to either grow or retain revenue from that segment.”

Here’s what a result might look like:

• VIPs (Top 5% by LTV): Predicted 15% growth next quarter based on repeat behavior
• One-time Buyers: 70% churn risk flagged—time to trigger a win-back campaign
• Discount-Only Shoppers: Revenue likely to dip unless incentive strategy changes

Instead of flying blind, you’re making decisions with clarity — and backing them with data that scales.

AI for sales 

When used strategically, AI becomes a proactive sales agent that can identify opportunities in real-time: recommending the right product to the right shopper at the right moment.

Here’s how ecommerce brands are using AI to drive revenue across every part of the funnel.

Dynamic pricing that responds to the market (and the shopper)

Your prices shouldn’t be static — especially when your competitors, inventory, and customer behavior are anything but.

AI-powered pricing tools like AI Agent for Sales help brands automatically adjust pricing based on shopper behavior. The goal is to make the right offer to the right customer.

For example:

• Show a discount to a price-sensitive shopper who’s hesitating at checkout
• Recommend premium add-ons to high-LTV customers who are more likely to spend

With dynamic pricing, you can protect your margins and boost conversions — without relying on blanket sales.

Turning chat into a personal shopper (that never sleeps)

AI-powered chat is no longer just a glorified FAQ. Today, it can act as a real-time shopping assistant — guiding customers, boosting conversions, and helping your team reclaim time.

That’s exactly what Pepper did with “Penelope,” their AI Agent built on Gorgias.

With a rapidly growing product catalog (22 new SKUs in 2024 alone), Pepper knew shoppers needed help discovering the right products. Customers often had questions about styles, materials, or sizing, and if they didn’t get answers right away, they’d abandon carts and move on.

Instead of hiring more agents to keep up, Pepper deployed Penelope to live chat and email.

Her job?

• Instantly answer questions about fit, fabric, or product differences
• Guide shoppers toward the best option for their needs
• Recommend complementary products (like matching panties or bottoms)
• Free up agents to focus on higher-value 1:1 moments, like virtual fit sessions

“With AI Agent, we’re not just putting information in our customer’s hands; we’re putting bras in their hands... We’re turning customer support from a cost center to a revenue generator.”
—Gabrielle McWhirter, CX Operations Lead at Pepper

Let’s look at how Penelope performs on the floor:

Real-time recommendations

A shopper asked about the difference between two wire-free bras. Penelope broke down the styles, support level, and fabric in plain language — then followed up with personalized suggestions based on the shopper’s preferences.

Proactive engagement

Using Gorgias Convert chat campaigns, Pepper triggers targeted messages to shoppers based on behavior. If someone is browsing white bras? Penelope jumps in and offers assistance, often leading to faster decisions and fewer abandoned carts.

Intelligent upsells

If a customer adds a swimsuit top to their cart, Penelope suggests matching bottoms. No full-screen popups, no awkward sales scripts — just thoughtful, helpful guidance.

Support and sales in one

Penelope also handles WISMO tickets and return inquiries. If a shopper is dealing with a sizing issue, Penelope walks them through the return process and links to Pepper’s Fit Guide to make sure the next purchase is spot on.

By implementing AI into chat, Pepper saw a 19% conversion rate from AI-assisted chats, an 18% uplift in AOV, and a 92.1% decrease in resolution time.

With Penelope handling repetitive and revenue-driving tasks, Pepper’s team now has more time to offer truly personalized touches — like virtual fit sessions that have turned refunds into exchanges and even upsells.

Curating bundles with AI-powered sales data

Bundling is a proven tactic for increasing AOV — but most brands still rely on subjective judgment calls or static reports to decide which products to group.

AI can take this a step further.

Instead of just looking at what’s bought together in the same cart, AI can analyze purchase sequences. For example, what people tend to buy as a follow-up 30 days after their first order. This gives you powerful clues into natural buying behavior and bundling opportunities you might’ve missed.

If you’re looking to explore this at scale, you can use anonymized sales data and feed it into AI tools to surface patterns in:

• Frequently bundled items
• Follow-up purchases within a set time frame
• High-value product pairings with repeat potential

Try this prompt:

 "Analyze this spreadsheet of order data and identify product bundle opportunities. Look for: (1) products frequently purchased together in the same order, (2) items commonly bought as a second purchase within 30 days of the first, and (3) patterns in high-value or high-frequency product pairings. Provide insights on the most promising bundles and why they might work well together."

Just make sure you’re keeping customer data anonymous — and always double-check the insights with your team.

Related: Ecommerce product categorization: How to organize your products

AI for support

AI isn’t just here to deflect tickets. From quality assurance to proactive outreach, AI can elevate the entire support experience — on both sides of the conversation.

Quality checks powered by AI

Manual QA is slow, selective, and often feels like it’s chasing the wrong tickets.

That’s where Auto QA comes in. Instead of reviewing just a handful of conversations each week, Auto QA evaluates 100% of private messages, whether they’re handled by a human or an AI agent.

Every message is scored on key metrics like:

• Resolution completeness
• Brand voice
• Empathy and tone
• Accuracy

It gives support leaders a full picture of how their team is performing, so they can coach with clarity, not just gut feeling.

Here’s what brands can do with automated QA:

• Save time by focusing only on the conversations that need attention
• Ensure consistency across agents and AI with a single scoring standard
• Improve agent performance with targeted coaching and feedback
• Deliver higher-quality support that customers actually notice

Let’s walk through a real example.

Customer: “Hi, my device broke, and I bought it less than a month ago.”

Agent: “Hi Kelly, please send us a photo or a video so we can determine the issue with your device.”

Auto QA flags this interaction with:

• Communication Score: 3/5 — The agent was clear, but could have shown more empathy in tone.
• Resolution Score: Complete — The issue was addressed effectively.

Proactive support that reaches out first

Reactive support is table stakes. AI takes it a step further by anticipating issues before they happen — and proactively helping customers.

Let’s say login errors spike after a product update. AI detects the surge and automatically triggers an email to affected customers with a simple fix. No need for them to dig through help docs or wait on chat — support meets them right where they are.

Proactive AI can also be used for:

• Order delay notifications with live tracking updates
• Subscription renewal reminders
• Back-in-stock alerts with support follow-up for next steps

This saves the time of your agents because the AI will spot problems before they turn into tickets.

Understanding sentiment at scale

Your customers are telling you what they think. AI just helps you hear it more clearly.

By analyzing reviews, support tickets, post-purchase surveys, and social comments, AI can spot sentiment trends that might otherwise fly under the radar.

For example:

• Multiple reviews mention “runs small”? AI flags it, so your team can update the product description or add a sizing chart.
• A sudden rise in “frustrated” language in support tickets? Time to check if something’s off with your shipping or product quality.

Related: 12 ways to upgrade your data and trend analysis with Ticket Fields 

Personalization at scale starts with the right AI stack

Whether you’re enriching customer data, making smarter product recommendations, triggering dynamic pricing, or proactively resolving support issues, AI gives your team the power to scale personalization without sacrificing quality.

With Gorgias, you can bring many of these use cases to life — from AI-powered chat that drives conversions to automated support that still feels human. 

And with our app store, you can tap into additional AI tools for data enrichment, direct mail, bundling insights, and more.

Personalized ecommerce doesn’t have to mean more work. With the right AI tools in your corner, it means smarter work — and better results.

{{lead-magnet-1}}