The Galera Crossroads, Part 2: Swap the Binaries, Keep the Cluster
Zero downtime, zero new hardware: converting MySQL Galera Cluster to Percona XtraDB Cluster node by node 
In my previous blog I have highlighted how the ones that were using the Codership version of MySQL + Galera are at a crossroad and must take a decision about their freedom and future.
Percona with the new PXC 8.4.11 had made things very simple for users to keep shifting from the dismissed Codership version to the fully maintained Percona Xtradb Cluster (PXC).
While you can get more information about how to cover different cases reading the Percona PXC Manual I am going to focus on a specific case, the one that for me is going to be the most useful when in production, the change in flight.
First of all try to upgrade to one of the latest version of Codership/Galera, for this test I was using:
Codership MySQL (wsrep) mysql-wsrep-8.4.6-26.24-linux-x86_64.tar.gz
Galera galera-4-26.4.24-Linux-x86_64.tar.gz
As said for PXC I am on the 8.4.11 release of PXC, specifically Percona-XtraDB-Cluster_8.4.11-11.1_Linux.x86_64.glibc2.28-minimal.tar.gz.
The process
Preliminary steps
This is an inflight process, a bit as if while you are at 33000 feets flying at 600mph someone starts to replace the engines one by one. It can be done but you need to be mindful. In our case this means, no DDL, low write rate, no long transaction, no data load.
Just to be sure we do not keep it unsaid, this is a migration so you must start and finish it and do not keep things mixed.
Take a full backup before start, and validate it! A backup that is not validated is just a waste of disk space.
PXC come with two settings that we must take into account:
--pxc-strict-mode
--pxcencrypt-cluster-traffic
About the first we need to disable it for the time of the migration so in our my.cnf we need to add:
--pxc-strict-mode=DISABLED
For pxcencrypt-cluster-traffic we have two options:
- No encryption currently in place: loose_pxcencrypt-cluster-traffic=OFF we disable the encryption in PXC
- With encryption currently in place then read Encrypt PXC traffic to correctly set your PXC instance to use the certificates from the current environment.
Keep in mind that by default pxcencrypt-cluster-traffic in PXC is on so to disable you must explicitly set it.
Last thing to be sure about is the SST method, let us use Xtrabackup to be sure as such be sure to have: wsrep-sst-method = xtrabackup-v2 and to have the Xtrabackup binaries in place. To be clear, we should NOT go for an SST, but just in case.
Swapping the binaries
To make my life simpler when I need to update MySQL or like in this case do tests I use symbolic links to binaries:
lrwxrwxrwx. 1 mysql mysql 36 Jul 7 14:24 COD84P -> mysql-wsrep-8.4.6-26.24-linux-x86_64 lrwxrwxrwx. 1 mysql mysql 65 Oct 14 18:15 PXC8P -> Percona-XtraDB-Cluster_8.4.11-11.1_Linux.x86_64.glibc2.35-minimal
So what I will do to swap the binaries on the node will be to have the link COD84P pointing to PXC8P and restart the instance, easy peasy.
Initial state
Ok this is normally what you have:
I have put ProxySQL here but it could have been HAProxy or any other proxy/tool that allows you to define a writer and N reader, because you are doing that right? You are NOT writing over all the nodes, correct? Because if you do so, well stop here and read these instructions. Do you remember … flying at 33K feet 600mph? Something simply cannot be done.
sysbench oltp_write.lua --mysql-host=<> --mysql-port=6033 --mysql-user=app_test --mysql-password=<> --mysql-db=migrate --db-driver=mysql --skip_trx=on --report-interval=1 --mysql-ignore-errors=none --histogram --table_name=mill --stats_format=csv --db-ps-mode=disable --mysql-ssl=preferred --threads=256 --tables=5 --table_size=10000 run
Step 1
Now while we have the app running and modifying data, we will stop one of the non primary nodes. Remember the concept of “Primary” is artificial when using Galera, and is achieved by electing one of the nodes as writer in ProxySQL:
+--------------+---------------+------+--------+--------------------------+ | hostgroup_id | hostname | port | status | comment | +--------------+---------------+------+--------+--------------------------+ | 100 | 192.168.4.205 | 3308 | ONLINE | Node1 main primary | | 101 | 192.168.4.205 | 3308 | ONLINE | Node1 | | 101 | 192.168.4.21 | 3308 | ONLINE | Node2 first failover | | 101 | 192.168.4.231 | 3308 | ONLINE | Node3 second failover | +--------------+---------------+------+--------+--------------------------+
Once we stop the node n3:
View: id: 036d186c-77cf-11f1-8338-fb6a1aa4f205:470059 status: primary protocol_version: 7 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 0 members(2): 0: 88c98ce9-c97e-11f1-ba76-9a0a085461a2, cnode1 1: f0a3125c-cca4-11f1-8e8a-b611d8c7e61a, cnode2
We have the new node view in the cluster that will continue to serve traffic, while we can modify the binaries in the stopped node.
lrwxrwxrwx. 1 mysql mysql 36 Jul 7 14:24 COD84P -> mysql-wsrep-8.4.6-26.24-linux-x86_64 root@pxc3:/opt/mysql_templates# rm -f COD84P; ln -s PXC8P COD84P;chown -R mysql:mysql COD84P; root@pxc3:/opt/mysql_templates# ll COD84P lrwxrwxrwx. 1 mysql mysql 5 Oct 21 10:58 COD84P -> PXC8P
Done, let us restart the instance:
2026-10-21T09:03:17.806872Z 0 [System] [MY-000000] [WSREP] P: Member 0.2 (cnode3) requested state transfer from '*any*'. Selected 2.2 (cnode2)(SYNCED) as donor. 2026-10-21T09:03:19.122175Z 0 [System] [MY-000000] [WSREP] P: 2.2 (cnode2): State transfer to 0.2 (cnode3) complete. 2026-10-21T09:03:19.125662Z 0 [System] [MY-000000] [WSREP] P: Member 2.2 (cnode2) synced with group. 2026-10-21T09:04:00.976655Z 0 [System] [MY-000000] [WSREP] P: 0.2 (cnode3): State transfer from 2.2 (cnode2) complete. 2026-10-21T09:04:07.909508Z 0 [System] [MY-000000] [WSREP] P: Member 0.2 (cnode3) synced with group.
After a few seconds of synchronization, we can see that our node 3 is back online and working using Percona PXC binaries:
Server version:8.4.11-11.1 Percona XtraDB Cluster binary (GPL) 8.4.11, Revision 33f7510, WSREP version 26.1.4.3
Step 2
Let us do the same for node2 and stop it:
View: id: 036d186c-77cf-11f1-8338-fb6a1aa4f205:758293 status: primary protocol_version: 7 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 1 members(2): 0: 3f0059e6-cd2e-11f1-9129-d24205451337, cnode3 1: 88c98ce9-c97e-11f1-ba76-9a0a085461a2, cnode1
And cluster on proxysql is:
+--------------+---------------+------+--------+--------------------------------------------------------------------------+ | hostgroup_id | hostname | port | status | comment | +--------------+---------------+------+--------+--------------------------------------------------------------------------+ | 100 | 192.168.4.205 | 3308 | ONLINE | Node1 main primary | | 101 | 192.168.4.205 | 3308 | ONLINE | Node1 | | 101 | 192.168.4.231 | 3308 | ONLINE | Node3 second failover 100_W_101_R_retry_up=0;100_W_101_R_retry_down=0; | +--------------+---------------+------+--------+--------------------------------------------------------------------------+
We change the link:
root@pxc2:/opt/mysql_templates# ll COD84P lrwxrwxrwx. 1 mysql mysql 5 Oct 21 11:31 COD84P -> PXC8P
And again we can see the new node joining and cluster synchronizing:
View: id: 036d186c-77cf-11f1-8338-fb6a1aa4f205:825441 status: primary protocol_version: 7 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 2 members(3): 0: 3f0059e6-cd2e-11f1-9129-d24205451337, cnode3 1: 5153a9e8-cd32-11f1-9034-17a540af26a2, cnode2 2: 88c98ce9-c97e-11f1-ba76-9a0a085461a2, cnode1 ================================================= 2026-10-21T09:32:23.708142Z 12 [System] [MY-000000] [WSREP] wsrep_notify_cmd is not defined, skipping notification. 2026-10-21T09:32:23.712580Z 12 [System] [MY-000000] [WSREP] P: Lowest cert index boundary for CC from group: 825392 2026-10-21T09:32:23.712662Z 12 [System] [MY-000000] [WSREP] P: Min available from gcache for CC from group: 1 2026-10-21T09:32:25.281302Z 0 [System] [MY-000000] [WSREP] P: (88c98ce9-ba76, 'tcp://10.0.0.205:4569') turning message relay requesting off 2026-10-21T09:32:26.880929Z 0 [System] [MY-000000] [WSREP] P: Member 1.2 (cnode2) requested state transfer from '*any*'. Selected 0.2 (cnode3)(SYNCED) as donor. 2026-10-21T09:32:30.851066Z 0 [System] [MY-000000] [WSREP] P: 0.2 (cnode3): State transfer to 1.2 (cnode2) complete. 2026-10-21T09:32:30.853938Z 0 [System] [MY-000000] [WSREP] P: Member 0.2 (cnode3) synced with group. 2026-10-21T09:33:54.295264Z 0 [System] [MY-000000] [WSREP] P: 1.2 (cnode2): State transfer from 0.2 (cnode3) complete. 2026-10-21T09:34:04.065755Z 0 [System] [MY-000000] [WSREP] P: Member 1.2 (cnode2) synced with group.
Node using Percona PXC:
Server version: 8.4.11-11.1 Percona XtraDB Cluster binary (GPL) 8.4.11, Revision 33f7510, WSREP version 26.1.4.3
Step 3
Now is the moment to shift the writes from current Primary to new primary.
To do so we will shift the current primary to OFFLINE_SOFT to give the connections time to complete the operations, and at the same time we will move to writer group node2:
UPDATE mysql_servers set status='OFFLINE_SOFT' where hostgroup_id=100 and hostname='192.168.4.205';
INSERT INTO mysql_servers (hostname,hostgroup_id,port,weight,max_connections,comment) VALUES ('192.168.4.21',100,3308,998,2000,'Node2 Writer');
LOAD MYSQL SERVERS TO RUNTIME; SAVE MYSQL SERVERS TO DISK;
+--------------+---------------+------+--------------+----------------------------------------------------------------------------+
| hostgroup_id | hostname | port | status | comment |
+--------------+---------------+------+--------------+----------------------------------------------------------------------------+
| 100 | 192.168.4.205 | 3308 | OFFLINE_SOFT | Node1 main primary |
| 100 | 192.168.4.21 | 3308 | ONLINE | Node2 Writer |
| 101 | 192.168.4.205 | 3308 | ONLINE | Node1 |
| 101 | 192.168.4.21 | 3308 | ONLINE | Node2 first failover 100_W_101_R_retry_up=0;100_W_101_R_retry_down=0; |
| 101 | 192.168.4.231 | 3308 | ONLINE | Node3 second failover 100_W_101_R_retry_up=0;100_W_101_R_retry_down=0; |
+--------------+---------------+------+--------------+----------------------------------------------------------------------------+
Once the connections on Node1 are gone to 0:
select c.hostgroup,c.srv_host,c.srv_port,c.status,c.ConnUsed from stats_mysql_connection_pool c left JOIN runtime_mysql_servers b ON c.hostgroup=b.hostgroup_id and c.srv_host=b.hostname and c.srv _port = b.port order by hostgroup,srv_host desc; +-----------+---------------+----------+--------------+----------+ | hostgroup | srv_host | srv_port | status | ConnUsed | +-----------+---------------+----------+--------------+----------+ | 100 | 192.168.4.21 | 3308 | ONLINE | 257 | | 100 | 192.168.4.205 | 3308 | OFFLINE_SOFT | 0 |<---- | 101 | 192.168.4.231 | 3308 | ONLINE | 0 | | 101 | 192.168.4.21 | 3308 | ONLINE | 0 | | 101 | 192.168.4.205 | 3308 | ONLINE | 0 | +-----------+---------------+----------+--------------+----------+
We are ready to start and do the final swap.
Step 4
I stop the node1 and cluster creates a new view and continue to serve the traffic as expected:
View: id: 036d186c-77cf-11f1-8338-fb6a1aa4f205:1079040 status: primary protocol_version: 7 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 1 members(2): 0: 3f0059e6-cd2e-11f1-9129-d24205451337, cnode3 1: 5153a9e8-cd32-11f1-9034-17a540af26a2, cnode2
Now let me do the binary swap:
root@pxc1:/opt/mysql_templates# ll COD84P lrwxrwxrwx. 1 mysql mysql 5 Oct 21 11:56 COD84P -> PXC8P
And after the usual synchronization we have the cluster fully using Percona PXC:
View: id: 036d186c-77cf-11f1-8338-fb6a1aa4f205:1109495 status: primary protocol_version: 7 capabilities: MULTI-MASTER, CERTIFICATION, PARALLEL_APPLYING, REPLAY, ISOLATION, PAUSE, CAUSAL_READ, INCREMENTAL_WS, UNORDERED, PREORDERED, STREAMING, NBO final: no own_index: 1 members(3): 0: 3f0059e6-cd2e-11f1-9129-d24205451337, cnode3 1: 5153a9e8-cd32-11f1-9034-17a540af26a2, cnode2 2: b7669a8f-cd35-11f1-8ea4-527375537a0b, cnode1 ================================================= 2026-10-21T09:56:43.482088Z 1 [Note] [MY-000000] [WSREP] wsrep_notify_cmd is not defined, skipping notification. 2026-10-21T09:56:43.508133Z 1 [Note] [MY-000000] [Galera] Recording CC from group: 1109495 2026-10-21T09:56:43.508222Z 1 [Note] [MY-000000] [Galera] Lowest cert index boundary for CC from group: 1109364 2026-10-21T09:56:43.508260Z 1 [Note] [MY-000000] [Galera] Min available from gcache for CC from group: 4 2026-10-21T09:56:45.000170Z 0 [Note] [MY-000000] [Galera] (5153a9e8-9034, 'tcp://10.0.0.21:4569') turning message relay requesting off 2026-10-21T09:56:47.022659Z 0 [Note] [MY-000000] [Galera] Member 2.2 (cnode1) requested state transfer from '*any*'. Selected 0.2 (cnode3)(SYNCED) as donor. 2026-10-21T09:56:49.599440Z 0 [Note] [MY-000000] [Galera] 0.2 (cnode3): State transfer to 2.2 (cnode1) complete. 2026-10-21T09:56:49.601423Z 0 [Note] [MY-000000] [Galera] Member 0.2 (cnode3) synced with group. 2026-10-21T09:57:49.384391Z 0 [Note] [MY-000000] [Galera] 2.2 (cnode1): State transfer from 0.2 (cnode3) complete. 2026-10-21T09:57:56.826355Z 0 [Note] [MY-000000] [Galera] Member 2.2 (cnode1) synced with group.
Final refining
Now all the cluster is using PXC so we can configure the variables we have left out like
--pxc-strict-mode read instructions
--pxcencrypt-cluster-traffic read instructions
Also if not really relevant we may also wish to switch back the write to Node1, which can be done as easily as we have done the previous shift:
UPDATE mysql_servers set status='OFFLINE_SOFT' where hostgroup_id=100 and hostname='192.168.4.21';
INSERT INTO mysql_servers (hostname,hostgroup_id,port,weight,max_connections,comment) VALUES ('192.168.4.205',100,3308,1000,2000,'Node1 Primary Writer');
LOAD MYSQL SERVERS TO RUNTIME; SAVE MYSQL SERVERS TO DISK;
+--------------+---------------+------+--------------+------------------------------------------------------------------------------+
| hostgroup_id | hostname | port | status | comment |
+--------------+---------------+------+--------------+------------------------------------------------------------------------------+
| 100 | 192.168.4.205 | 3308 | ONLINE | Node1 Primary Writer |
| 100 | 192.168.4.21 | 3308 | OFFLINE_SOFT | Node2 Writer |
| 101 | 192.168.4.205 | 3308 | ONLINE | Node1 100_W_101_R_retry_up=0;100_W_101_R_retry_down=0; |
| 101 | 192.168.4.21 | 3308 | ONLINE | Node2 first failover 100_W_101_R_retry_up=0;100_W_101_R_retry_down=0; |
| 101 | 192.168.4.231 | 3308 | ONLINE | Node3 second failover 100_W_101_R_retry_up=0;100_W_101_R_retry_down=0; |
+--------------+---------------+------+--------------+------------------------------------------------------------------------------+
Done!
Rollback
There is no compatibility if you cannot rollback what you have done if you do not like it.
So it is important to have the process clear. Yes we can rollback the installation, however given a current limitation in galera itself, once the cluster you have upgraded has all nodes using a specific Galera Protocol version you cannot inject a node with a lower protocol version number. This is not dependent on using Codership version or PXC but by the galera library used.
In those cases the only option we have is to put the cluster down and restart it with the different libraries/binaries.
A reliable compatibility plan requires a clear rollback process. While rolling back an installation is possible, a limitation within the core Galera library regardless of whether you use Codership or Percona XtraDB Cluster (PXC) complicates the procedure. Once an upgraded cluster has all nodes running on a newer Galera Protocol version, it is impossible to introduce a node running a lower protocol version. To perform a rollback in this scenario, the entire cluster must be taken offline and completely restarted using the older binaries and libraries.
Full Cluster Downgrade Procedure
Because a rolling downgrade is not possible when the protocol versions differ, you must perform a full cluster restart.
- Halt Application Traffic: Stop all application writes to the cluster to prevent data loss or inconsistencies during the shutdown process.
- Shut Down the Cluster: Gracefully stop the database service on every node in the cluster one at a time.
- Verify grastate.dat: Inspect the grastate.dat file (typically located in /var/lib/mysql/) on all nodes. Ensure the seqno is not -1 (which indicates an unclean shutdown) and identify the node with the highest seqno. This node holds the most up-to-date data and must be used to bootstrap the downgraded cluster.
- Downgrade the Binaries/Library
- Bootstrap the First Node: On the node you identified with the highest seqno, bootstrap the new cluster.
- Verify Bootstrap Success and data correctness.
- Start Remaining Nodes: Start the database service on the other nodes one by one. Monitor the MySQL error logs and wait for each node to fully join and synchronize (via SST or IST) before starting the next one.
- Restore Traffic: Once wsrep_cluster_size matches your total node count and all nodes report wsrep_ready = ON, allow application traffic back into the cluster.
Conclusion
What I have just done is, honestly, not much more than a rolling restart.
One node at a time we stopped the service, pointed a symbolic link to the new binaries, started it again, and watched it rejoin the cluster through IST.
No new hardware, no logical dump and reload, no new cluster to build alongside the old one, and above all no downtime for the application: sysbench kept writing for the whole migration, and the only “visible” event was a controlled writer switch in ProxySQL.
If you have ever performed a minor version upgrade on a Galera cluster, you already have all the skills needed to move to PXC.
This is possible because PXC is not “another product” for a Galera user. It is Galera, kept alive, maintained, and open.
Percona has made sure that a MySQL Galera node and a PXC 8.4.11 node can sit in the same cluster, talk to each other, and exchange state transfers as if nothing had changed.
While the Codership MySQL distribution is being dismissed, PXC gives us a fully open source, actively developed path forward, with the extra value Percona adds on top: Percona Server underneath, PXC strict mode, built-in traffic encryption, integration with XtraBackup and PMM, and a community that is not going away.
Choosing PXC is not abandoning Galera, it is the way we keep Galera going.
That said, let us not forget the plane at 33,000 feet.
Replacing the engines in flight works, but only if we fly carefully while doing it. During the migration we must accept a few limitations: no DDL, no long running transactions, no bulk data loads, and a reduced write rate to keep contention and certification conflicts low. Writes must go to a single node, and a proper validated backup must be in place before we start. Features like pxc_strict_mode and cluster traffic encryption must be relaxed for the duration of the swap and re-enabled only once the whole cluster is on PXC.
Finally, a migration is a migration: once started, we finish it. A mixed cluster is a transition state, not a place to live.
Respect these rules, and the move from MySQL Galera Cluster to Percona XtraDB Cluster becomes what it should be: a quiet, almost boring operation. And in production, boring is exactly what we want.
Great MySQL to everyone!





At Percona Live Europe 2026, Dmitry Lenev and I presented 

