MHA手动切换示例

[root@MHA-Manager ~]# masterha_master_switch --master_state=alive --conf=/etc/mha/app1.conf --new_master_host=192.168.124.202

Thu Oct 22 23:26:06 2020 - [info] MHA::MasterRotate version 0.58.

Thu Oct 22 23:26:06 2020 - [info] Starting online master switch..

Thu Oct 22 23:26:06 2020 - [info]

Thu Oct 22 23:26:06 2020 - [info] * Phase 1: Configuration Check Phase..

Thu Oct 22 23:26:06 2020 - [info]

Thu Oct 22 23:26:06 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.

Thu Oct 22 23:26:06 2020 - [info] Reading application default configuration from /etc/mha/app1.conf..

Thu Oct 22 23:26:06 2020 - [info] Reading server configuration from /etc/mha/app1.conf..

Thu Oct 22 23:26:07 2020 - [info] GTID failover mode = 0

Thu Oct 22 23:26:07 2020 - [info] Current Alive Master: 192.168.124.201(192.168.124.201:3306)

Thu Oct 22 23:26:07 2020 - [info] Alive Slaves:

Thu Oct 22 23:26:07 2020 - [info] 192.168.124.202(192.168.124.202:3306) Version=5.7.26-log (oldest major version between slaves) log-bin:enabled

Thu Oct 22 23:26:07 2020 - [info] GTID ON

Thu Oct 22 23:26:07 2020 - [info] Replicating from 192.168.124.201(192.168.124.201:3306)

Thu Oct 22 23:26:07 2020 - [info] Primary candidate for the new Master (candidate_master is set)

Thu Oct 22 23:26:07 2020 - [info] 192.168.124.203(192.168.124.203:3306) Version=5.7.26-log (oldest major version between slaves) log-bin:enabled

Thu Oct 22 23:26:07 2020 - [info] GTID ON

Thu Oct 22 23:26:07 2020 - [info] Replicating from 192.168.124.201(192.168.124.201:3306)

Thu Oct 22 23:26:07 2020 - [info] Primary candidate for the new Master (candidate_master is set)

It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on 192.168.124.201(192.168.124.201:3306)? (YES/no): yes

Thu Oct 22 23:26:47 2020 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..

Thu Oct 22 23:26:47 2020 - [info] ok.

Thu Oct 22 23:26:47 2020 - [info] Checking MHA is not monitoring or doing failover..

Thu Oct 22 23:26:47 2020 - [info] Checking replication health on 192.168.124.202..

Thu Oct 22 23:26:47 2020 - [info] ok.

Thu Oct 22 23:26:47 2020 - [info] Checking replication health on 192.168.124.203..

Thu Oct 22 23:26:47 2020 - [info] ok.

Thu Oct 22 23:26:47 2020 - [info] 192.168.124.202 can be new master.

Thu Oct 22 23:26:47 2020 - [info]

From:

192.168.124.201(192.168.124.201:3306) (current master)

+--192.168.124.202(192.168.124.202:3306)

+--192.168.124.203(192.168.124.203:3306)

To:

192.168.124.202(192.168.124.202:3306) (new master)

+--192.168.124.203(192.168.124.203:3306)

Starting master switch from 192.168.124.201(192.168.124.201:3306) to 192.168.124.202(192.168.124.202:3306)? (yes/NO): yes

Thu Oct 22 23:26:57 2020 - [info] Checking whether 192.168.124.202(192.168.124.202:3306) is ok for the new master..

Thu Oct 22 23:26:57 2020 - [info] ok.

Thu Oct 22 23:26:57 2020 - [info] ** Phase 1: Configuration Check Phase completed.

Thu Oct 22 23:26:57 2020 - [info]

Thu Oct 22 23:26:57 2020 - [info] * Phase 2: Rejecting updates Phase..

Thu Oct 22 23:26:57 2020 - [info]

Thu Oct 22 23:26:57 2020 - [info] Executing master ip online change script to disable write on the current master:

Thu Oct 22 23:26:57 2020 - [info] /etc/mha/master_ip_online_change --command=stop --orig_master_host=192.168.124.201 --orig_master_ip=192.168.124.201 --orig_master_port=3306 --orig_master_user='ha_monitor' --new_master_host=192.168.124.202 --new_master_ip=192.168.124.202 --new_master_port=3306 --new_master_user='ha_monitor' --orig_master_ssh_user=root --new_master_ssh_user=root --orig_master_password=xxx --new_master_password=xxx

Thu Oct 22 23:26:57 2020 201795 Set read_only on the new master.. ok.

Thu Oct 22 23:26:57 2020 233300 Drpping app user on the orig master..

Thu Oct 22 23:26:57 2020 233836 Waiting all running 2 threads are disconnected.. (max 1500 milliseconds)

{'Time' => '642','db' => undef,'Id' => '9','User' => 'repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Command' => 'Binlog Dump GTID','Info' => undef,'Host' => '192.168.124.202:53526'}

{'Time' => '623','db' => undef,'Id' => '10','User' => 'repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Command' => 'Binlog Dump GTID','Info' => undef,'Host' => '192.168.124.203:41798'}

Thu Oct 22 23:26:57 2020 738930 Waiting all running 3 threads are disconnected.. (max 1000 milliseconds)

{'Time' => '642','db' => undef,'Id' => '9','User' => 'repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Command' => 'Binlog Dump GTID','Info' => undef,'Host' => '192.168.124.202:53526'}

{'Time' => '623','db' => undef,'Id' => '10','User' => 'repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Command' => 'Binlog Dump GTID','Info' => undef,'Host' => '192.168.124.203:41798'}

{'Time' => '0','db' => undef,'Id' => '18','User' => 'ha_monitor','State' => '','Command' => 'Sleep','Info' => undef,'Host' => '192.168.124.204:41112'}

Thu Oct 22 23:26:58 2020 244830 Waiting all running 2 threads are disconnected.. (max 500 milliseconds)

{'Time' => '643','db' => undef,'Id' => '9','User' => 'repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Command' => 'Binlog Dump GTID','Info' => undef,'Host' => '192.168.124.202:53526'}

{'Time' => '624','db' => undef,'Id' => '10','User' => 'repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Command' => 'Binlog Dump GTID','Info' => undef,'Host' => '192.168.124.203:41798'}

Thu Oct 22 23:26:58 2020 747863 Set read_only=1 on the orig master.. ok.

Thu Oct 22 23:26:58 2020 749457 Waiting all running 3 queries are disconnected.. (max 500 milliseconds)

{'Time' => '643','db' => undef,'Id' => '9','User' => 'repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Command' => 'Binlog Dump GTID','Info' => undef,'Host' => '192.168.124.202:53526'}

{'Time' => '624','db' => undef,'Id' => '10','User' => 'repl','State' => 'Master has sent all binlog to slave; waiting for more updates','Command' => 'Binlog Dump GTID','Info' => undef,'Host' => '192.168.124.203:41798'}

{'Time' => '0','db' => undef,'Id' => '18','User' => 'ha_monitor','State' => '','Command' => 'Sleep','Info' => undef,'Host' => '192.168.124.204:41112'}

disable the VIP on old master: 192.168.124.201

SIOCSIFFLAGS: Cannot assign requested address

Thu Oct 22 23:26:59 2020 705196 Killing all application threads..

Thu Oct 22 23:26:59 2020 722604 done.

Thu Oct 22 23:26:59 2020 - [info] ok.

Thu Oct 22 23:26:59 2020 - [info] Locking all tables on the orig master to reject updates from everybody (including root):

Thu Oct 22 23:26:59 2020 - [info] Executing FLUSH TABLES WITH READ LOCK..

Thu Oct 22 23:26:59 2020 - [info] ok.

Thu Oct 22 23:26:59 2020 - [info] Orig master binlog:pos is log-bin.000012:154.

Thu Oct 22 23:26:59 2020 - [info] Waiting to execute all relay logs on 192.168.124.202(192.168.124.202:3306)..

Thu Oct 22 23:26:59 2020 - [info] master_pos_wait(log-bin.000012:154) completed on 192.168.124.202(192.168.124.202:3306). Executed 0 events.

Thu Oct 22 23:26:59 2020 - [info] done.

Thu Oct 22 23:26:59 2020 - [info] Getting new master's binlog name and position..

Thu Oct 22 23:26:59 2020 - [info] log-bin.000001:154

Thu Oct 22 23:26:59 2020 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='192.168.124.202', MASTER_PORT=3306, MASTER_LOG_FILE='log-bin.000001', MASTER_LOG_POS=154, MASTER_USER='repl', MASTER_PASSWORD='xxx';

Thu Oct 22 23:26:59 2020 - [info] Executing master ip online change script to allow write on the new master:

Thu Oct 22 23:26:59 2020 - [info] /etc/mha/master_ip_online_change --command=start --orig_master_host=192.168.124.201 --orig_master_ip=192.168.124.201 --orig_master_port=3306 --orig_master_user='ha_monitor' --new_master_host=192.168.124.202 --new_master_ip=192.168.124.202 --new_master_port=3306 --new_master_user='ha_monitor' --orig_master_ssh_user=root --new_master_ssh_user=root --orig_master_password=xxx --new_master_password=xxx

Thu Oct 22 23:27:00 2020 038734 Set read_only=0 on the new master.

Thu Oct 22 23:27:00 2020 039901 Creating app user on the new master..

enable the VIP: 192.168.124.205 on the new master: 192.168.124.202

Thu Oct 22 23:27:03 2020 - [info] ok.

Thu Oct 22 23:27:03 2020 - [info]

Thu Oct 22 23:27:03 2020 - [info] * Switching slaves in parallel..

Thu Oct 22 23:27:03 2020 - [info]

Thu Oct 22 23:27:03 2020 - [info] -- Slave switch on host 192.168.124.203(192.168.124.203:3306) started, pid: 9943

Thu Oct 22 23:27:03 2020 - [info]

Thu Oct 22 23:27:04 2020 - [info] Log messages from 192.168.124.203 ...

Thu Oct 22 23:27:04 2020 - [info]

Thu Oct 22 23:27:03 2020 - [info] Waiting to execute all relay logs on 192.168.124.203(192.168.124.203:3306)..

Thu Oct 22 23:27:03 2020 - [info] master_pos_wait(log-bin.000012:154) completed on 192.168.124.203(192.168.124.203:3306). Executed 0 events.

Thu Oct 22 23:27:03 2020 - [info] done.

Thu Oct 22 23:27:03 2020 - [info] Resetting slave 192.168.124.203(192.168.124.203:3306) and starting replication from the new master 192.168.124.202(192.168.124.202:3306)..

Thu Oct 22 23:27:03 2020 - [info] Executed CHANGE MASTER.

Thu Oct 22 23:27:03 2020 - [info] Slave started.

Thu Oct 22 23:27:04 2020 - [info] End of log messages from 192.168.124.203 ...

Thu Oct 22 23:27:04 2020 - [info]

Thu Oct 22 23:27:04 2020 - [info] -- Slave switch on host 192.168.124.203(192.168.124.203:3306) succeeded.

Thu Oct 22 23:27:04 2020 - [info] Unlocking all tables on the orig master:

Thu Oct 22 23:27:04 2020 - [info] Executing UNLOCK TABLES..

Thu Oct 22 23:27:04 2020 - [info] ok.

Thu Oct 22 23:27:04 2020 - [info] All new slave servers switched successfully.

Thu Oct 22 23:27:04 2020 - [info]

Thu Oct 22 23:27:04 2020 - [info] * Phase 5: New master cleanup phase..

Thu Oct 22 23:27:04 2020 - [info]