[pgsql-jp: 34388] Re: pgpool 2.3.1 で縮退運転に失敗します

[Ishikawa Tomoyuki]

石川です。

> 1) マスタ側が落ちたとき
>
> SIGQUITで落ちるときのケースを想定してませんでした．SIGQUITではなく，
> SIGINT(fast shutdown)でマスタを落としたときにうまくいくかどうか，うま
> くいかないときはデバッグ出力を見せていただけますか？

試してみたところ、うまく動きました。
参考になるかわかりませんが以下がその時のデバッグログです。

●Kill -INT でMaster 側のpostmaster を停止してデータベースへアクセス
---
postgres=# select * from test;
ERROR:  kind mismatch between backends
HINT:  check data consistency between master and secondary
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.
postgres=#
---
DEBUG: pid 706: read kind from frontend Q(51)
DEBUG: pid 706: Query: select * from test;
DEBUG: pid 706: read kind from backend E
DEBUG: pid 706: pool_process_query: waiting for secondary for data ready
ERROR: pid 706: pool_process_query: kind does not match between backends
master(
E) secondary(T)
DEBUG: pid 706: len:109
LOG: pid 706: notice_backend_error: master: 1 fail over request from pid 706
DEBUG: pid 674: failover_handler called
LOG: pid 674: starting degeneration. shutdown master host Master_Host(5432)
LOG: pid 706: do_child: exits with status 1 due to error
DEBUG: pid 717: I am 717
DEBUG: pid 717: I am 717 accept fd 6
DEBUG: pid 717: Protocol Major: 3 Minor: 0 database: postgres user: postgres
DEBUG: pid 718: I am 718
DEBUG: pid 719: I am 719
DEBUG: pid 720: I am 720
DEBUG: pid 721: I am 721
DEBUG: pid 722: I am 722
LOG: pid 674: degeneration done. shutdown master host Master_Host(5432)
DEBUG: pid 723: I am 723
DEBUG: pid 724: I am 724
DEBUG: pid 717: name: client_encoding value: SQL_ASCII
DEBUG: pid 717: name: DateStyle value: ISO, MDY
DEBUG: pid 717: name: is_superuser value: on
DEBUG: pid 717: name: server_version value: 7.4.3
DEBUG: pid 717: name: session_authorization value: postgres
DEBUG: pid 717: read kind from backend pending data Z len: 5 po: 160
DEBUG: pid 717: ReadyForQuery: message length: 5
DEBUG: pid 717: ReadyForQuery: transaction state: I
DEBUG: pid 725: I am 725
DEBUG: pid 725: I am 726
：
：（中略）
：
DEBUG: pid 748: I am 748
---


> 2) セカンダリ側が落ちたとき
>
> 子プロセスが死んだときにあがるSIGCHLDと，フェイルオーバリクエスト用の
> シグナルSIGUSR2の上がるタイミングが悪さをして，wait(2)の中でブロックし
> ているようです．添付のパッチをmain.cにあて，状況が改善されるかどうか確
> 認していただけますでしょうか?

添付されたパッチをあてて再インストールしてみましたが、失敗しました。

●Kill -QUIT でSecondary 側のpostmaster を停止してデータベースへアクセス
---
postgres=# select * from test;
ERROR:  kind mismatch between backends
HINT:  check data consistency between master and secondary
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>
---
DEBUG: pid 3148: read kind from frontend Q(51)
DEBUG: pid 3148: Query: select * from test;
DEBUG: pid 3148: read kind from backend T
DEBUG: pid 3148: pool_process_query: waiting for secondary for data ready
ERROR: pid 3148: pool_process_query: kind does not match between backends
master
(T) secondary(N)
DEBUG: pid 3148: len:386
LOG: pid 3148: do_child: exits with status 1 due to error
DEBUG: pid 3171: I am 3171 accept fd 6
DEBUG: pid 3171: Protocol Major: 3 Minor: 0 database: postgres user:
postgres
DEBUG: pid 3139: child 3148 exits with status 256 by signal 0
ERROR: pid 3171: connect_inet_domain_socket: connect() failed: Connection
refused
LOG: pid 3171: notice_backend_error: master: 0 fail over request from pid
3171
DEBUG: pid 3184: I am 3184
DEBUG: pid 3139: failover_handler called
LOG: pid 3139: starting degeneration. shutdown secondary host
Secondary_Host(5432)
DEBUG: pid 3139: exit_handler called
Terminated
---

また、Kill -INT を使用しても同様に失敗で、下記のようなログが出力されました。

●Kill -INT でSecondary 側のpostmaster を停止してデータベースへアクセス
---
DEBUG: pid 3413: read kind from frontend Q(51)
DEBUG: pid 3413: Query: select * from test;
DEBUG: pid 3413: read kind from backend T
DEBUG: pid 3413: pool_process_query: waiting for secondary for data ready
ERROR: pid 3413: pool_process_query: kind does not match between backends
master
(T) secondary(E)
DEBUG: pid 3413: len:109
LOG: pid 3413: notice_backend_error: master: 0 fail over request from pid
3413
DEBUG: pid 3381: failover_handler called
LOG: pid 3381: starting degeneration. shutdown secondary host
Secondary_Host(5432)
LOG: pid 3413: do_child: exits with status 1 due to error
DEBUG: pid 3381: exit_handler called
Terminated
---

さらに、上の　1) マスタ側が落ちたとき　でうまくいったはずの
Master 側のpostmaster を落とした場合も動かなくなってしまいました。

●Kill -INT でMaster 側のpostmaster を停止してデータベースへアクセス
---
postgres=# select * from test;
ERROR:  kind mismatch between backends
HINT:  check data consistency between master and secondary
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>
---
DEBUG: pid 485: read kind from frontend Q(51)
DEBUG: pid 485: Query: select * from test;
DEBUG: pid 485: read kind from backend E
DEBUG: pid 485: pool_process_query: waiting for secondary for data ready
ERROR: pid 485: pool_process_query: kind does not match between backends
master(
E) secondary(T)
DEBUG: pid 485: len:109
LOG: pid 485: notice_backend_error: master: 1 fail over request from pid 485
DEBUG: pid 453: failover_handler called
LOG: pid 453: starting degeneration. shutdown master host Master_Host(5432)
LOG: pid 485: do_child: exits with status 1 due to error
DEBUG: pid 453: exit_handler called
Terminated
---

以上です。