Allow to chose between cp/ap

Registered by markus_albe

We had at least two customers that would like to have their nodes continue running even after inconsistency was detected on a table. I understand this won't make sense for most people, but still given we had legit customer requests here's the FR:

Introduce new setting wsrep_cap_mode which accepts CP or AP as possible values. In CP mode PXC would keep it's current behavior, and in AP mode:

- if a write fails to a table, add table name to "inconsistent_tables" list and log warning to error log
- before a table is opened for any query, check if it's listed in "inconsistent_tables" and return "Unkown error"
- if failed query is part of multi-statement trx, rollback trx
- have a global status variable signaling inconsistent state (wsrep_is_consistent or such).

They understand the cons of this:
- In a single-writer setup if an inconsistent node is promoted it will reject all writes to the inconsistent tables; And if this promotion happens because the non-inconsistent node went down, the writes that happened there between the time inconsistency was introduced and when the non-inconsistent node went down could be lost.

- Only way to actually restore the table is through SST
-

Blueprint information

Status:
Not started
Approver:
None
Priority:
Not
Drafter:
markus_albe
Direction:
Needs approval
Assignee:
Vadim Tkachenko
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Rejected due to incompatibility with PXC architectural goals

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.