Support change the make up replica behavior online

[disksing]

PCP Issue

• difficult: Easy
• score: 100
• min score required: 0
• Recommended Skills: Go

Description

The default behavior we handle offline/down peer is to create an operator to replace it. It is safe but could be very slow. Sometimes the user may want to delete it directly (for example, disk broken, data is gone for sure), we can add a configuration to help.

[rleungx]

I think we need to provide a way to delete the peers directly on a specified store which can not be recovered.

[disksing]

@rleungx I think that is a better idea!

[miyuri-fox]

/pick-up-challenge

[sre-bot]

@miyuri-fox pick up issue success

[sourcelliu]

/pick-up-challenge

[sre-bot]

@mantuliu pick up issue success

[sourcelliu]

/give-up-challenge

[sre-bot]

@mantuliu give up issue success

[gzgywh]

/pick-up-challenge

[sre-bot]

@gzgywh pick up issue success

[you06]

@gzgywh Dear contributor, our rules has been updated. We will remove your pick if there is no update in 7 days(but feel free to pick up again).

[sre-bot]

This pick has been automatically canceled after more than a week.

[JmPotato]

@disksing @rleungx Do we have a way in PD to automatically judge whether a store is in specified states (for example, disk broken, data is gone for sure) which can not be recovered now? If the answer is yes, maybe we can just do some work in scheduler to do so. If it's not, providing a way like API or command line tool may be a good idea.

[disksing]

@JmPotato I think there is no such mechanism. I have a second thought here. It seems a global configuration to control the behaviour could work too, especially if it introduces a lot complexity to add APIs to manipulate some store states.

[JmPotato]

@disksing I have the same feeling that maybe introducing a configuration used to control the scheduler's behaviour in this situation is better. In more detail, with this configuration enabled, scheduler will not try to replace the offline/down peer, but just delete it directly. And the making up work will be done by the replica checker.