MongoDB: How we can reduce the Load spikes that is happening in database ?

Question

We Have a 3 node mongodb setup with a primary,secondary and an arbiter. If a Primary node goes down the election process to choose a new primary is taking around 30 seconds. During this period all the Create and Update operations fail. So is there anyway to reduce this failover time so that the client application can resume normal operations in less time.

What is the ideal way to handle Create and Update failures during the failover time? Should it be queued and retried? if the number of operations are high the queue size will not be enough. So please suggest a solution for this problem.If the failover time can be reduced it would solve this problem.

We tried reducing the heartbeat interval in the replica configuration. It did not help either

Answer 1

Typically, you will need to review the logs as well as run the mongodb diagnostic tools such as mongotop,mongostat and mongoperf in order to determine the cause of the spikes. Mtools is another great diagnostic tool.
With knowledge of the cause of the spikes, it becomes easier to address the problem.