From the radio prospective, too early handover and too late handover could be the two reasons when handover gets failed.
Another possible reason could be that target eNodeB responds back with failure to source eNodeB during handover signalling.
If you are looking for any other specific reason then let me know.