Oriented object detection has a very wide range of application scenarios. In recent years, a lot of rotation detectors have been designed to achieve high-performance oriented object detection. Intersection-over-union (IoU) is the commonly used indicator to evaluate the accuracy of detection performance. Many methods introduce IoU into the bounding box regression loss to achieve the aligned training and evaluation process for better performance. However, in this article, we demonstrate several drawbacks of rotated IoU loss through both experiments and theoretical derivation: 1) there is a negative correlation between the loss gradient and the angular error and 2) the optimization process of rotated IoU loss suffers from scale sensitivity, which is not conducive to model convergence. To solve the problems, we propose a gradient calibration loss (GCL) that optimizes the rotated IoU loss via gradient analysis and correction. We construct the optimized gradient in GCL to avoid IoU loss oscillation and scale sensitivity, thereby accelerating model convergence. Models supervised by GCL have a more stable training process, faster convergence, and better performance. Moreover, GCL can be easily introduced into the existing rotation detectors to achieve performance gains without extra inference overhead. Extensive experiments on multiple oriented object detection datasets and models demonstrate the superiority of our method. Our method achieves state-of-the-art performance on the mainstream benchmark datasets. The source code and models are available at https://github.com/ming71/GCL.