It (BFGS) is also more numerically stable, and has very effective “self-correcting
properties” not shared by DFP, which may account for its superior performance in practice. (查看原文)
In a sense, the line search and trust-region approaches differ in the order in which they
choose the direction and distance of the move to the next iterate. Line search starts by fixing the direction pk and then identifying an appropriate distance, namely the step length αk . In trust region, we first choose a maximum distance—the trust-region radius k —and then seek a direction and step that attain the best improvement possible subject to this distance constraint. If this step proves to be unsatisfactory, we reduce the distance measure k and try again. (查看原文)
Most of these bits (say t) are devoted to storing the fractional part, while the remainder encode the exponent e and other information, such as the sign of the number, or an indication of whether it is zero or "undefined." (查看原文)
This phenomenon is known as cancellation. It can also be explained (less formally) by noting that if both x and y are accurate to k digits, and if they agree in the frist k' digits, then their difference will contain only about k-k' significant digits–the first k' digits cancel each other out. This observation is the reason for the well-known adage of numerical computing–that one should avoid taking the difference of two similar numbers if at all possible. (查看原文)
