In numerical analysis, Brent's method is a complicated but popular root-finding algorithm combining the bisection method, the secant method and inverse quadratic interpolation. It has the reliability of bisection but it can be as quick as some of the less reliable methods.
在数值分析领域，Brent方法是一个复杂的、但是却很流行的寻根算法，它结合了二分法、割线法以及反向二次插值法的特点。它具有二分法的稳定性，但是它的速度却可与一些不太稳定的方法相比拟。

Detail on Brent's method：

http://en.wikipedia.org/wiki/Brent%27s_method

Why mention Brent's method：

Powell's method, strictly Powell's conjugate gradient descent method（Powell的共轭梯度下降法）, is an algorithm for finding the local minimum of a function. The function need not be differentiable（可微）, and no derivatives（导数） are taken.

The function must be a real-valued function of a fixed number of real-valued inputs, creating an N-dimensional hypersurface or Hamiltonian.^{[disambiguation needed]} The caller passes in the initial point. The callers also passes in a set of initial search vectors. Typically N search vectors are passed in which are simply the normals aligned to each axis.

The method minimises the function by a bi-directional search along each search vector, in turn. The new position can then be expressed as a linear combination of the search vectors. The new displacement vector becomes a new search vector, and is added to the end of the search vector list. Meanwhile the search vector which contributed most to the new direction, i.e. the one which was most successful, is deleted from the search vector list. The algorithm iterates an arbitrary number of times until no significant improvement is made.

The method is useful for calculating the local minimum of a continuous but complex function, especially one without an underlying mathematical definition, because it is not necessary to take derivatives. The basic algorithm is simple, the complexity is in the linear searches along the search vectors, which can be achieved via Brent's method.

Extended reading：

http://math.fullerton.edu/mathews/n2003/BrentMethodMod.html

http://mathworld.wolfram.com/BrentsMethod.html

http://reference.wolfram.com/mathematica/tutorial/UnconstrainedOptimizationBrentsMethod.html

Detail on Powell's method：

http://en.wikipedia.org/wiki/Powell's_method

Something more related with Powell's method：

http://www.efunda.com/math/leastsquares/leastsquares.cfm#PageTop

NULL

文章来源：https://www.codelast.com/
➤➤ 版权声明 ➤➤ 
转载需注明出处：codelast.com 
感谢关注我的微信公众号（微信扫一扫）：

       即：对已知的一个数据集 ${x_i}(i = 1,2, \cdots ,n)$ ，能极小化该式的 $S$ 就是最优参数。但是这个式子是怎么来的呢？

它是从最大似然估计方法得到的：对参数 $S$ ，能使已知数据集发生的概率越大，那么就说明我们取的 $S$ 越优良。注意，对于一组已知的数据集，参数 $S$ 几乎不可能使每个 ${x_i}$ 都满足我们假设的数学模型，因此这里所说的“使已知数据集发生的概率越大”，这个“发生”，是指 ${y_i} \in \left[ {f({x_i},S) - \delta ,f({x_i},S) + \delta } \right]$ ，其中δ为允许的误差。

假设所有数据点的测量误差独立、符合正态分布，且标准差相等，则每一个数据点发生的概率为：

整个数据集同时发生的概率为各数据点概率之积：

文章来源：http://www.codelast.com/

     如前文所述：对参数 $S$ ，能使已知数据集发生的概率越大，那么就说明我们取的 $S$ 越优良。因此，使上式最大化就是我们的目标。由于 $\delta$ 为正常数， $f(x) = {e^x}$ 为单调递增函数，因此，想要：

就等于：

文章来源：http://www.codelast.com/

       等同于：

       继续化简：

       相当于：

文章来源：http://www.codelast.com/

       现在，由于 $\sigma$ 是常数，上式就等同于：

       这就得到了我们要推导的结论。

文章来源：https://www.codelast.com/
➤➤ 版权声明 ➤➤ 
转载需注明出处：codelast.com 
感谢关注我的微信公众号（微信扫一扫）：

NULL