实际优化问题的目标函数往往比较复杂。为了使问题简化，通常将目标函数在某点附近展开为泰勒(Taylor)多项式来逼近原函数。
- 一元函数在点\(x_k\)处的泰勒展开式为：
\[\begin{equation} \begin{aligned} f(x) &= f(x_k)+(x-x_k)f'(x_k)+\frac{1}{2!}(x-x_k)^2f''(x_k)+o^n \end{aligned} \end{equation}\]
- 二元函数在点\((x_k,y_k)\)处的泰勒展开式为：
\[\begin{equation} \begin{aligned} f(x,y) &= f(x_k,y_k)+(x-x_k)f'_x(x_k,y_k)+(y-y_k)f'_y(x_k,y_k) \\ &+ \frac1{2!}(x-x_k)^2f''_{xx}(x_k,y_k) \\ &+ \frac1{2!}(x-x_k)(y-y_k)f''_{xy}(x_k,y_k) \\ &+ \frac1{2!}(x-x_k)(y-y_k)f''_{yx}(x_k,y_k) \\ &+ \frac1{2!}(y-y_k)^2f''_{yy}(x_k,y_k) \\ &+ o^n \\ \end{aligned} \end{equation}\]
- 多元函数(n)在点\(x_k\)处的泰勒展开式为：
\[\begin{equation} \begin{aligned} f(x^1,x^2,\ldots,x^n) &= f(x^1_k,x^2_k,\ldots,x^n_k)\\ &+ \sum_{i=1}^n(x^i-x_k^i)f'_{x^i}(x^1_k,x^2_k,\ldots,x^n_k)\\ &+ \frac1{2!}\sum_{i,j=1}^n(x^i-x_k^i)(x^j-x_k^j)f''_{ij}(x^1_k,x^2_k,\ldots,x^n_k)\\ &+ o^n \\ \end{aligned} \end{equation}\]
- 把Taylor展开式写成矩阵的形式： \[ f(\mathbf x) = f(\mathbf x_k)+[\nabla f(\mathbf x_k)]^T(\mathbf x-\mathbf x_k)+\frac1{2!}[\mathbf x-\mathbf x_k]^TH(\mathbf x_k)[\mathbf x-\mathbf x_k]+o^n \]
\[ H(\mathbf x_k)= \left[ \begin{matrix} \frac{\partial^2f(x_k)}{\partial x_1^2} & \frac{\partial^2f(x_k)}{\partial x_1\partial x_2} & \cdots & \frac{\partial^2f(x_k)}{\partial x_1\partial x_n} \\ \frac{\partial^2f(x_k)}{\partial x_2 \partial x_1} & \frac{\partial^2f(x_k)}{\partial x_2^2} & \cdots & \frac{\partial^2f(x_k)}{\partial x_2\partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2f(x_k)}{\partial x_n\partial x_1} & \frac{\partial^2f(x_k)}{\partial x_n\partial x_2} & \cdots & \frac{\partial^2f(x_k)}{\partial x_n^2} \\ \end{matrix} \right] \]