IMPROVED LIPSCHITZ BOUNDS WITH THE FIRST NORM FOR FUNCTION VALUES OVER MULTIDIMENSIONAL SIMPLEX

. A branch and bound algorithm for global optimization is proposed, where the maximum of an upper bounding function based on Lipschitz condition and the ﬁrst norm over a simplex is used as the upper bound of function. In this case the graph of bounding function is intersection of n -dimensional pyramids and its maximum point is found solving a system of linear equations. The eﬃciency of the proposed global optimization algorithm is evaluated experimentally


Introduction
Our aim is to find at least one globally optimal solution to the problem where an objective function f (x), f : R n → R, is a real-valued Lipschitz function, D ⊆ R n is a feasible region, n is the number of variables.A function f : D → R, D ⊆ R n , is said to be Lipschitz if it satisfies the condition where L > 0 is a constant called Lipschitz constant, D is a compact set and • denotes the norm.The Euclidean norm is used most often, but other norms could also be considered.In [15] we showed that for Lipschitz function f (x) where L p = sup{ ▽f (x) p : x ∈ D} is Lipschitz constant, the gradient of the function f (x) is denoted by ▽f (x) = ( ∂f ∂x1 , . . ., ∂f ∂xn ), and 1/p + 1/q = 1, 1 ≤ p, q ≤ ∞.
The most often studied case of problem (1.1) is the univariate one (n = 1), for which many algorithms have been proposed, compared and theoretically investigated.In the presented paper, we are interested in the multivariate case (n ≥ 2).
In Lipschitz optimization the upper bound of the optimal value f * is evaluated exploiting Lipschitz condition.It follows from (1.2) that, for all x, y ∈ D Almost all methods for solving the multivariate problem fall into two main classes.The first class contains the direct extensions of Piyavskii's method [19] to the multivariate case and various modifications with different norms or approximations [7,11,12,13,14,21].Note that when the Euclidean norm is used in the multivariate case, the upper bounding functions are envelopes of circular cones with parallel symmetry axes.A problem of finding maximum of such a bounding function becomes a difficult global optimization problem involving systems of quadratic and linear equations.Most of these algorithms can be improved by interpreting them as branch and bound methods [5,7,8].
The second class contains many simplicial and rectangular branch and bound techniques [4,6,17,18,20].They differ in the ways how branching is performed and bounds are computed.Simplicial partitions are preferable when the values of an objective function at the vertices of partitions are used to compute bounds [22,27].Another advantage of simplicial partitions is that they may be used to vertex triangulate feasible regions of non rectangular shape defined by linear inequality constraints [27], what allows reduction of search space of problems with symmetric objective functions [26].
In general, bounds belong to the following two simple families µ 1 (P ) and µ 2 (P ).Let δ (P ) = max { x − y : x, y ∈ P } denotes the diameter of P .For example, if P = {x ∈ R n : a ≤ x ≤ b} is nrectangle, then δ (P ) = b − a and if P is an n-simplex then diameter δ (P ) is the length of its longest edge.Then simpler upper bound can be derived: here T ⊂ P is finite sample of points in P where the function values of f have been evaluated.When P is a rectangle or a simplex the set T often coincides with the vertex set V (P ).The more tight but computationally more expensive than (1.3) is the estimate Performance of optimization algorithms depends on tightness of bounds [23].For D ⊆ P , the sharpest upper bound given the knowledge of the function values f (y) , y ∈ T , and of the Lipschitz constant L, is provided by (1.5) Although it may by more natural to formulate this problem as min-max problem, we keep formulation used in [5,13,14].Anyway (1.5) is a difficult optimization problem when the space has dimension n ≥ 2.
The Euclidean norm is used most often, but other norms could be also considered.In [15,16] we investigated how different norms and corresponding Lipschitz constants influence speed of algorithms for global optimization when (1.4) is used on vertices of simplex: where v is a vertex of the simplex I and values of the function at all vertices of this simplex V (I) are used.Experiments have shown that better results may be achieved when non Euclidean norms are used.Therefore for better upper bound we proposed the combination of two extreme (infinite and first) and the Euclidean norms: where the first norm is defined as In this paper we investigate improved bounds with the first norm.For finding upper bound we exploit (1.5) formula: In this case the graph of F is the intersection of n-dimensional pyramids and the maximum point is found solving system of linear equations.Branch and bound algorithm with simplicial partitions and face to face vertex triangulation [27] is used to find the global maximum.

Improved Upper Bound
Let us formulate two propositions, which are used for solving global optimization problem (1.1) with (1.8) for evaluation of improved upper bound. where and all points x in the intersection are closer to the vertex v 1 than to v 2 , i.e.
. ., n one case from the following three possibilities is true: Therefore from (2.3) we get (2.1). Because Proposition 1.The maximum point of F can be found solving a system of n linear equations.
Proof.Let us define numeration of vertices v i so that f -manifold defined by (2.1) (see Proposition 1).Taking into account (2.2), it is possible to consider only part of the manifold, which is defined by linear equation and constraints.Therefore it is possible to form a system of n linear equations defining intersections If the solution of this system satisfies the constraints (see Example 1) then the upper bounding function is maximal at the solution point.
If the solution of this system does not satisfy the constraints, then the maximum of the upper bounding function is the minimum of the function value at the intersections (see Example 2).

Algorithm
Branch and bound is a technique for implementation of covering global optimization methods [27] as well as combinatorial optimization algorithms.An iteration of a classical branch and bound algorithm processes a node in the search tree representing a not yet explored subspace of the solution space.The iteration has three main components: selection of the node to process, branching of the search tree and bound calculation.Bound may be estimated using interval arithmetic [24,25,28,29]  Choose and exclude I k ∈ I from the set of non-solved simplices I.
Let us consider the first simplex Intersection of pyramids F v1 = F v2 gives the following equation However as 0 ≤ x 1 ≤ 1, this equation can be simplified and solved: Analogously intersection of pyramids F v1 = F v3 may be found: or (taking into account that From (4.1), (4.2) and (2.2) it follows that and improved upper bound is given by F (I 1 ) = F vi (p 1 ) (see, Fig. 1): Let us verify, that this upper bound is better, than (1.6):From (4.5), (4.6) and (2.2) we get a system which defines an empty set of solutions (see Fig. 2).Therefore improved upper bound is achieved in intersection line belonging to the upper bounding function (see Fig. 2), i.e.

Results of Experiments
Various test functions for global optimization from [5,9,10] have been used in our experiments.Lipschitz constants have been estimated using Theorem 1 from [15].Test functions with (n = 2 and n = 3) are numbered according to [5,10].For (n ≥ 4) functions names from [9] are used.The speed of global optimization is measured using the number of function evaluations criterion.The results are presented in Table 1.
The improved upper bound F calculated by (1.8) gives better results for Lipschitz optimization than U B calculated by (1.6).When n = 2, the number of function evaluations on average is 11% smaller when improved upper bound F is used than when U B is used.When n = 3, the number of function evaluations on average is 15% smaller.When n ≥ 4, it is on average 17% smaller.
Results of combination of bounds with different norms [15,16] are also given in Table 1.Combination of bounds gives better results when the first norm is not preferable.However for some functions improved bound with first norm produces even better results than the combination.The results suggest to include improved bound with first norm into combination instead of simpler bound with first norm.It is also promising to develop improved bounds for other norms.

Conclusions
In this paper an improved Lipschitz bound with first norm has been proposed and applied in multidimensional Lipschitz global optimization.Test functions of different dimensionality (n = 2, 3, 4, 5, 6) have been used for experimental investigation of branch and bound algorithm.

Algorithm 1 1 :
as well as Lipschitz condition.Subspaces which cannot contain a global minimum are discarded from further search pruning the branches of the search tree.Although covering, selection, branching and bounding rules differ in different branch and bound algorithms, the structure of the algorithm remains the same, what enables implementation of branch and bound algorithms using prepared templates [1, 2, 3].In the proposed algorithm partitions, face to face vertex triangulation for initial covering, subdivision through the middle of the longest edge of simplex, the breadth first selection strategy and improved bounds (1.8) with first norm are used.The proposed algorithm is shown in Algorithm 1. Branch and bound algorithm with simplicial partitions.An n-dimensional hyper-rectangle D is face-to-face vertex triangulated into set of n-dimensional simplices I = {I k |D ⊆ ∪I k , k = 1, . . ., n! }. 2: LB(D) = −∞.3: while (I is not empty: I = Ø) do 4:

Figure 1 .
Figure 1.a) Projection of intersection lines; b) Visualization of upper bounding functions.

Figure 2 .
Figure 2. a) Projection of intersection lines; b) Visualization of upper bounding functions.