Initialization

this doc describes initialization

CobraInitializer

The initialization in SACOBRA_Py consists of the following steps:

  • pass in the specification of the COP and all the options in SACoptions s_opt

  • (optional, if s_opts.ID.rescale==True) rescale the problem in input space

  • create the initial design, see InitDesigner

  • adjust several elements according to constraint range, see CobraInitializer.adCon()

  • calculate for each initial design point numViol, the number of violated constraints, and maxViol, the maximum constraint violation. If equality constraints are involved, calculate \(\mu_{init}\), the radius for an artificial feasibility tube around each equality constraint (see EQUoptions) and base the calculation of numViol and maxViol on this artificial feasibility.

  • calculate the so-far best (artificial) feasible point. If no point fulfills (artificial) feasibility, take from the set of points with minimum numViol the one with the best objective.

  • set up result dictionary self.sac_res

  • adjust DRC according to objective range, see CobraInitializer.adDRC()

class cobraInit.CobraInitializer(x0, fn: object, fName: str, lower: ~numpy.ndarray, upper: ~numpy.ndarray, is_equ: ~numpy.ndarray, solu=None, s_opts: ~opt.sacOptions.SACoptions = <opt.sacOptions.SACoptions object>)

Initialize SACOBRA optimization:

  • problem formulation: x0, fn, lower, upper, is_equ, solu

  • parameter settings: s_opts via SACoptions

  • create initial design: A, Fres, Gres via InitDesigner

Parameters:

  • x0 (np.ndarray or None) – start point, if given, then its dim has to be the same as lower. If it is/has NaN or None on input, it is replaced by a random point from [lower, upper].

  • fn – function returning (1+nConstraints)-dim vector: [objective to minimize, constraints]

  • fName – function name

  • lower – lower bound, its dimension defines input space dimension

  • upper – upper bound (same dim as lower)

  • is_equ – boolean vector with dim nConstraints: which constraints are equality constraints?

  • solu (np.ndarray or None) – (optional, for diagnostics) true solution vector or solution matrix (one solution per row): one or several feasible x that deliver the minimal objective value

  • s_opts (SACoptions) – the options

adCon()

Adjust several elements according to constraint range.

The following elements of self.sac_res may be changed: ‘fn’, ‘Gres’, ‘Grange’, ‘GrangeEqu’

adDRC()

Adjust DRC (distance requirement cycle), based on range of Fres

get_fbest()

Return the original objective function value at the best feasible solution.

Note: We cannot take the best function value via sac_res['fn'], because this may be modified by PLOG or others.

get_xbest()
Returns:

best solution in original space

Return type:

np.ndarray

get_xbest_cobra()
Returns:

best solution in COBRA space (maybe rescaled)

Return type:

np.ndarray

class initDesigner.InitDesigner(x0: ndarray, fn, rng, lower: ndarray, upper: ndarray, s_opts: SACoptions)

Create matrix self.A with shape (P, d) of sample points in (potentially rescaled) input space [lower, upper] \(\subset \mathbb{R}^d\), where P = initDesPoints and d = input space dimension.

Apply fn to these points and split the result in objective function values self.Fres (with shape (P,)) and constraint function values self.Gres (with shape (P,nC) where nC = number of constraints).

Parameters:

  • x0 – the last point self.A[-1,:] is x0

  • fn – see parameter fn in cobraInit.CobraInitializer

  • lower – vector of shape (d,)

  • upper – vector of shape (d,)

  • s_opts (SACoptions) – object of class SACoptions. Here we use from element IDoptions s_opts.ID the elements initDesign and initDesPoints.

__call__() → tuple[ndarray, ndarray, ndarray]

Return the three results A, Fres and Gres of the initial design

Returns:

(self.A, self.Fres, self.Gres)

Return type:

(np.ndarray, np.ndarray, np.ndarray)

TODO: Describe Types of Initial Design (LHS, Random, …)

Options

All paramters (options) for SACOBRA_Py have sensible defaults defined. The user has only to specify those parameters where a setting different from the defaults is desired.

class opt.sacOptions.SACoptions(feval=100, XI=None, skipPhaseI=True, saveIntermediate=False, saveSurrogates=False, verbose=1, verboseIter=10, important=True, cobraSeed=42, ID=<opt.idOptions.IDoptions object>, RBF=<opt.rbfOptions.RBFoptions object>, SEQ=<opt.seqOptions.SEQoptions object>, EQU=<opt.equOptions.EQUoptions object>, ISA=<opt.isaOptions.ISAoptions object>, MS=<opt.msOptions.MSoptions object>, TR=<opt.trOptions.TRoptions object>)

The collection of all parameters (options) for SACOBRA_Py. Except for some general parameters defined in this class, they are hierarchically organized in nested option classes.

Parameters:

  • feval – number of function evaluations

  • XI – Distance-Requirement-Cycle (DRC) that controls exploration: Each infill point has a forbidden-sphere of radius XI[c] around it. c loops cyclically through XI’s inidices. If XI==None, then CobraInitializer will set it, depending on objective range, to short DRC [0.001, 0.0] or long DRC [0.3, 0.05, 0.001, 0.0005, 0.0].

  • skipPhaseI – whether to skip SACOBRA_Py phase I or not

  • DOSAC – controls the default options for ISAoptions ISA. 0: take plain COBRA settings, 1: full SACOBRA settings, 2: reduced SACOBRA settings

  • saveIntermediate – whether to save intermediate results or not (TODO)

  • saveSurrogates – whether to save surrogate models or not (TODO)

  • verbose – verbosity level: 0: print nothing. 1: print only important messages. 2: print everything

  • verboseIter – an integer value, after how many iterations to print summarized results.

  • important – controls the importance level for some verboseprint’s in updateInfoAndCounters

  • cobraSeed – the seed for RNGs. SACOBRA_Py guarantees the same results for the same seed

  • ID (IDoptions) – nested options for initial design

  • RBF (RBFoptions) – nested options for radial basis functions

  • SEQ (SEQoptions) – nested options for sequential optimizer

  • EQU (EQUoptions) – nested options for equality constraints

  • ISA (ISAoptions) – nested Internal SACOBRA options

  • MS (MSoptions) – nested options for model selection

  • TR (TRoptions) – nested options for trust region

Distance Requirement Cycle

The Distance Requirement Cycle (DRC) is the vector XI that controls exploration: Each already evaluated infill point is surrounded by a forbidden-sphere of radius XI[c] with c = i mod XI.size (c loops cyclically through XI’s inidices`, that’s where the name cycle comes from). A new infill point is searched under the additional constraint that it has to be a distance XI[c] away from all other already evaluted infill points. The larger XI[c], the more exploration.

If XI==None, then CobraInitializer will set it, depending on objective range, to short DRC [0.001, 0.0] or long DRC [0.3, 0.05, 0.001, 0.0005, 0.0]. Both vectors contain XI[c] = 0 which enforces exploitation. (If all entries were XI[c] > 0 then a good region would never be exploited further, since the search could never continue in the close vicinity of an already good infill point.)

class opt.idOptions.IDoptions(initDesign='RANDOM', initDesPoints: int = None, initDesOptP: int = None, initBias=0.005, rescale=True, newLower=-1, newUpper=1)

Options for the initial design (\(d\) = input dimension of problem).

Parameters:

  • initDesign – options: “RANDOM”, “RAND_R”, “RAND_REP”, “LHS”, … (see initDesigner)

  • initDesPoints – number of initial design points. If None, cobraInit will set it to \(d+1\) if RBF.degree=1 or to \((d+1)(d+2)/2\) if RBF.degree=2

  • initDesOptP – if None, cobraInit will set it to initDesPoints

  • initBias

  • rescale – if True, rescale input space from [lower, upper] to \([\) newLower, newUpper \(]^d\)

  • newLower – common new lower bound for each of the \(d\) input dimensions

  • newUpper – common new upper bound for each of the \(d\) input dimensions

class opt.rbfOptions.RBFoptions(model='cubic', degree=None, rho=0.0, rhoDec=2.0, rhoGrow=0, width=-1, widthFactor=1.0, gaussRule='One')

Options for the RBF surrogate models

Parameters:

  • model – RBF kernel type (currently only "cubic", but others will follow soon)

  • degree – degree of polynomial tail for RBF kernel. If None, then RBFInterpolator will set it depending on kernel type. See SciPy’s RBFInterpolator documentation for details.

  • rho – 0: interpolating RBFs, > 0: approximating (spline-like) RBFs. The larger rho the smoother

  • rhoDec – exponential decay factor for rho

  • rhoGrow – every rhoGrow (e.g. 100) iterations, re-enlarge rho. If 0, then re-enlarge never

  • width – only relevant for scalable (e.g. Gaussian) kernels. Determines the width \(\sigma\)

  • widthFactor – only for scalable kernels. Additional constant factor applied to each width \(\sigma\)

  • gaussRule – only relevant for Gaussian kernels, see trainGaussRBF

class opt.seqOptions.SEQoptions(optimizer='COBYLA', feval=1000, tol=1e-06, conTol=0.0, penaF=[3.0, 1.7, 300000.0], sigmaD=[3.0, 2.0, 100], epsilonInit=None, epsilonMax=None, finalEpsXiZero=True, trueFuncForSurrogates=False)

Options for the sequential optimization

Parameters:

  • optimizer – string defining the optimization method for SACOBRA_Py phase I and II. One out of [“COBYLA”,”ISRESN”]

  • feval – maximum number of function evaluations on the surrogate model

  • tol – convergence tolerance for sequential optimizer

  • conTol – constraint violation tolerance

  • penaF – (TODO)

  • sigmaD – (TODO)

  • epsilonInit – initial constant added to each constraint to maintain a certain margin to boundary

  • epsilonMax – maximum for constant added to each constraint

  • finalEpsXiZero – if True, set in final iteration EPS and XI to zero for best exploitation

  • trueFuncForSurrogates – if True, use the true (constraint & fitness) functions instead of surrogates (only for debug analysis)

class opt.equOptions.EQUoptions(active=True, initType='TAV', muType='expFunc', muDec=1.5, muFinal=1e-07, mu4inequality=False, muGrow=0, refine=True, refineMaxit=1000, refineAlgo='L-BGFS-B', refinePrint=False)

Equality handling options

Parameters:

  • active – if set to TRUE, the equality-handling (EH) technique is activated. The EH technique transforms each equality constraint \(h(\mathbf{x})=0\) into two inequality constraints \(h(\mathbf{x})-\mu <0\) and \(-h(\mathbf{x})-\mu<0\) with an adaptive (normally decaying) margin \(\mu\).

  • initType – the equality margin \(\mu\) is initialized with one of these choices: ["TAV"|"TMV"|"EMV"|"useGrange"]

  • muType – type of function used to shrink margin \(\mu\) during the optimization process. One out of ["SAexpFunc"|"expFunc"|"funcDim"|"funcSDim"|"Zhang"|"CONS"], see modifyMu

  • muDec – decay factor for margin \(\mu\), see modifyMu

  • muFinal – lower bound for margin \(\mu\). muFinal should be set to a small but non-zero value (larger than machine accuracy).

  • muGrow – every muGrow (e.g. 100) iterations, re-enlarge the \(\mu\)-band. If 0, then re-enlarge never

  • mu4inequality – use the artificial feasibility band also for inequalities (experimental)

  • refine – enables the refine mechanism for equality handling

  • refineMaxit – maximum number of iterations used in the refine step. Note that the refine step runs on the surrogate models and does not impose any extra real function evaluation

  • refineAlgo – optimizer for refine step [“COBYLA”|”L-BFGS-B”]

  • refinePrint – whether to print “cg-values (before,after,true)” after each refine step

TODO: Describe Refine Algo

class opt.isaOptions.ISAoptions(DOSAC=1, RS=True, RStype='CONSTANT', RSauto=False, RSmax=0.3, RSmin=0.05, RS_Cs=10, RS_rep=False, RS_verb=False, aDRC=True, aFF=True, aCF=True, TFRange=100000.0, TGR=1000.0, conPLOG=False, conFitPLOG=False, adaptivePLOG=False, onlinePLOG=False, onlineFreqPLOG=10, pEffectInit=0, minMaxNormal=False, onlineMinMax=False)

Internal self-adjusting (ISA) options for SACOBRA.

Defaults for DOSAC=1 (SACOBRA settings with fewer online adjustments). Other choices are

  • DOSAC=0 (ISAoptions0, COBRA-R settings = turn off SACOBRA), and

  • DOSAC=2 (ISAoptions2, SACOBRA settings with fewer parameters and more online adjustments).

Parameters:

  • DOSAC

  • RS – flag to enable/disable random start algorithm

  • RStype – type of function to calculate probability to start the internal optimizer with a random start point. One out of [“CONSTANT”, “SIGMOID”]

  • RSauto

  • RSmax – maximum probability of a random start

  • RSmin – minimum probability of a random start

  • RS_Cs – if RS_Cs iterations w/o progress, do a random start

  • RS_rep – if True, generate reproducible random numbers with my_rng2 (R and Python)

  • RS_verb – if True, be verbose in RandomStarter.random_start

  • aDRC – flag for automatic DRC adjustment

  • aFF – flag for automatic objective function transformation

  • aCF – flag for automatic constraint function transformation

  • TFRange – threshold, if the range of Fres is larger than TFRange, then apply automatic objective function transformation

  • TGR – threshold: GRatio > TGR, then apply automatic constraint function transformation. GRatio is the ratio “largest GR / smallest GR” where GR is the min-max range of a specific constraint. If TGR < 1, then the transformation is always performed.

  • conPLOG

  • conFitPLOG

  • adaptivePLOG – (experimental) flag for objective function transformation with plog, where the parameter pShift is adapted during iterations

  • onlinePLOG – flag for online decision marking whether to use plog or not according to p-effect

  • onlineFreqPLOG – after how many iterations the online plog check is done again

  • pEffectInit

  • minMaxNormal

  • onlineMinMax

class opt.isaOptions.ISAoptions0(DOSAC=0, RS=False, RStype='SIGMOID', RSauto=False, RSmax=0.3, RSmin=0.05, RS_Cs=10, RS_rep=False, RS_verb=False, aDRC=False, aFF=False, aCF=False, TFRange=inf, TGR=inf, conPLOG=False, conFitPLOG=False, adaptivePLOG=False, onlinePLOG=False, onlineFreqPLOG=10, pEffectInit=0, minMaxNormal=False, onlineMinMax=False)

Internal self-adjusting options for DOSAC=0 (COBRA-R settings = turn off SACOBRA).

Differs only in default settings, parameters and their meaning are the same as in ISAoptions.

class opt.isaOptions.ISAoptions2(DOSAC=2, RS=True, RStype='CONSTANT', RSauto=True, RSmax=0.3, RSmin=0.05, RS_Cs=10, RS_rep=False, RS_verb=False, aDRC=True, aFF=True, aCF=True, TFRange=-1, TGR=-1, conPLOG=False, conFitPLOG=False, adaptivePLOG=False, onlinePLOG=True, onlineFreqPLOG=10, pEffectInit=3, minMaxNormal=False, onlineMinMax=False)

Internal self-adjusting options for DOSAC=2 (SACOBRA with fewer parameters and more online adjustments).

Differs only in default settings, parameters and their meaning are the same as in ISAoptions.