## How do I set the tolerance in scipy.optimize.minimize()?

scipy optimize maximize
scipy optimize example
scipy minimize multiple variables
scipy optimize minimize step size
scipy optimize minimize args
scipy minimize debug
sc optimize minimize
scipy optimize initial guess

I am using scipy.optimize.minimize to solve for an efficient portfolio.

With the default settings I am frequently running into "BaseException: Positive directional derivative for linesearch" errors for when using certain inputs. I noticed if I set the tolerance high enough, the problem becomes less but doesn't go away. Any advice?

```import numpy as np
import pandas as pd
import scipy.optimize

def fx(TOLERANCE):
#TOLERANCE = 1.5

def solve_weights(R, C, rf, b_):
def port_mean_var(W,R,C):
return sum(R*W), np.dot(np.dot(W, C), W)
def fitness(W, R, C, rf):
mean, var = port_mean_var(W, R, C)    # calculate mean/variance of the portfolio
util = (mean - rf) / np.sqrt(var)        # utility = Sharpe ratio
return 1/util                        # maximize the utility, minimize its inverse value
n = len(R)
W = np.ones([n])/n                        # start optimization with equal weights
#b_ = [(0.,1.) for i in range(n)]    # weights for boundaries between 0%..100%. No leverage, no shorting
c_ = ({'type':'eq', 'fun': lambda W: sum(W)-1. })    # Sum of weights must be 100%
optimized = scipy.optimize.minimize(fitness, W, (R, C, rf),
method='SLSQP', constraints=c_,
bounds=b_, tol=TOLERANCE)
if not optimized.success:
raise BaseException(optimized.message)
return optimized.x

def mean_var_opt2(ret_df, upper_bounds=None):
R = (ret_df.mean(0)*252).values
C = (ret_df.cov()*252).values
rf = 0.0
if upper_bounds == None:
upper_bounds = pd.Series(1.0,index=ret_df.columns)
b_ = [(0.0,float(num)) for num in upper_bounds]
wgts = solve_weights(R, C, rf, b_)
return pd.Series(wgts, index=ret_df.columns)

np.random.seed(45)
rets = []
for i in range(10000):
rets.append(pd.DataFrame(np.random.randn(100,4)/100.0))

try:
for i in range(10000):
mean_var_opt2(rets[i])
except BaseException as e:
print e
finally: print "Tolerance: %s, iter: %s" % (TOLERANCE,i)

for k in [0.001, 0.01, 0.025, 0.05, 0.1, 0.5, 1.0, 5.0, 50.0, 500.0]:
fx(k)

Positive directional derivative for linesearch
Tolerance: 0.001, iter: 0
Positive directional derivative for linesearch
Tolerance: 0.01, iter: 30
Positive directional derivative for linesearch
Tolerance: 0.025, iter: 77
Inequality constraints incompatible
Tolerance: 0.05, iter: 212
Positive directional derivative for linesearch
Tolerance: 0.1, iter: 444
Positive directional derivative for linesearch
Tolerance: 0.5, iter: 444
Positive directional derivative for linesearch
Tolerance: 1.0, iter: 1026
Positive directional derivative for linesearch
Tolerance: 5.0, iter: 1026
Positive directional derivative for linesearch
Tolerance: 50.0, iter: 1026
Positive directional derivative for linesearch
Tolerance: 500.0, iter: 1026
```

python, Tuning the tolerance until it doesn't crash is a very weak solution, a small change on your data or function, and you are bound to crash. See here what does your  Minimization of scalar function of one or more variables. The objective function to be minimized. where x is an 1-D array with shape (n,) and args is a tuple of the fixed parameters needed to completely specify the function. Array of real elements of size (n,), where ‘n’ is the number of independent variables.

`SLSQP` requires the cost function to be twice derivable, and it may fail hard if that condition is not met. No amount of tolerance increase will help you unless it's so large that you'll always get the initial solution.

You may want to try `COBYLA` instead -- note, however, that this would require you to transform your bounds into constraints, as`COBYLA` does not accept a separate bounds argument.

scipy.optimize.minimize, Tolerance for termination. For detailed Set to True to print convergence messages. from scipy.optimize import minimize, rosen, rosen_der. The provided method callable must be able to accept (and possibly ignore) arbitrary parameters; the set of parameters accepted by minimize may expand in future versions and then these parameters will be passed to the method. You can find an example in the scipy.optimize tutorial.

I am providing an answer at this late date because it seems that the source code for this example is still up on the web, here.

In order to maximize the Sharpe ratio as in the function `fitness()` it is not necessary to minimize the reciprocal of it; you can simply minimize the negative of it.

Thus if I replace `return 1/util` with `return -util` in that function and run the questioner's code--- I did that--- I find that there are no errors for any of the 10,000 iterations for any `TOLERANCE`.

scipy.optimize.minimize, scipy.optimize.minimize(fun, x0, args=(), method=None, jac=None, hess=None, Tolerance for termination. It performs sequential one-dimensional minimizations along each vector of the directions set (direc field in options and info), which  The scipy.optimize.minimize method offers an interface to several minimizers. It defines a tol argument, for which the docs say:. Tolerance for termination. For detailed control, use solver-specific options.

scipy.optimize.show_options, Type of optimization solver. One of 'minimize', 'minimize_scalar', 'root'. method : str, optional. If not given, shows all methods of the specified solver. Otherwise  I have a simple optimization problem that, with some specific data, makes scipy.optimize.minimize ignore the tol argument. From the documentation, tol determines the "tolerance for termination", that is, the

minimize(method='SLSQP'), scipy.optimize. minimize (fun, x0, args=(), method='SLSQP', jac=None, bounds=​None, Set to True to print convergence messages. If False  Problem Description: An objective function is set up to take mooring line lenghs as an input array and return the sum of absolute differences between target and achieved pretension values. Now, I am using the scipy.optimize.minimize function with following options:

scipy.optimize.minimize_scalar, scipy.optimize.minimize_scalar(fun, bracket=None, bounds=None, args=(), method='brent', tol=None, options=None)[source]¶ Tolerance for termination. ignore) arbitrary parameters; the set of parameters accepted by minimize may  Your tolerance should be set to whatever tolerance you need. Setting it higher just tells the optimiser to stop sooner and doesn't actually speed it up. That being said, allowing it to go to a greater tollerence might be a waste of your time if not needed. Possible ways to reduce the time required are as follows: Use a different optimiser