%% Copyright (C) 2005, 2011 Carnegie Mellon University and others.
%%
%% The first version of this file was contributed to the Ipopt project
%% on Aug 1, 2005, by Yoshiaki Kawajiri
%%                    Department of Chemical Engineering
%%                    Carnegie Mellon University
%%                    Pittsburgh, PA 15213
%%
%% Since then, the content of this file has been updated significantly by
%%     Carl Laird and Andreas Waechter        IBM
%%     Stefan Vigerske                        GAMS
%%
%%
%% $Id: documentation.tex 2617 2015-11-26 16:00:20Z stefan $
%%
\documentclass[10pt]{article}
\setlength{\textwidth}{6.3in}       % Text width
\setlength{\textheight}{9.4in}      % Text height
\setlength{\oddsidemargin}{0.1in}     % Left margin for even-numbered pages
\setlength{\evensidemargin}{0.1in}    % Left margin for odd-numbered pages
\setlength{\topmargin}{-0.5in}         % Top margin
\renewcommand{\baselinestretch}{1.1}
\usepackage{amsfonts}
\usepackage{amsmath}
\PassOptionsToPackage{hidelinks,pdftitle={Ipopt documentation},pdflang=en}{hyperref}
\usepackage{url}
\usepackage{html}
\usepackage{xspace}

%\usepackage{showlabels}

\newcommand{\RR}{\mathbb{R}}
\newcommand{\Ipopt}{\textsc{Ipopt}\xspace}
\newcommand{\JIpopt}{\textsc{JIpopt}\xspace}
\newcommand{\ipoptr}{\texttt{ipoptr}\xspace}
\newcommand{\sIpopt}{\textsc{sIpopt}\xspace}
\newcommand{\Matlab}{\textsc{MATLAB}\xspace}

%\htmltitle{Ipopt documentation}

\begin{document}
\title{Introduction to \Ipopt:\\
A tutorial for downloading, installing, and using \Ipopt}

\author{Revision number of this document: $Revision: 2617 $}

%\date{\today}
\maketitle

\begin{abstract}
  This document is a guide to using \Ipopt 3.12.  It includes
  instructions on how to obtain and compile \Ipopt, a description of
  the interface, user options, etc., as well as a tutorial on how to
  solve a nonlinear optimization problem with \Ipopt.
\end{abstract}

\section*{History of this document}
The initial version of this document was created by Yoshiaki
Kawajir\footnote{then Department of Chemical Engineering, Carnegie Mellon
  University, Pittsburgh PA} as a course project for \textit{47852
  Open Source Software for Optimization}, taught by Prof. Fran\c{c}ois
Margot at Tepper School of Business, Carnegie Mellon University.
After this, Carl Laird\footnote{then Department of Chemical
  Engineering, Carnegie Mellon University, Pittsburgh PA} has added
significant portions, including the very nice tutorials.  The current
version is maintained by Stefan Vigerske\footnote{GAMS Software GmbH} and
  Andreas W\"achter\footnote{Department of Industrial Engineering and
  Management Sciences, Northwestern University}.

\tableofcontents

\vspace{\baselineskip}
\begin{small}
\noindent
The following names used in this document are trademarks or registered
trademarks: AMPL, IBM, Intel, Matlab, Microsoft, MKL, Visual Studio C++,
Visual Studio C++ .NET
\end{small}

\section{Introduction}
\Ipopt (\underline{I}nterior \underline{P}oint \underline{Opt}imizer,
pronounced ``Eye--Pea--Opt'') is an open source software package for
large-scale nonlinear optimization. It can be used to solve general
nonlinear programming problems of the form
%\begin{subequations}\label{NLP}
\begin{eqnarray}
\min_{x\in\RR^n} &&f(x) \label{eq:obj} \\
\mbox{s.t.} \;  &&g^L \leq g(x) \leq g^U \label{eq:constraints}\\
                &&x^L \leq x \leq x^U, \label{eq:bounds}
\end{eqnarray}
%\end{subequations}
where $x \in \RR^n$ are the optimization variables (possibly with
lower and upper bounds, $x^L\in(\RR\cup\{-\infty\})^n$ and
$x^U\in(\RR\cup\{+\infty\})^n$), $f:\RR^n\longrightarrow\RR$ is the
objective function, and $g:\RR^n\longrightarrow \RR^m$ are the general
nonlinear constraints.  The functions $f(x)$ and $g(x)$ can be linear
or nonlinear and convex or non-convex (but should be twice
continuously differentiable). The constraints, $g(x)$, have lower and
upper bounds, $g^L\in(\RR\cup\{-\infty\})^m$ and
$g^U\in(\RR\cup\{+\infty\})^m$. Note that equality constraints of the
form $g_i(x)=\bar g_i$ can be specified by setting
$g^L_{i}=g^U_{i}=\bar g_i$.

\subsection{Mathematical Background}
\Ipopt implements an interior point line search filter method that
aims to find a local solution of (\ref{eq:obj})-(\ref{eq:bounds}).  The
mathematical details of the algorithm can be found in several
publications
\cite{NocWaeWal:adaptive,WaechterPhD,WaecBieg06:mp,WaeBie05:filterglobal,WaeBie05:filterlocal}.

\subsection{Availability}
The \Ipopt package is available from COIN-OR
(\url{http://www.coin-or.org}) under the EPL (Eclipse Public License)
open-source license and includes the source code for \Ipopt.  This
means, it is available free of charge, also for commercial purposes.
However, if you give away software including \Ipopt code (in source
code or binary form) and you made changes to the \Ipopt source code,
you are required to make those changes public and to clearly indicate
which modifications you made.  After all, the goal of open source
software is the continuous development and improvement of software.
For details, please refer to the Eclipse Public License.

Also, if you are using \Ipopt to obtain results for a publication, we
politely ask you to point out in your paper that you used \Ipopt, and
to cite the publication \cite{WaecBieg06:mp}.  Writing high-quality
numerical software takes a lot of time and effort, and does usually
not translate into a large number of publications, therefore we
believe this request is only fair :).  We also have space at the
\Ipopt project home page where we list publications, projects, etc.,
in which \Ipopt has been used.  We would be very happy to hear about
your experiences.

\subsection{Prerequisites}\label{sec:prerequisites}
In order to build \Ipopt, some third party components are required:
\begin{itemize}
\item BLAS (Basic Linear Algebra Subroutines).  Many vendors of
  compilers and operating systems provide precompiled and optimized
  libraries for these dense linear algebra subroutines.  You can also
  get the source code for a simple reference implementation from {\tt
    www.netlib.org} and have the \Ipopt distribution compile it
  automatically.  However, it is strongly recommended to use some
  optimized BLAS implemetion, for large problems this can make a
  runtime difference of an order of magnitude!

  Examples for efficient BLAS implementations are:
  \begin{itemize}
  \item From hardware vendors:
    \begin{itemize}
    \item ACML (AMD Core Math Library) by AMD
    \item ESSL (Engineering Scientific Subroutine Library) by IBM
    \item MKL (Math Kernel Library) by Intel
    \item Sun Performance Library by Sun
    \end{itemize}
  \item Generic:
    \begin{itemize}
    \item Atlas (Automatically Tuned Linear Algebra Software)
    \item GotoBLAS
    \end{itemize}
  \end{itemize}
  You find more information on the web by googling them.

  Note: BLAS libraries distributed with Linux are usually not
  optimized.
\item LAPACK (Linear Algebra PACKage).  Also for LAPACK, some vendors
  offer precompiled and optimized libraries.  But like with BLAS, you
  can get the source code from \url{http://www.netlib.org} and have the
  \Ipopt distribution compile it automatically.

  Note that currently LAPACK is only required if you intend to use the
  quasi-Newton options in \Ipopt.  You can compile the code without
  LAPACK, but an error message will then occur if you try to run the
  code with an option that requires LAPACK.  Currently, the LAPACK
  routines that are used by \Ipopt are only {\tt DPOTRF}, {\tt
    DPOTRS}, and {\tt DSYEV}.

  Note: LAPACK libraries distributed with Linux are usually not
  optimized.
\item A sparse symmetric indefinite linear solver. \Ipopt needs
  to obtain the solution of sparse, symmetric, indefinite linear
  systems, and for this it relies on third-party code.  

  Currently, the following linear solvers can be used:
  \begin{itemize}
  \item MA27 from the HSL Mathematical Software Library\\ (see \url{http://www.hsl.rl.ac.uk}).
  \item MA57 from the HSL Mathematical Software Library\\ (see \url{http://www.hsl.rl.ac.uk}).
  \item HSL\_MA77 from the HSL Mathematical Software Library\\ (see \url{http://www.hsl.rl.ac.uk}).
  \item HSL\_MA86 from the HSL Mathematical Software Library\\ (see \url{http://www.hsl.rl.ac.uk}).
  \item HSL\_MA97 from the HSL Mathematical Software Library\\ (see \url{http://www.hsl.rl.ac.uk}).
  \item MUMPS (MUltifrontal Massively Parallel sparse direct Solver)\\
    (see \url{http://graal.ens-lyon.fr/MUMPS})
  \item The Parallel Sparse Direct Solver (PARDISO)\\ (see \url{http://www.pardiso-project.org}).%\\
    %Note: The Pardiso version in Intel's MKL library does not yet
    %support the features necessary for \Ipopt.
  \item The Watson Sparse Matrix Package (WSMP)\\ (see \url{http://researcher.ibm.com/view_project.php?id=1426})
  \end{itemize}
  You should include at least one of the linear solvers above in order
  to run \Ipopt, and if you want to be able to switch easily between
  different alternatives, you can compile \Ipopt with all of them.

  The \Ipopt library also has mechanisms to load the linear solvers MA27, 
  MA57, HSL\_MA77, HSL\_MA86, HSL\_MA97, and Pardiso from a shared library at 
  runtime, if the library has not been compiled with them (see
  Section~\ref{sec:linear_solver_loader}).

  \textbf{NOTE: The solution of the linear systems is a central
    ingredient in \Ipopt and the optimizer's performance and
    robustness depends on your choice.  The best choice depends on
    your application, and it makes sense to try different options.
    Most of the solvers also rely on efficient BLAS code (see above),
    so you should use a good BLAS library tailored to your system.
    Please keep this in mind, particularly when you are comparing
    \Ipopt with other optimization codes.}

  If you are compiling MA57, HSL\_MA77, HSL\_MA86, HSL\_MA97, or MUMPS within
  the \Ipopt build
  system, you should also include the METIS linear system ordering package.

  Interfaces to other linear solvers might be added in the future; if
  you are interested in contributing such an interface please contact
  us!  Note that \Ipopt requires that the linear solver is able to
  provide the inertia (number of positive and negative eigenvalues) of
  the symmetric matrix that is factorized.

\item Furthermore, \Ipopt can also use the HSL package MC19
  for scaling of the linear systems before they are passed to the
  linear solver.  This may be particularly useful if \Ipopt is used
  with MA27 or MA57.  However, it is not required to have MC19 to
  compile \Ipopt; if this routine is missing, the scaling is never
  performed.
  %\footnote{There are more recent scaling routines in the
  %  HSL, but they have not (yet) been integrated.  Contributions are
  %  welcome!}.
\item ASL (AMPL Solver Library).  The source code is available at {\tt
    www.netlib.org}, and the \Ipopt makefiles will automatically
  compile it for you if you put the source code into a designated
  space.  NOTE: This is only required if you want to use \Ipopt from
  AMPL and want to compile the \Ipopt AMPL solver executable.
\end{itemize}
For more information on third-party components and how to obtain them,
see Section~\ref{ExternalCode}.

Since the \Ipopt code is written in C++, you will need a C++ compiler
to build the \Ipopt library.  We tried very hard to write the code as
platform and compiler independent as possible.

In addition, the configuration script also searches for a Fortran
compiler, since some of the dependencies above are written in Fortran.
If all third party dependencies are available as self-contained
libraries, those compilers are in principle not necessary.  Also, it
is possible to use the Fortran-to-C compiler {\tt f2c} from
\url{http://www.netlib.org/f2c} to convert Fortran 77 code to C, and compile the
resulting C files with a C compiler and create a library containing
the required third party dependencies.  %We have tested and used this
%in connection with the Microsoft Visual C++ compiler, and instructions
%on how to use it in this context are given below.

When using GNU compilers, we recommend you use the same version numbers for {\tt gcc}, {\tt g++}, and {\tt gfortran}. For {\tt gfortran} specifically, we recommend versions newer than 4.5.2 (versions 4.5.1, 4.5.2, and before 4.2.0 are known to have bugs that caused issues with some of the newer Fortran 90 HSL linear solvers).

\subsection{How to use \Ipopt}
If desired, the \Ipopt distribution generates an executable for the
modeling environment AMPL. As well, you can link your problem
statement with \Ipopt using interfaces for C++, C, or Fortran.
\Ipopt can be used with most Linux/Unix environments, and on Windows
using Visual Studio .NET, Cygwin or MSYS/MinGW.  In
Section~\ref{sec:tutorial-example} this document demonstrates how to
solve problems using \Ipopt. This includes installation and
compilation of \Ipopt for use with AMPL as well as linking with your
own code.

Additionally, the \Ipopt distribution includes interfaces for
\begin{itemize}
\item {\tt CUTEr}\footnote{see \url{http://cuter.rl.ac.uk/cuter-www}} (for
  solving problems modeled in SIF),

\item {\tt Java}, which allows you to use \Ipopt from Java, see the files in the
\texttt{Ipopt/contrib/JavaInterface} directory,
  
\item {\tt Matlab} (mex interface), which allows you to use \Ipopt from Matlab, see

\centerline{\url{https://projects.coin-or.org/Ipopt/wiki/MatlabInterface},}

\item and the {\tt R} project for statistical computing, see the files in the
\texttt{Ipopt/contrib/RInterface} directory.
\end{itemize}

There is also software that facilitates use of \Ipopt maintained by other people, among them are:
\begin{itemize}
\item ADOL-C (automatic differentiation)

  ADOL-C facilitates the evaluation of first and higher derivatives of
  vector functions that are defined by computer programs written in C or C++.
  It comes with examples that show how to use it in connection with \Ipopt, see
  \url{https://projects.coin-or.org/ADOL-C}.
    
\item AIMMS (modeling environment)

  The AIMMSlinks project on COIN-OR, maintained by Marcel Hunting,
  provides an interface for \Ipopt within the AIMMS modeling tool, see
  \url{https://projects.coin-or.org/AIMMSlinks}.

\item APMonitor

  MATLAB, Python, and Web Interface to Ipopt for Android, Linux, MacOS X,
  and Windows, see \url{http://apmonitor.com}.

\item CasADi

  CasADi is a symbolic framework for automatic differentiation and
  numeric optimization and comes with \Ipopt, see
  \url{http://casadi.org}.

\item CppAD (automatic differentiation)

  Given a C++ algorithm that computes function values, CppAD generates an
  algorithm that computes corresponding derivative values (of arbitrary
  order using either forward or reverse mode).
  It comes with an example that shows how to use it in connection with \Ipopt, see
  \url{https://projects.coin-or.org/CppAD}.

\item GAMS (modeling environment)

  The GAMSlinks project on COIN-OR includes a GAMS interface for \Ipopt, see
  \url{https://projects.coin-or.org/GAMSlinks}.
  
\item JuliaOpt

  Julia is a high-level, high-performance dynamic programming language for technical computing.
  JuliaOpt, see \url{http://juliaopt.org}, is an umbrella group for Julia-based optimization-
  related projects. It includes the algebraic modeling language JuMP
  (\url{https://github.com/JuliaOpt/JuMP.jl}) and an interface to \Ipopt
  (\url{https://github.com/JuliaOpt/Ipopt.jl}).

\item mexIPOPT

  A rewrite of the above mentioned MATLAB Interface:
  \url{https://github.com/ebertolazzi/mexIPOPT}

\item MADOPT (Modelling and Automatic Differentiation for Optimisation)

  Light-weight C++ and Python modelling interfaces implementing
  expression building using operator overloading and automatic differentiation, see
  \url{https://github.com/stanle/madopt}

\item .NET :
  
  An interface to the C\# language is available here: 
  \url{https://github.com/cureos/csipopt}
  
\item OPTimization Interface (OPTI) Toolbox

  OPTI is a free {\tt Matlab} toolbox for constructing and solving linear, 
  nonlinear, continuous and discrete optimization problem and comes with 
  \Ipopt.

\item Optimization Services

  The Optimization Services (OS) project provides a set of standards 
  for representing optimization instances, results, solver options, 
  and communication between clients and solvers, incl.\ \Ipopt, in a 
  distributed environment using Web Services, see
  \url{https://projects.coin-or.org/OS}.

\item PyIpopt:

  An interface to the python language is available here: 
  \url{https://github.com/xuy/pyipopt}
    
\item Scilab (free Matlab-like environment):

  %Edson Cordeiro do Valle has written an interface to use \Ipopt from
  %Scilab.\url{http://www.scilab.org/contrib/displayContribution.php?fileID=839}

  A Scilab interface is available here: 
  \url{http://forge.scilab.org/index.php/p/sci-ipopt}
\end{itemize}

\subsection{More Information and Contributions}
More and up-to-date information can be found at the \Ipopt homepage,

\begin{center}
\url{http://projects.coin-or.org/Ipopt}.
\end{center}

Here, you can find FAQs, some (hopefully useful) hints, a bug report
system etc.  The website is managed with Wiki, which means that every
user can edit the webpages from the regular web browser.  {\bf In
  particular, we encourage \Ipopt users to share their experiences
  and usage hints on the ``Success Stories'' and ``Hints and Tricks''
  pages, or to list the publications discussing applications of
  \Ipopt in the ``Papers related to Ipopt'' page}\footnote{Since we
  had some malicious hacker attacks destroying the content of the web
  pages in the past, you are now required to enter a user name and
  password; simply follow the instructions on top of the main project
  page.}.  In particular, if you have trouble getting \Ipopt work
well for your optimization problem, you might find some ideas here.
Also, if you had some difficulties to solve a problem and found a way
around it (e.g., by reformulating your problem or by using certain
\Ipopt options), it would be very nice if you help other users by
sharing your experience at the ``Hints and Tricks'' page.

\Ipopt is an open source project, and we encourage people to
contribute code (such as interfaces to appropriate linear solvers,
modeling environments, or even algorithmic features).  If you are
interested in contributing code, please have a look at the COIN-OR
contributions webpage\footnote{see \url{http://www.coin-or.org/contributions.html}} and contact the \Ipopt
project leader.

There is also a mailing list for \Ipopt, available from the webpage
\begin{center}
\url{http://list.coin-or.org/mailman/listinfo/ipopt},
\end{center}
where you can subscribe to get notified of updates, to ask general
questions regarding installation and usage, or to share your
experience with \Ipopt. You might want to look at the archives before
posting a question.  An easy way to search the archive with Google is
to specify
\url{site:http://list.coin-or.org/pipermail/ipopt}
in addition to your keywords in the search string.

We try to answer questions posted to the mailing list in a reasonable
manner.  Please understand that we cannot answer all questions in
detail, and because of time constraints, we are not able to help
you model and debug your particular optimization problem.

Another way for discussion and contributing to \Ipopt is via its presence
on GitHub:
\begin{center}
\url{https://github.com/coin-or/Ipopt}
\end{center}
The git repository of \Ipopt at GitHub is only a mirror of the
subversion repository at the COIN-OR server, but the page allows you to
file issues and send small patches. This may be suitable for users who
find using the conventional mailing list and bug tracking system too
cumbersome.

A short tutorial on getting started with \Ipopt is also available 
\cite{Waechter90Minutes}.

\subsection{History of \Ipopt}
The original \Ipopt (Fortran version) was a product of the
dissertation research of Andreas W\"achter \cite{WaechterPhD}, under
the supervision of Lorenz T. Biegler at the Chemical Engineering
Department at Carnegie Mellon University. The code was made open
source and distributed by the COIN-OR initiative, which is now a
non-profit corporation.  \Ipopt has been actively developed under
COIN-OR since 2002.

To continue natural extension of the code and allow easy addition of
new features, IBM Research decided to invest in an open source
re-write of \Ipopt in C++.  With the help of Carl Laird, who came to
the Mathematical Sciences Department at IBM Research as a summer
intern in 2004 and 2005 during his PhD studies, the code was
re-implemented from scratch.

The new C++ version of the \Ipopt optimization code (\Ipopt 3.0.0
and beyond) was maintained at IBM Research and remains part of the
COIN-OR initiative. The development on the Fortran version has
ceased, but the source code can still be downloaded from \url{http://www.coin-or.org/download/source/Ipopt-Fortran}.

\section{Installing \Ipopt}\label{Installing}

The following sections describe the installation procedures on
UNIX/Linux systems.  For installation instructions on Windows
see Section~\ref{WindowsInstall}.

Additional hints on installing \Ipopt and its various interfaces is available
on the \Ipopt and CoinHelp wiki pages, in particular
\begin{itemize}
\item \Ipopt compilation hints:\\
  \url{https://projects.coin-or.org/Ipopt/wiki/CompilationHints}
\item Current configuration and installation issues for COIN-OR projects:\\
  \url{https://projects.coin-or.org/BuildTools/wiki/current-issues}
\end{itemize}

\subsection{Getting System Packages (Compilers, ...)}

Many Linux distributions will come with all necessary tools. All you should need to do is check the compiler versions. On a Debian-based distribution, you can obtain all necessary tools with the following command:
\begin{verbatim}
  sudo apt-get install gcc g++ gfortran subversion patch wget
\end{verbatim}
Replace {\tt apt-get} with your relevant package manager, e.g. {\tt yum} for Red Hat-based distributions, {\tt zypper} for SUSE, etc. The {\tt g++} and {\tt gfortran} compilers may need to be specified respectively as {\tt gcc-c++} and {\tt gcc-gfortran} with some package managers.

On Mac OS X, you need either the Xcode Command Line Tools, available at \url{https://developer.apple.com/downloads} after registering as an Apple Developer, or a community alternative such as \url{https://github.com/kennethreitz/osx-gcc-installer/downloads} to install the gcc and g++ compilers.
It has been reported, that gcc/g++ 4.2 and older is not sufficient for using the HSL codes.
If you have a recent version of Xcode installed, the Command Line Tools are available under Preferences, Downloads. In Xcode 3.x, the Command Line Tools are contained in the optional item ``UNIX Dev Support'' during Xcode installation.
%If you are using OS X 10.6 or earlier, then see the instructions at \url{http://kitcambridge.tumblr.com/post/17778742499/installing-the-xcode-command-line-tools-on-snow-leopard} on how to install the Xcode Command Line Tools without needing to download the rest of Xcode. 
These items unfortunately do not come with a Fortran compiler, but you can get {\tt gfortran} from {\tt http://gcc.gnu.org/wiki/GFortranBinaries\#MacOS}. We have been able to compile \Ipopt using default Xcode versions of {\tt gcc} and {\tt g++} and a newer version of {\tt gfortran} from this link, but consistent version numbers may be an issue in future cases.

\subsection{Getting the \Ipopt Code}
\Ipopt is available from the COIN-OR subversion repository and the COIN-OR
group at GitHub (for expert users). You can
either download the code using \texttt{svn} (the
\textit{subversion} client) or \texttt{git} or
simply retrieve a tarball (compressed archive file).  While the
tarball is an easy method to retrieve the code, using the
\textit{subversion} and \textit{git} system allows users the benefits of the version
control system, including easy updates and revision control.

\subsubsection{Getting the \Ipopt code via subversion}

Of course, the \textit{subversion} client must be installed on your
system if you want to obtain the code this way (the executable is
called \texttt{svn}); it is already installed by default for many
recent Linux distributions.  Information about \textit{subversion} and
how to download it can be found at
\url{http://subversion.apache.org}.

To obtain the \Ipopt source code via subversion, change into the
directory in which you want to create a subdirectory {\tt CoinIpopt} with
the \Ipopt source code.  Then follow the steps below:
\begin{enumerate}
\item{Download the code from the repository}\\
{\tt \$ svn co https://projects.coin-or.org/svn/Ipopt/stable/3.12 CoinIpopt} \\
Note: The {\tt \$} indicates the command line
prompt, do not type {\tt \$}, only the text following it.
\item Change into the root directory of the \Ipopt distribution\\
{\tt \$ cd CoinIpopt}
\end{enumerate}

In the following, ``\texttt{\$IPOPTDIR}'' will refer to the directory in
which you are right now (output of \texttt{pwd}).

\subsubsection{Getting the \Ipopt code via git (expert users only)}

Of course, the \textit{git} client must be installed on your
system if you want to obtain the code this way (the executable is
called \texttt{git}).  Information about \textit{git} and
how to download it can be found at
\url{http://git-scm.com}.

\textbf{NOTE:} Currently, cloning the code from the GitHub mirror does
not automatically retrieve the \Ipopt dependencies that are essential
for building \Ipopt (BuildTools and build system for external codes).
You will have to obtain them manually.

To obtain the \Ipopt source code via git, change into the
directory in which you want to create a subdirectory {\tt CoinIpopt} with
the \Ipopt source code.  Then follow the steps below:
\begin{enumerate}
\item{Download the code from the repository}\\
{\tt \$ git clone -b stable/3.12 https://github.com/coin-or/Ipopt.git CoinIpopt} \\
Note: The {\tt \$} indicates the command line
prompt, do not type {\tt \$}, only the text following it.
\item Change into the root directory of the \Ipopt distribution\\
{\tt \$ cd CoinIpopt}
\end{enumerate}

In the following, ``\texttt{\$IPOPTDIR}'' will refer to the directory in
which you are right now (output of \texttt{pwd}).

\subsubsection{Getting the \Ipopt code as a tarball}

To use the tarball, follow the steps below:
\begin{enumerate}
\item Download the desired tarball from
  \url{http://www.coin-or.org/download/source/Ipopt}, it has the form {\tt
    Ipopt-{\em x.y.z}.tgz}, where {\tt\em x.y.z} is the version
  number, such as {\tt 3.12.0}.  There might also be daily snapshot
  from the stable branch.  The number of the latest official release
  can be found on the \Ipopt Trac page.
\item Issue the following commands to unpack the archive file: \\
\texttt{\$ gunzip Ipopt-{\em x.y.z}.tgz} \\
\texttt{\$ tar xvf Ipopt-{\em x.y.z}.tar} \\
Note: The {\tt \$} indicates the command line
prompt, do not type {\tt \$}, only the text following it.
\item Rename the directory you just extracted:\\
\texttt{\$ mv Ipopt-{\em x.y.z} CoinIpopt}
\item Change into the root directory of the \Ipopt distribution\\
{\tt \$ cd CoinIpopt}
\end{enumerate}

In the following, ``\texttt{\$IPOPTDIR}'' will refer to the directory in
which you are right now (output of \texttt{pwd}).

\subsection{Download External Code}\label{ExternalCode}
\Ipopt uses a few external packages that are not included in the
\Ipopt source code distribution, namely ASL (the AMPL Solver Library
if you want to compile the \Ipopt AMPL solver executable), Blas,
Lapack.

\Ipopt also requires at least one linear solver for sparse symmetric
indefinite matrices. There are
different possibilities, see Sections~\ref{sec:HSL}--\ref{sec:WSMP}.
{\bf It is important to keep in mind that usually
the largest fraction of computation time in the optimizer is spent for
solving the linear system, and that your choice of the linear solver
impacts \Ipopt's speed and robustness.  It might be worthwhile to try
different linear solver to experiment with what is best for your
application.}


Since this third party software is released under different licenses than
\Ipopt, we cannot distribute their code together with the \Ipopt
packages and have to ask you to go through the hassle of obtaining it
yourself (even though we tried to make it as easy for you as we
could).  Keep in mind that it is still your responsibility to ensure
that your downloading and usage of the third party components conforms
with their licenses.

Note that you only need to obtain the ASL if you intend to use \Ipopt
from AMPL.  It is not required if you want to specify your
optimization problem in a programming language (C++, C, or Fortran).
Also, currently, Lapack is only required if you intend to use the
quasi-Newton options implemented in \Ipopt.

\subsubsection{Download BLAS, LAPACK and ASL}
Note: It is \textbf{highly recommended that you obtain an efficient
  implementation of the BLAS library}, tailored to your hardware;
Section~\ref{sec:prerequisites} lists a few options.  Assuming that
your precompiled efficient BLAS library is \texttt{libmyblas.a} in
\texttt{\$HOME/lib}, you need to add the flag
\texttt{--with-blas="-L\$HOME/lib -lmyblas"} when you run
\texttt{configure} (see Section~\ref{sec.comp_and_inst}).  Some of
those libraries also include LAPACK.

If you have the download utility \texttt{wget} installed on your
system (or \texttt{ftp} on Mac OS X), retrieving source code for
BLAS (the inefficient reference
implementation, not required if you have a precompiled library), as
well as LAPACK and ASL is straightforward using scripts included with
the ipopt distribution.  These scripts download the required files
from the Netlib Repository
(\url{http://www.netlib.org}).
\medskip

\noindent
{\tt \$ cd \$IPOPTDIR/ThirdParty/Blas}\\
{\tt \$ ./get.Blas}\\
{\tt \$ cd ../Lapack}\\
{\tt \$ ./get.Lapack}\\
{\tt \$ cd ../ASL}\\
{\tt \$ ./get.ASL}
\medskip

\noindent
If you do not have \texttt{wget} (or \texttt{ftp} on Mac OS X) installed
on your system, please read the \texttt{INSTALL.*} files in the
\texttt{\$IPOPTDIR/ThirdParty/Blas},
\texttt{\$IPOPTDIR/ThirdParty/Lapack} and
\texttt{\$IPOPTDIR/ThirdParty/ASL} directories for alternative
instructions.

If you are having firewall issues with {\tt wget}, try opening the {\tt get.<library>} scripts and replace the line {\tt wgetcmd=wget} with {\tt wgetcmd="wget --passive-ftp"}.

If you are getting permissions errors from tar, try opening the {\tt get.<library>} scripts and replace any instances of {\tt tar xf} with {\tt tar --no-same-owner -xf}.

\subsubsection{Download HSL Subroutines}
\label{sec:HSL}

\noindent
There are two versions of HSL available:
\begin{description}
\item[HSL Archive] contains outdated codes that are freely available for
personal commercial or non-com\-mer\-cial usage. Note that you may not
redistribute these codes in either source or binary form without purchasing a
licence from the authors. This version includes MA27, MA28, and MC19.
\item[HSL 2011] contains more modern codes that are freely available for
academic use only. This version includes the codes from the HSL Archive and 
additionally MA57, HSL\_MA77, HSL\_MA86, and HSL\_MA97. \Ipopt supports the
HSL 2011 codes from 2012 and 2013, the support for the versions from 2012 may be
dropped in a future release.
\end{description}

% The use of alternative linear solvers is described in
% Sections~\ref{sec:MUMPS}--\ref{sec:WSMP}.  You do not necessarily
% have to use a HSL code such as MA27 as described in this section, but at least
% one linear solver is required for \Ipopt to function.
To obtain the HSL code, you can follow the following steps:
\begin{enumerate}
\item Go to \url{http://hsl.rl.ac.uk/ipopt}.
\item Choose whether to download either the Archive code or the HSL 2011
   code. To download, select the relevant
   ``source'' link.
\item Follow the instructions on the website, read the license, and
   submit the registration form.
\item Wait for an email containing a download link (this should take no
   more than one working day).
\end{enumerate}

\noindent
You may either:
\begin{itemize}
   \item Compile the HSL code as part of \Ipopt. See the instructions below.
   \item Compile the HSL code separately either before or after the \Ipopt code
      and use the shared library loading mechanism. See the documentation
      distributed with the HSL package for information on how to do so.
\end{itemize}

To compile the HSL code as part of \Ipopt, unpack the archive, then move and
rename the resulting directory so that it becomes
{\tt \$IPOPTDIR/ThirdParty/HSL/coinhsl}. \Ipopt may then be configured as
normal.

Note: Whereas it is essential to have at least one linear solver, the
package MC19 could be omitted (with the consequence that you cannot
use this method for scaling the linear systems arising inside the
\Ipopt algorithm).  By default, MC19 is only used to scale the linear
system when using one of the HSL solvers, but it can also be
switched on for other linear solvers (which usually have internal
scaling mechanisms).
Further, also the package MA28 can be omitted, since it is used only
in the experimental dependency detector, which is not used by default.

Note: If you are an academic or a student, we recommend you download the
HSL 2011 package as this ensures you have access to the full range of solvers.
MA57 can be considerably faster than MA27 on some problems.

Yet another note: If you have a precompiled library containing the
HSL codes, you can specify the directory with the header files and
the linker flags for this library with the \verb|--with-hsl-incdir| and
\verb|--with-hsl-lib| flags for the {\tt configure} script described in
Section~\ref{sec.comp_and_inst}.

\subsubsection{Obtaining the MUMPS Linear Solver}\label{sec:MUMPS}

You can also use the (public domain) sparse linear solver MUMPS.
Please visit the MUMPS home page \url{http://graal.ens-lyon.fr/MUMPS}
for more information about the solver. MUMPS is provided as Fortran 90
and C source code.  You need to have a Fortran 90 compiler (for
example, the GNU compiler {\tt gfortran} is a free one) to be able to
use it.

You can obtain the MUMPS code by running the script
{\tt \$IPOPTDIR/ThirdParty/Mumps/get.Mumps} if you have {\tt wget} 
(or {\tt ftp} on Mac OS X) installed in your system. 
Alternatively, you can get the latest version
from the MUMPS home page and extract the archive in the
directory {\tt \$IPOPTDIR/ThirdParty/Mumps}.  The extracted
directory usually has the MUMPS version number in it, so you need to
rename it to {\tt MUMPS} such that you have a file called
{\tt \$IPOPTDIR/ThirdParty/Mumps/MUMPS/README}.

Once you put the MUMPS source code into the correct place, the \Ipopt
configuration scripts will automatically detect it and compile MUMPS
together with \Ipopt, \emph{if your Fortran compiler is able to compile
Fortran 90 code}.

Note: MUMPS will perform better with METIS, see
Section~\ref{sec:METIS}.

Note: MUMPS uses interally a fake implementation of MPI.  If you are
using \Ipopt within an MPI program together with MUMPS, the code will
not run.  You will have to modify the MUMPS sources so that the MPI
symbols inside the MUMPS code are renamed.

\subsubsection{Obtaining the Linear Solver Pardiso}\label{sec:Pardiso}

If you would like to compile \Ipopt with the Parallel Sparse Direct
Linear Solver (Pardiso), you need to obtain either Intel's MKL library
or the Pardiso library from \url{http://www.pardiso-project.org} for
your operating system.

From \url{http://www.pardiso-project.org},
you can obtain a limited time license of Pardiso for academic or
evaluation purposes or buy a non-profit or commercial
license. Make sure you read the license agreement before filling out
the download form.

%Note: Pardiso is included in Intel's MKL library.  However, that
%version does not include the changes done by the Pardiso developers to
%make the linear solver work smoothly with \Ipopt.

Please consult Appendix~\ref{ExpertInstall} to find out how to
configure your \Ipopt installation to work with Pardiso.

\subsubsection{Obtaining the Linear Solver WSMP}\label{sec:WSMP}

If you would like to compile \Ipopt with the Watson Sparse Matrix
Package (WSMP), you need to obtain the WSMP library for your operating
system.  Information about WSMP can be found at
\url{http://www.research.ibm.com/projects/wsmp}.
%http://researcher.ibm.com/view_project.php?id=1426

At this website you can download the library for several operating systems
including a trial license key for 90 days that allows you to use WSMP
for ``educational, research, and benchmarking purposes by
non-profit academic institutions'' or evaluation purposes by commercial
organizations;
make sure you read the license agreement before using the library.
Once you obtained the library and license, please check if the version
number of the library matches the one on the WSMP website.

If a newer version is announced on that website, you can (and
probably should) request the current version by sending a message to
\verb|wsmp@watson.ibm.com|.  Please include the operating system and
other details to describe which particular version of WSMP you need.

% You can use the bugfix releases with the license you obtained from alphaWorks.

Note: Only the interface to the shared-memory version of WSMP is
currently supported.

Please consult Appendix~\ref{ExpertInstall} to find out how to
configure your \Ipopt installation to work with WSMP.

\subsubsection{Using the Linear Solver Loader}\label{sec:linear_solver_loader}
By default, \Ipopt will be compiled with a mechanism, the Linear
Solver Loader, which can dynamically load shared libraries with MA27,
MA57, HSL\_MA77, HSL\_MA86, HSL\_MA97, or the Pardiso linear solver at 
runtime\footnote{This is not
  enabled if you compile \Ipopt with the MS Visual Studio project files
  provided in the \Ipopt distribution.  Further, if you have problems
  compiling this new feature, you can disable this by specifying
  \texttt{--disable-linear-solver-loader} for the \texttt{configure}
  script}. This means, if you obtain one of those solvers after you
compiled \Ipopt, you don't need to recompile to
use it.  Instead, you can just put a shared library called
\texttt{libhsl.so} or \texttt{libpardiso.so} into the shared library
search path, \texttt{LD\_LIBRARY\_PATH}.  These are the names on most
UNIX platforms, including Linux.  On Mac OS X, the names are
\texttt{libhsl.dylib}, \texttt{libpardiso.dylib}, and
\texttt{DYLD\_LIBRARY\_PATH}.  On Windows, the names are \texttt{libhsl.dll},
\texttt{libpardiso.dll}, and \texttt{PATH}.

The Pardiso shared library can be downloaded from the Pardiso website.
To create a shared library containing the HSL linear solvers, read the
instructions in \texttt{\$IPOPTDIR/ThirdParty/HSL/INSTALL.HSL}.

\subsubsection{Obtaining METIS}\label{sec:METIS}

The linear solvers MA57, HSL\_MA77, HSL\_MA86, HSL\_MA97, and MUMPS can make 
use of the matrix ordering algorithms implemented in METIS (see
\url{http://glaros.dtc.umn.edu/gkhome/metis/metis/overview}).  If
you are using one of these linear solvers, you should obtain the METIS
source code and put it into \texttt{\$IPOPTDIR/ThirdParty/Metis}.
Read the \texttt{INSTALL.Metis} file in that directory, and if you
have the \texttt{wget} utility (or \texttt{ftp} on Mac OS X) installed on your 
system, you can download the code by running the \texttt{./get.Metis} script.

Note, that {\bf only the older METIS 4.x version\footnote{\url{http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/OLD/metis-4.0.3.tar.gz}} is supported} by
MA57, HSL\_MA77, HSL\_MA86, HSL\_MA97, MUMPS, and the build system.
The \texttt{./get.Metis} script takes care of downloading the right METIS version.

\subsection{Compiling and Installing \Ipopt} \label{sec.comp_and_inst}

\Ipopt can be easily compiled and installed with the usual {\tt
  configure}, {\tt make}, {\tt make install} commands.  We follow the
procedure that is used for most of the COIN-OR projects, based on the
GNU autotools.  At \url{https://projects.coin-or.org/CoinHelp}
you can find a general description of the tools.

Below are the basic steps for the \Ipopt compilation that should work
on most systems.  For special compilations and for some
troubleshooting see Appendix~\ref{ExpertInstall} and consult the
generic COIN-OR help page
\url{https://projects.coin-or.org/CoinHelp} before submitting a
ticket or sending a message to the mailing list.

\begin{enumerate}
\item Create a directory where you want to compile \Ipopt, for example\\
  {\tt \$ mkdir \$IPOPTDIR/build}\\
  and go into this direcrory\\
  {\tt \$ cd \$IPOPTDIR/build}

  Note: You can choose any location, including {\tt \$IPOPTDIR}
  itself, as the location of your compilation.  However, on COIN-OR we
  recommend to keep the source and compiled files separate.

\item Run the configure script\\
  {\tt \$ \$IPOPTDIR/configure}

  One might have to give options to the configure script, e.g., in
  order to choose a non-default compiler, or to tell it where some
  third party code is installed, see Appendix~\ref{ExpertInstall}.

  If the last output line reads ``\texttt{configure:\ Main configuration of Ipopt successful}'' then everything worked
  fine.  Otherwise, look at the screen output, have a look at the
  \texttt{config.log} output files and/or consult
  Appendix~\ref{ExpertInstall}.

  The default configure (without any options) is sufficient for most
  users that downloaded the source code for the linear solver. If you
  want to see the configure options, consult
  Appendix~\ref{ExpertInstall}, and also visit the generic COIN-OR
  configuration instruction page at

  \centerline{\url{https://projects.coin-or.org/CoinHelp/wiki/user-configure}}

\item Build the code \\
  {\tt \$ make}

  Note: If you are using GNU make, you can also try to speed up the
  compilation by using the {\tt -jN} flag (e.g., {\tt make -j3}),
  where {\tt N} is the number of parallel compilation jobs.  A good
  number for {\tt N} is the number of available processors plus one.
  Under some circumstances, this fails, and you might have to re-issue
  the command, or omit the {\tt -j} flag.
\item If you want, you can run a short test to verify that the
  compilation was successful.  For this, you just
  enter\\
  {\tt \$ make test}\\
  This will test if the AMPL solver executable works (if you got the
  ASL code) and if the included C++, C, and Fortran examples work.

  Note: The {\tt configure} script is not able to automatically
  determine the C++ runtime libraries for the C++ compiler.  For
  certain compilers we enabled default values for this, but those
  might not exist or be wrong for your compiler.  In that case, the C
  and Fortran example in the test will most probably fail to compile.
  If you don't want to hook up the compiled \Ipopt library to some
  Fortran or C code that you wrote you don't need to worry about this.
  If you do want to link the \Ipopt library with a C or Fortran
  compiler, you need to find out the C++ runtime libraries (e.g., by
  running the C++ compiler in verbose mode for a simple example
  program) and run {\tt configure} again, and this time specify all
  C++ runtime libraries with the {\tt CXXLIBS} variable (see also
  Appendix~\ref{ExpertInstall}).
\item Install \Ipopt \\
  {\tt \$ make install}\\
  This installs
  \begin{itemize}
  \item the \Ipopt AMPL solver executable (if ASL source was
    downloaded) in \texttt{\$IPOPTDIR/build/bin},
  \item the \Ipopt library (\texttt{libipopt.so}, \texttt{libipopt.a}
    or similar) and all its dependencies (MUMPS, HSL, Metis libraries)
    in \texttt{\$IPOPTDIR/build/lib},
  \item text files {\tt ipopt\_addlibs\_cpp.txt}, {\tt ipopt\_addlibs\_c.txt}, 
      and {\tt ipopt\_addlibs\_f.txt} in\\ 
    \texttt{\$IPOPTDIR/build/share/coin/doc/Ipopt} each 
    containing a line with linking flags that are required for
    linking code with the \Ipopt library for C++, C, and Fortran main
    programs, respectively. (This is only for convenience if you want
    to find out what additional flags are required, for example, to
    include the Fortran runtime libraries with a C++ compiler.)
  \item the necessary header files in
    \texttt{\$IPOPTDIR/build/include/coin}.
  \end{itemize}
  You can change the default installation directory (here
  \texttt{\$IPOPTDIR/build}) to something else (such as \texttt{/usr/local})
  by using the \verb|--prefix| switch for \texttt{configure}.
\item (Optional) Install \Ipopt for use with {\tt CUTEr}\\
  If you have {\tt CUTEr} already installed on your system and you
  want to use \Ipopt as a solver for problems modeled in {\tt SIF},
  type\\
  {\tt \$ make cuter}\\
  This assumes that you have the environment variable {\tt MYCUTER}
  defined according to the {\tt CUTEr} instructions.  After this, you
  can use the script {\tt sdipo} as the {\tt CUTEr} script to solve a
  {\tt SIF} model.
\end{enumerate}

Note: The above procedures show how to compile the code in directories
separate from the source files.  This comes in handy when you want to
compile the code with different compilers, compiler options, or
different operating system that share a common file system.  To use
this feature, change into the directory where you want to compile the
code, and then type {\tt \$IPOPTDIR/configure} with all the options.
For this, the directories with the \Ipopt source must not have any
configuration and compiled code.

\subsection{Installation on Windows}\label{WindowsInstall}

There are several ways to install \Ipopt on Windows systems.  The
first two options, described in Sections~\ref{CygwinInstall} and
\ref{CygwinInstallNative}, are to use Cygwin (see
\url{http://www.cygwin.com}), which offers a comprehensive UNIX-like
environment on Windows and in which the installation procedure
described earlier in this section can be used.  If you want to use the
(free) GNU compilers, follow the instructions in
Section~\ref{CygwinInstall}.  If you have the Microsoft C++ compiler
and possibly a ``native'' Fortran compiler (e.g., the Intel Fortran
compiler) and want to use those to compile \Ipopt, please see
Section~\ref{CygwinInstallNative}.  If you use MSYS/MinGW (a
light-weight UNIX-like environment for Windows), please consider the
notes in Section~\ref{MinGWInstall}.
If you want to compile the \Ipopt\ {\tt mex} interface to \Matlab, then we recommend to use the MSYS/MinGW option.

%The \Ipopt distribution also includes projects files for
%Microsoft Visual Studio. % (see Section~\ref{VisualStudioInstall}).

Note: Some binaries for \Ipopt are available on the COIN-OR website at
\url{http://www.coin-or.org/download/binary/Ipopt}.
There, also precompiled versions of \Ipopt as DLLs (generated from
the MSVS solution in \Ipopt's subdirectory
\texttt{\$IPOPTDIR/Ipopt/MSVisualStudio/v8-ifort}) are available.  Look at the
\texttt{README} files for details.  An example how to use these DLLs
from your own MSVS project is in
\texttt{\$IPOPTDIR/Ipopt/MSVisualStudio/BinaryDLL-Link-Example}.


\subsubsection{Installation with Cygwin using GNU compilers}\label{CygwinInstall}

Cygwin is a Linux-like environment for Windows; if you don't know what
it is you might want to have a look at the Cygwin homepage,
\url{http://www.cygwin.com}.

It is possible to build the \Ipopt AMPL solver executable in Cygwin
for general use in Windows.  You can also hook up \Ipopt to your own
program if you compile it in the Cygwin environment\footnote{It is
  also possible to build an \Ipopt DLL that can be used from
  non-cygwin compilers, but this is not (yet?) supported.}.

If you want to compile \Ipopt under Cygwin, you first have to install
Cygwin on your Windows system.  This is pretty straight forward; you
simply download the ``setup'' program from
\url{http://www.cygwin.com} and start it.

Then you do the following steps (assuming here that you don't have any
complications with firewall settings etc - in that case you might have
to choose some connection settings differently):

\begin{enumerate}
\item Click next
\item Select ``install from the internet'' (default) and click next
\item Select a directory where Cygwin is to be installed (you can
  leave the default) and choose all other things to your liking, then
  click next
\item Select a temp dir for Cygwin setup to store some files (if you
  put it on your desktop you will later remember to delete it)
\item Select ``direct connection'' (default) and click next
\item Select some mirror site that seems close by to you and click next
\item OK, now comes the complicated part:\\
  You need to select the packages that you want to have installed.  By
  default, there are already selections, but the compilers are usually
  not pre-chosen.  You need to make sure that you select the GNU
  compilers (for Fortran, C, and C++), Subversion, and some additional tools.
  For this, get the following packages from the associated branches:
  \begin{itemize}
  \item ``Devel'': {\tt gcc4}
  \item ``Devel'': {\tt gcc4-fortran}
  \item ``Devel'': {\tt pkg-config}
  \item ``Devel'': {\tt subversion}
  \item ``Archive'': {\tt unzip}
  \item ``Utils'': {\tt patch}
  \item ``Web'': {\tt wget}
  \end{itemize}
  When a Resolving Dependencies window comes up, be sure to
  ``Select required packages (RECOMMENDED)''.
  This will automatically also select some other packages.
  
\item\label{it:cyg_done} Then you click on next, and Cygwin will be
  installed (follow the rest of the instructions and choose everything
  else to your liking).  At a later point you can easily add/remove
  packages with the setup program.

\item The version of the GNU Make utility provided by the Cygwin installer 
   will not work. Therefore, you need to download the fixed version from 
  \url{http://www.cmake.org/files/cygwin/make.exe} and save it to {\tt C:$\backslash$cygwin$\backslash$bin}.
   Double-check this new version by typing {\tt make --version} in a Cygwin 
   terminal (see next point).
   If you get an error {\tt -bash: /usr/bin/make: Bad address}, then try 
   \url{http://www.cmake.org/files/cygwin/make.exe-cygwin1.7} instead, rename 
   it to {\tt make.exe} and move it to {\tt C:$\backslash$cygwin$\backslash$bin}.
   (Replace {\tt C:$\backslash$cygwin} with your installation location if different.)

\item Now that you have Cygwin, you can open a Cygwin window, which is
  like a UNIX shell window.

\item\label{it:cyg_inst} Now you just follow the instructions in the
  beginning of Section~\ref{Installing}: You download the \Ipopt
  code into your Cygwin home directory (from the Windows explorer that
  is usually something like
  \texttt{C:$\backslash$Cygwin$\backslash$home$\backslash$your\_user\_name}).\
  After that you obtain the third party code (as on Linux/UNIX),
  type

  \texttt{./configure}

  and

  \texttt{make install}

  in the correct directories, and hopefully that will work.  The
  \Ipopt AMPL solver executable will be in the subdirectory
  \texttt{bin} (called ``\texttt{ipopt.exe}'').  If you want to set
  the installation, type

  \texttt{make test}

% I think this is outdated nowadays:
%  \textbf{NOTE:} By default, the compiled binaries (library and
%  executables) will be ``Cygwin-native'', i.e., in order to run
%  executables using this, the {\tt Cygwin1.dll} has to be present
%  (e.g., in a Cygwin window).  If you want to compile things in a way
%  that allow your executables to run outside of Cygwin, e.g., in a
%  regular DOS prompt, you need to specify the option ``{\tt
%    --enable-doscompile}'' when you run {\tt configure}.
\end{enumerate}

\subsubsection{Installation with Cygwin using the MSVC++ compiler}
\label{CygwinInstallNative}

This section describes how you can compile \Ipopt with the Microsoft
Visual C++ compiler under Cygwin.  Here you have two options for
compiling the Fortran code in the third party dependencies:

\begin{itemize}
\item Using a Windows Fortran compiler, e.g., the Intel Fortran
  compiler, which is also able to compile Fortran 90 code.  This would
  allow you to compile the MUMPS linear solver if you desire to do so.
\item Using the {\tt f2c} Fortran to C compiler, available for free at
  Netlib (see \url{http://www.netlib.org/f2c}).  This can only compile
  Fortran 77 code (i.e., you won't be able to compile MUMPS).  Before
  doing the following installation steps, you need to follow the
  instructions in\\ {\tt\$IPOPTDIR/BuildTools/compile\_f2c/INSTALL}.
\end{itemize}

\noindent
Once you have settled on this, do the following:

\begin{enumerate}
\item Follow the instructions in Section~\ref{CygwinInstall} until
  Step~\ref{it:cyg_inst} and stop after your downloaded the third
  party code.

\item\label{it:setupmsvcpath} Now you need to make sure that Cygwin
  knows about the native compilers.  For this you need to edit the
  file {\tt cygwin.bat} in the Cygwin base directory (usually {\tt
    C:$\backslash$cygwin}).  Here you need to add a line like the
  following:

  \texttt{call "C:$\backslash$Program Files$\backslash$Microsoft Visual
    Studio 8$\backslash$VC$\backslash$vcvarsall.bat"}

  On my computer, this sets the environment variables so that I can use
  the MSVC++ compiler.

  If you want to use also a native Fortran compiler, you need to
  include something like this

  \texttt{call "C:$\backslash$Program
    Files$\backslash$Intel$\backslash$Fortran$\backslash$compiler80$\backslash$IA32$\backslash$BIN$\backslash$ifortvars.bat"}

  You might have to search around a bit.  The important thing is that,
  after your change, you can type ``{\tt cl}'' in a newly opened
  Cygwin windows, and it finds the Microsoft C++ compiler (and if you
  want to use it, the Fortran compiler, such as the Intel's {\tt
    ifort}).

\item Run the configuration script, and tell it that you want to use
  the native compilers:

  \texttt{./configure --enable-doscompile=msvc}

  Make sure the last message is

  \texttt{Main Ipopt configuration successful}

%\item\label{it:ASLcompile} If want to compile the AMPL solver
%  executable, you need to compile the ASL library from a script.  For
%  this you need to change into the ASL compilation directory, execute
%  the script \texttt{compile\_MS\_ASL}, and go back to the directory
%  where you were:
%
%  \texttt{cd ThirdParty/ASL}
%
%  \texttt{./compile\_MS\_ASL}
%
%  \texttt{cd -}

\item Now you can compile the code with

  \texttt{make},

  test the installation with

  \texttt{make test},

  and install everything with

  \texttt{make install}
\end{enumerate}

\subsubsection{Installation with MSYS/MinGW}\label{MinGWInstall}

You can compile \Ipopt also under MSYS/MinGW, which is another, more
light-weight UNIX-like environment for Windows.  It can be obtained
from \url{http://www.mingw.org/}.

If you want to use MSYS/MinGW to compile \Ipopt with native Windows
compilers (see Section~\ref{CygwinInstallNative}), all you need to
install is the basic version\footnote{a convenient Windows install
program is available from \url{http://sourceforge.net/projects/mingw/files/Installer/mingw-get-inst/}}.
If you also want to use the GNU
compilers, you need to install those as well, of course.

A compilation with the GNU compilers works just like with any other
UNIX system, as described in Section~\ref{sec.comp_and_inst}.
That is, during the installation, select (at least) the C Compiler, C++ Compiler, Fortran Compiler, MSYS Basic System, and the MinGW Developer ToolKit. 
Additionally, {\tt wget} and {\tt unzip} should be installed with the following command in an MSYS terminal:
\begin{verbatim}
  mingw-get install msys-wget msys-unzip
\end{verbatim}

If you want to use the native MSVC++ compiler (with {\tt f2c} or a native
Fortran compiler), you essentially follow the steps outlined in
Section~\ref{CygwinInstallNative}. 
Additionally, you need to make sure that the environment variables are set for 
the compilers (see step~\ref{it:setupmsvcpath}), this time adding the line to 
the {\tt msys.bat} file.

For a 64-bit build, you will need to install also a MinGW-64 distribution.
We recommend TDM-GCC, which is available from \url{http://sourceforge.net/projects/tdm-gcc/files/TDM-GCC\%20Installer/tdm-gcc-webdl.exe/download}.
Install MinGW-64 in a different folder than your existing 32-bit MinGW installation!
The components you need are: {\tt core} (under {\tt gcc}), {\tt c++} (under {\tt gcc}), {\tt fortran} (under {\tt gcc}), {\tt openmp} (under {\tt gcc}, necessary if you want to use any multi-threaded linear solvers), {\tt binutils}, and {\tt mingw64-runtime}.

After MinGW-64 is installed, open the file {\tt C:$\backslash$MinGW$\backslash$msys$\backslash$1.0$\backslash$etc$\backslash$fstab}, and replace the line
\begin{verbatim}
  C:\MinGW\     /mingw
\end{verbatim}
with
\begin{verbatim}
  C:\MinGW64\   /mingw
\end{verbatim}
(Replace paths with your installation locations if different.) 

  %Also, you need to run the
%\texttt{compile\_MS\_ASL} script in the \texttt{ThirdParty/ASL}
%immediately after you run the configuration script.

%\subsubsection{Using Microsoft Visual Studio}\label{VisualStudioInstall}

%\textbf{NEW:} Some binaries for \Ipopt are available on the COIN-OR website at
%\url{http://www.coin-or.org/download/binary/Ipopt}.

%There are also precompiled versions of Ipopt as DLLs (generated from
%the MSVC solution in \Ipopt's subdirectory
%\texttt{\$IPOPTDIR/Ipopt/MSVisualStudio/v8-ifort}).  Look at the
%\texttt{README} files for details.  An example how to use these DLLs
%from your own MSVC project is in\\
%\texttt{\$IPOPTDIR/Ipopt/MSVisualStudio/BinaryDLL-Link-Example}.

%The \Ipopt distribution includes project files that can be used to
%compile the \Ipopt library, the AMPL solver executable, and a C++
%example within the Microsoft Visual Studio.  The project files have
%been created with Microsoft Visual 8 Express.  Fortran files in the
%third party dependencies need to be converted to C code using the {\tt
%  f2c} Fortran to C compiler\footnote{Projects files for a previous
%  version of \Ipopt that used the Intel Fortran compiler are in
%  {\$IPOPTDIR$\backslash$Ipopt$\backslash$NoLongerMaintainedWindows},
%  but they are probably outdated, and you will have to correct them.}.

%In order to use those project files, download the \Ipopt source code,
%as well as the required third party code (put it into the {\tt
%  ThirdParty$\backslash$Blas}, {\tt ThirdParty$\backslash$Lapack},
%{\tt ThirdParty$\backslash$HSL}, {\tt  ThirdParty$\backslash$ASL}
%directories).  Detailed step-by-step instructions on how to install
%{\tt f2c}, translate the Fortran code to C files, and further details
%are described in the file

%\texttt{\$IPOPTDIR$\backslash$Ipopt$\backslash$MSVisualStudio$\backslash$v8$\backslash$README.TXT}

%After that, you can open the solution file

%\texttt{\$IPOPTDIR$\backslash$Ipopt$\backslash$MSVisualStudio$\backslash$v8$\backslash$Ipopt.sln}

%If you are compiling \Ipopt with different linear solvers, you need
%to edit the configuration header file
%\texttt{Ipopt$\backslash$src$\backslash$Common$\backslash$IpoptConfig.h},
%in the section after
%\begin{verbatim}
%/***************************************************************************/
%/*             HERE DEFINE THE CONFIGURATION SPECIFIC MACROS               */
%/***************************************************************************/
%\end{verbatim}
%and include the corresponding source files in
%\texttt{Ipopt$\backslash$src$\backslash$Algorithm$\backslash$LinearSolvers}
%and add the corresponding libraries to your project.

\subsection{Compiling and Installing the Java Interface \JIpopt}  
\label{sec.jipopt.build}
\hfill \textit{based on documentation by Rafael de Pelegrini Soares}%
\footnote{VRTech Industrial Technologies}
\medskip

\JIpopt uses the Java Native Interface (JNI), which is a programming framework 
that allows Java code running in the Java Virtual Machine (JVM) to call and be 
called by native applications and libraries written in languages such as C and 
C++.
\JIpopt requires Java 5 or higher.

After building and installing \Ipopt, the \JIpopt interface can be build by 
setting the environment variable {\tt JAVA\_HOME} to the directory that 
contains your JDK, changing to the \JIpopt directory in your \Ipopt build, and 
issuing {\tt make}, e.g.,
\begin{verbatim}
  export JAVA_HOME=/usr/lib/jvm/java-1.5.0
  cd $IPOPTDIR/build/Ipopt/contrib/JavaInterface
  make
\end{verbatim}

This will generate the Java class {\tt org/coinor/Ipopt.class}, which you will 
need to make available in your Java code (i.e., add {\tt \$IPOPTDIR/build/Ipopt/contrib/JavaInterface}
to your {\tt CLASSPATH}) and the shared object
{\tt lib/libjipopt.so} (on Linux/UNIX) or {\tt lib/libjipopt.dylib} (on Mac OS 
X) or the DLL {\tt lib/jipopt.dll} (on Windows).
In order to test your \JIpopt library you can run two example problems by 
issuing the command {\tt make test} inside the \JIpopt directory.

\textbf{NOTE}: The \JIpopt build procedure currently cannot deal with spaces 
in the path to the JDK. If you are on Windows and have Java in a path like 
\verb|C:\Program Files\Java|, try setting {\tt JAVA\_HOME} to the DOS 
equivalent \verb|C:\Progra~1| (or similar).

\textbf{NOTE}: \JIpopt needs to be able to load the \Ipopt library dynamically 
at runtime. Therefore, \Ipopt must have been compiled with the {\tt -fPIC} 
compiler flag. While per default, an Ipopt shared library is compiled with 
this flag, for a configuration of \Ipopt in debug mode ({\tt --enable-debug}) 
or as static library ({\tt --disable-shared}), the configure flag
{\tt --with-pic} need to be used to enable compilation with {\tt -fPIC}.

\subsection{Compiling and Installing the R Interface \ipoptr} 
\label{sec.ipoptr.build}

The \ipoptr interface can be build after \Ipopt has been build and installed.
In the best case, it is sufficient to execute the following command in R:
\begin{verbatim}
install.packages('$IPOPTDIR/build/Ipopt/contrib/RInterface', repos=NULL, type='source')
\end{verbatim}

In certain situations, however, it can be necessary to setup the dynamic 
library load path to the path where the \Ipopt library has been installed, 
e.g.,
\begin{verbatim}
  LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$IPOPTDIR/build/lib
\end{verbatim}

\textbf{NOTE}: R needs to be able to load the \Ipopt library dynamically 
at runtime. Therefore, \Ipopt must have been compiled with the {\tt -fPIC} 
compiler flag. While per default, an Ipopt shared library is compiled with 
this flag, for a configuration of \Ipopt in debug mode ({\tt --enable-debug}) 
or as static library ({\tt --disable-shared}), the configure flag {\tt --with-
pic} need to be used to enable compilation with {\tt -fPIC}.

After installation of the \ipoptr package, it should be possible to load the 
package in R and to view the help page:
\begin{verbatim}
> library('ipoptr')
> ?ipoptr
\end{verbatim}

\subsection{Compiling and Installing the \Matlab interface}
\label{sec.matlab.build}
\hfill \textit{based on documentation by Peter Carbonetto\footnote{University of British Columbia}, Tony Kelman\footnote{University of California, Berkeley}, and Ray Zimmerman}%
\medskip

The \Matlab interface to \Ipopt uses the {\tt mex} interface of \Matlab.
It has been tested on \Matlab versions 7.2 through 7.7. It might very well 
work on earlier versions of \Matlab, but there is also a good chance that it 
will not. It is unlikely that the software will run with versions prior to 
\Matlab 6.5.

\textbf{NOTE}: The \Matlab interface \textbf{does not support \Matlab 8.3} (aka, R2014a), as
it has not been adapted to \Matlab changes in the buildsystem for MEX files.

First, note that some binaries of \Ipopt\ {\tt mex} files are available for download at
\url{http://www.coin-or.org/download/binary/Ipopt}.
Further, the OPTI Toolbox (\url{http://www.i2c2.aut.ac.nz/Wiki/OPTI}) comes with a 
precompiled \Matlab interface for \Ipopt on Windows.

\subsubsection{Setting up {\tt mex}}

To build the interface by yourself, you will need to have \Matlab installed on 
your system and have it configured to build {\tt mex} files,  see 
\url{http://www.mathworks.com/support/tech-notes/1600/1605.html} for detail on 
how to set this up.

Ipopt 3.11 has added Makefile options to automate fixes for commonly-encountered
issues with building the \Matlab interface. On Mac OS X or Windows, the file
{\tt mexopts.sh} ({\tt mexopts.bat} on Windows) will need to be modified.
This is performed automatically by calling {\tt make mexopts} in the
{\tt \$IPOPTDIR/build/Ipopt/contrib/MatlabInterface/src} directory. No changes
will be made if you already have a {\tt mexopts.sh} file in that directory.
If you need to make these modifications manually, follow the steps below.


For Mac OS X, the following procedure has been reported: First, one executes a command like
\begin{verbatim}
/Applications/MATLAB_R2012.app/bin/mex -setup
\end{verbatim}
This creates a {\tt mexopts.sh} file in the {\tt ~/.matlab/R2010} directory. 
Copy that file to the directory {\tt \$IPOPTDIR/build/Ipopt/contrib/MatlabInterface/src}
and modify it as follows.
\begin{itemize}
\item In the {\tt maci} section (32 bit builds) or the {\tt maci64} section
(64 bit builds), change both instances of {\tt libgfortran.dylib} to
{\tt libgfortran.a} in the {\tt FC\_LIBDIR} line (in case your Fortran
compiler only comes with static libraries).
\item Remove all occurrences of {\tt -isysroot \$SDKROOT} or
{\tt -Wl,-syslibroot,\$SDKROOT} in case the hard-coded version of the Xcode
SDK that Matlab expects does not match what you have installed on your system.
\item Remove all occurrences of {\tt -arch \$ARCHS} in case you are using a
GNU compiler that does not recognize these Apple-specific flags.
\end{itemize}

On Windows, if you are using the GNU compilers via MinGW, then you will need
to use the {\tt gnumex} project. First, execute the script {\tt ./get.Gnumex}
from the {\tt \$IPOPTDIR/Ipopt/contrib/MatlabInterface} directory. Then, after
configuring \Ipopt, go to {\tt \$IPOPTDIR/build/contrib/MatlabInterface/src}
and execute {\tt make gnumex}. This will start an instance of \Matlab and open
the {\tt gnumex} tool. Check that the options are filled out appropriately for
your MinGW installation, click ``Make options file," then close this new instance
of \Matlab after {\tt gnumex} has created {\tt mexopts.bat}.

Calling {\tt make mexopts} will automatically make the necessary changes to this
new {\tt mexopts.bat} file. If you would like to do so manually, the changes are
as follows.
\begin{itemize}
\item Change {\tt COMPILER=gcc} to {\tt COMPILER=g++}
\item Change {\tt GM\_MEXLANG=c} to {\tt GM\_MEXLANG=cxx}
\item Add the contents of the {\tt LIBS=} line from the \Matlab interface Makefile
to {\tt GM\_ADD\_LIBS}
\item If you want to statically link the standard libraries into the 
{\tt mex} file, add {\tt -static} to {\tt GM\_ADD\_LIBS}
\end{itemize}

\subsubsection{Adjusting configuration and build of \Ipopt}

The configure script of \Ipopt attempts to automatically locate the directory 
where \Matlab is installed by querying the location of the {\tt matlab} 
executable. You can also manually specify the \Matlab home directory when 
calling the configure script with the flag {\tt --with-matlab-home}. You can
determine this home directory by the command {\tt matlabroot} within \Matlab.


In practice, it has been found easier to install and use the \Matlab interface 
by disabling compilation of the shared libraries, and use only static 
libraries instead. However, these static libraries need to be built in a way 
that allow using them in a shared library, i.e., they need to build with 
position-independent code.
This is achieved with the configure script flags
\begin{verbatim}
  --disable-shared --with-pic
\end{verbatim}
On Mac OS X, it has been reported that additionally the following flags for 
configure should be used:
\begin{verbatim}
  ADD_CFLAGS="-fno-common -fexceptions -no-cpp-precomp"
  ADD_CXXFLAGS="-fno-common -fexceptions -no-cpp-precomp"
  ADD_FFLAGS="-fexceptions -fbackslash" 
\end{verbatim}

With \Ipopt 3.11, a \emph{site script for configure} has been added to the 
\Matlab interface. This script takes care of setting configure options in a 
way that is appropriate for building an \Ipopt { \tt mex} file that is useable via
\Matlab. Therefore, instead of setting configure options as described in the 
previous section, it should be sufficient to create a directory {\tt \$IPOPTDIR/build/share}, copy the site file {\tt \$IPOPTDIR/contrib/MatlabInterface/MatlabInterface.site} to that directory, and rename it to {\tt config.site} before running configure.
Alternatively, you can set an environment variable 
{\tt CONFIG\_SITE} that points to the site file.

This site script sets the configure flags (if not specified by the user)
{\tt --disable-shared --with-pic --with-blas=BUILD --with-lapack=BUILD}.
The first two flags are discussed above. We also specify that the reference
versions of BLAS and LAPACK should be used by default because of a commonly
observed issue on 64-bit Linux systems. If \Ipopt configure finds BLAS and/or
LAPACK libraries already installed then it will use them. However, \Matlab
includes its own versions of BLAS and LAPACK, which on 64-bit systems are
incompatible with the expected interface used by \Ipopt and the BLAS and LAPACK
packages available in many Linux distributions. If the \Ipopt { \tt mex} file is
compiled in such a way that the BLAS and LAPACK libraries are dynamically linked
as shared libraries (as found in installed Linux packages), those library
dependencies will be overridden by \Matlab's incompatible versions. This can
be avoided by statically linking BLAS and LAPACK into the \Ipopt { \tt mex} file,
which the above combination of configure flags will do. Note, that this issue does
not appear to affect Mac OS X versions of \Matlab, so if you would like to use
the Apple optimized BLAS and LAPACK libraries you can override these settings and
specify {\tt --with-blas='-framework vecLib' --with-lapack='-framework vecLib'}.

The site script also tests whether the compilers on your system are capable of
statically linking the standard C++ and Fortran libraries into a shared library.
This is possible with GCC versions 4.5.0 or newer on Mac OS X or Windows, 4.7.3 or newer
(when GCC itself is built {\tt --with-pic}) on Linux. If this is the case, then
the site script will set appropriate configure flags and options in the \Matlab
interface {\tt Makefile} to statically link all standard libraries into the \Ipopt { \tt mex}
file. This should allow a single {\tt mex} file to work with a variety of versions of
\Matlab, and on computers that do not have the same compiler versions installed.
If this static linking of standard libraries causes any issues, you can disable
it with the configure flag {\tt --disable-matlab-static}.

\subsubsection{Building the \Matlab interface}

After configuring, building, and installing \Ipopt itself, it is time to build 
the \Matlab interface.
For that, \Ipopt's configure has setup a directory
{\tt \$IPOPTDIR/build/contrib/MatlabInterface/src} which contains a
{\tt Makefile}.
You may need to edit this file a little bit to suit for your system setup. You 
will find that most of the variables such as {\tt CXX} and {\tt CXXFLAGS} have 
been automatically (and hopefully, correctly) set according to the flags 
specified during your initial call to configure script.
However, you may need to modify {\tt MATLAB\_HOME}, {\tt MEXSUFFIX} and
{\tt MEX} as explained in the comments of the Makefile.
For example, on Mac OS X, it has been reported that all duplicates of strings 
like {\tt -L/usr/lib/gcc/i686-apple-darwin11/4.2.1/../../..} should be removed 
from the {\tt LIBS} line.

Once you think you've set up the {\tt Makefile} properly, type
{\tt make install} in the same directory as the {\tt Makefile}. If you didn't 
get any errors, then you should have ended up with a {\tt mex} file. The {\tt mex}
file will be called {\tt ipopt.\$MEXEXT}, where {\tt \$MEXEXT} is {\tt mexglx}
for 32-bit Linux, {\tt mexa64} for 64-bit Linux, {\tt mexw32} for 32-bit Windows, etc.

\subsubsection{Making \Matlab aware of the {\tt mex} file} 

In order to use the {\tt mex} file in \Matlab, you need to tell \Matlab where 
to find it. The best way to do this is to type
\begin{verbatim}
  addpath sourcedir
\end{verbatim}
in the \Matlab command prompt, where {\tt sourcedir} is the location of the 
{\tt mex} file you created. (For more information, type {\tt help addpath} in 
\Matlab. You can also achieve the same thing by modifying the {\tt MATLABPATH} 
environment variable in the UNIX command line, using either the {\tt export} 
command (in Bash shell) or the {\tt setenv} command (in C-shell).


%There's a great possibility you will encounter problems with the installation instructions we have just described here. I'm afraid some resourcefulness will be required on your part, as the installation will be slightly different for each person. Please consult the troubleshooting section on this webpage, and the archives of the IPOPT mailing list. If you can't find the answer at either of these locations, try sending an email to the IPOPT mailing list.

\subsubsection{Additional notes}

Starting with version 7.3, \Matlab can handle 64-bit 
addressing, and the authors of \Matlab have modified the implementation of 
sparse matrices to reflect this change. However, the row and column indices in 
the sparse matrix are converted to signed integers, and this could potentially 
cause problems when dealing with large, sparse matrices on 64-bit platforms 
with \Matlab version 7.3 or greater.

As \Matlab (version R2008a or newer) includes its own HSL MA57
library, \Ipopt's configure can be setup to enable using this library in
\Ipopt's MA57 solver interface. To enable this, one should specify the
configure option {\tt --enable-matlab-ma57}. Note, that using this option is
not advisable if one also provides the source for MA57 via ThirdParty/HSL.

\subsubsection{Troubleshooting}

The installation procedure described above does involve a certain amount of 
expertise on the part of the user. If you are encountering problems, it is 
highly recommended that you follow the standard installation of \Ipopt first, 
and then test the installation by running some examples, either in C++ or in 
AMPL.

What follows are a list of common errors encountered, along with a suggested remedy.
\medskip

\textbf{Problem:} compilation is successful, but \Matlab crashes

\textbf{Remedy:} Even if you didn't get any errors during compilation, there's 
still a possibility that you didn't link the {\tt mex} file properly. In this 
case, executing \Ipopt in \Matlab will cause \Matlab to crash. This is a 
common problem, and usually arises because you did not choose the proper 
compiler or set the proper compilation flags (e.g. {\tt --with-pic}) when you 
ran the configure script at the very beginning.
\medskip

\textbf{Problem:} \Matlab fails to link to \Ipopt shared library

\textbf{Remedy:} You might encounter this problem if you try to execute one of 
the examples in \Matlab, and \Matlab complains that it cannot find the \Ipopt 
shared library. The installation script has been set up so that the {\tt mex} 
file you are calling knows where to look for the \Ipopt shared library. 
However, if you moved the library then you will run into a problem. One way to 
fix this problem is to modify the {\tt LDFLAGS} variable in the \Matlab 
interface {\tt Makefile} (see above) so that the correct path of the \Ipopt 
library is specified. Alternatively, you could modify the
{\tt LD\_LIBRARY\_PATH} environment variable so that the location of the 
\Ipopt library is included in the path. If none of this is familiar to you, 
you might want to do a web search to find out more.
\medskip

\textbf{Problem:} {\tt mwIndex} is not defined

\textbf{Remedy:} You may get a compilation error that says something to the 
effect that {\tt mwIndex} is not defined. This error will surface on a version 
of \Matlab prior to 7.3. The solution is to add the flag {\tt -DMWINDEXISINT} 
to the {\tt CXXFLAGS} variable in the \Matlab interface {\tt Makefile} (see 
above).

%\subsubsection{More Information}

%The difficulties with building and using the \Matlab interface of \Ipopt have 
%lead to several installation instructions on the web:
%\begin{itemize}
%\item Peter Carbonetto's original instructions on how to build the \Matlab 
%interface:\\ \url{https://projects.coin-or.org/Ipopt/wiki/MatlabInterface}
%\item Ray Zimmerman's instructions on how to build the \Matlab interface on 
%Mac OS X:\\ \url{https://projects.coin-or.org/Ipopt/wiki/Ipopt_on_Mac_OS_X}
%\item Giacomo Perantoni's experiences and help with the \Matlab interface on 
%Windows:\\ \url{http://users.ox.ac.uk/~newc3480}
%\end{itemize}


\subsection{Expert Installation Options for \Ipopt}\label{ExpertInstall}

The configuration script and Makefiles in the \Ipopt distribution
have been created using GNU's {\tt autoconf} and {\tt automake}.  They
attempt to automatically adapt the compiler settings etc.\ to the
system they are running on.  We tested the provided scripts for a
number of different machines, operating systems and compilers, but you
might run into a situation where the default setting does not work, or
where you need to change the settings to fit your particular
environment.

In general, you can see the list of options and variables that can be
set for the {\tt configure} script by typing \verb/configure --help/.
Also, the generic COIN-OR help pages are a valuable resource of
information:

\centerline{\url{https://projects.coin-or.org/CoinHelp}}

Below a few particular options are discussed:

\begin{itemize}
\item The {\tt configure} script tries to determine automatically, if
  you have BLAS and/or LAPACK already installed on your system (trying
  a few default libraries), and if it does not find them, it makes
  sure that you put the source code in the required place.

  However, you can specify a BLAS library (such as your local ATLAS
  library\footnote{see \url{http://math-atlas.sourceforge.net}})
  explicitly, using the \verb/--with-blas/ flag for {\tt configure}.
  For example,

  \verb|./configure --with-blas="-L$HOME/lib -lf77blas -lcblas -latlas"| %$

  To tell the configure script to compile and use the downloaded BLAS
  source files even if a BLAS library is found on your system, specify
  \verb|--with-blas=BUILD|.

  Similarly, you can use the \verb/--with-lapack/ switch to specify
  the location of your LAPACK library, or use the keyword {\tt BUILD}
  to force the \Ipopt makefiles to compile LAPACK together with
  \Ipopt.

\item Similarly, if you have a precompiled library containing the
  HSL packages, you can specify the directory with the
  \texttt{CoinHslConfig.h} header file with the \verb|--with-hsl-incdir| flag and
  the linker flags with the \verb|--with-hsl-lib| flag. 
  Analogously, use \verb|--with-asl-incdir| and \verb|--with-asl-lib| for
  building against a precompiled AMPL solver library.
  
\item The HSL codes HSL\_MA86 and HSL\_MA97 can run in parallel if 
compiled with OpenMP support. By default, this is not enabled by 
\Ipopt's configure so far. To enable OpenMP with GNU compilers, it 
has been reported that the following configure flags should be used:

  \verb|ADD_CFLAGS=-fopenmp ADD_FFLAGS=-fopenmp ADD_CXXFLAGS=-fopenmp|

\item If you want to compile \Ipopt with the linear solver Pardiso
  (see Section~\ref{sec:Pardiso}) from the Pardiso project website,
  you need to specify the link flags
  for the library with the \verb|--with-pardiso| flag, including
  required additional libraries and flags.  For example, if you want
  to compile \Ipopt with the parallel version of Pardiso (located in
  {\tt \$HOME/lib}) on an AIX system in 64bit mode, you should add the
  flag

  \verb|--with-pardiso="-qsmp=omp $HOME/lib/libpardiso_P4AIX51_64_P.so"|  %$

  If you are using the parallel version of Pardiso, you need to
  specify the number of processors it should run on with the
  environment variable \verb|OMP_NUM_THREADS|, as described in the
  Pardiso manual.
  
  If you want to compile \Ipopt with the Pardiso library that is included
  in Intel MKL, it should be sufficient to ensure that MKL is used for
  the linear algebra routines (Blas/Lapack). On some systems, configure
  is able to find MKL automatically when looking for Blas. On other systems,
  one has to specify the MKL libraries with the \verb|--with-blas| option.

\item If you want to compile \Ipopt with the linear solver WSMP (see
  Section~\ref{sec:WSMP}), you need to specify the link flags for the
  library with the \verb|--with-wsmp| flag, including required
  additional libraries and flags.  For example, if you want to compile
  \Ipopt with WSMP (located in {\tt \$HOME/lib}) on an Intel IA32
  Linux system, you should add the flag

  \verb|--with-wsmp="$HOME/lib/wsmp/wsmp-Linux/lib/IA32/libwsmp.a -lpthread"| %$

\item If you want to compile \Ipopt with a precompiled MUMPS library
  (see Section~\ref{sec:MUMPS}), you need to specify the directory containing
  the MUMPS header files with the \verb|--with-mumps-incdir| flag,
  e.g.,

  \verb|--with-mumps-incdir="$HOME/MUMPS/include"|  %$

  and you also need to provide the link flags for MUMPS with the
  \verb|--with-mumps-lib| flag.

\item If you want to specify that you want to use particular
  compilers, you can do so by adding the variables definitions for
  {\tt CXX}, {\tt CC}, and {\tt F77} to the {\tt ./configure} command
  line, to specify the C++, C, and Fortran compiler, respectively.
  For example,

  {\tt ./configure CXX=g++-4.2.0 CC=gcc-4.2.0 F77=gfortran-4.2.0}

  In order to set the compiler flags, you should use the variables
  {\tt CXXFLAGS}, {\tt CFLAGS}, {\tt FFLAGS}.  Note, that the \Ipopt
  code uses ``{\tt dynamic\_cast}''.  Therefore it is necessary that
  the C++ code is compiled including RTTI (Run-Time Type Information).
  Some compilers need to be given special flags to do that (e.g.,
  ``{\tt -qrtti=dyna}'' for the AIX {\tt xlC} compiler).

  Please also check the generic COIN-OR help page at

  \centerline{\url{https://projects.coin-or.org/CoinHelp/wiki/user-configure\#GivingOptions}}

  for the description of more variables that can be set for {\tt
    configure}.

\item By default, the \Ipopt library is compiled as a shared library,
  on systems where this is supported.  If you want to generate a
  static library, you need to specify the {\tt --disable-shared}
  flag.  If you want to compile both shared and static libraries, you
  should specify the {\tt --enable-static} flag.

\item If you want to link the \Ipopt library with a main program
  written in C or Fortran, the C and Fortran compiler doing the
  linking of the executable needs to be told about the C++ runtime
  libraries.  Unfortunately, the current version of {\tt autoconf}
  does not provide the automatic detection of those libraries.  We
  have hard-coded some default values for some systems and compilers,
  but this might not work all the time.

  If you have problems linking your Fortran or C code with the \Ipopt
  library {\tt libipopt.a} and the linker complains about missing
  symbols from C++ (e.g., the standard template library), you should
  specify the C++ libraries with the {\tt CXXLIBS} variable.  To find out
  what those libraries are, it is probably helpful to link a  simple C++
  program with verbose compiler output.

  For example, for the Intel compilers on a Linux system, you
  might need to specify something like

  {\tt ./configure CC=icc F77=ifort CXX=icpc $\backslash$\\ \hspace*{14ex} CXXLIBS='-L/usr/lib/gcc-lib/i386-redhat-linux/3.2.3 -lstdc++'}

\item Compilation in 64bit mode sometimes requires some special
  consideration.  For example, for compilation of 64bit code on AIX,
  we recommend the following configuration

  {\tt ./configure AR='ar -X64' NM='nm -X64' $\backslash$\\
    \hspace*{14ex} CC='xlc -q64' F77='xlf -q64' CXX='xlC
    -q64'$\backslash$\\ \hspace*{14ex} CFLAGS='-O3
    -bmaxdata:0x3f0000000'
    $\backslash$\\ \hspace*{14ex} FFLAGS='-O3 -bmaxdata:0x3f0000000' $\backslash$\\
    \hspace*{14ex} CXXFLAGS='-qrtti=dyna -O3 -bmaxdata:0x3f0000000'}

  (Alternatively, a simpler solution for AIX is to set the environment variable {\tt OBJECT\_MODE} to 64.)

% \item To build library/archive files (with the ending {\tt .a})
%   including C++ code in some environments, it is necessary to use the
%   C++ compiler instead of {\tt ar} to build the archive.  This is for
%   example the case for some older compilers on SGI and SUN.  For this,
%   the {\tt configure} variables {\tt AR}, {\tt ARFLAGS}, and {\tt
%     AR\_X} are provided.  Here, {\tt AR} specifies the command for the
%   archiver for creating an archive, and {\tt ARFLAGS} specifies
%   additional flags.  {\tt AR\_X} contains the command for extracting
%   all files from an archive.  For example, the default setting for SUN
%   compilers for our configure script is

%   {\tt AR='CC -xar' ARFLAGS='-o' AR\_X='ar x'}

\item It is possible to compile the \Ipopt library in a debug
  configuration, by specifying \verb|--enable-debug|.  Then the
  compilers will use the debug flags (unless the compilation flag
  variables are overwritten in the {\tt configure} command line)

  Also, you can tell \Ipopt to do some additional runtime sanity
  checks, by specifying the flag {\tt --with-ipopt-checklevel=1}.

  This usually leads to a significant slowdown of the code, but might
  be helpful when debugging something.

% We assume vpath installations already during this chapter.
%\item It is not necessary to produce the binary files in the
%  directories where the source files are.  If you want to compile the
%  code on different systems or with different compilers/options on a
%  shared file system, you can keep one single copy of the source files
%  in one directory, and the binary files for each configuration in
%  separate directories.  For this, simply run the configure script in
%  the directory where you want the base directory for the \Ipopt
%  binary files.  For example:
%
%  {\tt \$ mkdir \$HOME/Ipopt-objects}\\
%  {\tt \$ cd \$HOME/Ipopt-objects}\\
%  {\tt \$ \$HOME/CoinIpopt/configure}

\end{itemize}


\section{Interfacing your NLP to \Ipopt}
\label{sec:tutorial-example}

\Ipopt has been designed to be flexible for a wide variety of
applications, and there are a number of ways to interface with \Ipopt
that allow specific data structures and linear solver
techniques. Nevertheless, the authors have included a standard
representation that should meet the needs of most users.

This tutorial will discuss six interfaces to \Ipopt, namely the AMPL
modeling language \cite{FouGayKer:AMPLbook} interface, and the C++, C,
Fortran, Java, and R code interfaces.  AMPL is a 3rd party modeling language
tool that allows users to write their optimization problem in a syntax
that resembles the way the problem would be written mathematically.
Once the problem has been formulated in AMPL, the problem can be
easily solved using the (already compiled) \Ipopt AMPL solver
executable, {\tt ipopt}. Interfacing your problem by directly linking
code requires more effort to write, but can be far more efficient for
large problems.

We will illustrate how to use each of the four interfaces using an
example problem, number 71 from the Hock-Schittkowsky test suite \cite{HS},
%\begin{subequations}\label{HS71}
  \begin{eqnarray}
    \min_{x \in \Re^4} &&x_1 x_4 (x_1 + x_2 + x_3)  +  x_3 \label{eq:ex_obj} \\
    \mbox{s.t.}  &&x_1 x_2 x_3 x_4 \ge 25 \label{eq:ex_ineq} \\
    &&x_1^2 + x_2^2 + x_3^2 + x_4^2  =  40 \label{eq:ex_equ} \\
    &&1 \leq x_1, x_2, x_3, x_4 \leq 5, \label{eq:ex_bounds}
  \end{eqnarray}
%\end{subequations}
with the starting point
\begin{equation}
x_0 = (1, 5, 5, 1) \label{eq:ex_startpt}
\end{equation}
and the optimal solution
\[
x_* = (1.00000000, 4.74299963, 3.82114998, 1.37940829). \nonumber
\]

You can find further, less documented examples for using \Ipopt from
your own source code in the {\tt Ipopt/examples} subdirectory.

\subsection{Using \Ipopt through AMPL} \label{sec.ipoptampl}
Using the AMPL solver executable is by far the easiest way to
solve a problem with \Ipopt. The user must simply formulate the problem
in AMPL syntax, and solve the problem through the AMPL environment.
There are drawbacks, however. AMPL is a 3rd party package and, as
such, must be appropriately licensed (a free student version for
limited problem size is available from the AMPL website,
\url{http://www.ampl.com}). Furthermore, the AMPL environment may be 
prohibitive for very large problems. Nevertheless, formulating the problem in 
AMPL is straightforward and even for large problems, it is often used as a
prototyping tool before using one of the code interfaces.

This tutorial is not intended as a guide to formulating models in
AMPL. If you are not already familiar with AMPL, please consult
\cite{FouGayKer:AMPLbook}.

The problem presented in equations
(\ref{eq:ex_obj})--(\ref{eq:ex_startpt}) can be solved with \Ipopt with
the following AMPL model.

\begin{verbatim}
# tell ampl to use the ipopt executable as a solver
# make sure ipopt is in the path!
option solver ipopt;

# declare the variables and their bounds, 
# set notation could be used, but this is straightforward
var x1 >= 1, <= 5; 
var x2 >= 1, <= 5; 
var x3 >= 1, <= 5; 
var x4 >= 1, <= 5;

# specify the objective function
minimize obj:
    x1 * x4 * (x1 + x2 + x3) + x3;
        
# specify the constraints
s.t.
  inequality:
    x1 * x2 * x3 * x4 >= 25;
                
  equality:
    x1^2 + x2^2 + x3^2 +x4^2 = 40;

# specify the starting point            
let x1 := 1;
let x2 := 5;
let x3 := 5;
let x4 := 1;

# solve the problem
solve;

# print the solution
display x1;
display x2;
display x3;
display x4;
\end{verbatim}
  
The line, ``{\tt option solver ipopt;}'' tells AMPL to use \Ipopt as
the solver. The \Ipopt executable (installed in
Section~\ref{sec.comp_and_inst}) must be in the {\tt PATH} for AMPL to
find it. The remaining lines specify the problem in AMPL format. The
problem can now be solved by starting AMPL and loading the mod file:
\begin{verbatim}
$ ampl
> model hs071_ampl.mod;
.
.
.
\end{verbatim}
%$
The problem will be solved using \Ipopt and the solution will be
displayed.

At this point, AMPL users may wish to skip the sections about
interfacing with code, but should read Section \ref{sec:options}
concerning \Ipopt options, and Section \ref{sec:output} which
explains the output displayed by \Ipopt.

\subsubsection{Using \Ipopt from the command line}

It is possible to solve AMPL problems with \Ipopt directly from
the command line. However, this requires a file in format {\tt .nl}
produced by {\tt ampl}.  If you have a model and data loaded in {\tt
  Ampl}, you can create the corresponding {\tt .nl} file with name,
say, {\tt myprob.nl} by using the {\tt Ampl} command:

{\tt write gmyprob}

There is a small {\tt .nl} file available in the \Ipopt distribution. It is
located at {\tt Ipopt/test/mytoy.nl}. 
We use this file in the remainder of this section. We assume that the file
{\tt mytoy.nl} is in the current directory and that the command 
{\tt ipopt} is a shortcut for running the {\tt ipopt} binary available
in the {\tt bin} directory of the installation of \Ipopt.

We list below commands to perform basic tasks from the Linux prompt.

\begin{itemize}
\item To solve {\tt mytoy.nl} from the Linux prompt, use:

{\tt ipopt mytoy}

\item To see all command line options for \Ipopt, use:

{\tt ipopt -=}

\item To see more detailed information on all options for \Ipopt:

{\tt ipopt mytoy 'print\_options\_documentation yes'}

\item To run {\tt ipopt}, setting the maximum number of iterations to 2 and 
print level to 4:

{\tt ipopt mytoy 'max\_iter 2 print\_level 4'}
\end{itemize}

If many options are to be set, they can be collected in a file {\tt
  ipopt.opt} that is automatically read by \Ipopt if present in
the current directory, see Section \ref{sec:options}.


\subsection{Interfacing with \Ipopt through code}
\label{sec.required_info}
In order to solve a problem, \Ipopt needs more information than just
the problem definition (for example, the derivative information). If
you are using a modeling language like AMPL, the extra information is
provided by the modeling tool and the \Ipopt interface. When
interfacing with \Ipopt through your own code, however, you must
provide this additional information.
The following information is required by \Ipopt:
\begin{enumerate}
\item Problem dimensions \label{it.prob_dim}
  \begin{itemize}
  \item number of variables
  \item number of constraints
  \end{itemize}
\item Problem bounds
  \begin{itemize}
  \item variable bounds
  \item constraint bounds
  \end{itemize}
\item Initial starting point
  \begin{itemize}
  \item Initial values for the primal $x$ variables
  \item Initial values for the multipliers (only
    required for a warm start option)
  \end{itemize}
\item Problem Structure \label{it.prob_struct}
  \begin{itemize}
  \item number of nonzeros in the Jacobian of the constraints
  \item number of nonzeros in the Hessian of the Lagrangian function
  \item sparsity structure of the Jacobian of the constraints
  \item sparsity structure of the Hessian of the Lagrangian function
  \end{itemize}
\item Evaluation of Problem Functions \label{it.prob_eval} \\
  Information evaluated using a given point ($x,
  \lambda, \sigma_f$ coming from \Ipopt)
  \begin{itemize}
  \item Objective function, $f(x)$
  \item Gradient of the objective $\nabla f(x)$
  \item Constraint function values, $g(x)$
  \item Jacobian of the constraints, $\nabla g(x)^T$
  \item Hessian of the Lagrangian function, 
    $\sigma_f \nabla^2 f(x) + \sum_{i=1}^m\lambda_i\nabla^2
    g_i(x)$ \\
    (this is not required if a quasi-Newton options is chosen to
    approximate the second derivatives)
  \end{itemize}
\end{enumerate}
The problem dimensions and bounds are
straightforward and come solely from the problem definition. The
initial starting point is used by the algorithm when it begins
iterating to solve the problem. If \Ipopt has difficulty converging, or
if it converges to a locally infeasible point, adjusting the starting
point may help.  Depending on the starting point, \Ipopt may also
converge to different local solutions.

Providing the sparsity structure of derivative matrices is a bit more
involved. \Ipopt is a nonlinear programming solver that is designed
for solving large-scale, sparse problems. While \Ipopt can be
customized for a variety of matrix formats, the triplet format is used
for the standard interfaces in this tutorial. For an overview of the
triplet format for sparse matrices, see Appendix~\ref{app.triplet}.
Before solving the problem, \Ipopt needs to know the number of
nonzero elements and the sparsity structure (row and column indices of
each of the nonzero entries) of the constraint Jacobian and the
Lagrangian function Hessian. Once defined, this nonzero structure MUST
remain constant for the entire optimization procedure. This means that
the structure needs to include entries for any element that could ever
be nonzero, not only those that are nonzero at the starting point.

As \Ipopt iterates, it will need the values for
Item~\ref{it.prob_eval} in Section \ref{sec.required_info} evaluated at
particular points. Before we can begin coding the interface, however,
we need to work out the details of these equations symbolically for
example problem (\ref{eq:ex_obj})-(\ref{eq:ex_bounds}).

The gradient of the objective $f(x)$ is given by
\[
\left[
\begin{array}{c}
x_1 x_4 + x_4 (x_1 + x_2 + x_3) \\
x_1 x_4 \\
x_1 x_4 + 1 \\
x_1 (x_1 + x_2 + x_3)
\end{array}
\right]
\]
and the Jacobian of the constraints $g(x)$ is
\[
\left[
\begin{array}{cccc}
x_2 x_3 x_4     & x_1 x_3 x_4   & x_1 x_2 x_4   & x_1 x_2 x_3   \\
2 x_1           & 2 x_2         & 2 x_3         & 2 x_4
\end{array}
\right].
\]

We also need to determine the Hessian of the Lagrangian\footnote{If a
  quasi-Newton option is chosen to approximate the second derivatives,
  this is not required.  However, if second derivatives can be
  computed, it is often worthwhile to let \Ipopt use them, since the
  algorithm is then usually more robust and converges faster.  More on
  the quasi-Newton approximation in Section~\ref{sec:quasiNewton}.}.
The Lagrangian function for the NLP
(\ref{eq:ex_obj})-(\ref{eq:ex_bounds}) is defined as $f(x) + g(x)^T
\lambda$ and the Hessian of the Lagrangian function is, technically, $
\nabla^2 f(x_k) + \sum_{i=1}^m\lambda_i\nabla^2 g_i(x_k)$.  However,
we introduce a factor ($\sigma_f$) in front of the objective term
so that \Ipopt can ask for the Hessian of the objective or the
constraints independently, if required.
%
Thus, for \Ipopt the symbolic form of the Hessian of the
Lagrangian is
\begin{equation}\label{eq:IpoptLAG}
\sigma_f \nabla^2 f(x_k) + \sum_{i=1}^m\lambda_i\nabla^2 g_i(x_k)
\end{equation}
and for the example problem this becomes
%\begin{eqnarray}
%{\cal L}(x,\lambda) &{=}& f(x) + c(x)^T \lambda \nonumber \\
%&{=}& \left(x_1 x_4 (x_1 + x_2 + x_3)  +  x_3\right) 
%+ \left(x_1 x_2 x_3 x_4\right) \lambda_1 \nonumber \\
%&& \;\;\;\;\;+ \left(x_1^2 + x_2^2 + x_3^2 + x_4^2\right) \lambda_2 
%- \displaystyle \sum_{i \in 1..4} z^L_i + \sum_{i \in 1..4} z^U_i
%\end{eqnarray}
\[%\begin{equation}
\sigma_f \left[
\begin{array}{cccc}
2 x_4           & x_4           & x_4           & 2 x_1 + x_2 + x_3     \\
x_4             & 0             & 0             & x_1                   \\
x_4             & 0             & 0             & x_1                   \\
2 x_1+x_2+x_3   & x_1           & x_1           & 0
\end{array}
\right]
+
\lambda_1
\left[
\begin{array}{cccc}
0               & x_3 x_4       & x_2 x_4       & x_2 x_3       \\
x_3 x_4         & 0             & x_1 x_4       & x_1 x_3       \\
x_2 x_4         & x_1 x_4       & 0             & x_1 x_2       \\
x_2 x_3         & x_1 x_3       & x_1 x_2       & 0 
\end{array}
\right]
+
\lambda_2
\left[
\begin{array}{cccc}
2       & 0     & 0     & 0     \\
0       & 2     & 0     & 0     \\
0       & 0     & 2     & 0     \\
0       & 0     & 0     & 2
\end{array}
\right]
\]%\end{equation}
where the first term comes from the Hessian of the objective function,
and the second and third term from the Hessian of the constraints
(\ref{eq:ex_ineq}) and (\ref{eq:ex_equ}), respectively. Therefore, the
dual variables $\lambda_1$ and $\lambda_2$ are the multipliers
for constraints (\ref{eq:ex_ineq}) and (\ref{eq:ex_equ}), respectively.

%C =============================================================================
%C
%C     This is an example for the usage of IPOPT.
%C     It implements problem 71 from the Hock-Schittkowsky test suite:
%C
%C     min   x1*x4*(x1 + x2 + x3)  +  x3
%C     s.t.  x1*x2*x3*x4                   >=  25
%C           x1**2 + x2**2 + x3**2 + x4**2  =  40
%C           1 <=  x1,x2,x3,x4  <= 5
%C
%C     Starting point:
%C        x = (1, 5, 5, 1)
%C
%C     Optimal solution:
%C        x = (1.00000000, 4.74299963, 3.82114998, 1.37940829)
%C
%C =============================================================================
\vspace{\baselineskip}

The remaining sections of the tutorial will lead you through
the coding required to solve example problem
(\ref{eq:ex_obj})--(\ref{eq:ex_bounds}) using, first C++, then C, and finally
Fortran. Completed versions of these examples can be found in {\tt
\$IPOPTDIR/Ipopt/examples} under {\tt hs071\_cpp}, {\tt hs071\_c}, {\tt
hs071\_f}.

As a user, you are responsible for coding two sections of the program
that solves a problem using \Ipopt: the main executable (e.g., {\tt
  main}) and the problem representation.  Typically, you will write an
executable that prepares the problem, and then passes control over to
\Ipopt through an {\tt Optimize} or {\tt Solve} call. In this call,
you will give \Ipopt everything that it requires to call back to your
code whenever it needs functions evaluated (like the objective
function, the Jacobian of the constraints, etc.).  In each of the
three sections that follow (C++, C, and Fortran), we will first
discuss how to code the problem representation, and then how to code
the executable.

\subsection{The C++ Interface} \label{sec.cppinterface}
This tutorial assumes that you are familiar with the C++ programming
language, however, we will lead you through each step of the
implementation. For the problem representation, we will create a class
that inherits off of the pure virtual base class, {\tt TNLP} ({\tt
  IpTNLP.hpp}). For the executable (the {\tt main} function) we will
make the call to \Ipopt through the {\tt IpoptApplication} class
({\tt IpIpoptApplication.hpp}). In addition, we will also be using the
{\tt SmartPtr} class ({\tt IpSmartPtr.hpp}) which implements a reference
counting pointer that takes care of memory management (object
deletion) for you (for details, see Appendix~\ref{app.smart_ptr}).

After ``\texttt{make install}'' (see Section~\ref{sec.comp_and_inst}),
the header files are installed in \texttt{\$IPOPTDIR/include/coin}
(or in \texttt{\$PREFIX/include/coin} if the switch
\verb|--prefix=$PREFIX| was used for {\tt configure}).  %$

\subsubsection{Coding the Problem Representation}\label{sec.cpp_problem}
We provide the required information
by coding the {\tt HS071\_NLP} class, a specific implementation of the
{\tt TNLP} base class. In the executable, we will create an instance
of the {\tt HS071\_NLP} class and give this class to \Ipopt so it can
evaluate the problem functions through the {\tt TNLP} interface. If
you have any difficulty as the implementation proceeds, have a look at
the completed example in the {\tt Ipopt/examples/hs071\_cpp} directory.

Start by creating a new directory {\tt MyExample} under {\tt examples} and 
create the files {\tt hs071\_nlp.hpp} and {\tt
  hs071\_nlp.cpp}. In {\tt hs071\_nlp.hpp}, include {\tt IpTNLP.hpp}
(the base class), tell the compiler that we are using the \Ipopt
namespace, and create the declaration of the {\tt HS071\_NLP} class,
inheriting off of {\tt TNLP}. Have a look at the {\tt TNLP} class in
{\tt IpTNLP.hpp}; you will see eight pure virtual methods that we must
implement. Declare these methods in the header file.  Implement each
of the methods in {\tt HS071\_NLP.cpp} using the descriptions given
below. In {\tt hs071\_nlp.cpp}, first include the header file for your
class and tell the compiler that you are using the \Ipopt namespace.
A full version of these files can be found in the {\tt
  Ipopt/examples/hs071\_cpp} directory.

It is very easy to make mistakes in the implementation of the function
evaluation methods, in particular regarding the derivatives.  \Ipopt
has a feature that can help you to debug the derivative code, using
finite differences, see Section~\ref{sec:deriv-checker}.

Note that the return value of any {\tt bool}-valued function should be
{\tt true}, unless an error occurred, for example, because the value of
a problem function could not be evaluated at the required point.

\paragraph{Method {\texttt{get\_nlp\_info}}} with prototype
\begin{verbatim}
virtual bool get_nlp_info(Index& n, Index& m, Index& nnz_jac_g,
                          Index& nnz_h_lag, IndexStyleEnum& index_style)
\end{verbatim}
Give \Ipopt the information about the size of the problem (and hence,
the size of the arrays that it needs to allocate). 
\begin{itemize}
\item {\tt n}: (out), the number of variables in the problem (dimension of $x$).
\item {\tt m}: (out), the number of constraints in the problem (dimension of $g(x)$).
\item {\tt nnz\_jac\_g}: (out), the number of nonzero entries in the Jacobian.
\item {\tt nnz\_h\_lag}: (out), the number of nonzero entries in the Hessian.
\item {\tt index\_style}: (out), the numbering style used for row/col entries in the sparse matrix
format ({\tt C\_STYLE}: 0-based, {\tt FORTRAN\_STYLE}: 1-based; see
also Appendix~\ref{app.triplet}).
\end{itemize}
\Ipopt uses this information when allocating the arrays that
it will later ask you to fill with values. Be careful in this method
since incorrect values will cause memory bugs which may be very
difficult to find.

Our example problem has 4 variables (n), and 2 constraints (m). The
constraint Jacobian for this small problem is actually dense and has 8
nonzeros (we still need to represent this Jacobian using the sparse
matrix triplet format). The Hessian of the Lagrangian has 10
``symmetric'' nonzeros (i.e., nonzeros in the lower left triangular
part.).  Keep in mind that the number of nonzeros is the total number
of elements that may \emph{ever} be nonzero, not just those that are
nonzero at the starting point. This information is set once for the
entire problem.

\begin{footnotesize}
\begin{verbatim}
bool HS071_NLP::get_nlp_info(Index& n, Index& m, Index& nnz_jac_g, 
                             Index& nnz_h_lag, IndexStyleEnum& index_style)
{
  // The problem described in HS071_NLP.hpp has 4 variables, x[0] through x[3]
  n = 4;

  // one equality constraint and one inequality constraint
  m = 2;

  // in this example the Jacobian is dense and contains 8 nonzeros
  nnz_jac_g = 8;

  // the Hessian is also dense and has 16 total nonzeros, but we
  // only need the lower left corner (since it is symmetric)
  nnz_h_lag = 10;

  // use the C style indexing (0-based)
  index_style = TNLP::C_STYLE;

  return true;
}
\end{verbatim}
\end{footnotesize}

\paragraph{Method {\texttt{get\_bounds\_info}}} with prototype
\begin{verbatim}
virtual bool get_bounds_info(Index n, Number* x_l, Number* x_u,
                             Index m, Number* g_l, Number* g_u)
\end{verbatim}
Give \Ipopt the value of the bounds on the variables and constraints.
\begin{itemize}
\item {\tt n}: (in), the number of variables in the problem (dimension of $x$). 
\item {\tt x\_l}: (out) the lower bounds $x^L$ for $x$. 
\item {\tt x\_u}: (out) the upper bounds $x^U$ for $x$.
\item {\tt m}: (in), the number of constraints in the problem (dimension of $g(x)$).
\item {\tt g\_l}: (out) the lower bounds $g^L$ for $g(x)$. 
\item {\tt g\_u}: (out) the upper bounds $g^U$ for $g(x)$.
\end{itemize}
The values of {\tt n} and {\tt m} that you specified in {\tt
  get\_nlp\_info} are passed to you for debug checking.  Setting a
lower bound to a value less than or equal to the value of the option
\htmlref{\tt nlp\_lower\_bound\_inf}{opt:nlp_lower_bound_inf} will
cause \Ipopt to assume no lower bound. Likewise, specifying the upper
bound above or equal to the value of the option
\htmlref{\tt nlp\_upper\_bound\_inf}{opt:nlp_upper_bound_inf} will cause \Ipopt to
assume no upper bound.  These options,
\htmlref{\tt nlp\_lower\_bound\_inf}{opt:nlp_lower_bound_inf}
and \htmlref{\tt nlp\_upper\_bound\_inf}{opt:nlp_upper_bound_inf},
are set to $-10^{19}$ and $10^{19}$,
respectively, by default, but may be modified by changing the options
(see Section \ref{sec:options}).

In our example, the first constraint has a lower bound of $25$ and no upper
bound, so we set the lower bound of constraint {\tt [0]} to $25$ and
the upper bound to some number greater than $10^{19}$. The second
constraint is an equality constraint and we set both bounds to
$40$. \Ipopt recognizes this as an equality constraint and does not
treat it as two inequalities.

\begin{footnotesize}
\begin{verbatim}
bool HS071_NLP::get_bounds_info(Index n, Number* x_l, Number* x_u,
                                Index m, Number* g_l, Number* g_u)
{
  // here, the n and m we gave IPOPT in get_nlp_info are passed back to us.
  // If desired, we could assert to make sure they are what we think they are.
  assert(n == 4);
  assert(m == 2);

  // the variables have lower bounds of 1
  for (Index i=0; i<4; i++)
    x_l[i] = 1.0;

  // the variables have upper bounds of 5
  for (Index i=0; i<4; i++)
    x_u[i] = 5.0;

  // the first constraint g1 has a lower bound of 25
  g_l[0] = 25;
  // the first constraint g1 has NO upper bound, here we set it to 2e19.
  // Ipopt interprets any number greater than nlp_upper_bound_inf as 
  // infinity. The default value of nlp_upper_bound_inf and nlp_lower_bound_inf
  // is 1e19 and can be changed through ipopt options.
  g_u[0] = 2e19;

  // the second constraint g2 is an equality constraint, so we set the 
  // upper and lower bound to the same value
  g_l[1] = g_u[1] = 40.0;

  return true;
}
\end{verbatim}
\end{footnotesize}

\paragraph{Method {\texttt{get\_starting\_point}}} with prototype
\begin{verbatim}
virtual bool get_starting_point(Index n, bool init_x, Number* x,
                                bool init_z, Number* z_L, Number* z_U,
                                Index m, bool init_lambda, Number* lambda)
\end{verbatim}
Give \Ipopt the starting point before it begins iterating.
\begin{itemize}
\item {\tt n}: (in), the number of variables in the problem (dimension of $x$). 
\item {\tt init\_x}: (in), if true, this method must provide an initial value for $x$.
\item {\tt x}: (out), the initial values for the primal variables, $x$.
\item {\tt init\_z}: (in), if true, this method must provide an initial value 
        for the bound multipliers $z^L$ and $z^U$.
\item {\tt z\_L}: (out), the initial values for the bound multipliers, $z^L$.
\item {\tt z\_U}: (out), the initial values for the bound multipliers, $z^U$.
\item {\tt m}: (in), the number of constraints in the problem (dimension of $g(x)$).
\item {\tt init\_lambda}: (in), if true, this method must provide an initial value 
        for the constraint multipliers, $\lambda$.
\item {\tt lambda}: (out), the initial values for the constraint multipliers, $\lambda$.
\end{itemize}

The variables {\tt n} and {\tt m} are passed in for your convenience.
These variables will have the same values you specified in {\tt
  get\_nlp\_info}.

Depending on the options that have been set, \Ipopt may or may not
require bounds for the primal variables $x$, the bound multipliers
$z^L$ and $z^U$, and the constraint multipliers $\lambda$. The boolean
flags {\tt init\_x}, {\tt init\_z}, and {\tt init\_lambda} tell you
whether or not you should provide initial values for $x$, $z^L$, $z^U$, or
$\lambda$ respectively. The default options only require an initial
value for the primal variables $x$.  Note, the initial values for
bound multiplier components for ``infinity'' bounds
($x_L^{(i)}=-\infty$ or $x_U^{(i)}=\infty$) are ignored.

In our example, we provide initial values for $x$ as specified in the
example problem. We do not provide any initial values for the dual
variables, but use an assert to immediately let us know if we are ever
asked for them.

\begin{footnotesize}
\begin{verbatim}
bool HS071_NLP::get_starting_point(Index n, bool init_x, Number* x,
                                   bool init_z, Number* z_L, Number* z_U,
                                   Index m, bool init_lambda,
                                   Number* lambda)
{
  // Here, we assume we only have starting values for x, if you code
  // your own NLP, you can provide starting values for the dual variables
  // if you wish to use a warmstart option
  assert(init_x == true);
  assert(init_z == false);
  assert(init_lambda == false);

  // initialize to the given starting point
  x[0] = 1.0;
  x[1] = 5.0;
  x[2] = 5.0;
  x[3] = 1.0;

  return true;
}
\end{verbatim}
\end{footnotesize}

\paragraph{Method {\texttt{eval\_f}}} with prototype
\begin{verbatim}
virtual bool eval_f(Index n, const Number* x, 
                    bool new_x, Number& obj_value)
\end{verbatim}
Return the value of the objective function at the point $x$.
\begin{itemize}
\item {\tt n}: (in), the number of variables in the problem (dimension
  of $x$).
\item {\tt x}: (in), the values for the primal variables, $x$, at which
  $f(x)$ is to be evaluated.
\item {\tt new\_x}: (in), false if any evaluation method was
  previously called with the same values in {\tt x}, true otherwise.
\item {\tt obj\_value}: (out) the value of the objective function
  ($f(x)$).
\end{itemize}

The boolean variable {\tt new\_x} will be false if the last call to
any of the evaluation methods ({\tt eval\_*}) used the same $x$
values. This can be helpful when users have efficient implementations
that calculate multiple outputs at once. \Ipopt internally caches
results from the {\tt TNLP} and generally, this flag can be ignored.

The variable {\tt n} is passed in for your convenience. This variable
will have the same value you specified in {\tt get\_nlp\_info}.

For our example, we ignore the {\tt new\_x} flag and calculate the objective.

\begin{footnotesize}
\begin{verbatim}
bool HS071_NLP::eval_f(Index n, const Number* x, bool new_x, Number& obj_value)
{
  assert(n == 4);

  obj_value = x[0] * x[3] * (x[0] + x[1] + x[2]) + x[2];

  return true;
}
\end{verbatim}
\end{footnotesize}

\paragraph{Method {\texttt{eval\_grad\_f}}} with prototype
\begin{verbatim}
virtual bool eval_grad_f(Index n, const Number* x, bool new_x, 
                         Number* grad_f)
\end{verbatim}
Return the gradient of the objective function at the point $x$.
\begin{itemize}
\item {\tt n}: (in), the number of variables in the problem (dimension of $x$). 
\item {\tt x}: (in), the values for the primal variables, $x$, at which
  $\nabla f(x)$ is to be evaluated.
\item {\tt new\_x}: (in), false if any evaluation method was previously called 
        with the same values in {\tt x}, true otherwise.
\item {\tt grad\_f}: (out) the array of values for the gradient of the 
        objective function ($\nabla f(x)$).
\end{itemize}

The gradient array is in the same order as the $x$ variables (i.e., the
gradient of the objective with respect to {\tt x[2]} should be put in
{\tt grad\_f[2]}).

The boolean variable {\tt new\_x} will be false if the last call to
any of the evaluation methods ({\tt eval\_*}) used the same $x$
values. This can be helpful when users have efficient implementations
that calculate multiple outputs at once. \Ipopt internally caches
results from the {\tt TNLP} and generally, this flag can be ignored.

The variable {\tt n} is passed in for your convenience. This
variable will have the same value you specified in {\tt
get\_nlp\_info}.

In our example, we ignore the {\tt new\_x} flag and calculate the
values for the gradient of the objective.

\begin{footnotesize}
\begin{verbatim}
bool HS071_NLP::eval_grad_f(Index n, const Number* x, bool new_x, Number* grad_f)
{
  assert(n == 4);

  grad_f[0] = x[0] * x[3] + x[3] * (x[0] + x[1] + x[2]);
  grad_f[1] = x[0] * x[3];
  grad_f[2] = x[0] * x[3] + 1;
  grad_f[3] = x[0] * (x[0] + x[1] + x[2]);

  return true;
}
\end{verbatim}
\end{footnotesize}

\paragraph{Method {\texttt{eval\_g}}} with prototype
\begin{verbatim}
virtual bool eval_g(Index n, const Number* x, 
                    bool new_x, Index m, Number* g)
\end{verbatim}
Return the value of the constraint function at the point $x$.
\begin{itemize}
\item {\tt n}: (in), the number of variables in the problem (dimension of $x$). 
\item {\tt x}: (in), the values for the primal variables, $x$, at
  which the constraint functions,
  $g(x)$, are to be evaluated.
\item {\tt new\_x}: (in), false if any evaluation method was previously called 
        with the same values in {\tt x}, true otherwise.
\item {\tt m}: (in), the number of constraints in the problem (dimension of $g(x)$).
\item {\tt g}: (out) the array of constraint function values, $g(x)$.
\end{itemize}

The values returned in {\tt g} should be only the $g(x)$ values, 
do not add or subtract the bound values $g^L$ or $g^U$.

The boolean variable {\tt new\_x} will be false if the last call to
any of the evaluation methods ({\tt eval\_*}) used the same $x$
values. This can be helpful when users have efficient implementations
that calculate multiple outputs at once. \Ipopt internally caches
results from the {\tt TNLP} and generally, this flag can be ignored.

The variables {\tt n} and {\tt m} are passed in for your convenience.
These variables will have the same values you specified in {\tt
  get\_nlp\_info}.

In our example, we ignore the {\tt new\_x} flag and calculate the
values of constraint functions.

\begin{footnotesize}
\begin{verbatim}
bool HS071_NLP::eval_g(Index n, const Number* x, bool new_x, Index m, Number* g)
{
  assert(n == 4);
  assert(m == 2);

  g[0] = x[0] * x[1] * x[2] * x[3];
  g[1] = x[0]*x[0] + x[1]*x[1] + x[2]*x[2] + x[3]*x[3];

  return true;
} 
\end{verbatim}
\end{footnotesize}

\paragraph{Method {\texttt{eval\_jac\_g}}} with prototype
\begin{verbatim}
virtual bool eval_jac_g(Index n, const Number* x, bool new_x,
                        Index m, Index nele_jac, Index* iRow, 
                        Index *jCol, Number* values)
\end{verbatim}
Return either the sparsity structure of the Jacobian of the
constraints, or the values for the Jacobian of the constraints at the
point $x$.
\begin{itemize}
\item {\tt n}: (in), the number of variables in the problem (dimension of $x$). 
\item {\tt x}: (in), the values for the primal variables, $x$, at which
  the constraint Jacobian, $\nabla g(x)^T$, is to be evaluated.
\item {\tt new\_x}: (in), false if any evaluation method was previously called 
        with the same values in {\tt x}, true otherwise.
\item {\tt m}: (in), the number of constraints in the problem (dimension of $g(x)$).
\item {\tt n\_ele\_jac}: (in), the number of nonzero elements in the 
        Jacobian (dimension of {\tt iRow}, {\tt jCol}, and {\tt values}).
\item {\tt iRow}: (out), the row indices of entries in the Jacobian of the constraints.
\item {\tt jCol}: (out), the column indices of entries in the Jacobian of the constraints.
\item {\tt values}: (out), the values of the entries in the Jacobian of the constraints.
\end{itemize}

The Jacobian is the matrix of derivatives where the derivative of
constraint $g^{(i)}$ with respect to variable $x^{(j)}$ is placed in
row $i$ and column $j$. See Appendix \ref{app.triplet} for a
discussion of the sparse matrix format used in this method.

If the {\tt iRow} and {\tt jCol} arguments are not {\tt NULL}, then
\Ipopt wants you to fill in the sparsity structure of the Jacobian
(the row and column indices only). At this time, the {\tt x} argument
and the {\tt values} argument will be {\tt NULL}.

If the {\tt x} argument and the {\tt values} argument are not {\tt
  NULL}, then \Ipopt wants you to fill in the values of the Jacobian
as calculated from the array {\tt x} (using the same order as you used
when specifying the sparsity structure). At this time, the {\tt iRow}
and {\tt jCol} arguments will be {\tt NULL};

The boolean variable {\tt new\_x} will be false if the last call to
any of the evaluation methods ({\tt eval\_*}) used the same $x$
values. This can be helpful when users have efficient implementations
that calculate multiple outputs at once. \Ipopt internally caches
results from the {\tt TNLP} and generally, this flag can be ignored.

The variables {\tt n}, {\tt m}, and {\tt nele\_jac} are passed in for
your convenience. These arguments will have the same values you
specified in {\tt get\_nlp\_info}.

In our example, the Jacobian is actually dense, but we still
specify it using the sparse format.

\begin{footnotesize}
\begin{verbatim}
bool HS071_NLP::eval_jac_g(Index n, const Number* x, bool new_x,
                           Index m, Index nele_jac, Index* iRow, Index *jCol,
                           Number* values)
{
  if (values == NULL) {
    // return the structure of the Jacobian

    // this particular Jacobian is dense
    iRow[0] = 0; jCol[0] = 0;
    iRow[1] = 0; jCol[1] = 1;
    iRow[2] = 0; jCol[2] = 2;
    iRow[3] = 0; jCol[3] = 3;
    iRow[4] = 1; jCol[4] = 0;
    iRow[5] = 1; jCol[5] = 1;
    iRow[6] = 1; jCol[6] = 2;
    iRow[7] = 1; jCol[7] = 3;
  }
  else {
    // return the values of the Jacobian of the constraints
    
    values[0] = x[1]*x[2]*x[3]; // 0,0
    values[1] = x[0]*x[2]*x[3]; // 0,1
    values[2] = x[0]*x[1]*x[3]; // 0,2
    values[3] = x[0]*x[1]*x[2]; // 0,3

    values[4] = 2*x[0]; // 1,0
    values[5] = 2*x[1]; // 1,1
    values[6] = 2*x[2]; // 1,2
    values[7] = 2*x[3]; // 1,3
  }

  return true;
}
\end{verbatim}
\end{footnotesize}

\paragraph{Method {\texttt{eval\_h}}} with prototype
\begin{verbatim}
virtual bool eval_h(Index n, const Number* x, bool new_x,
                    Number obj_factor, Index m, const Number* lambda,
                    bool new_lambda, Index nele_hess, Index* iRow,
                    Index* jCol, Number* values)
\end{verbatim}
Return either the sparsity structure of the Hessian of the Lagrangian, or the values of the 
Hessian of the Lagrangian (\ref{eq:IpoptLAG}) for the given values for $x$,
$\sigma_f$, and $\lambda$.
\begin{itemize}
\item {\tt n}: (in), the number of variables in the problem (dimension
  of $x$).
\item {\tt x}: (in), the values for the primal variables, $x$, at which
  the Hessian is to be evaluated.
\item {\tt new\_x}: (in), false if any evaluation method was previously called 
        with the same values in {\tt x}, true otherwise.
\item {\tt obj\_factor}: (in), factor in front of the objective term
  in the Hessian, $\sigma_f$.
\item {\tt m}: (in), the number of constraints in the problem (dimension of $g(x)$).
\item {\tt lambda}: (in), the values for the constraint multipliers,
  $\lambda$, at which the Hessian is to be evaluated.
\item {\tt new\_lambda}: (in), false if any evaluation method was
  previously called with the same values in {\tt lambda}, true
  otherwise.
\item {\tt nele\_hess}: (in), the number of nonzero elements in the
  Hessian (dimension of {\tt iRow}, {\tt jCol}, and {\tt values}).
\item {\tt iRow}: (out), the row indices of entries in the Hessian.
\item {\tt jCol}: (out), the column indices of entries in the Hessian.
\item {\tt values}: (out), the values of the entries in the Hessian.
\end{itemize}

The Hessian matrix that \Ipopt uses is defined in
(\ref{eq:IpoptLAG}).  See Appendix \ref{app.triplet} for a
discussion of the sparse symmetric matrix format used in this method.

If the {\tt iRow} and {\tt jCol} arguments are not {\tt NULL}, then
\Ipopt wants you to fill in the sparsity structure of the Hessian
(the row and column indices for the lower or upper triangular part
only). In this case, the {\tt x}, {\tt lambda}, and {\tt values}
arrays will be {\tt NULL}.

If the {\tt x}, {\tt lambda}, and {\tt values} arrays are not {\tt
  NULL}, then \Ipopt wants you to fill in the values of the Hessian
as calculated using {\tt x} and {\tt lambda} (using the same order as
you used when specifying the sparsity structure). In this case, the
{\tt iRow} and {\tt jCol} arguments will be {\tt NULL}.

The boolean variables {\tt new\_x} and {\tt new\_lambda} will both be
false if the last call to any of the evaluation methods ({\tt
  eval\_*}) used the same values. This can be helpful when users have
efficient implementations that calculate multiple outputs at once.
\Ipopt internally caches results from the {\tt TNLP} and generally,
this flag can be ignored.

The variables {\tt n}, {\tt m}, and {\tt nele\_hess} are passed in for
your convenience. These arguments will have the same values you
specified in {\tt get\_nlp\_info}.

In our example, the Hessian is dense, but we still specify it using the
sparse matrix format. Because the Hessian is symmetric, we only need to 
specify the lower left corner.

\begin{footnotesize}
\begin{verbatim}
bool HS071_NLP::eval_h(Index n, const Number* x, bool new_x,
                       Number obj_factor, Index m, const Number* lambda,
                       bool new_lambda, Index nele_hess, Index* iRow,
                       Index* jCol, Number* values)
{
  if (values == NULL) {
    // return the structure. This is a symmetric matrix, fill the lower left
    // triangle only.

    // the Hessian for this problem is actually dense
    Index idx=0;
    for (Index row = 0; row < 4; row++) {
      for (Index col = 0; col <= row; col++) {
        iRow[idx] = row; 
        jCol[idx] = col;
        idx++;
      }
    }
    
    assert(idx == nele_hess);
  }
  else {
    // return the values. This is a symmetric matrix, fill the lower left
    // triangle only

    // fill the objective portion
    values[0] = obj_factor * (2*x[3]); // 0,0

    values[1] = obj_factor * (x[3]);   // 1,0
    values[2] = 0;                     // 1,1

    values[3] = obj_factor * (x[3]);   // 2,0
    values[4] = 0;                     // 2,1
    values[5] = 0;                     // 2,2

    values[6] = obj_factor * (2*x[0] + x[1] + x[2]); // 3,0
    values[7] = obj_factor * (x[0]);                 // 3,1
    values[8] = obj_factor * (x[0]);                 // 3,2
    values[9] = 0;                                   // 3,3


    // add the portion for the first constraint
    values[1] += lambda[0] * (x[2] * x[3]); // 1,0
    
    values[3] += lambda[0] * (x[1] * x[3]); // 2,0
    values[4] += lambda[0] * (x[0] * x[3]); // 2,1

    values[6] += lambda[0] * (x[1] * x[2]); // 3,0
    values[7] += lambda[0] * (x[0] * x[2]); // 3,1
    values[8] += lambda[0] * (x[0] * x[1]); // 3,2

    // add the portion for the second constraint
    values[0] += lambda[1] * 2; // 0,0

    values[2] += lambda[1] * 2; // 1,1

    values[5] += lambda[1] * 2; // 2,2

    values[9] += lambda[1] * 2; // 3,3
  }

  return true;
}
\end{verbatim}
\end{footnotesize}

\paragraph{Method \texttt{finalize\_solution}} with prototype

\begin{verbatim}
virtual void finalize_solution(SolverReturn status, Index n,
                               const Number* x, const Number* z_L,
                               const Number* z_U, Index m, const Number* g,
                               const Number* lambda, Number obj_value,
                               const IpoptData* ip_data,
                               IpoptCalculatedQuantities* ip_cq)
\end{verbatim}
This is the only method that is not mentioned in Section
\ref{sec.required_info}. This method is called by \Ipopt after the
algorithm has finished (successfully or even with most errors).
\begin{itemize}
\item {\tt status}: (in), gives the status of the algorithm as
  specified in {\tt IpAlgTypes.hpp},
  \begin{itemize}
  \item {\tt SUCCESS}: Algorithm terminated successfully at a locally
    optimal point, satisfying the convergence tolerances (can be
    specified by options).
  \item {\tt MAXITER\_EXCEEDED}: Maximum number of iterations exceeded
    (can be specified by an option).
  \item {\tt CPUTIME\_EXCEEDED}: Maximum number of CPU seconds exceeded
    (can be specified by an option).
  \item {\tt STOP\_AT\_TINY\_STEP}: Algorithm proceeds with very
    little progress.
  \item {\tt STOP\_AT\_ACCEPTABLE\_POINT}: Algorithm stopped at a
    point that was converged, not to ``desired'' tolerances, but to
    ``acceptable'' tolerances (see the {\tt acceptable-...} options).
  \item {\tt LOCAL\_INFEASIBILITY}: Algorithm converged to a point of
    local infeasibility. Problem may be infeasible.
  \item {\tt USER\_REQUESTED\_STOP}: The user call-back function {\tt
      intermediate\_callback} (see Section~\ref{sec:add_meth})
    returned {\tt false}, i.e., the user code requested a premature
    termination of the optimization.
  \item {\tt DIVERGING\_ITERATES}: It seems that the iterates diverge.
  \item {\tt RESTORATION\_FAILURE}: Restoration phase failed,
    algorithm doesn't know how to proceed.
  \item {\tt ERROR\_IN\_STEP\_COMPUTATION}: An unrecoverable error
    occurred while \Ipopt tried to compute the search direction.
  \item {\tt INVALID\_NUMBER\_DETECTED}:  Algorithm received an
    invalid number (such as {\tt NaN} or {\tt Inf}) from the NLP; see
    also option \htmlref{\tt check\_derivatives\_for\_naninf}{opt:check_derivatives_for_naninf}.
  \item {\tt
      INTERNAL\_ERROR}: An unknown internal error occurred.  Please
    contact the \Ipopt authors through the mailing list.
  \end{itemize}
\item {\tt n}: (in), the number of variables in the problem (dimension
  of $x$).
\item {\tt x}: (in), the final values for the primal variables, $x_*$.
\item {\tt z\_L}: (in), the final values for the lower bound
  multipliers, $z^L_*$.
\item {\tt z\_U}: (in), the final values for the upper bound
  multipliers, $z^U_*$.
\item {\tt m}: (in), the number of constraints in the problem
  (dimension of $g(x)$).
\item {\tt g}: (in), the final value of the constraint function
  values, $g(x_*)$.
\item {\tt lambda}: (in), the final values of the constraint
  multipliers, $\lambda_*$.
\item {\tt obj\_value}: (in), the final value of the objective,
  $f(x_*)$.
\item {\tt ip\_data} and {\tt ip\_cq} are provided for expert users.
\end{itemize}

This method gives you the return status of the algorithm
(SolverReturn), and the values of the variables, 
the objective and constraint function values when the algorithm exited.

In our example, we will print the values of some of the variables to 
the screen.

\begin{footnotesize}
\begin{verbatim}
void HS071_NLP::finalize_solution(SolverReturn status,
                                  Index n, const Number* x, const Number* z_L,
                                  const Number* z_U, Index m, const Number* g,
                                  const Number* lambda, Number obj_value,
                                  const IpoptData* ip_data, IpoptCalculatedQuantities* ip_cq)                                  
{
  // here is where we would store the solution to variables, or write to a file, etc
  // so we could use the solution. 

  // For this example, we write the solution to the console
  printf("\n\nSolution of the primal variables, x\n");
  for (Index i=0; i<n; i++) {
    printf("x[%d] = %e\n", i, x[i]); 
  }

  printf("\n\nSolution of the bound multipliers, z_L and z_U\n");
  for (Index i=0; i<n; i++) {
    printf("z_L[%d] = %e\n", i, z_L[i]); 
  }
  for (Index i=0; i<n; i++) {
    printf("z_U[%d] = %e\n", i, z_U[i]); 
  }

  printf("\n\nObjective value\n");
  printf("f(x*) = %e\n", obj_value); 
}
\end{verbatim}
\end{footnotesize}

This is all that is required for our {\tt HS071\_NLP} class and 
the coding of the problem representation.
 
\subsubsection{Coding the Executable (\texttt{main})}
Now that we have a problem representation, the {\tt HS071\_NLP} class,
we need to code the main function that will call \Ipopt and ask \Ipopt
to find a solution.

Here, we must create an instance of our problem ({\tt HS071\_NLP}),
create an instance of the \Ipopt solver (\texttt{IpoptApplication}),
initialize it, and ask the solver to find a solution. We always use
the \texttt{SmartPtr} template class instead of raw C++ pointers when
creating and passing \Ipopt objects. To find out more information
about smart pointers and the {\tt SmartPtr} implementation used in
\Ipopt, see Appendix \ref{app.smart_ptr}.

Create the file {\tt MyExample.cpp} in the MyExample directory.
Include the header files {\tt HS071\_NLP.hpp} and {\tt IpIpoptApplication.hpp}, tell
the compiler to use the {\tt Ipopt} namespace, and implement the {\tt
  main} function.

\begin{footnotesize}
\begin{verbatim}
#include "IpIpoptApplication.hpp"
#include "hs071_nlp.hpp"

using namespace Ipopt;

int main(int argv, char* argc[])
{
  // Create a new instance of your nlp 
  //  (use a SmartPtr, not raw)
  SmartPtr<TNLP> mynlp = new HS071_NLP();

  // Create a new instance of IpoptApplication
  //  (use a SmartPtr, not raw)
  // We are using the factory, since this allows us to compile this
  // example with an Ipopt Windows DLL
  SmartPtr<IpoptApplication> app = IpoptApplicationFactory();

  // Change some options
  // Note: The following choices are only examples, they might not be
  //       suitable for your optimization problem.
  app->Options()->SetNumericValue("tol", 1e-9);
  app->Options()->SetStringValue("mu_strategy", "adaptive");
  app->Options()->SetStringValue("output_file", "ipopt.out");

  // Intialize the IpoptApplication and process the options
  ApplicationReturnStatus status;
  status = app->Initialize();
  if (status != Solve_Succeeded) {
    printf("\n\n*** Error during initialization!\n");
    return (int) status;
  }

  // Ask Ipopt to solve the problem
  status = app->OptimizeTNLP(mynlp);

  if (status == Solve_Succeeded) {
    printf("\n\n*** The problem solved!\n");
  }
  else {
    printf("\n\n*** The problem FAILED!\n");
  }

  // As the SmartPtrs go out of scope, the reference count
  // will be decremented and the objects will automatically 
  // be deleted.

  return (int) status;
}
\end{verbatim} 
\end{footnotesize}

The first line of code in {\tt main} creates an instance of {\tt
  HS071\_NLP}. We then create an instance of the \Ipopt solver, {\tt
  IpoptApplication}. You could use \texttt{new} to create a new
application object, but if you want to make sure that your code would
also work with a Windows DLL, you need to use the factory, as done in
the example above. The call to {\tt app->Initialize(...)} will
initialize that object and process the options (particularly the output
related options). The call to {\tt app->OptimizeTNLP(...)}  will
run \Ipopt and try to solve the problem. By default, \Ipopt will
write its progress to the console, and return the {\tt
  SolverReturn} status.

\subsubsection{Compiling and Testing the Example}
Our next task is to compile and test the code. If you are familiar
with the compiler and linker used on your system, you can build the
code, telling the linker about the 
%\Ipopt library {\tt libipopt.so} (or {\tt libipopt.a}), as well as other
 necessary libraries, as listed
in the {\tt ipopt\_addlibs\_cpp.txt} file.
If you are using the autotools based build system, then a sample makefile 
created by configure already exists.  Copy {\tt
  Ipopt/examples/hs071\_cpp/Makefile} into your {\tt MyExample}
directory.  This makefile was created for the {\tt hs071\_cpp} code,
but it can be easily modified for your example problem. Edit the file,
making the following changes,

\begin{itemize}
\item change the {\tt EXE} variable \\
{\tt EXE = my\_example}
\item change the {\tt OBJS} variable \\
{\tt OBJS = HS071\_NLP.o MyExample.o}
\end{itemize}
and the problem should compile easily with, \\
{\tt \$ make} \\
Now run the executable,\\ 
{\tt \$ ./my\_example} \\
and you should see output resembling the following,

\begin{footnotesize}
\begin{verbatim}
******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
 Ipopt is released as open source code under the Eclipse Public License (EPL).
         For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************

Number of nonzeros in equality constraint Jacobian...:        4
Number of nonzeros in inequality constraint Jacobian.:        4
Number of nonzeros in Lagrangian Hessian.............:       10

Total number of variables............................:        4
                     variables with only lower bounds:        0
                variables with lower and upper bounds:        4
                     variables with only upper bounds:        0
Total number of equality constraints.................:        1
Total number of inequality constraints...............:        1
        inequality constraints with only lower bounds:        1
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.6109693e+01 1.12e+01 5.28e-01   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  1.7410406e+01 8.38e-01 2.25e+01  -0.3 7.97e-01    -  3.19e-01 1.00e+00f  1
   2  1.8001613e+01 1.06e-02 4.96e+00  -0.3 5.60e-02   2.0 9.97e-01 1.00e+00h  1
   3  1.7199482e+01 9.04e-02 4.24e-01  -1.0 9.91e-01    -  9.98e-01 1.00e+00f  1
   4  1.6940955e+01 2.09e-01 4.58e-02  -1.4 2.88e-01    -  9.66e-01 1.00e+00h  1
   5  1.7003411e+01 2.29e-02 8.42e-03  -2.9 7.03e-02    -  9.68e-01 1.00e+00h  1
   6  1.7013974e+01 2.59e-04 8.65e-05  -4.5 6.22e-03    -  1.00e+00 1.00e+00h  1
   7  1.7014017e+01 2.26e-07 5.71e-08  -8.0 1.43e-04    -  1.00e-00 1.00e+00h  1
   8  1.7014017e+01 4.62e-14 9.09e-14  -8.0 6.95e-08    -  1.00e+00 1.00e+00h  1

Number of Iterations....: 8

Number of objective function evaluations             = 9
Number of objective gradient evaluations             = 9
Number of equality constraint evaluations            = 9
Number of inequality constraint evaluations          = 9
Number of equality constraint Jacobian evaluations   = 9
Number of inequality constraint Jacobian evaluations = 9
Number of Lagrangian Hessian evaluations             = 8
Total CPU secs in IPOPT (w/o function evaluations)   =      0.220
Total CPU secs in NLP function evaluations           =      0.000

EXIT: Optimal Solution Found.


Solution of the primal variables, x
x[0] = 1.000000e+00
x[1] = 4.743000e+00
x[2] = 3.821150e+00
x[3] = 1.379408e+00


Solution of the bound multipliers, z_L and z_U
z_L[0] = 1.087871e+00
z_L[1] = 2.428776e-09
z_L[2] = 3.222413e-09
z_L[3] = 2.396076e-08
z_U[0] = 2.272727e-09
z_U[1] = 3.537314e-08
z_U[2] = 7.711676e-09
z_U[3] = 2.510890e-09


Objective value
f(x*) = 1.701402e+01


*** The problem solved!
\end{verbatim}
\end{footnotesize}

This completes the basic C++ tutorial, but see Section
\ref{sec:output} which explains the standard console output of \Ipopt
and Section \ref{sec:options} for information about the use of options
to customize the behavior of \Ipopt.

The {\tt Ipopt/examples/ScalableProblems} directory contains other NLP
problems coded in C++.

\subsubsection{Additional methods in {\tt TNLP}}\label{sec:add_meth}

The following methods are available to additional features that are
not explained in the example.  Default implementations for those
methods are provided, so that a user can safely ignore them, unless
she wants to make use of those features.  From these features, only the 
intermediate callback is already available in the C and Fortran interfaces.

\paragraph{Method \texttt{intermediate\_callback}} with prototype
\begin{verbatim}
virtual bool intermediate_callback(AlgorithmMode mode,
                                   Index iter, Number obj_value,
                                   Number inf_pr, Number inf_du,
                                   Number mu, Number d_norm,
                                   Number regularization_size,
                                   Number alpha_du, Number alpha_pr,
                                   Index ls_trials,
                                   const IpoptData* ip_data,
                                   IpoptCalculatedQuantities* ip_cq)
\end{verbatim}
It is not required to implement (overload) this method.  This method
is called once per iteration (during the convergence check), and can
be used to obtain information about the optimization status while
\Ipopt solves the problem, and also to request a premature
termination.

The information provided by the entities in the argument list
correspond to what \Ipopt prints in the iteration summary (see also
Section~\ref{sec:output}).  Further information can be obtained from
the {\tt ip\_data} and {\tt ip\_cq} objects (in the C++ interface and for experts only :)).

You you let this method return {\tt false}, \Ipopt will terminate
with the {\tt User\_Requested\_Stop} status.  If you do not implement
this method (as we do in this example), the default implementation
always returns {\tt true}.

A frequently asked question is how to access the values of the primal and dual variables in this callback. The values are stored in the {\tt ip\_cq} object for the \emph{internal representation} of the problem.
To access the values in a form that corresponds to those used in the evaluation routines, the user has to request \Ipopt's {\tt TNLPAdapter} object to ``resort'' the data vectors and to fill in information about possibly filtered out fixed variables.
The {\tt TNLPAdapter} can be accessed as follows.
First, add the following includes to your {\tt TNLP} implementation:
\begin{verbatim}
  #include "IpIpoptCalculatedQuantities.hpp"
  #include "IpIpoptData.hpp"
  #include "IpTNLPAdapter.hpp"
  #include "IpOrigIpoptNLP.hpp"
\end{verbatim}
Next, add the following code to your implementation of the {\tt intermediate\_callback}:
\begin{verbatim}
  Ipopt::TNLPAdapter* tnlp_adapter = NULL;
  if( ip_cq != NULL )
  {
    Ipopt::OrigIpoptNLP* orignlp;
    orignlp = dynamic_cast<OrigIpoptNLP*>(GetRawPtr(ip_cq->GetIpoptNLP()));
    if( orignlp != NULL )
      tnlp_adapter = dynamic_cast<TNLPAdapter*>(GetRawPtr(orignlp->nlp()));
  }
\end{verbatim}
Note, that retrieving the {\tt TNLPAdapter} will fail (i.e., {\tt orignlp} will be {\tt NULL}) if \Ipopt is currently in restoration mode.
If, however, {\tt tnlp\_adapter} is not {\tt NULL}, then it can be used to obtain primal variable values $x$ and the dual values for the constraints \ref{eq:constraints} and the variable bounds \ref{eq:bounds} as follows.
\begin{verbatim}
  double* primals = new double[n];
  double* dualeqs = new double[m];
  double* duallbs = new double[n];
  double* dualubs = new double[n];
  tnlp_adapter->ResortX(*ip_data->curr()->x(), primals);
  tnlp_adapter->ResortG(*ip_data->curr()->y_c(), *ip_data->curr()->y_d(), dualeqs);
  tnlp_adapter->ResortBnds(*ip_data->curr()->z_L(), duallbs,
                           *ip_data->curr()->z_U(), dualubs);
\end{verbatim}
Additionally, information about scaled violation of constraint 
\ref{eq:constraints} and violation of complementarity constraints can be 
obtained via
\begin{verbatim}
  tnlp_adapter->ResortG(*ip_data->curr_c(), *ip_data->curr_d_minus_s(), ...)
  tnlp_adapter->ResortBnds(*ip_data->curr_compl_x_L(), ...,
                           *ip_data->curr_compl_x_U(), ...)
  tnlp_adapter->ResortG(*ip_data->curr_compl_s_L(), ...)
  tnlp_adapter->ResortG(*ip_data->curr_compl_s_U(), ...)
\end{verbatim}

\paragraph{Method \texttt{get\_scaling\_parameters}} with prototype
\begin{verbatim}
virtual bool get_scaling_parameters(Number& obj_scaling,
                                    bool& use_x_scaling, Index n,
                                    Number* x_scaling,
                                    bool& use_g_scaling, Index m,
                                    Number* g_scaling)
\end{verbatim}

This method is called if the {\tt nlp\_scaling\_method} is chosen as
{\tt user-scaling}.  The user has to provide scaling factors for
the objective function as well as for the optimization variables
and/or constraints.  The return value should be true, unless an error
occurred, and the program is to be aborted.

The value returned in {\tt obj\_scaling} determines, how \Ipopt
should internally scale the objective function.  For example, if this
number is chosen to be 10, then \Ipopt solves internally an
optimization problem that has 10 times the value of the original
objective function provided by the {\tt TNLP}.  In particular, if this
value is negative, then \Ipopt will maximize the objective function
instead of minimizing it.

The scaling factors for the variables can be returned in {\tt
  x\_scaling}, which has the same length as {\tt x} in the other {\tt
  TNLP} methods, and the factors are ordered like {\tt x}.  You need
to set {\tt use\_x\_scaling} to {\tt true}, if you want \Ipopt so scale
the variables.  If it is {\tt false}, no internal scaling of the
variables is done.  Similarly, the scaling factors for the constraints
can be returned in {\tt g\_scaling}, and this scaling is activated by
setting {\tt use\_g\_scaling} to {\tt true}.

As a guideline, we suggest to scale the optimization problem (either
directly in the original formulation, or after using scaling factors)
so that all sensitivities, i.e., all non-zero first partial
derivatives, are typically of the order $0.1-10$.

\paragraph{Method \texttt{get\_number\_of\_nonlinear\_variables}} with prototype
\begin{verbatim}
virtual Index get_number_of_nonlinear_variables()
\end{verbatim}

This method is only important if the limited-memory quasi-Newton
options is used, see Section~\ref{sec:quasiNewton}.  It is used
to return the number of variables that appear nonlinearly in the
objective function or in at least one constraint function.  If a
negative number is returned, \Ipopt assumes that all variables are
nonlinear.

If the user doesn't overload this method in her implementation of the
class derived from {\tt TNLP}, the default implementation returns -1,
i.e., all variables are assumed to be nonlinear.

\paragraph{Method \texttt{get\_list\_of\_nonlinear\_variables}} with prototype
\begin{verbatim}
virtual bool get_list_of_nonlinear_variables(Index num_nonlin_vars,
                                             Index* pos_nonlin_vars)
\end{verbatim}

This method is called by \Ipopt only if the limited-memory
quasi-Newton options is used and if the {\tt
  get\_number\_of\_nonlinear\_variables} method returns a positive
number; this number is then identical with {\tt num\_nonlin\_vars} and
the length of the array {\tt pos\_nonlin\_vars}.  In this call, you
need to list the indices of all nonlinear variables in {\tt
  pos\_nonlin\_vars}, where the numbering starts with 0 order 1,
depending on the numbering style determined in {\tt get\_nlp\_info}.

\paragraph{Method \texttt{get\_variables\_linearity}} with prototype
\begin{verbatim}
virtual bool get_variables_linearity(Index n,
                                     LinearityType* var_types)
\end{verbatim}

This method is never called by \Ipopt, but is used by \textsc{Bonmin} to get information about which variables occur only in linear terms.
\Ipopt passes a {\tt var\_types} array of size {\tt n}, which the user should fill with the appropriate linearity type of the variables ({\tt TNLP::LINEAR} or {\tt TNLP::NON\_LINEAR}).

If the user doesn't overload this method in her implementation of the class derived from {\tt TNLP}, the default implementation returns {\tt false}.

\paragraph{Method \texttt{get\_constraints\_linearity}} with prototype
\begin{verbatim}
virtual bool get_constraints_linearity(Index m,
                                       LinearityType* const_types)
\end{verbatim}

This method is never called by \Ipopt, but is used by \textsc{Bonmin} to get information about which constraints are linear.
\Ipopt passes a {\tt const\_types} array of size {\tt m}, which the user should fill with the appropriate linearity type of the constraints ({\tt TNLP::LINEAR} or {\tt TNLP::NON\_LINEAR}).

If the user doesn't overload this method in her implementation of the class derived from {\tt TNLP}, the default implementation returns {\tt false}.

\paragraph{Method \texttt{get\_var\_con\_metadata}} with prototype
\begin{verbatim}
virtual bool get_var_con_metadata(Index n,
                                  StringMetaDataMapType& var_string_md,
                                  IntegerMetaDataMapType& var_integer_md,
                                  NumericMetaDataMapType& var_numeric_md,
                                  Index m,
                                  StringMetaDataMapType& con_string_md,
                                  IntegerMetaDataMapType& con_integer_md,
                                  NumericMetaDataMapType& con_numeric_md)
\end{verbatim}

This method is used to pass meta data about variables or constraints to 
\Ipopt. The data can be either of integer, numeric, or string type.
\Ipopt passes this data on to its internal problem representation.
The meta data type is a {\tt std::map} with {\tt std::string} as key type and 
a {\tt std::vector} as value type.
So far, \Ipopt itself makes only use of string meta data under the key {\tt 
idx\_names}. With this key, variable and constraint names can be passed to 
\Ipopt, which are shown when printing internal vector or matrix data structures 
if \Ipopt is run with a high value for the \htmlref{\tt print\_level}{opt:print_level} 
option. This allows a user to identify the original variables and constraints
corresponding to \Ipopt's internal problem representation.

If the user doesn't overload this method in her implementation of the class 
derived from {\tt TNLP}, the default implementation does not set any meta data 
and returns {\tt false}.

\paragraph{Method \texttt{finalize\_metadata}} with prototype
\begin{verbatim}
virtual void finalize_metadata(Index n,
                               const StringMetaDataMapType& var_string_md,
                               const IntegerMetaDataMapType& var_integer_md,
                               const NumericMetaDataMapType& var_numeric_md,
                               Index m,
                               const StringMetaDataMapType& con_string_md,
                               const IntegerMetaDataMapType& con_integer_md,
                               const NumericMetaDataMapType& con_numeric_md)
\end{verbatim}

This method is called just before {\tt finalize\_solution} and is used to
return any meta data collected during the algorithms run, including the meta 
data provided by the user with the {\tt get\_var\_con\_metadata} method.

If the user doesn't overload this method in her implementation of the class 
derived from {\tt TNLP}, the default implementation does nothing.

\paragraph{Method \texttt{get\_warm\_start\_iterate}} with prototype
\begin{verbatim}
virtual bool get_warm_start_iterate(IteratesVector& warm_start_iterate)
\end{verbatim}

Overload this method to provide an \Ipopt iterate which is already in the form 
\Ipopt requires it internally for a warm starts.
This method is only for expert users.

If the user doesn't overload this method in her implementation of the class 
derived from {\tt TNLP}, the default implementation does not provide a warm 
start iterate and returns {\tt false}.


\subsection{The C Interface} \label{sec.cinterface}
The C interface for \Ipopt is declared in the header file {\tt
  IpStdCInterface.h}, which is found in\\
\texttt{\$IPOPTDIR/include/coin} (or in
\texttt{\$PREFIX/include/coin} if the switch
\verb|--prefix=$PREFIX| was used for {\tt configure}); while  %$
reading this section, it will be helpful to have a look at this file.

In order to solve an optimization problem with the C interface, one
has to create an {\tt IpoptProblem}\footnote{{\tt IpoptProblem} is a
  pointer to a C structure; you should not access this structure
  directly, only through the functions provided in the C interface.}
with the function {\tt CreateIpoptProblem}, which later has to be
passed to the {\tt IpoptSolve} function.

The {\tt IpoptProblem} created by {\tt CreateIpoptProblem} contains
the problem dimensions, the variable and constraint bounds, and the
function pointers for callbacks that will be used to evaluate the NLP
problem functions and their derivatives (see also the discussion of
the C++ methods {\tt get\_nlp\_info} and {\tt get\_bounds\_info} in
Section~\ref{sec.cpp_problem} for information about the arguments of
{\tt CreateIpoptProblem}).

The prototypes for the callback functions, {\tt Eval\_F\_CB}, {\tt
  Eval\_Grad\_F\_CB}, etc., are defined in the header file {\tt
  IpStdCInterface.h}.  Their arguments correspond one-to-one to the
arguments for the C++ methods discussed in
Section~\ref{sec.cpp_problem}; for example, for the meaning of $\tt
n$, $\tt x$, $\tt new\_x$, $\tt obj\_value$ in the declaration of {\tt
  Eval\_F\_CB} see the discussion of ``{\tt eval\_f}''.  The callback
functions should return {\tt TRUE}, unless there was a problem doing
the requested function/derivative evaluation at the given point {\tt
  x} (then it should return {\tt FALSE}).

Note the additional argument of type {\tt UserDataPtr} in the callback
functions.  This pointer argument is available for you to communicate
information between the main program that calls {\tt IpoptSolve} and
any of the callback functions.  This pointer is simply passed
unmodified by \Ipopt among those functions.  For example, you can
use this to pass constants that define the optimization problem and
are computed before the optimization in the main C program to the
callback functions.

After an {\tt IpoptProblem} has been created, you can set algorithmic
options for \Ipopt (see Section~\ref{sec:options}) using the {\tt
  AddIpopt...Option} functions.  Finally, the \Ipopt algorithm is
called with {\tt IpoptSolve}, giving \Ipopt the {\tt IpoptProblem},
the starting point, and arrays to store the solution values (primal
and dual variables), if desired.  Finally, after everything is done,
you should call {\tt FreeIpoptProblem} to release internal memory that
is still allocated inside \Ipopt.

In the remainder of this section we discuss how the example problem
(\ref{eq:ex_obj})--(\ref{eq:ex_bounds}) can be solved using the C
interface.  A completed version of this example can be found in {\tt
  Ipopt/examples/hs071\_c}.

% We first create the necessary callback
% functions for evaluating the NLP. As just discussed, the \Ipopt C
% interface required callbacks to evaluate the objective value,
% constraints, gradient of the objective, Jacobian of the constraints,
% and the Hessian of the Lagrangian.  These callbacks are implemented
% using function pointers.  Have a look at the C++ implementation for
% {\tt eval\_f}, {\tt eval\_g}, {\tt eval\_grad\_f}, {\tt eval\_jac\_g},
% and {\tt eval\_h} in Section \ref{sec.cpp_problem}. The C
% implementations have somewhat different prototypes, but are
% implemented almost identically to the C++ code.

\vspace{\baselineskip}

In order to implement the example problem on your own, create a new
directory {\tt MyCExample} and create a new file, {\tt
  hs071\_c.c}.  Here, include the interface header file {\tt
  IpStdCInterface.h}, along with other necessary header files, such as
{\tt stdlib.h} and {\tt assert.h}.  Add the prototypes and
implementations for the five callback functions.  Have a look at the
C++ implementation for {\tt eval\_f}, {\tt eval\_g}, {\tt
  eval\_grad\_f}, {\tt eval\_jac\_g}, and {\tt eval\_h} in Section
\ref{sec.cpp_problem}. The C implementations have somewhat different
prototypes, but are implemented almost identically to the C++ code.
See the completed example in {\tt Ipopt/examples/hs071\_c/hs071\_c.c} if you
are not sure how to do this.

We now need to implement the {\tt main} function, create the {\tt
  IpoptProblem}, set options, and call {\tt IpoptSolve}. The {\tt
  CreateIpoptProblem} function requires the problem dimensions, the
variable and constraint bounds, and the function pointers to the
callback routines. The {\tt IpoptSolve} function requires the {\tt
  IpoptProblem}, the starting point, and allocated arrays for the
solution.  The {\tt main} function from the example is shown next and
discussed below.

\begin{verbatim}
int main()
{
  Index n=-1;                          /* number of variables */
  Index m=-1;                          /* number of constraints */
  Number* x_L = NULL;                  /* lower bounds on x */
  Number* x_U = NULL;                  /* upper bounds on x */
  Number* g_L = NULL;                  /* lower bounds on g */
  Number* g_U = NULL;                  /* upper bounds on g */
  IpoptProblem nlp = NULL;             /* IpoptProblem */
  enum ApplicationReturnStatus status; /* Solve return code */
  Number* x = NULL;                    /* starting point and solution vector */
  Number* mult_x_L = NULL;             /* lower bound multipliers at the solution */
  Number* mult_x_U = NULL;             /* upper bound multipliers at the solution */
  Number obj;                          /* objective value */
  Index i;                             /* generic counter */
  
  /* set the number of variables and allocate space for the bounds */
  n=4;
  x_L = (Number*)malloc(sizeof(Number)*n);
  x_U = (Number*)malloc(sizeof(Number)*n);
  /* set the values for the variable bounds */
  for (i=0; i<n; i++) {
    x_L[i] = 1.0;
    x_U[i] = 5.0;
  }

  /* set the number of constraints and allocate space for the bounds */
  m=2;
  g_L = (Number*)malloc(sizeof(Number)*m);
  g_U = (Number*)malloc(sizeof(Number)*m);
  /* set the values of the constraint bounds */
  g_L[0] = 25; g_U[0] = 2e19;
  g_L[1] = 40; g_U[1] = 40;

  /* create the IpoptProblem */
  nlp = CreateIpoptProblem(n, x_L, x_U, m, g_L, g_U, 8, 10, 0, 
			   &eval_f, &eval_g, &eval_grad_f, 
			   &eval_jac_g, &eval_h);
  
  /* We can free the memory now - the values for the bounds have been
     copied internally in CreateIpoptProblem */
  free(x_L);
  free(x_U);
  free(g_L);
  free(g_U);

  /* set some options */
  AddIpoptNumOption(nlp, "tol", 1e-9);
  AddIpoptStrOption(nlp, "mu_strategy", "adaptive");

  /* allocate space for the initial point and set the values */
  x = (Number*)malloc(sizeof(Number)*n);
  x[0] = 1.0;
  x[1] = 5.0;
  x[2] = 5.0;
  x[3] = 1.0;

  /* allocate space to store the bound multipliers at the solution */
  mult_x_L = (Number*)malloc(sizeof(Number)*n);
  mult_x_U = (Number*)malloc(sizeof(Number)*n);

  /* solve the problem */
  status = IpoptSolve(nlp, x, NULL, &obj, NULL, mult_x_L, mult_x_U, NULL);

  if (status == Solve_Succeeded) {
    printf("\n\nSolution of the primal variables, x\n");
    for (i=0; i<n; i++)
      printf("x[%d] = %e\n", i, x[i]); 

    printf("\n\nSolution of the bound multipliers, z_L and z_U\n");
    for (i=0; i<n; i++)
      printf("z_L[%d] = %e\n", i, mult_x_L[i]); 
    for (i=0; i<n; i++)
      printf("z_U[%d] = %e\n", i, mult_x_U[i]); 

    printf("\n\nObjective value\nf(x*) = %e\n", obj); 
  }
 
  /* free allocated memory */
  FreeIpoptProblem(nlp);
  free(x);
  free(mult_x_L);
  free(mult_x_U);

  return 0;
}
\end{verbatim}

Here, we declare all the necessary variables and set the dimensions of
the problem.  The problem has 4 variables, so we set {\tt n} and
allocate space for the variable bounds (don't forget to call {\tt
  free} for each of your {\tt malloc} calls before the end of the
program). We then set the values for the variable bounds.

The problem has 2 constraints, so we set {\tt m} and allocate space
for the constraint bounds. The first constraint has a lower bound of
$25$ and no upper bound.  Here we set the upper bound to
\texttt{2e19}. \Ipopt interprets any number greater than or equal to
\htmlref{\tt nlp\_upper\_bound\_inf}{opt:nlp_upper_bound_inf} as infinity.
The default value of \htmlref{\tt nlp\_lower\_bound\_inf}{opt:nlp_lower_bound_inf}
and \htmlref{\tt nlp\_upper\_bound\_inf}{opt:nlp_upper_bound_inf} is
\texttt{-1e19} and \texttt{1e19}, respectively, and can be changed
through \Ipopt options.  The second constraint is an equality with
right hand side 40, so we set both the upper and the lower bound to
40.

We next create an instance of the {\tt IpoptProblem} by calling {\tt
CreateIpoptProblem}, giving it the problem dimensions and the variable
and constraint bounds. The arguments {\tt nele\_jac} and {\tt
nele\_hess} are the number of elements in Jacobian and the Hessian,
respectively. See Appendix~\ref{app.triplet} for a description of the
sparse matrix format. The {\tt index\_style} argument specifies whether
we want to use C style indexing for the row and column indices of the
matrices or Fortran style indexing. Here, we set it to {\tt 0} to
indicate C style.  We also include the references to each of our
callback functions. \Ipopt uses these function pointers to ask for
evaluation of the NLP when required.

After freeing the bound arrays that are no longer required, the next
two lines illustrate how you can change the value of options through
the interface.  \Ipopt options can also be changed by creating a {\tt
ipopt.opt} file (see Section~\ref{sec:options}). We next allocate
space for the initial point and set the values as given in the problem
definition.

The call to {\tt IpoptSolve} can provide us with information about the
solution, but most of this is optional. Here, we want values for the
bound multipliers at the solution and we allocate space for these.

We can now make the call to {\tt IpoptSolve} and find the solution of
the problem. We pass in the {\tt IpoptProblem}, the starting point
{\tt x} (\Ipopt will use this array to return the solution or final
point as well).  The next 5 arguments are pointers so \Ipopt can fill
in values at the solution.  If these pointers are set to {\tt NULL},
\Ipopt will ignore that entry.  For example, here, we do not want the
constraint function values at the solution or the constraint
multipliers, so we set those entries to {\tt NULL}. We do want the
value of the objective, and the multipliers for the variable bounds.
The last argument is a {\tt void*} for user data. Any pointer you give
here will also be passed to you in the callback functions.

The return code is an {\tt ApplicationReturnStatus} enumeration, see
the header file {\tt ReturnCodes\_inc.h} which is installed along {\tt
  IpStdCInterface.h} in the \Ipopt include directory.

After the optimizer terminates, we check the status and print the
solution if successful. Finally, we free the {\tt IpoptProblem} and
the remaining memory and return from {\tt main}.

\subsection{The Fortran Interface}

The Fortran interface is essentially a wrapper of the C interface
discussed in Section~\ref{sec.cinterface}.  The way to hook up \Ipopt
in a Fortran program is very similar to how it is done for the C
interface, and the functions of the Fortran interface correspond
one-to-one to the those of the C and C++ interface, including their
arguments.  You can find an implementation of the example problem
(\ref{eq:ex_obj})--(\ref{eq:ex_bounds}) in {\tt
  \$IPOPTDIR/Ipopt/examples/hs071\_f}.

The only special things to consider are:
\begin{itemize}
\item The return value of the function {\tt IPCREATE} is of an {\tt
    INTEGER} type that must be large enough to capture a pointer
  on the particular machine.  This means, that you have to declare
  the ``handle'' for the IpoptProblem as {\tt INTEGER*8} if your
  program is compiled in 64-bit mode.  All other {\tt INTEGER}-type
  variables must be of the regular type.
\item For the call of {\tt IPSOLVE} (which is the function that is to
  be called to run \Ipopt), all arrays, including those for the dual
  variables, must be given (in contrast to the C interface).  The
  return value {\tt IERR} of this function indicates the outcome of
  the optimization (see the include file {\tt IpReturnCodes.inc} in
  the \Ipopt include directory).
\item The return {\tt IERR} value of the remaining functions has to be
  set to zero, unless there was a problem during execution of the
  function call.
\item The callback functions ({\tt EV\_*} in the example) include the
  arguments {\tt IDAT} and {\tt DAT}, which are {\tt INTEGER} and {\tt
    DOUBLE PRECISION} arrays that are passed unmodified between the
  main program calling {\tt IPSOLVE} and the evaluation subroutines
  {\tt EV\_*} (similarly to {\tt UserDataPtr} arguments in the C
  interface).  These arrays can be used to pass ``private'' data
  between the main program and the user-provided Fortran subroutines.

  The last argument of the {\tt EV\_*} subroutines, {\tt IERR}, is to
  be set to 0 by the user on return, unless there was a problem
  during the evaluation of the optimization problem
  function/derivative for the given point {\tt X} (then it should
  return a non-zero value).
\end{itemize}

\subsection{The Java Interface}
\hfill \textit{based on documentation by Rafael de Pelegrini Soares}%
\footnote{VRTech Industrial Technologies}
\medskip

The Java interface offers an abstract base class {\tt Ipopt} with basic 
methods to specify an NLP, set a number of \Ipopt options, to request \Ipopt 
to solve the NLP, and to retrieve a found solution, if any.
A HTML documentation of all available interface methods of the {\tt Ipopt} 
class can be generated via {\tt javadoc} by executing {\tt make doc} in the 
\JIpopt build directory.

In the following, we discuss necessary steps to implement the HS071 example 
with \JIpopt.

First, we create a new directory and therein sub directories {\tt 
org/coinor/}. Into {\tt org/coinor/} we copy the file {\tt Ipopt.java}, which 
contains the Java code of the interface, from the corresponding 
\JIpopt source directory
(\verb|$IPOPTDIR/Ipopt/contrib/JavaInterface/org/coinor|). %$
Further, we create a directory {\tt lib} next to the {\tt org} directory and 
place the previously build \JIpopt library into it ({\tt libjipopt.so} on 
Linux/UNIX, {\tt libjipopt.dylib} on Mac OS X, {\tt jipopt.dll} on Windows), 
see also Section \ref{sec.jipopt.build}.

Next, we create a new Java source file {\tt HS071.java} and define a class 
{\tt HS071} that extends the class {\tt Ipopt} of \JIpopt.
In the class constructor, we call the {\tt create()} method of \JIpopt, which 
works analogously to {\tt get\_nlp\_info()} of the C++ interface.
It initializes an {\tt IpoptApplication} object and informs \JIpopt about the
problem size (number of variables, constraints, nonzeros in Jacobian and 
Hessian).

\begin{verbatim}
/** Initialize the bounds and create the native Ipopt problem. */
public HS071() {
  /* Number of nonzeros in the Jacobian of the constraints */
  int nele_jac = 8;
  
  /* Number of nonzeros in the Hessian of the Lagrangian (lower or
   * upper triangual part only) */
  int nele_hess = 10;

  /* Number of variables */
  int n = 4;
        
  /* Number of constraints */
  int m = 2;
        
  /* Index style for the irow/jcol elements */
  int index_style = Ipopt.C_STYLE;
        
  /* create the IpoptProblem */
  create(n, m,  nele_jac, nele_hess, index_style);
}
\end{verbatim}        

\noindent Next, we add callback functions that are called by \JIpopt to obtain
variable bounds, constraint sides, and a starting point:
\begin{verbatim}
protected boolean get_bounds_info(int n, double[] x_L, double[] x_U,
                                  int m, double[] g_L, double[] g_U) {
  /* set the values of the variable bounds */
  for( int i = 0; i < x_L.length; i++ ) {
    x_L[i] = 1.0;
    x_U[i] = 5.0;
  }
  
  /* set the values of the constraint bounds */
  g_L[0] = 25.0;
  g_U[0] = 2e19;
  g_L[1] = 40.0;
  g_U[1] = 40.0;
        
  return true;
}
    
protected boolean get_starting_point(int n, boolean init_x, double[] x,
                                     boolean init_z, double[] z_L, double[] z_U,
                                     int m, boolean init_lambda,double[] lambda) {
  assert init_z == false;
  assert init_lambda = false;
  
  if( init_x ) {
    x[0] = 1.0;
    x[1] = 5.0;
    x[2] = 5.0;
    x[3] = 1.0;
  }
        
  return true;
}
\end{verbatim}

\noindent In the following, we implement the evaluation methods in a way that 
is very similar to the C++ interface:
\begin{verbatim}
protected boolean eval_f(int n, double[] x, boolean new_x, double[] obj_value) {
  obj_value[0] = x[0] * x[3] * (x[0] + x[1] + x[2]) + x[2];

  return true;
}

protected boolean eval_grad_f(int n, double[] x, boolean new_x, double[] grad_f) {
  grad_f[0] = x[0] * x[3] + x[3] * (x[0] + x[1] + x[2]);
  grad_f[1] = x[0] * x[3];
  grad_f[2] = x[0] * x[3] + 1;
  grad_f[3] = x[0] * (x[0] + x[1] + x[2]);

  return true;
}

protected boolean eval_g(int n, double[] x, boolean new_x, int m, double[] g) {
  g[0] = x[0] * x[1] * x[2] * x[3];
  g[1] = x[0] * x[0] + x[1] * x[1] + x[2] * x[2] + x[3] * x[3];

  return true;
}

protected boolean eval_jac_g(int n, double[] x, boolean new_x, int m, int nele_jac,
                             int[] iRow, int[] jCol, double[] values) {
  if( values == null ) {
    /* return the structure of the jacobian */
    /* this particular jacobian is dense */
    iRow[0] = 0;  jCol[0] = 0;
    iRow[1] = 0;  jCol[1] = 1;
    iRow[2] = 0;  jCol[2] = 2;
    iRow[3] = 0;  jCol[3] = 3;
    iRow[4] = 1;  jCol[4] = 0;
    iRow[5] = 1;  jCol[5] = 1;
    iRow[6] = 1;  jCol[6] = 2;
    iRow[7] = 1;  jCol[7] = 3;
  }
  else {
    /* return the values of the jacobian of the constraints */
    values[0] = x[1]*x[2]*x[3]; /* 0,0 */
    values[1] = x[0]*x[2]*x[3]; /* 0,1 */
    values[2] = x[0]*x[1]*x[3]; /* 0,2 */
    values[3] = x[0]*x[1]*x[2]; /* 0,3 */

    values[4] = 2*x[0];         /* 1,0 */
    values[5] = 2*x[1];         /* 1,1 */
    values[6] = 2*x[2];         /* 1,2 */
    values[7] = 2*x[3];         /* 1,3 */
  }

  return true;
}

protected boolean eval_h(int n, double[] x, boolean new_x, double obj_factor,
                         int m, double[] lambda, boolean new_lambda,
                         int nele_hess, int[] iRow, int[] jCol, double[] values) {
  int idx = 0; /* nonzero element counter */
  int row = 0; /* row counter for loop */
  int col = 0; /* col counter for loop */
  
  if( values == null ) {
    /* return the structure. This is a symmetric matrix, fill the lower left
     * triangle only. */
    /* the hessian for this problem is actually dense */
    idx = 0;
    for( row = 0; row < 4; row++ ) {
      for( col = 0; col <= row; col++ ) {
        iRow[idx] = row;
        jCol[idx] = col;
        idx++;
      }
    }
  }
  else {
    /* return the values. This is a symmetric matrix, fill the lower left
     * triangle only */

    /* fill the objective portion */
    values[0] = obj_factor * (2*x[3]);               /* 0,0 */
    values[1] = obj_factor * (x[3]);                 /* 1,0 */
    values[2] = 0;                                   /* 1,1 */
    values[3] = obj_factor * (x[3]);                 /* 2,0 */
    values[4] = 0;                                   /* 2,1 */
    values[5] = 0;                                   /* 2,2 */
    values[6] = obj_factor * (2*x[0] + x[1] + x[2]); /* 3,0 */
    values[7] = obj_factor * (x[0]);                 /* 3,1 */
    values[8] = obj_factor * (x[0]);                 /* 3,2 */
    values[9] = 0;                                   /* 3,3 */

    /* add the portion for the first constraint */
    values[1] += lambda[0] * (x[2] * x[3]);          /* 1,0 */
    values[3] += lambda[0] * (x[1] * x[3]);          /* 2,0 */
    values[4] += lambda[0] * (x[0] * x[3]);          /* 2,1 */
    values[6] += lambda[0] * (x[1] * x[2]);          /* 3,0 */
    values[7] += lambda[0] * (x[0] * x[2]);          /* 3,1 */
    values[8] += lambda[0] * (x[0] * x[1]);          /* 3,2 */

    /* add the portion for the second constraint */
    values[0] += lambda[1] * 2;                      /* 0,0 */
    values[2] += lambda[1] * 2;                      /* 1,1 */
    values[5] += lambda[1] * 2;                      /* 2,2 */
    values[9] += lambda[1] * 2;                      /* 3,3 */
  }
  return true;
}
\end{verbatim}

\noindent Finally, we add a main routine to run this example. The main 
routines creates an instance of our object and calls the solve method {\tt 
OptimizeNLP}:
\begin{verbatim}
public static void main(String[] args) {
  // Create the problem
  HS071 hs071 = new HS071();

  // solve the problem
  int status = hs071.OptimizeNLP(x);
  
  // print the status and optimal value
  System.out.println("Status    = " + status);
  System.out.println("Obj Value = " + hs071.getObjectiveValue());
}
\end{verbatim}
The {\tt OptimizeNLP} method returns the \Ipopt solve status as integer, which 
indicates whether the problem was solved successfully.
Further, the methods {\tt getObjectiveValue()}, {\tt getVariableValues()}, and {\tt 
getConstraintMultipliers()}, {\tt getLowerBoundMultipliers()}, {\tt 
getUpperBoundMultipliers()} can be used to obtain the objective value, the primal 
solution value of the variables, and dual solution values, respectively.

\subsection{The R Interface} 
\hfill \textit{based on documentation by Jelmer Ypma}%
\footnote{University College London}
\medskip

The \ipoptr package (see Section \ref{sec.ipoptr.build} for installation 
instructions) offers a R function {\tt ipoptr} which takes an NLP 
specification, a starting point, and \Ipopt options as input and returns 
information about a \Ipopt run (status, message, ...) and a solution point.

In the following, we discuss necessary steps to implement the HS071 example 
with \ipoptr.
A more detailed documentation of \ipoptr is available in
{\tt Ipopt/contrib/RInterface/inst/doc/ipoptr.pdf}.

First, we define the objective function and its gradient
\begin{verbatim}
> eval_f <- function( x ) { 
     return( x[1]*x[4]*(x[1] + x[2] + x[3]) + x[3] ) 
 }
> eval_grad_f <- function( x ) {
     return( c( x[1] * x[4] + x[4] * (x[1] + x[2] + x[3]),
                x[1] * x[4],
                x[1] * x[4] + 1.0,
                x[1] * (x[1] + x[2] + x[3]) ) )		  
 }
\end{verbatim}
Then we define a function that returns the value of the two constraints. We 
define the bounds of the constraints (in this case the $g_L$ and $g_U$ are 
$25$ and $40$) later.
\begin{verbatim}
> # constraint functions
> eval_g <- function( x ) {
     return( c( x[1] * x[2] * x[3] * x[4],
                x[1]^2 + x[2]^2 + x[3]^2 + x[4]^2 ) )
 }
\end{verbatim}
Then we define the structure of the Jacobian, which is a dense matrix in this 
case, and function to evaluate it
\begin{verbatim}
> eval_jac_g_structure <- list( c(1,2,3,4), c(1,2,3,4) )
> eval_jac_g <- function( x ) {
     return( c ( x[2]*x[3]*x[4],
                 x[1]*x[3]*x[4],
                 x[1]*x[2]*x[4],
                 x[1]*x[2]*x[3],
                 2.0*x[1],
                 2.0*x[2],
                 2.0*x[3],
                 2.0*x[4] ) )
 }
\end{verbatim}
The Hessian is also dense, but it looks slightly more complicated because we 
have to take into account the Hessian of the objective function and of the 
constraints at the same time, although you could write a function to calculate 
them both separately and then return the combined result in \texttt{eval\_h}.
\begin{verbatim}
> # The Hessian for this problem is actually dense, 
> # This is a symmetric matrix, fill the lower left triangle only.
> eval_h_structure <- list( c(1), c(1,2), c(1,2,3), c(1,2,3,4) )
> eval_h <- function( x, obj_factor, hessian_lambda ) {
 
     values <- numeric(10)
     values[1] = obj_factor * (2*x[4]) # 1,1
 
     values[2] = obj_factor * (x[4])   # 2,1
     values[3] = 0                     # 2,2
 
     values[4] = obj_factor * (x[4])   # 3,1
     values[5] = 0                     # 4,2
     values[6] = 0                     # 3,3
 
     values[7] = obj_factor * (2*x[1] + x[2] + x[3]) # 4,1
     values[8] = obj_factor * (x[1])                 # 4,2
     values[9] = obj_factor * (x[1])                 # 4,3
     values[10] = 0                                  # 4,4
 
 
     # add the portion for the first constraint
     values[2] = values[2] + hessian_lambda[1] * (x[3] * x[4]) # 2,1
     
     values[4] = values[4] + hessian_lambda[1] * (x[2] * x[4]) # 3,1
     values[5] = values[5] + hessian_lambda[1] * (x[1] * x[4]) # 3,2
 
     values[7] = values[7] + hessian_lambda[1] * (x[2] * x[3]) # 4,1
     values[8] = values[8] + hessian_lambda[1] * (x[1] * x[3]) # 4,2
     values[9] = values[9] + hessian_lambda[1] * (x[1] * x[2]) # 4,3
 
     # add the portion for the second constraint
     values[1] = values[1] + hessian_lambda[2] * 2 # 1,1
     values[3] = values[3] + hessian_lambda[2] * 2 # 2,2
     values[6] = values[6] + hessian_lambda[2] * 2 # 3,3
     values[10] = values[10] + hessian_lambda[2] * 2 # 4,4
 
     return ( values )
 }
\end{verbatim}
After the hard part is done, we only have to define the initial values, the 
lower and upper bounds of the control variables, and the lower and upper 
bounds of the constraints. If a variable or a constraint does not have lower 
or upper bounds, the values \texttt{-Inf} or \texttt{Inf} can be used. If the 
upper and lower bounds of a constraint are equal, Ipopt recognizes this as an 
equality constraint and acts accordingly.
\begin{verbatim}
> # initial values
> x0 <- c( 1, 5, 5, 1 )
> # lower and upper bounds of control
> lb <- c( 1, 1, 1, 1 )
> ub <- c( 5, 5, 5, 5 )
> # lower and upper bounds of constraints
> constraint_lb <- c(  25, 40 )
> constraint_ub <- c( Inf, 40 )
\end{verbatim}
Finally, we can call \Ipopt with the {\tt ipoptr} function.
In order to redirect the \Ipopt output into a file, we use \Ipopt's
\htmlref{\tt output\_file}{opt:output_file} and
\htmlref{\tt print\_level}{opt:print_level} options.
\begin{verbatim}
> opts <- list("print_level" = 0,
              "file_print_level" = 12,
              "output_file" = "hs071_nlp.out")
> print( ipoptr( x0 = x0, 
                eval_f = eval_f, 
                eval_grad_f = eval_grad_f, 
                lb = lb, 
                ub = ub, 
                eval_g = eval_g, 
                eval_jac_g = eval_jac_g, 
                constraint_lb = constraint_lb, 
                constraint_ub = constraint_ub, 
                eval_jac_g_structure = eval_jac_g_structure,
                eval_h = eval_h,
                eval_h_structure = eval_h_structure,
                opts = opts) )

Call:

ipoptr(x0 = x0, eval_f = eval_f, eval_grad_f = eval_grad_f, lb = lb, 
  ub = ub, eval_g = eval_g, eval_jac_g = eval_jac_g,
  eval_jac_g_structure = eval_jac_g_structure, constraint_lb = constraint_lb,
  constraint_ub = constraint_ub, eval_h = eval_h, eval_h_structure = eval_h_structure,
  opts = opts)

Ipopt solver status: 0 ( SUCCESS: Algorithm terminated 
successfully at a locally optimal point, satisfying the 
convergence tolerances (can be specified by options). )

Number of Iterations....: 8 
Optimal value of objective function:  17.0140171451792 
Optimal value of controls: 1 4.743 3.82115 1.379408
\end{verbatim}

To pass additional data to the evaluation routines, one can either supply 
additional arguments to the user defined functions and {\tt ipoptr} or define 
an environment that holds the data and pass this environment to {\tt ipoptr}.
Both methods are shown in the file \texttt{tests/parameters.R} that comes with 
\ipoptr.

As a very simple example, suppose we want to find the minimum of
\[
f( x ) = a_1 x^2 + a_2 x + a_3
\]
for different values of the parameters $a_1$, $a_2$ and $a_3$. 

First, we define the objective function and its gradient using, assuming that 
there is some variable \texttt{params} that contains the values of the 
parameters.
\begin{verbatim}
> eval_f_ex1 <- function(x, params) { 
     return( params[1]*x^2 + params[2]*x + params[3] ) 
 }
> eval_grad_f_ex1 <- function(x, params) { 
     return( 2*params[1]*x + params[2] ) 
 }
\end{verbatim}
Note, that the first parameter should always be the control variable. All of 
the user-defined functions should contain the same set of additional 
parameters. You have to supply them as input argument to all functions, even 
if you're not using them in some of the functions. 

Then we can solve the problem for a specific set of parameters, in this case 
$a_1=1$, $a_2=2$ and $a_3=3$, from initial value $x_0=0$, with the following 
command
\begin{verbatim}
> # solve using ipoptr with additional parameters
> ipoptr(x0          = 0, 
         eval_f      = eval_f_ex1, 
         eval_grad_f = eval_grad_f_ex1,
         opts        = list("print_level"=0),
         params      = c(1,2,3) )
         
Call:

ipoptr(x0 = 0, eval_f = eval_f_ex1, eval_grad_f = eval_grad_f_ex1, 
    opts = list(print_level = 0), params = c(1, 2, 3))


Ipopt solver status: 0 ( SUCCESS: Algorithm terminated 
successfully at a locally optimal point, satisfying the 
convergence tolerances (can be specified by options). )

Number of Iterations....: 1 
Optimal value of objective function:  2 
Optimal value of controls: -1
\end{verbatim}

For the second method, we don't have to supply the parameters as additional 
arguments to the function.
\begin{verbatim}
> eval_f_ex2 <- function(x) { 
     return( params[1]*x^2 + params[2]*x + params[3] ) 
 }
> eval_grad_f_ex2 <- function(x) { 
     return( 2*params[1]*x + params[2] ) 
 }
\end{verbatim}
Instead, we define an environment that contains specific values of 
\texttt{params}
\begin{verbatim}
> # define a new environment that contains params
> auxdata        <- new.env()
> auxdata$params <- c(1,2,3)
\end{verbatim}
To solve this we supply \texttt{auxdata} as an argument to \texttt{ipoptr}, 
which will take care of evaluating the functions in the correct environment, 
so that the auxiliary data is available.
\begin{verbatim}
> # pass the environment that should be used to evaluate functions to ipoptr
> ipoptr(x0                 = 0, 
         eval_f             = eval_f_ex2, 
         eval_grad_f        = eval_grad_f_ex2, 
         ipoptr_environment = auxdata,
         opts               = list("print_level"=0) )
         
Call:

ipoptr(x0 = 0, eval_f = eval_f_ex2, eval_grad_f = eval_grad_f_ex2, 
    opts = list(print_level = 0), ipoptr_environment = auxdata)


Ipopt solver status: 0 ( SUCCESS: Algorithm terminated 
successfully at a locally optimal point, satisfying the 
convergence tolerances (can be specified by options). )

Number of Iterations....: 1 
Optimal value of objective function:  2 
Optimal value of controls: -1
\end{verbatim}

\subsection{The \Matlab Interface} 
\hfill \textit{based on documentation by Peter Carbonetto\footnote{University of British Columbia}}%
\medskip

See Section \ref{sec.matlab.build} for instructions on how to build a
{\tt mex} file of the \Matlab interface for \Ipopt and how to make it known to 
\Matlab.

The {\tt \$IPOPTDIR/contrib/MatlabInterface/examples} directory contains 
several illustrative examples on how to use the \Matlab interface.
The best way to understand how to use the interface is to carefully go over 
these examples.

For more information, type {\tt help ipopt} in the \Matlab prompt.

Further, Jonas Asprion assembled information about the \Matlab\ {\tt ipopt} 
function and its arguments: 
\url{http://www.idsc.ethz.ch/Downloads/IPOPT_InstallMatlab/IPOPT_MatlabInterface_V0p1.pdf}

Note, that this document refers to \Ipopt versions before 3.11.
With 3.11, the {\tt auxdata} option has been removed from the {\tt mex} code. The new wrapper function {\tt ipopt\_auxdata} implements the same functionality as the previous {\tt ipopt} function, but uses \Matlab function handles to do so.

\section{Special Features}
\subsection{Derivative Checker}\label{sec:deriv-checker}
When writing code for the evaluation of derivatives it is very easy to
make mistakes (much easier than writing it correctly the first time
:)).  As a convenient feature, \Ipopt provides the option to run a
simple derivative checker, based on finite differences, before the
optimization is started.

To use the derivative checker, you need to use the option
\htmlref{\tt derivative\_test}{opt:derivative_test}.
By default, this option is set to {\tt none},
i.e., no finite difference test is performed,  It is set to {\tt
  first-order}, then the first derivatives of the objective function
and the constraints are verified, and for the setting {\tt
  second-order}, the second derivatives are tested as well.

The verification is done by a simple finite differences approximation,
where each component of the user-provided starting point is perturbed
one of the other.  The relative size of the perturbation is determined
by the option \htmlref{\tt derivative\_test\_perturbation}{opt:derivative_test_perturbation}.  The default value
($10^{-8}$, about the square root of the machine precision) is
probably fine in most cases, but if you believe that you see wrong
warnings, you might want to play with this parameter.  When the test is
performed, \Ipopt prints out a line for every partial derivative, for
which the user-provided derivative value deviates too much from the
finite difference approximation.  The relative tolerance for deciding
when a warning should be issued, is determined by the option
\htmlref{\tt derivative\_test\_tol}{opt:derivative_test_tol}.
If you want to see the user-provided and
estimated derivative values with the relative deviation for each
single partial derivative, you can switch the
\htmlref{\tt derivative\_test\_print\_all}{opt:derivative_test_print_all}
option to {\tt yes}.

A typical output is:

\begin{footnotesize}
\begin{verbatim}
Starting derivative checker.

* grad_f[          2] = -6.5159999999999991e+02    ~ -6.5559997134793468e+02  [ 6.101e-03]
* jac_g [    4,    4] =  0.0000000000000000e+00    ~  2.2160643690464592e-02  [ 2.216e-02]
* jac_g [    4,    5] =  1.3798494268463347e+01 v  ~  1.3776333629422766e+01  [ 1.609e-03]
* jac_g [    6,    7] =  1.4776333636790881e+01 v  ~  1.3776333629422766e+01  [ 7.259e-02]

Derivative checker detected 4 error(s).
\end{verbatim}
\end{footnotesize}

The star (``\verb|*|'') in the first column indicates that this line
corresponds to some partial derivative for which the error tolerance
was exceeded.  Next, we see which partial derivative is concerned in
this output line.  For example, in the first line, it is the second
component of the objective function gradient (or the third, if the
{\tt C\_STYLE} numbering is used, i.e., when counting of indices
starts with 0 instead of 1).  The first floating point number is the
value given by the user code, and the second number (after
``\verb|~|'') is the finite differences estimation.  Finally, the
number in square brackets is the relative difference between these two
numbers.

For constraints, the first index after {\tt jac\_g} is the index of
the constraint, and the second one corresponds to the variable index
(again, the choice of the numbering style matters).

Since also the sparsity structure of the constraint Jacobian has to be
provided by the user, it can be faulty as well.  For this, the ``{\tt
  v}'' after a user-provided derivative value indicates that this
component of the Jacobian is part of the user provided sparsity
structure.  If there is no ``{\tt v}'', it means that the user did not
include this partial derivative in the list of non-zero elements.  In
the above output, the partial derivative ``{\tt jac\_g[4,4]}'' is
non-zero (based on the finite difference approximation), but it is not
included in the list of non-zero elements (missing ``{\tt v}''), so
that the user probably made a mistake in the sparsity structure.  The
other two Jacobian entries are provided in the non-zero structure but
their values seem to be off.

For second derivatives, the output looks like:

\begin{footnotesize}
\begin{verbatim}
*             obj_hess[    1,    1] =  1.8810000000000000e+03 v  ~  1.8820000036612328e+03  [ 5.314e-04]
*     3-th constr_hess[    2,    4] =  1.0000000000000000e+00 v  ~  0.0000000000000000e+00  [ 1.000e+00]
\end{verbatim}
\end{footnotesize}

There, the first line shows the deviation of the user-provided partial
second derivative in the Hessian for the objective function, and the
second line show an error in a partial derivative for the Hessian of
the third constraint (again, the numbering style matters).

Since the second derivatives are approximates by finite differences of
the first derivatives, you should first correct errors for the first
derivatives.  Also, since the finite difference approximations are
quite expensive, you should try to debug a small instance of your
problem if you can.  

Another useful option is 
\htmlref{\tt derivative\_test\_first\_index}{opt:derivative_test_first_index}
which allows your to start the derivative test with variables with a larger
index.
%
Finally, it is of course always a good idea to run your code through
some memory checker, such as {\tt valgrind} on Linux.

\subsection{Quasi-Newton Approximation of Second Derivatives}
\label{sec:quasiNewton}

\Ipopt has an option to approximate the Hessian of the Lagrangian by
a limited-memory quasi-Newton method (L-BFGS).  You can use this
feature by setting the \htmlref{\tt hessian\_approximation}{opt:hessian_approximation} 
option to the value {\tt limited-memory}.  In this case, it is not necessary to
implement the Hessian computation method {\tt eval\_h} in {\tt TNLP}.
If you are using the C or Fortran interface, you still need to
implement these functions, but they should return {\tt false} or {\tt
  IERR=1}, respectively, and don't need to do anything else.

In general, when second derivatives can be computed with reasonable
computational effort, it is usually a good idea to use them, since
then \Ipopt normally converges in fewer iterations and is more
robust.  An exception might be in cases, where your optimization
problem has a dense Hessian, i.e., a large percentage of non-zero entries
in the Hessian. In such a case, using the quasi-Newton approximation might be
better, even if it increases the number of iterations, since with exact
second derivatives the computation time per iteration might be significantly 
higher due to the very large number of non-zero elements in the linear systems 
that \Ipopt solve in order to compute the search direction.

Since the Hessian of the Lagrangian is zero for all variables that
appear only linearly in the objective and constraint functions, the
Hessian approximation should only take place in the space of all
nonlinear variables.  By default, it is assumed that all variables are
nonlinear, but you can tell \Ipopt explicitly which variables are
nonlinear, using the {\tt get\_number\_of\_nonlinear\_variables} and
{\tt get\_list\_of\_nonlinear\_variables} method of the {\tt TNLP}
class, see Section~\ref{sec:add_meth}.  (Those methods have been
implemented for the AMPL interface, so you would automatically only
approximate the Hessian in the space of the nonlinear variables, if
you are using the quasi-Newton option for AMPL models.)  Currently,
those two methods are not available through the C or Fortran
interface.

\subsection{Warm-Starting Capabilities via AMPL}
\hfill \textit{based on documentation by Victor M. Zavala}\footnote{Department of Chemical Engineering, Carnegie Mellon University}
\medskip

Warm-starting an interior-point algorithm is an important issue. One of the main 
difficulties arises from the fact that full-space variable information is required to 
generate the warm-starting point. While \Ipopt is currently equipped to retrieve and receive 
this type of information through the {\tt TNLP} interface, there exist some communication 
barriers in the AMPL interface. When the user solves the problem 
\eqref{eq:obj}--\eqref{eq:bounds}, \Ipopt will only return the optimal values of the primal 
variables $x$ and of the constraint multipliers corresponding to the active bounds of $g(x)$ 
(see \eqref{eq:constraints}). The constraint multiplier values can be accessed through the 
{\tt .dual} suffix or through the {\tt .sol} file. If this information is used to solve the 
same problem again, you will notice that \Ipopt will take some iterations in finding the 
same solution. The reason for this is that we are missing the input information of the 
multipliers $z^L$ and $z^U$ corresponding to the variable bounds (see \eqref{eq:bounds}).

However, \Ipopt also passes the values of the bound multipliers $z^L$ and $z^U$ to AMPL. 
This will be communicated to the AMPL user through the suffixes {\tt ipopt\_zL\_out} and 
{\tt ipopt\_zU\_out}, respectively. The user does not need to declare these suffixes, they 
will be generated automatically in the AMPL interface. The user can use the suffix values to 
initialize the bound multipliers for subsequent calls. In order to pass this information to 
\Ipopt, the user will need to declare and assign values to the suffixes {\tt ipopt\_zL\_in} 
and {\tt ipopt\_zU\_in}. For instance, for a given variable {\tt x[i]}, this can be done by 
setting:
\begin{verbatim}
  let x[i].ipopt_zL_in := x[i].ipopt_zL_out;
  let x[i].ipopt_zU_in := x[i].ipopt_zU_out;
\end{verbatim}

If the user does not specify some of these values, \Ipopt will set these multipliers to 1.0 
(as before). In order to make the warm-start effective, the user has control over the 
following options from AMPL:\\
\htmlref{\tt warm\_start\_init\_point}{opt:warm_start_init_point} \\
\htmlref{\tt warm\_start\_bound\_push}{opt:warm_start_bound_push} \\
\htmlref{\tt warm\_start\_mult\_bound\_push}{opt:warm_start_mult_bound_push}

Note, that the use of this feature is far from solving the complicated issue of warm-
starting interior-point algorithms. As a general advice, this feature will be useful if the 
user observes that the solution of subsequent problems (i.e., for different data instances) 
preserves the same set of active inequalities and bounds (monitor the values of $z^L$ and 
$z^U$ for subsequent solutions). In this case, initializing the bound multipliers and 
setting \htmlref{\tt warm\_start\_init\_point}{opt:warm_start_init_point} to {\tt yes} and 
setting \htmlref{\tt warm\_start\_bound\_push}{opt:warm_start_bound_push},
\htmlref{\tt warm\_start\_mult\_bound\_push}{opt:warm_start_mult_bound_push} and
\htmlref{\tt mu\_init}{opt:mu_init} to a small value ($10^{-6}$ or so) will reduce 
significantly the number of iterations. This is particularly useful in setting up on-line 
applications and high-level optimization strategies in AMPL.
If active-set changes are observed between subsequent solutions, then this strategy might 
not decrease the number of iterations (in some cases, it might even tend to increase the 
number of iterations).

You might also want to try the adaptive barrier update (instead of the default monotone one where above we chose the initial value $10^{-6}$) when doing the warm start. This can be activated by setting the \htmlref{\tt mu\_strategy}{opt:mu_strategy} option to {\tt adaptive}. Also the option \htmlref{\tt mu\_oracle}{opt:mu_oracle} gives some alternative choices. In general, the adaptive choice often leads to less iterations, but the computational cost per iteration might be higher.

The file {\tt \$IPOPTDIR/Ipopt/doc/hs071\_warmstart.mod} illustrates the use of the warm-start feature on the HS071 problem, see also Section \ref{sec.ipoptampl}.

\subsection{\sIpopt: Optimal Sensitivity Based on \Ipopt}
\hfill \textit{based on documentation by Hans Pirnay\footnote{RWTH Aachen, {\tt hans.pirnay@avt.rwth-aachen.de}} and Rodrigo L\'opez-Negrete\footnote{Carnegie Mellon University, {\tt rln@cmu.edu}}}
\medskip

The \sIpopt project provides a toolbox that uses NLP sensitivity theory to generate fast approximations to solutions when parameters in the problem change. It has been developed primarily by Hans Pirnay (RWTH-Aachen), Rodrigo L\'opez-Negrete (CMU), and Lorenz Biegler (CMU).

Sensitivity of nonlinear programming problems is a key step in any optimization study. Sensitivity provides information on regularity and curvature conditions at KKT points, assesses which variables play dominant roles in the optimization, and provides first order estimates for parametric nonlinear programs. Moreover, for NLP algorithms that use exact second derivatives, sensitivity can be implemented very efficiently within NLP solvers and provide valuable information with very little added computation. This implementation provides \Ipopt with the capabilities to calculate sensitivities, and approximate perturbed solutions with them.

The basic sensitivity strategy implemented here is based on the application of the Implicit Function Theorem (IFT) to the KKT conditions of the NLP. As shown by Fiacco (1983), sensitivities can be obtained from a solution with suitable regularity conditions merely by solving a linearization of the KKT conditions. More details can be found in \cite{PLNB:sIpopt}.
If you are using \sIpopt for your research, please cite \cite{PLNB:sIpopt}.

The \sIpopt project is available in the \Ipopt repository under
{\tt \$IPOPTDIR/Ipopt/contrib/sIPOPT}.
After having installed \Ipopt successfully, \sIpopt can be build and installed by changing to the directory {\tt \$IPOPTDIR/build/Ipopt/contrib/sIPOPT} and executing {\tt make install}. 
This should copy the generated libraries {\tt libsipopt.*} to {\tt \$IPOPTDIR/build/lib} and an AMPL executable {\tt ipopt\_sens} to {\tt \$IPOPTDIR/build/bin}.

The files {\tt \$IPOPTDIR/Ipopt/contrib/sIPOPT/examples/parametric\_ampl/parametric.\{mod,run\}} are an example that shows how to use \sIpopt to solve the NLP
\begin{align}
\min\quad & x_1^2 + x_2^2 + x_3^2, \\
\mathrm{such~that}\quad & 6x_1 + 3x_2 + 2x_3 = p_1, \\
 & p_2 x_1 + x_2 - x_3  = 1, \\
 & x_1, x_2, x_3 \geq 0,
\end{align}
where we perturb the parameters $p_1$ and $p_2$ from $p_a = (p_1, p_2) = (5, 1)$ to $p_b = (4.5, 1)$.


Note, that \sIpopt has been developed under the constraint that it must work with the regular \Ipopt code. Due to this constraint, some compromises had to be made.
However, there is an ongoing effort to develop \sIpopt 2, which is a fork of the \Ipopt code that allows for the explicit definition of parametric NLPs. This code can be found at \url{https://github.com/athrpf/sipopt2}. If you have questions about \sIpopt 2, please contact ​Hans Pirnay.


\subsection{Inertia-Free Curvature Test}
\hfill \textit{contributed by Nai-Yuan Chiang\footnote{Argonne National Laboratory, \url{http://www.mcs.anl.gov/~nychiang/}} and Victor M. Zavala Tejeda\footnote{University of Wisconsin-Madison, \url{http://zavalab.engr.wisc.edu/}}}
\medskip

In a filter line-search setting it is necessary to detect the presence of
negative curvature and to regularize the Hessian of the Lagrangian when
such is present. Regularization ensures that the computed step is a descent
direction for the objective function when the constraint violation is
sufficiently small, which in turn is necessary to guarantee global
convergence.

To detect the presence of negative curvature, the default method
implemented in IPOPT requires inertia information of the augmented system.
The inertia of the augmented system is the number of positive, negative,
and zero eigenvalues. Inertia is currently estimated using symmetric
indefinite factorization routines implemented in powerful packages such as
MA27, MA57, or Pardiso. When more general linear
algebra strategies/packages are used (e.g., iterative, parallel
decomposition), however, inertia information is difficult (if not
impossible) to obtain.

In \cite{ChiangZavala2014}, we present acceptance tests for the search step
that do not require inertia information of the linear system and prove that
such tests are sufficient to ensure global convergence. Similar tests were
proposed in the exact penalty framework reported in \cite{CouHubSchWae:inexact}.
The inertia-free approach also enables the use of a wider range of linear
algebra strategies and packages. We have performed significant benchmarks
and found satisfactory performance compared to the inertia-based
counterpart.  Moreover, we have found that this test can yield significant
improvements in computing time because it provides more flexibility to
accept steps. This flexibility is particularly beneficial in problems that
are inherently ill-conditioned and require significant amounts of
regularization.

The inertia-free capability implemented in \Ipopt is controlled by the
options \htmlref{\tt neg\_curv\_test\_tol}{opt:neg_curv_test_tol} and
\htmlref{\tt neg\_curv\_test\_reg}{opt:neg_curv_test_reg}.


\section{\Ipopt Options}\label{sec:options}
\Ipopt has many (maybe too many) options that can be adjusted for the
algorithm.  Options are all identified by a string name, and their
values can be of one of three types: Number (real), Integer, or
String. Number options are used for things like tolerances, integer
options are used for things like maximum number of iterations, and
string options are used for setting algorithm details, like the NLP
scaling method. Options can be set through code, through the AMPL
interface if you are using AMPL, or by creating a {\tt ipopt.opt}
file in the directory you are executing \Ipopt.

The {\tt ipopt.opt} file is read line by line and each line should
contain the option name, followed by whitespace, and then the
value. Comments can be included with the {\tt \#} symbol. For example,
\begin{verbatim}
# This is a comment

# Turn off the NLP scaling
nlp_scaling_method none

# Change the initial barrier parameter
mu_init 1e-2

# Set the max number of iterations
max_iter 500
\end{verbatim}
is a valid {\tt ipopt.opt} file.

Options can also be set in code. Have a look at the examples to see
how this is done. 

A subset of \Ipopt options are available through AMPL. To set options
through AMPL, use the internal AMPL command {\tt options}.  For
example, \\
{\tt options ipopt\_options "nlp\_scaling\_method=none mu\_init=1e-2
  max\_iter=500"} \\
is a valid options command in AMPL. The most important options are
referenced in Appendix~\ref{app.options_ref}. To see which options are
available through AMPL, you can run the AMPL solver executable with
the ``{\tt -=}'' flag from the command prompt.  To specify other
options when using AMPL, you can always create {\tt ipopt.opt}.  Note,
the {\tt ipopt.opt} file is given preference when setting options.
This way, you can easily override any options set in a particular
executable or AMPL model by specifying new values in {\tt ipopt.opt}.

For a list of the most important valid options, see the Appendix
\ref{app.options_ref}. You can print the documentation for all \Ipopt
options by using the option
\medskip

\htmlref{\tt print\_options\_documentation}{opt:print_options_documentation} {\tt~yes}
\medskip

and running \Ipopt (like the AMPL solver executable, for
instance). This will output the documentation of almost all options to the
console.

\section{\Ipopt Output}\label{sec:output}
This section describes the standard \Ipopt console output with the
default setting for \htmlref{\tt print\_level}{opt:print_level}. The output is designed to
provide a quick summary of each iteration as \Ipopt solves the problem.

Before \Ipopt starts to solve the problem, it displays the problem
statistics (number of nonzero-elements in the matrices, number of
variables, etc.). Note that if you have fixed variables (both upper
and lower bounds are equal), \Ipopt may remove these variables from
the problem internally and not include them in the problem statistics.

Following the problem statistics, \Ipopt will begin to solve the
problem and you will see output resembling the following,
\begin{verbatim}
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  1.6109693e+01 1.12e+01 5.28e-01   0.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  1.8029749e+01 9.90e-01 6.62e+01   0.1 2.05e+00    -  2.14e-01 1.00e+00f  1
   2  1.8719906e+01 1.25e-02 9.04e+00  -2.2 5.94e-02   2.0 8.04e-01 1.00e+00h  1
\end{verbatim}
and the columns of output are defined as,
\begin{description}
\item[{\tt iter}:] The current iteration count. This includes regular
  iterations and iterations during the restoration phase. If the
  algorithm is in the restoration phase, the letter {\tt r'} will be
  appended to the iteration number.
\item[{\tt objective}:] The unscaled objective value at the current
  point. During the restoration phase, this value remains the unscaled
  objective value for the original problem.
\item[{\tt inf\_pr}:] The unscaled constraint violation at the current
  point.  This quantity is the infinity-norm (max) of the (unscaled)
  constraints (\ref{eq:constraints}).  During the restoration phase,
  this value remains the constraint violation of the original problem
  at the current point.  The option \htmlref{\tt inf\_pr\_output}{opt:inf_pr_output} can
  be used to switch to the printing of a different quantity.
\item[{\tt inf\_du}:] The scaled dual infeasibility at the current
  point.  This quantity measure the infinity-norm (max) of the
  internal dual infeasibility, Eq.~(4a) in the implementation paper
  \cite{WaecBieg06:mp}, including inequality constraints reformulated
  using slack variables and problem scaling. During the restoration
  phase, this is the value of the dual infeasibility for the
  restoration phase problem.
\item[{\tt lg(mu)}:] $\log_{10}$ of the value of the barrier parameter
  $\mu$.
\item[{\tt ||d||}:] The infinity norm (max) of the primal step (for
  the original variables $x$ and the internal slack variables $s$).
  During the restoration phase, this value includes the values of
  additional variables, $p$ and $n$ (see Eq.~(30) in
  \cite{WaecBieg06:mp}).
\item[{\tt lg(rg)}:] $\log_{10}$ of the value of the regularization
  term for the Hessian of the Lagrangian in the augmented system
  ($\delta_w$ in Eq.~(26) and Section 3.1 in \cite{WaecBieg06:mp}).  A
  dash (``\texttt{-}'') indicates that no regularization was done.
\item[{\tt alpha\_du}:] The stepsize for the dual variables
  ($\alpha^z_k$ in Eq.~(14c) in \cite{WaecBieg06:mp}).
\item[{\tt alpha\_pr}:] The stepsize for the primal
  variables ($\alpha_k$ in Eq.~(14a) in \cite{WaecBieg06:mp}).  The
  number is usually followed by a character for additional diagnostic
  information regarding the step acceptance criterion, see
  Table~\ref{tab:alpha_pr}.
\item[{\tt ls}:] The number of backtracking line search steps (does
  not include second-order correction steps).
\end{description}

\begin{table}
  \centering
  \begin{tabular}{ll}
    Tag & Description \\
    \hline
    f & f-type iteration in the filter method w/o second order correction \\
    F & f-type iteration in the filter method w/ second order correction \\
    h & h-type iteration in the filter method w/o second order correction \\
    H & h-type iteration in the filter method w/ second order correction \\
    k & penalty value unchanged in merit function method w/o second order correction \\
    K & penalty value unchanged in merit function method w/ second order correction \\
    n & penalty value updated in merit function method w/o second order correction \\
    N & penalty value updated in merit function method w/ second order correction \\
    R & Restoration phase just started \\
    w &  in watchdog procedure \\
    s & step accepted in soft restoration phase \\
    t/T & tiny step accepted without line search \\
    r & some previous iterate restored \\
    % (forgot right now what that means ;-) )
    \hline \\
  \end{tabular} 

  \caption{Diagnostic output in {\tt alpha\_pr} column.}
  \label{tab:alpha_pr}
\end{table}

Note that the step acceptance mechanisms in \Ipopt consider the
barrier objective function (Eq~(3a) in \cite{WaecBieg06:mp}) which is
usually different from the value reported in the \texttt{objective}
column.  Similarly, for the purposes of the step acceptance, the
constraint violation is measured for the internal problem formulation,
which includes slack variables for inequality constraints and
potentially scaling of the constraint functions.  This value, too, is
usually different from the value reported in \texttt{inf\_pr}.  As a
consequence, a new iterate might have worse values both for the
objective function and the constraint violation as reported in the
iteration output, seemingly contradicting globalization procedure.

When the algorithm terminates, \Ipopt will output a message to the
screen based on the return status of the call to {\tt Optimize}. The following
is a list of the possible return codes, their corresponding output message
to the console, and a brief description.
\begin{description}
\item[{\tt Solve\_Succeeded}:] $\;$ \\
  Console Message: {\tt EXIT: Optimal Solution Found.} \\
  This message indicates that \Ipopt found a (locally) optimal point
  within the desired tolerances.
\item[{\tt Solved\_To\_Acceptable\_Level}:]  $\;$ \\
  Console Message: {\tt EXIT: Solved To Acceptable Level.} \\
  This indicates that the algorithm did not converge to the
  ``desired'' tolerances, but that it was able to obtain a point
  satisfying the ``acceptable'' tolerance level as specified by the
  \htmlref{\tt acceptable\_*}{opt:acceptable_tol} options.
  This may happen if the desired tolerances
  are too small for the current problem.
\item[{\tt Feasible\_Point\_Found}:]  $\;$ \\
  Console Message: {\tt EXIT: Feasible point for square problem found.} \\
  This message is printed if the problem is ``square'' (i.e., it has
  as many equality constraints as free variables) and \Ipopt found a
  feasible point.
\item[{\tt Infeasible\_Problem\_Detected}:]  $\;$ \\
  Console Message: {\tt EXIT: Converged to a point of
    local infeasibility. Problem may be infeasible.} \\
  The restoration phase converged to a point that is a minimizer for
  the constraint violation (in the $\ell_1$-norm), but is not feasible
  for the original problem. This indicates that the problem may be
  infeasible (or at least that the algorithm is stuck at a locally
  infeasible point).  The returned point (the minimizer of the
  constraint violation) might help you to find which constraint is
  causing the problem.  If you believe that the NLP is feasible,
  it might help to start the optimization from a different point.
\item[{\tt Search\_Direction\_Becomes\_Too\_Small}:]  $\;$ \\
  Console Message: {\tt EXIT: Search Direction is becoming Too Small.} \\
  This indicates that \Ipopt is calculating very small step sizes and
  is making very little progress.  This could happen if the problem has
  been solved to the best numerical accuracy possible given the
  current scaling.
\item[{\tt Diverging\_Iterates}:]  $\;$ \\
  Console Message: {\tt EXIT: Iterates divering; problem might be
    unbounded.} \\
  This message is printed if the max-norm of the iterates becomes
  larger than the value of the option
  \htmlref{\tt diverging\_iterates\_tol}{opt:diverging_iterates_tol}.
  This can happen if the problem is unbounded below and the iterates
  are diverging.
\item[{\tt User\_Requested\_Stop}:]  $\;$ \\
  Console Message: {\tt EXIT: Stopping optimization at current point
    as requested by user.} \\
  This message is printed if the user call-back method {\tt
    intermediate\_callback} returned {\tt false} (see
  Section~\ref{sec:add_meth}).
\item[{\tt Maximum\_Iterations\_Exceeded}:]  $\;$ \\
  Console Message: {\tt EXIT: Maximum Number of Iterations Exceeded.} \\
  This indicates that \Ipopt has exceeded the maximum number of
  iterations as specified by the option \htmlref{\tt max\_iter}{opt:max_iter}.
\item[{\tt Maximum\_CpuTime\_Exceeded}:]  $\;$ \\
  Console Message: {\tt EXIT: Maximum CPU time exceeded.} \\
  This indicates that \Ipopt has exceeded the maximum number of
  CPU seconds as specified by the option \htmlref{\tt max\_cpu\_time}{opt:max_cpu_time}.
\item[{\tt Restoration\_Failed}:]  $\;$ \\
  Console Message: {\tt EXIT: Restoration Failed!} \\
  This indicates that the restoration phase failed to find a feasible
  point that was acceptable to the filter line search for the original
  problem. This could happen if the problem is highly degenerate, does
  not satisfy the constraint qualification, or if your NLP code
  provides incorrect derivative information.
\item[{\tt Error\_In\_Step\_Computation}:]  $\;$ \\
  Console Output:\ {\tt EXIT:\ Error in step computation (regularization becomes too large?)!} \\
  This messages is printed if \Ipopt is unable to compute a search
  direction, despite several attempts to modify the iteration matrix.
  Usually, the value of the regularization parameter then becomes too
  large.  One situation where this can happen is when values in the
  Hessian are invalid ({\tt NaN} or {\tt Inf}).  You can check whether this
  is true by using the
  \htmlref{\tt check\_derivatives\_for\_naninf}{opt:check_derivatives_for_naninf} option.
\item[{\tt Invalid\_Option}:]  $\;$ \\
  Console Message: (details about the particular error
  will be output to the console) \\
  This indicates that there was some problem specifying the options.
  See the specific message for details.
\item[{\tt Not\_Enough\_Degrees\_Of\_Freedom}:]  $\;$ \\
  Console Message: {\tt EXIT: Problem has too few degrees of freedom.} \\
  This indicates that your problem, as specified, has too few degrees
  of freedom. This can happen if you have too many equality
  constraints, or if you fix too many variables (\Ipopt removes fixed
  variables by default, see also the \htmlref{\tt fixed\_variable\_treatment}{opt:fixed_variable_treatment} option).
\item[{\tt Invalid\_Problem\_Definition}:]  $\;$ \\
  Console Message: (no console message, this is a return code for the
  C and Fortran interfaces only.) \\
  This indicates that there was an exception of some sort when
  building the {\tt IpoptProblem} structure in the C or Fortran
  interface. Likely there is an error in your model or the {\tt main}
  routine.
\item[{\tt Unrecoverable\_Exception}:]  $\;$ \\
  Console Message: (details about the particular error
  will be output to the console) \\
  This indicates that \Ipopt has thrown an exception that does not
  have an internal return code. See the specific message for details.
\item[{\tt NonIpopt\_Exception\_Thrown}:]  $\;$ \\
  Console Message: {\tt Unknown Exception caught in Ipopt} \\
  An unknown exception was caught in \Ipopt. This exception could have
  originated from your model or any linked in third party code.
\item[{\tt Insufficient\_Memory}:]  $\;$ \\
  Console Message: {\tt EXIT: Not enough memory.} \\
  An error occurred while trying to allocate memory. The problem may
  be too large for your current memory and swap configuration.
\item[{\tt Internal\_Error}:]  $\;$ \\
  Console: {\tt EXIT: INTERNAL ERROR: Unknown SolverReturn
    value - Notify IPOPT Authors.} \\
  An unknown internal error has occurred. Please notify the authors of
  \Ipopt via the mailing list.

\end{description}

\subsection{Diagnostic Tags for \Ipopt}

To print additional diagnostic tags for each iteration of \Ipopt, set
the options \htmlref{\tt print\_info\_string}{opt:print_info_string}
to \texttt{yes}. With
this, a tag will appear at the end of an iteration line with the
following diagnostic meaning that are useful to flag difficulties for
a particular \Ipopt run.  A list of possible strings is given in
Table~\ref{tab:info_string}.

\begin{table}\centering
  \begin{tabular}{@{}lll@{}}
    Tag & Description & Reference \\
    \hline 
    ! & Tighten resto tolerance if only slightly infeasible & Section 3.3 in \cite{WaecBieg06:mp} \\ 
    A & Current iteration is acceptable & Alternate termination \\  
    a & Perturbation for PD singularity impossible, assume singular & Section 3.1 in \cite{WaecBieg06:mp}\\  
    C & Second Order Correction taken & Section 2.4 in \cite{WaecBieg06:mp} \\  
    Dh & Hessian degenerate based on multiple iterations & Section 3.1 in \cite{WaecBieg06:mp}\\  
    Dhj & Hessian/Jacobian degenerate based on multiple iterations & Section 3.1 in \cite{WaecBieg06:mp}\\  
    Dj & Jacobian degenerate based on multiple iterations & Section 3.1 in \cite{WaecBieg06:mp}\\  
    dx & $\delta_x$ perturbation too large & Section 3.1 in \cite{WaecBieg06:mp}\\  
    e & Cutting back $\alpha$ due to evaluation error & in backtracking line search \\  
    F- & Filter should be reset, but maximal resets exceeded & Section 2.3 in \cite{WaecBieg06:mp} \\  
    F+ & Resetting filter due to last few rejections of filter & Section 2.3 in \cite{WaecBieg06:mp} \\  
    L & Degenerate Jacobian, $\delta_c$ already perturbed & Section 3.1 in \cite{WaecBieg06:mp}\\  
    l & Degenerate Jacobian, $\delta_c$ perturbed & Section 3.1 in \cite{WaecBieg06:mp}\\  
    M & Magic step taken for slack variables & in backtracking line search \\  
    Nh & Hessian not yet degenerate & Section 3.1 in \cite{WaecBieg06:mp}\\  
    Nhj & Hessian/Jacobian not yet degenerate & Section 3.1 in \cite{WaecBieg06:mp}\\  
    Nj & Jacobian not yet degenerate & Section 3.1 in \cite{WaecBieg06:mp}\\  
    NW & Warm start initialization failed & in Warm Start Initialization \\
    q & PD system possibly singular, attempt improving sol.\ quality & Section 3.1 in \cite{WaecBieg06:mp}\\  
    R & Solution of restoration phase & Section 3.3 in \cite{WaecBieg06:mp} \\  
    S & PD system possibly singular, accept current solution & Section 3.1 in \cite{WaecBieg06:mp}\\  
    s & PD system singular & Section 3.1 in \cite{WaecBieg06:mp}\\  
    s & Square Problem. Set multipliers to zero & Default initialization routine \\  
    Tmax & Trial $\theta$ is larger than $\theta_{max}$ & filter parameter, see (21) in \cite{WaecBieg06:mp} \\  
    W & Watchdog line search procedure successful & Section 3.2 in \cite{WaecBieg06:mp} \\  
    w & Watchdog line search procedure unsuccessful, stopped & Section 3.2 in \cite{WaecBieg06:mp} \\  
    Wb & Undoing most recent SR1 update & Section 5.4.1 in \cite{Biegler:nlpbook} \\  
    We & Skip Limited-Memory Update in restoration phase  & Section 5.4.1 in \cite{Biegler:nlpbook} \\ 
    Wp & Safeguard $B^0 = \sigma I$ for  Limited-Memory Update &  Section 5.4.1 in \cite{Biegler:nlpbook} \\  
    Wr & Resetting Limited-Memory Update &  Section 5.4.1 in \cite{Biegler:nlpbook} \\  
    Ws & Skip Limited-Memory Update since $s^Ty$ is not positive &  Section 5.4.1 in \cite{Biegler:nlpbook} \\  
    WS & Skip Limited-Memory Update since $\Delta x$ is too small &  Section 5.4.1 in \cite{Biegler:nlpbook} \\  
    y & Dual infeasibility, use least square multiplier update & during ipopt algorithm \\                           
    z & Apply correction to bound multiplier if too large & during ipopt algorithm \\
  \end{tabular}
  \caption{Diagnostic output appended using \texttt{print\_info\_sting}.}
  \label{tab:info_string}
\end{table}


\appendix
\section{Triplet Format for Sparse Matrices}\label{app.triplet}
\Ipopt was designed for optimizing large sparse nonlinear programs.
Because of problem sparsity, the required matrices (like the
constraints Jacobian or Lagrangian Hessian) are not stored as dense
matrices, but rather in a sparse matrix format. For the tutorials in
this document, we use the triplet format.  Consider the matrix
\begin{equation}
\label{eqn.ex_matrix}
\left[
\begin{array}{ccccccc}
1.1     & 0             & 0             & 0             & 0             & 0             & 0.5 \\
0       & 1.9   & 0             & 0             & 0             & 0             & 0.5 \\
0       & 0             & 2.6   & 0             & 0             & 0             & 0.5 \\
0       & 0             & 7.8   & 0.6   & 0             & 0             & 0    \\
0       & 0             & 0             & 1.5   & 2.7   & 0             & 0     \\
1.6     & 0             & 0             & 0             & 0.4   & 0             & 0     \\
0       & 0             & 0             & 0             & 0             & 0.9   & 1.7 \\
\end{array}
\right]
\end{equation}

A standard dense matrix representation would need to store $7 \cdot
7{=} 49$ floating point numbers, where many entries would be zero. In
triplet format, however, only the nonzero entries are stored. The
triplet format records the row number, the column number, and the
value of all nonzero entries in the matrix. For the matrix above, this
means storing $14$ integers for the rows, $14$ integers for the
columns, and $14$ floating point numbers for the values. While this
does not seem like a huge space saving over the $49$ floating point
numbers stored in the dense representation, for larger matrices, the
space savings are very dramatic\footnote{For an $n \times n$ matrix,
the dense representation grows with the the square of $n$, while the
sparse representation grows linearly in the number of nonzeros.}.

The parameter {\tt index\_style} in {\tt get\_nlp\_info} tells \Ipopt if
you prefer to use C style indexing (0-based, i.e., starting the
counting at 0) for the row and column indices or Fortran style
(1-based). Tables \ref{tab.fortran_triplet} and \ref{tab.c_triplet}
below show the triplet format for both indexing styles, using the
example matrix (\ref{eqn.ex_matrix}).

\begin{footnotesize}
\begin{table}[ht]%[!h]
\begin{center}
\begin{tabular}{c c c}
row     		&       col     	&       value 			    \\
\hline
{\tt iRow[0] = 1}       &       {\tt jCol[0] = 1}       & {\tt values[0] = 1.1}     \\
{\tt iRow[1] = 1}       &       {\tt jCol[1] = 7}       & {\tt values[1] = 0.5}     \\
{\tt iRow[2] = 2}       &       {\tt jCol[2] = 2}       & {\tt values[2] = 1.9}     \\
{\tt iRow[3] = 2}       &       {\tt jCol[3] = 7}       & {\tt values[3] = 0.5}     \\
{\tt iRow[4] = 3}       &       {\tt jCol[4] = 3}       & {\tt values[4] = 2.6}     \\
{\tt iRow[5] = 3}       &       {\tt jCol[5] = 7}       & {\tt values[5] = 0.5}     \\
{\tt iRow[6] = 4}       &       {\tt jCol[6] = 3}       & {\tt values[6] = 7.8}     \\
{\tt iRow[7] = 4}       &       {\tt jCol[7] = 4}       & {\tt values[7] = 0.6}     \\
{\tt iRow[8] = 5}       &       {\tt jCol[8] = 4}       & {\tt values[8] = 1.5}     \\
{\tt iRow[9] = 5}       &       {\tt jCol[9] = 5}       & {\tt values[9] = 2.7}     \\
{\tt iRow[10] = 6}      &       {\tt jCol[10] = 1}      & {\tt values[10] = 1.6}     \\
{\tt iRow[11] = 6}      &       {\tt jCol[11] = 5}      & {\tt values[11] = 0.4}     \\
{\tt iRow[12] = 7}      &       {\tt jCol[12] = 6}      & {\tt values[12] = 0.9}     \\
{\tt iRow[13] = 7}      &       {\tt jCol[13] = 7}      & {\tt values[13] = 1.7}
\end{tabular}
\caption{Triplet Format of Matrix (\ref{eqn.ex_matrix}) 
with {\tt index\_style=FORTRAN\_STYLE}}
\label{tab.fortran_triplet}
\end{center}
\end{table}
\begin{table}[ht]%[!h]
\begin{center}
\begin{tabular}{c c c}
row     		&       col     	&       value 			    \\
\hline
{\tt iRow[0] = 0}       &       {\tt jCol[0] = 0}       & {\tt values[0] = 1.1}     \\
{\tt iRow[1] = 0}       &       {\tt jCol[1] = 6}       & {\tt values[1] = 0.5}     \\
{\tt iRow[2] = 1}       &       {\tt jCol[2] = 1}       & {\tt values[2] = 1.9}     \\
{\tt iRow[3] = 1}       &       {\tt jCol[3] = 6}       & {\tt values[3] = 0.5}     \\
{\tt iRow[4] = 2}       &       {\tt jCol[4] = 2}       & {\tt values[4] = 2.6}     \\
{\tt iRow[5] = 2}       &       {\tt jCol[5] = 6}       & {\tt values[5] = 0.5}     \\
{\tt iRow[6] = 3}       &       {\tt jCol[6] = 2}       & {\tt values[6] = 7.8}     \\
{\tt iRow[7] = 3}       &       {\tt jCol[7] = 3}       & {\tt values[7] = 0.6}     \\
{\tt iRow[8] = 4}       &       {\tt jCol[8] = 3}       & {\tt values[8] = 1.5}     \\
{\tt iRow[9] = 4}       &       {\tt jCol[9] = 4}       & {\tt values[9] = 2.7}     \\
{\tt iRow[10] = 5}      &       {\tt jCol[10] = 0}      & {\tt values[10] = 1.6}     \\
{\tt iRow[11] = 5}      &       {\tt jCol[11] = 4}      & {\tt values[11] = 0.4}     \\
{\tt iRow[12] = 6}      &       {\tt jCol[12] = 5}      & {\tt values[12] = 0.9}     \\
{\tt iRow[13] = 6}      &       {\tt jCol[13] = 6}      & {\tt values[13] = 1.7}
\end{tabular}
\caption{Triplet Format of Matrix (\ref{eqn.ex_matrix}) 
with {\tt index\_style=C\_STYLE}}
\label{tab.c_triplet}
\end{center}
\end{table}
\end{footnotesize}
The individual elements of the matrix can be listed in any order, and
if there are multiple items for the same nonzero position, the values
provided for those positions are added.

The Hessian of the Lagrangian is a symmetric matrix. In the case of a
symmetric matrix, you only need to specify the lower left triangular
part (or, alternatively, the upper right triangular part). For
example, given the matrix,
\begin{equation}
\label{eqn.ex_sym_matrix}
\left[
\begin{array}{ccccccc}
1.0	& 0	& 3.0	& 0	& 2.0 	\\
0	& 1.1	& 0	& 0	& 5.0	\\
3.0	& 0	& 1.2	& 6.0	& 0	\\
0	& 0	& 6.0	& 1.3	& 9.0	\\
2.0	& 5.0	& 0	& 9.0	& 1.4
\end{array}
\right]
\end{equation}
the triplet format is shown in Tables \ref{tab.sym_fortran_triplet}
and \ref{tab.sym_c_triplet}.

\begin{footnotesize}
\begin{table}[ht]%[!h]
\begin{center}
\begin{tabular}{c c c}
row     		&       col     	&       value 			    \\
\hline
{\tt iRow[0] = 1}       &       {\tt jCol[0] = 1}       & {\tt values[0] = 1.0}     \\
{\tt iRow[1] = 2}       &       {\tt jCol[1] = 1}       & {\tt values[1] = 1.1}     \\
{\tt iRow[2] = 3}       &       {\tt jCol[2] = 1}       & {\tt values[2] = 3.0}     \\
{\tt iRow[3] = 3}       &       {\tt jCol[3] = 3}       & {\tt values[3] = 1.2}     \\
{\tt iRow[4] = 4}       &       {\tt jCol[4] = 3}       & {\tt values[4] = 6.0}     \\
{\tt iRow[5] = 4}       &       {\tt jCol[5] = 4}       & {\tt values[5] = 1.3}     \\
{\tt iRow[6] = 5}       &       {\tt jCol[6] = 1}       & {\tt values[6] = 2.0}     \\
{\tt iRow[7] = 5}       &       {\tt jCol[7] = 2}       & {\tt values[7] = 5.0}     \\
{\tt iRow[8] = 5}       &       {\tt jCol[8] = 4}       & {\tt values[8] = 9.0}     \\
{\tt iRow[9] = 5}       &       {\tt jCol[9] = 5}       & {\tt values[9] = 1.4}
\end{tabular}
\caption{Triplet Format of Matrix (\ref{eqn.ex_sym_matrix}) 
with {\tt index\_style=FORTRAN\_STYLE}}
\label{tab.sym_fortran_triplet}
\end{center}
\end{table}
\begin{table}[ht]%[!h]
\begin{center}
\begin{tabular}{c c c}
row     		&       col     	&       value 			    \\
\hline
{\tt iRow[0] = 0}       &       {\tt jCol[0] = 0}       & {\tt values[0] = 1.0}     \\
{\tt iRow[1] = 1}       &       {\tt jCol[1] = 0}       & {\tt values[1] = 1.1}     \\
{\tt iRow[2] = 2}       &       {\tt jCol[2] = 0}       & {\tt values[2] = 3.0}     \\
{\tt iRow[3] = 2}       &       {\tt jCol[3] = 2}       & {\tt values[3] = 1.2}     \\
{\tt iRow[4] = 3}       &       {\tt jCol[4] = 2}       & {\tt values[4] = 6.0}     \\
{\tt iRow[5] = 3}       &       {\tt jCol[5] = 3}       & {\tt values[5] = 1.3}     \\
{\tt iRow[6] = 4}       &       {\tt jCol[6] = 0}       & {\tt values[6] = 2.0}     \\
{\tt iRow[7] = 4}       &       {\tt jCol[7] = 1}       & {\tt values[7] = 5.0}     \\
{\tt iRow[8] = 4}       &       {\tt jCol[8] = 3}       & {\tt values[8] = 9.0}     \\
{\tt iRow[9] = 4}       &       {\tt jCol[9] = 4}       & {\tt values[9] = 1.4}
\end{tabular}
\caption{Triplet Format of Matrix (\ref{eqn.ex_sym_matrix}) 
with {\tt index\_style=C\_STYLE}}
\label{tab.sym_c_triplet}
\end{center}
\end{table}
\end{footnotesize}


\section{The Smart Pointer Implementation: {\tt SmartPtr<T>}} \label{app.smart_ptr}

The {\tt SmartPtr} class is described in {\tt IpSmartPtr.hpp}. It is a
template class that takes care of counting references to objects and
deleting them when not references anymore. Instead of pointing to an object
with a raw C++ pointer (e.g. {\tt HS071\_NLP*}), we use a {\tt
  SmartPtr}.  Every time a {\tt SmartPtr} is set to reference an
object, it increments a counter in that object (see the {\tt
  ReferencedObject} base class if you are interested). If a {\tt
  SmartPtr} is done with the object, either by leaving scope or being
set to point to another object, the counter is decremented. When the
count of the object goes to zero, the object is automatically deleted.
{\tt SmartPtr}'s are very simple, just use them as you would use a
standard pointer.

It is very important to use {\tt SmartPtr}'s instead of raw pointers
when passing objects to \Ipopt. Internally, \Ipopt uses smart
pointers for referencing objects. If you use a raw pointer in your
executable, the object's counter will NOT get incremented. Then, when
\Ipopt uses smart pointers inside its own code, the counter will get
incremented. However, before \Ipopt returns control to your code, it
will decrement as many times as it incremented earlier, and the
counter will return to zero. Therefore, \Ipopt will delete the
object. When control returns to you, you now have a raw pointer that
points to a deleted object.

This might sound difficult to anyone not familiar with the use of
smart pointers, but just follow one simple rule; always use a SmartPtr
when creating or passing an \Ipopt object.

\section{Options Reference} \label{app.options_ref}

Options can be set using {\tt ipopt.opt}, through your own code, or
through the AMPL {\tt ipopt\_options} command. See Section
\ref{sec:options} for an explanation of how to use these commands.
Shown here is a list of the most important options for \Ipopt. To view
the full list of options, you can set the option 
\htmlref{\tt print\_options\_documentation}{opt:print_options_documentation}
to {\tt yes} or simply run the \Ipopt AMPL solver executable as
\begin{verbatim}
ipopt --print-options
\end{verbatim}

Usually, option values are identical for the regular mode of \Ipopt
and the restoration phase.  However, to set an option value
specifically for the restoration phase, the prefix ``\texttt{resto.}''
should be appended.  For example, to set the acceptable tolerance for
the restoration phase, use the keyword
``\texttt{resto.acceptable\_tol}''.

\medskip
\noindent
The most common options are:

\input{options.tex}

\section{Options available via the AMPL Interface}

The following is a list of options that is available via AMPL:
\input{options_ampl.tex}

%\bibliographystyle{plain}
%\bibliography{/home/andreasw/tex/andreas}
\input{documentation.bbl}

\end{document}
