--- title: "Getting started" author: "James Melville" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Getting started} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE, echo = FALSE, message = FALSE} knitr::opts_chunk$set(echo = TRUE, collapse = TRUE, comment = "#>") library(mize) ``` `mize` performs unconstrained numerical optimization when you can provide an objective function and its gradient. This guide follows one complete first-use path: define the problem, check the gradient, run the optimizer, and interpret the result. It ends with pointers to the specialist guides for convergence, stateful use, and method tuning. ## Define the objective and gradient `mize()` receives the objective and gradient together in a list. The `fn` function must return one numeric value; `gr` must return a numeric vector with the same length as its input. Here is the two-dimensional Rosenbrock function: ```{r define-rosenbrock} rb_fg <- list( fn = function(x) { 100 * (x[2] - x[1] * x[1])^2 + (1 - x[1])^2 }, gr = function(x) { c( -400 * x[1] * (x[2] - x[1] * x[1]) - 2 * (1 - x[1]), 200 * (x[2] - x[1] * x[1]) ) } ) ``` For a more involved problem, a factory function can return this list so that `fn` and `gr` share data through a closure. The list may also contain an `fg` function that computes both values together when they share expensive work. The [Metric MDS](mmds.html) article develops that pattern in a realistic example. ## Choose a starting point and check the gradient The parameters are a numeric vector. Before optimizing, check an analytic gradient at a representative, non-special starting point: ```{r check-gradient} rb0 <- c(-1.2, 1) gradient_check <- check_mize_gradient(rb_fg, rb0) gradient_summary <- data.frame( "Maximum absolute error" = formatC( gradient_check$max_abs_error, format = "e", digits = 2 ), "Maximum relative error" = formatC( gradient_check$max_rel_error, format = "e", digits = 2 ), check.names = FALSE ) knitr::kable(gradient_summary, align = c("r", "r")) ``` `check_mize_gradient()` compares `gr` with a finite-difference approximation of `fn`. Large errors are a strong reason to fix the gradient before trusting an optimization. At this point, the analytic and finite-difference gradients agree to approximately `r formatC(gradient_check$max_rel_error, format = "e", digits = 1)` in relative terms. The full result also identifies the worst coordinate and, when an optional combined `fg` function is present, checks it against the separate callbacks. ```{r assert-gradient, include=FALSE} stopifnot( is.finite(gradient_check$max_rel_error), gradient_check$max_rel_error < 1e-6 ) ``` ## Run the optimization The default method is limited-memory BFGS (`"L-BFGS"`), a useful general starting point for a smooth problem. Set `store_progress = TRUE` here so that we can inspect the trajectory later: ```{r run-mize} res <- mize(rb0, rb_fg, store_progress = TRUE) ``` Start with the fitted parameters, objective, and high-level outcome: ```{r result-summary} res[c("par", "f", "status", "message")] ``` The Rosenbrock function has a known minimum at `c(1, 1)`, where its objective is zero. The returned parameters are close to that point, the objective is close to zero, and `status = "converged"` says that a configured numerical tolerance was reached. Callback counts make the work behind the result inspectable: ```{r callback-summary} res[c("nf", "ng", "nh", "nhi")] ``` `nf` and `ng` count function and gradient callbacks. `nh` and `nhi` count accepted user-supplied Hessian and inverse-Hessian callback results. They are zero in this run because the default L-BFGS method builds its curvature model internally and `rb_fg` supplies neither curvature callback. ### Read the termination record `status` gives the quickest classification of any run: - `"converged"` means a configured tolerance was reached; - `"budget_exhausted"` means an iteration or callback limit was reached; - `"failed"` identifies failures such as a non-finite callback or line-search failure; and - `"terminated"` covers another recognized stopping reason. `converged` is `TRUE` only for the tolerance-based class, while `message` is a human-readable summary. Use the detailed record when code needs the precise reason: ```{r result-termination} res[c("converged", "terminate")] ``` ```{r assert-result, include=FALSE} initial_f <- rb_fg$fn(rb0) convergence_reasons <- c( "abs_tol", "rel_tol", "grad_tol", "ginf_tol", "step_tol" ) stopifnot( is.finite(res$best_f), res$best_f < initial_f, identical(res$status, "converged"), isTRUE(res$converged), res$terminate$what %in% convergence_reasons ) ``` ## Inspect progress With `store_progress = TRUE`, `res$progress` contains the observations and diagnostics available at stored iterations. Plotting the objective against the cumulative number of function evaluations gives a quick view of the run: ```{r progress-plot, echo=FALSE, fig.width=6.5, fig.height=4.2, fig.cap="Rosenbrock objective values over cumulative function evaluations. The vertical axis uses a logarithmic scale.", fig.alt="Line plot of the Rosenbrock objective decreasing from about 24 to near zero as cumulative function evaluations increase. The objective axis is logarithmic."} progress_for_plot <- subset(res$progress, is.finite(f) & f > 0) plot( progress_for_plot$nf, progress_for_plot$f, type = "o", pch = 16, cex = 0.6, log = "y", xlab = "Cumulative function evaluations", ylab = "Objective" ) stopifnot( nrow(progress_for_plot) > 1, tail(progress_for_plot$f, 1) < progress_for_plot$f[1] ) ``` The stored objective falls by several orders of magnitude during this run. For exact values, select the fields useful for the question and inspect a few rows from the beginning and end: ```{r compact-progress} progress_view <- res$progress[, c("f", "step", "nf", "ng"), drop = FALSE] ``` ```{r progress-table, echo=FALSE} progress_rows <- unique(c( seq_len(min(3L, nrow(progress_view))), seq.int(max(1L, nrow(progress_view) - 2L), nrow(progress_view)) )) progress_excerpt <- progress_view[progress_rows, , drop = FALSE] progress_table <- data.frame( Iteration = as.integer(rownames(progress_excerpt)), Objective = trimws(formatC(progress_excerpt$f, format = "g", digits = 4)), Step = trimws(formatC(progress_excerpt$step, format = "g", digits = 4)), "Function calls" = progress_excerpt$nf, "Gradient calls" = progress_excerpt$ng, check.names = FALSE ) knitr::kable( progress_table, align = rep("r", ncol(progress_table)) ) ``` The initial row represents the starting point. Here `f` is the current objective, `step` is the size of the parameter update, and `nf` and `ng` are cumulative function and gradient callback counts. The progress data also contains cumulative `nh` and `nhi`; the compact table omits them because both remain zero for this run. The available columns depend on the method and calculations requested; see the [Progress section of the `mize()` reference](../reference/mize.html#progress) for the complete dynamic schema. For live console output, use `verbose = TRUE`; `log_every` controls how often progress is printed or stored. Keeping every iteration can produce a large result on a long run. > **Returned points and hard budgets.** `mize()` normally returns the best > evaluated point. For nonmonotone runs, `best_*` and `last_*` distinguish that > point from the final iterate. Under an exact callback budget, some objective > fields may be unavailable. The [Convergence](convergence.html) article > explains these cases, the status classes, and the stopping controls in > detail. ## Where to go next The default L-BFGS method is a useful baseline for a smooth problem. Problem size and available derivative information may suggest another method: | Situation | Method to consider | Main trade-off | |---|---|---| | General smooth problem | `L-BFGS` | Limited-memory quasi-Newton; the default | | Smaller smooth problem | `BFGS` | Dense inverse-Hessian approximation uses quadratic storage | | Very tight memory budget | `CG` | Low storage; sensitive to scaling and line-search choices | | Extra gradient work is acceptable | `TN` | Approximates a Newton step without forming a Hessian | | Hessian or inverse Hessian is available | `NEWTON` | Requires an `hs` or `hi` callback in the function list | Treat these as starting points and benchmark promising methods on the problem at hand. Continue with the guide that matches your next task: - [Convergence](convergence.html) explains tolerances, limits, failures, and result interpretation. - [Stateful optimization](stateful.html) shows how to step, monitor, checkpoint, and resume an optimizer. - [Metric MDS](mmds.html) is a complete case study using closures, a checked gradient, and combined callbacks. - [Choosing methods and tuning](methods.html) covers algorithm selection and advanced controls, including line searches, step sizes, and momentum.