> ## Documentation Index > Fetch the complete documentation index at: https://docs.ionworks.com/llms.txt > Use this file to discover all available pages before exploring further. # Data Fitting Overview > Estimate battery model parameters by fitting to experimental data with iws.DataFit. `iws.DataFit` describes a parameter fit: which experiments to compare against, which parameters are free, and how to search. The schema is submitted as one element of a pipeline. For the theory (cost functions, identifiability, multi-start), see the [Data Fitting Guide](/guide/data-fitting/overview). ## A minimal fit ```python theme={null} import pybamm import ionworks_schema as iws from ionworks import Ionworks # Known parameters (everything not fit) known = iws.direct_entries.DirectEntry( parameters={"Ambient temperature [K]": 298.15}, ) # Objective: compare a current-driven SPMe simulation against measured voltage obj_1C = iws.objectives.CurrentDriven( data_input="file:examples/data/chen_synthetic_1C/time_series.csv", options={"model": pybamm.lithium_ion.SPMe()}, ) # Free parameters parameters = { "Negative particle diffusivity [m2.s-1]": iws.Parameter( "Negative particle diffusivity [m2.s-1]", initial_value=2e-14, bounds=(1e-14, 1e-13), ), "Positive particle diffusivity [m2.s-1]": iws.Parameter( "Positive particle diffusivity [m2.s-1]", initial_value=2e-15, bounds=(1e-15, 1e-14), ), } fit = iws.DataFit( objectives={"test_1C": obj_1C}, parameters=parameters, cost=iws.costs.SSE(), optimizer=iws.optimizers.DifferentialEvolution(), ) pipeline = iws.Pipeline({"known": known, "fit": fit}) client = Ionworks() submission = client.pipeline.create(pipeline) client.pipeline.wait_for_completion(submission.id) result = client.pipeline.result(submission.id) print(result.element_results["fit"]) ``` Configuration mistakes inside a `DataFit` (bad parameter names, malformed objectives, …) surface as `UserConfigurationError`. The job classifier maps these to a **Configuration error** in Studio so they're easy to distinguish from solver failures. ## Multiple objectives Pass multiple objectives to fit against several experiments simultaneously (e.g. discharge at different C-rates or temperatures): ```python theme={null} import pybamm fit = iws.DataFit( objectives={ "1C": iws.objectives.CurrentDriven( data_input="file:.../1C.csv", options={"model": pybamm.lithium_ion.SPMe()}, ), "0.5C": iws.objectives.CurrentDriven( data_input="file:.../0.5C.csv", options={"model": pybamm.lithium_ion.SPMe()}, ), }, parameters=parameters, ) ``` Each objective contributes to a single combined cost. ## Optimizers `iws.optimizers` exposes the optimisers available to `DataFit`. Pick the one that fits your problem: | Schema | Best for | | ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `iws.optimizers.ScipyMinimize(method="L-BFGS-B")` | Smooth problems, fast local optimisation | | `iws.optimizers.ScipyLeastSquares()` | Residual-based least-squares; good with priors. Uses an analytic parameter Jacobian when the model supplies one. | | `iws.optimizers.ScipyLsqLinear()` | Bounded **linear** least-squares (single-solve BVLS/TRF). Use as the inner solver of a `Nested` fit when the inner residual is linear in its parameters. | | `iws.optimizers.DifferentialEvolution()` | Global, no gradients required | | `iws.optimizers.CMAES()` | Global, many local minima, well-tested defaults | | `iws.optimizers.PSO()` | Global, parallelisable population search | | `iws.optimizers.BayesianOptimization()` | Expensive evaluations, ≤ \~10 parameters, small budget | | `iws.optimizers.TuRBO()` | Expensive evaluations run in parallel batches; higher-dimensional problems | | `iws.optimizers.SOBER()` | Wide parallel batches using quadrature-style recombination | See [Objective Functions](/pipelines/data-fitting/objective-functions) for the cost-function options. The surrogate optimisers (`BayesianOptimization`, `TuRBO`, `SOBER`) are available with no extra installation required — their `torch`, `botorch`, and `gpytorch` dependencies are handled automatically when the fit runs on the platform. ### TuRBO for expensive parallel problems When each evaluation is expensive, `TuRBO` proposes a batch of candidates per round and adapts a trust region around the current best point. Worker count is decided by the platform when the fit runs, so set the warm-up (`n_initial`) to the batch size you want evaluated in the first round: ```python theme={null} import ionworks_schema as iws fit = iws.DataFit( objectives=objectives, parameters=parameters, optimizer=iws.optimizers.TuRBO( max_iterations=12, population_size=64, algorithm_options={"noise_floor": "low", "n_initial": 64}, ), ) ``` `DataFit` no longer takes `parallel`, `num_workers`, or `max_batch_size` (and `AskTellOptimizer` no longer takes `async_mode`). Parallelism is decided by the platform when the fit runs, not set in the config. Stored configs that still carry the removed fields are migrated automatically at parse time, so no action is required for existing saved fits. Useful `algorithm_options` keys for surrogate optimisers include `n_initial` (warm-up sample count), `noise_floor` (`"low"`, `"standard"`, or a `(lo, hi)` interval), and — for TuRBO — trust-region controls (`tr_length_init`, `tr_length_min`, `tr_length_max`, `tr_success_tolerance`, `tr_failure_tolerance`, `n_candidates`). ### Strict option validation `algorithm_options` is validated against the optimiser you chose. Unknown keys — including typos — are rejected at submission time rather than silently ignored, so misconfigured fits fail fast instead of running with default behaviour: ```python theme={null} # Raises a validation error: "noise_flor" is not a recognised TuRBO option. iws.optimizers.TuRBO(algorithm_options={"noise_flor": "low"}) # AskTellOptimizer is the low-level class behind CMAES(), PSO(), XNES(), etc.; # the named optimisers above are thin wrappers that set `method` for you. # Raises a validation error: BO options passed to a CMAES fit. iws.optimizers.AskTellOptimizer( method="CMAES", algorithm_options=iws.optimizers.BayesianOptimizationOptions(n_initial=32), ) ``` The same check applies whether you pass a raw dict or one of the typed wrappers (`CMAESOptions`, `PSOOptions`, `DEOptions`, `XNESOptions` — for the `XNES` optimizer, available via `iws.optimizers.XNES()` / `AskTellOptimizer(method="XNES")` — `BayesianOptimizationOptions`, `SOBEROptions`, `TuRBOOptions`). Prefer the typed wrappers for editor autocomplete and inline documentation of each option. The only exception is `CMAESOptions`, which remains a passthrough to pycma's own option surface. #### No SciPy-style kwargs on native optimizers Native ask/tell optimizers (`CMAES`, `DifferentialEvolution`, `PSO`, `XNES`, `BayesianOptimization`, `TuRBO`, `SOBER`, and the underlying `AskTellOptimizer`) also reject unknown top-level keyword arguments at construction. SciPy-style keys such as `maxiter`, `popsize`, `seed`, and `tol` are **not** accepted — they previously had no effect and now fail validation immediately, so misconfigured fits surface at construction rather than silently: ```python theme={null} # Raises a validation error — "Extra inputs are not permitted" for each unknown # key (maxiter, popsize, tol). iws.optimizers.DifferentialEvolution(maxiter=10, popsize=5, tol=1e-6) # Correct: use the documented ask/tell parameters (tol -> population_convergence_tol). iws.optimizers.DifferentialEvolution( max_iterations=10, population_size=5, population_convergence_tol=1e-6, ) # Algorithm-specific settings go in `algorithm_options`. iws.optimizers.CMAES(algorithm_options={"seed": 42}) ``` SciPy-style keys still belong on the SciPy passthrough optimizers (`ScipyMinimize`, `ScipyLeastSquares`, `ScipyDifferentialEvolution`), which forward them directly to the underlying SciPy call: ```python theme={null} # OK: ScipyDifferentialEvolution forwards maxiter/popsize/seed/tol to scipy.optimize. iws.optimizers.ScipyDifferentialEvolution(maxiter=10, popsize=5, seed=0) ``` In short: put iteration, population, and tolerance limits in the named ask/tell arguments (`max_iterations`, `population_size`, `population_convergence_tol`); put algorithm internals in `algorithm_options`; and keep SciPy keywords on the `Scipy*` optimizers. ### Strict objective validation `DataFit.objectives` (and `Validation.objectives`) validates each entry against a discriminated union keyed on the objective's `type`. Configuration mistakes are rejected at submission time with a clear `ValidationError`, instead of being silently ignored or surfacing later as an opaque runtime crash. Objective **instances** (e.g. `iws.objectives.CurrentDriven(...)`) keep working unchanged — both the positional and keyword constructors are preserved. The strict-validation rules apply when you pass raw config **dicts**, which is common for configs loaded from JSON or produced by `to_config()`: ```python theme={null} # OK — concrete type, all keys recognised. fit = iws.DataFit( objectives={ "1C": { "type": "CurrentDriven", "data": "file:.../1C.csv", "options": {"model": {"type": "SPMe"}}, }, }, parameters=parameters, ) # Also OK — the legacy `objective` alias and top-level `model` shorthand # are normalised into `type` and `options.model` before validation. fit = iws.DataFit( objectives={ "1C": { "objective": "CurrentDriven", "model": {"type": "SPMe"}, "data": "file:.../1C.csv", }, }, parameters=parameters, ) ``` The following are now rejected up-front: | Mistake | Example | Why it fails | | ---------------------------- | ------------------------------------------------------------ | -------------------------------------------------------------------- | | Unknown objective type | `{"type": "NotAnObjective"}` | Type isn't a member of the union. | | Missing discriminator | `{"electrode": "positive", "data": "x.csv"}` | No `type` (or legacy `objective`) to dispatch on. | | Unknown inner key | `{"type": "OCPHalfCell", "definitely_not_a_field": 1}` | Schemas use `extra="forbid"`; typos are caught early. | | Non-concrete type | A generic/base name instead of a specific objective | Only concrete objective types (e.g. `OCPHalfCell`) validate. | | Conflicting discriminators | `{"type": "OCPHalfCell", "objective": "CurrentDriven", ...}` | Raises `Conflicting objective discriminators` — fix one of the keys. | | Non-dict, non-instance value | `{"bad": "RMSE"}` / `{"bad": 123}` | Objective values must be a config dict or an objective instance. | In a raw config dict the data key is `data`, not `data_input`. The Python constructor exposes it as the `data_input` keyword argument, but `to_config()` serialises it to `data` — so hand-written dicts and JSON configs must use `data`. Passing `data_input` in a dict is now rejected as an unknown key. `DesignObjective` configs are not accepted on the `DataFit` runtime path: they run on the separate design-optimization pipeline. Passing a `DesignObjective` dict to `DataFit.from_schema` raises a `UserConfigurationError` pointing you at the design-optimization pipeline instead of crashing later. ## Multi-start For problems with multiple local minima, run several optimisations from different starting points: ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, multistarts=20, ) ``` The pipeline generates initial guesses (Latin Hypercube by default), runs them in parallel, and returns every result sorted by cost. ### Choosing the initial-guess sampler By default multi-start uses Latin Hypercube sampling, which spreads initial guesses evenly across the parameter bounds. Override the sampler with `initial_guess_sampler` when you want a different sampling scheme — for example, plain uniform sampling for a baseline comparison: ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, multistarts=20, initial_guess_sampler=iws.distribution_samplers.Uniform(), ) ``` Available samplers: | Sampler | Use for | | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | | `iws.distribution_samplers.LatinHypercube()` | Default. Stratified samples that cover the parameter space more evenly than independent draws — recommended for most fits. | | `iws.distribution_samplers.Uniform()` | Independent uniform draws across the bounds. Useful as a baseline or when you want IID samples. | The sampler is validated against a discriminated union at submission time: passing an unknown sampler `type` or an unrecognised key raises a `ValidationError` immediately instead of failing later in the run. ## Nested (variable-projection) fits `iws.optimizers.Nested` is a bi-level optimizer: an **outer** optimizer searches over a chosen subset of parameters, and for every outer trial point an **inner** optimizer is run to optimality over the remaining parameters. This is the variable-projection formulation — the outer sees a concentrated objective $g(x_\text{outer}) = \min_{x_\text{inner}} f(x_\text{outer}, x_\text{inner})$ and the inner absorbs the parameters that are cheap to fit conditionally. Use `Nested` when: * One subset of parameters enters the model **linearly** (or near-linearly) and can be fit in closed form with a bounded least-squares inner solve, while the rest are nonlinear. * The full joint search is ill-conditioned or slow, but the reduced outer problem is well-behaved. * You want a global outer search (e.g. CMA-ES or `Nelder-Mead`) with a fast local inner refinement at every point. ```python theme={null} import ionworks_schema as iws fit = iws.DataFit( objectives=objectives, parameters=parameters, optimizer=iws.optimizers.Nested( parameters=["Negative particle diffusivity [m2.s-1]"], optimizer=iws.optimizers.ScipyMinimize(method="Nelder-Mead"), inner=iws.optimizers.ScipyLeastSquares(), ), ) ``` `parameters` lists the fit-parameter names this level's outer optimizer controls; every other free parameter in the `DataFit` is delegated to `inner`. `inner` can itself be a `Nested` for deeper nesting. The concentrated objective can be non-smooth where the inner optimum is non-unique, so derivative-free or finite-difference outer optimizers (`Nelder-Mead`, `CMAES`, `DifferentialEvolution`) are recommended for the outer slot. Samplers are rejected at construction — both slots must be optimizers. ### Bounded linear inner solves with `ScipyLsqLinear` When the inner parameters enter the residual **linearly** (for example, a linear combination of basis functions with bounded coefficients), pair the `Nested` estimator with `iws.optimizers.ScipyLsqLinear` as the inner solver. It reaches the bounded optimum in a single `scipy.optimize.lsq_linear` call rather than iterating with Gauss-Newton, and warm-starts its active set across successive outer evaluations so the concentrated objective stays smooth: ```python theme={null} optimizer=iws.optimizers.Nested( parameters=["k_nonlinear"], optimizer=iws.optimizers.CMAES(), inner=iws.optimizers.ScipyLsqLinear( method="bvls", # or "trf" warm_start=True, # reuse the previous active set (default) ), ) ``` Use `ScipyLsqLinear` only when the residual really is linear in the inner parameters; for nonlinear inner residuals, keep `ScipyLeastSquares`. ### Analytic parameter Jacobian When the outer or inner slot is a least-squares optimizer (`ScipyLeastSquares` or `ScipyLsqLinear`), the pipeline supplies an **analytic residual Jacobian** $J(x) = \partial r / \partial x$ built from the model's parameter sensitivities. That replaces SciPy's finite-difference Jacobian, so each iteration takes one solve plus one sensitivity evaluation instead of $O(n_\text{params})$ solves. In practice this makes gradient-based least-squares fits substantially faster and more robust to noisy finite-difference steps, especially inside a `Nested` variable-projection loop. No configuration is required — the analytic Jacobian is used automatically when the objective and model support it, and the fit transparently falls back to finite differences otherwise. Look for `using analytic residual Jacobian` in the fit logs to confirm it's active. ## Runtime options `iws.DataFit` accepts `options`, which tunes *how* a fit executes rather than what it fits. Every key is optional. | Key | Default | Description | | -------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `seed` | engine picks one from the clock | Random seed, for reproducible multi-start initial guesses and stochastic optimisers. Must be in `[0, 2**32 - 1]` — the range numpy accepts. | | `low_memory` | engine decides | Append a log entry only when the cost improves the best-so-far by ≥0.1%. Left unset, the engine enables it for deterministic optimisers and disables it for probabilistic ones. | | `max_iterations` | `None` | Per-job iteration cap. Must be positive. Only takes effect when the model's `convert_to_format` is `"casadi"`. | | `maxtime` | `None` | Per-job wall-time budget in seconds. Must be positive and finite. With multi-start the total may exceed this, since many jobs run. Only takes effect when `convert_to_format` is `"casadi"`. | | `validate` | `True` | Before the fit starts, check that every fit parameter is actually used by at least one objective's model. Catches a misspelt or orphaned parameter name up front, rather than after a fit that could never have moved it. | | `skip_objective_callbacks` | `False` | Skip the per-objective callbacks that simulate the model at the initial guess and at the fitted parameters. Faster, but leaves the initial/final fit results unpopulated. See the note below for cluster behaviour. | Pass a dict: ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, options={"seed": 42, "low_memory": True, "maxtime": 600}, ) ``` or `iws.DataFitOptions`, which gives you editor autocomplete and inline documentation of each field: ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, options=iws.DataFitOptions(seed=42, low_memory=True, maxtime=600), ) ``` Only the keys you set are sent. `iws.DataFitOptions().to_config()` is `{}`, so leaving a field alone defers to the engine's own default instead of freezing today's default into a stored config. ### Unknown option keys are rejected Options are validated when you build the config, not part-way through the job. A misplaced or misspelt key fails immediately: ```python theme={null} # Raises a validation error: "Extra inputs are not permitted" for multistarts. # multistarts is a DataFit field, not a runtime option. iws.DataFit(objectives=objectives, parameters=parameters, options={"multistarts": 5}) # Correct — it belongs one level up. iws.DataFit(objectives=objectives, parameters=parameters, multistarts=5) ``` The check applies whether you pass a raw dict or `iws.DataFitOptions`, and the value constraints are enforced too — `seed=-1`, `max_iterations=0`, or a non-finite `maxtime` all fail at construction. This mirrors the [strict validation on optimiser options](#strict-option-validation): the same principle, one level up the config. Pipelines submitted to the Ionworks cluster enable `skip_objective_callbacks` by default to reduce simulation cost. Set it explicitly to `False` in `options` if you need the initial- and final-fit simulation results returned with the run. ## Objective-level parallelism `DataFit` exposes `objective_parallelism` as a top-level field — a debugging escape hatch for objective-level scheduling. It is a direct `DataFit` argument, **not** a `Runtime options` key: | Value | Behaviour | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | | `"auto"` (default) | The execution engine decides. Multiple non-trivial objectives are flattened and evaluated in parallel; single-objective fits stay sequential. | | `"on"` | Force objective-level parallelism. Useful when the auto heuristic underestimates the per-objective cost. | | `"off"` | Force sequential objective evaluation. Useful for reproducing a single-threaded baseline or when debugging an objective in isolation. | ```python theme={null} fit = iws.DataFit( objectives=objectives, parameters=parameters, objective_parallelism="off", # force sequential evaluation while debugging ) ``` Leave it at `"auto"` unless you have a specific reason to override. The setting only affects scheduling — it does not change the cost the optimizer sees. ## Troubleshooting model failures If an objective's model can't be set up or evaluated during a fit — for example because the parameter set is incomplete, a custom model can't be discretised, or a state-of-charge initialisation fails — the pipeline raises a `ModelError` that names the offending objective and the underlying cause: ```text theme={null} Failed to set up the model for objective 'test_1C': Parameter 'Negative electrode thickness [m]' not found. ``` `ModelError` is distinct from a generic `CONFIGURATION_ERROR`: the failed job is tagged with the `MODEL_ERROR` code so the message can be shown to you directly without being routed through internal error monitoring. The fix is almost always to your fit configuration — supply the missing parameter, widen unrealistic bounds, or adjust the model options on the objective — rather than something to report. Already-clear errors pass through unchanged: solver-side numerical failures still surface as `pybamm.SolverError` / `SOLVER_ERROR`, and parameter-lookup failures still surface as `ParameterNotFoundError`. Those continue to be handled by the optimizer's normal fallback (NaN cost, skipped step) when they occur inside an evaluation, so a single bad iteration won't kill the whole fit. ### Every evaluation returned a non-finite cost If **every** cost evaluation across the whole fit is non-finite (NaN or ±infinity), the optimizer has no usable signal to move away from the initial guess. In that case the pipeline fails the fit explicitly instead of silently returning the initial parameters as if the fit had succeeded. The failure surfaces as: ```text theme={null} No fit was found: every evaluation failed (minimum cost is inf). Likely cause: ... ``` The `Likely cause:` clause is filled in with one of three diagnoses: * **The target data contains non-finite (inf/NaN) values**, so the cost can never be finite. The offending variables are named in the message. This usually means an invalid problem setup — for example a resistance or overpotential computed on a zero-current rest step, or inf/NaN in the input data. Remove or correct those points. * **A model evaluation raised an exception** (the exception type and message are included). Fix the underlying model, parameter, or bounds problem it reports. * **The model produced non-finite output for every parameter set** even though the target data is finite. Check that the model gives finite outputs at the initial parameter values and across the bounds (no divide-by-zero or invalid parameter combinations). If every evaluation returned the *identical* cost, the message says so — either the fit parameters don't affect the model outputs, or every solve fell back to the same failure cost. The fix is almost always to your fit configuration: tighten the bounds around a point where the model is known to solve, verify the objective data aligns with the experiment, or run a single simulation at the initial guess to confirm it produces finite output before submitting the fit. A fit where at least one evaluation is finite is unaffected — the optimizer's NaN fallback still handles occasional bad iterations. ## Retrieving results ```python theme={null} client.pipeline.wait_for_completion(submission.id) result = client.pipeline.result(submission.id) print(result.element_results["fit"]) ``` `result.element_results["fit"]` is a dict keyed by the data-fit's outputs (best parameter values, final cost, and any logged trajectories). See [`packages/ionworks-api/examples/pipeline/datafit.py`](https://github.com/ionworks/ionworks-api/tree/main/examples/pipeline/datafit.py) for an end-to-end example. Cost-function math, identifiability, multi-start strategy. Pick the right cost for your data shape. Stabilise fits with Gaussian priors. Quantify which parameters the fit actually constrains.