Conference Call

August 18, 2021 Slides

By phone:

Mike Beakes, Thad Bettner, Erica Bishop, Matt Brown, Erin Cain, Heather Casillas, Alison Collins, Megan Cook, Flora Cordoleani, Holly Dawley, Logan Day, Mary Beth Day, Matt Dekar, Jane Dolan, Adam Duarte, Jim Earley, Laurie Earley, Rebekah Funes, Michael Harris, Jason Hassrick, Kimberly Holley, Morgan Kilgour, Julie Leimbach, Priscilla Liang, Duane Linander, Keith Marine, Cyril Michel, Bryan Matthias, Sonya Nechanicky, Kirk Nelson, Jim Peterson, Corey Phillis, Regina Rieger, Derek Rupert, Alicia Seesholtz, Ian Smith, Kate Spear, Cory Starr, Susan Strachan, Erin Strange, Pamela Taber, Mark Tompkins, Spencer Walden, Heidi Williams, Rod Wittler, Michael Wright

Meeting Summary

The August 2021 SIT Background Workshop provided an overview of the evolution of the SIT structured decision making process, the Chinook salmon decision support models, and the development of priorities for the Near-term Restoration Strategy. The workshop closed with an overview of where we are headed with the SIT Process for the rest of 2021 and what to expect at the August 25 SIT meeting.

Detailed Meeting Notes

Below is an outline of the content presented during the workshop. Please refer to the workshop slides for full details, diagrams, and graphs.


SIT Background Workshop Goals

  • Provide refresher on:
    • SIT structured decision making process
    • Chinook salmon Decision Support Models
    • CVPIA Near-term Restoration Strategy priorities
  • Provide interactive forum for questions and answers
  • Equip SIT members with necessary background to support understanding of updated model results at Aug 25 call

Three phases to CVPIA Process

  • CVPIA Process Phase 1 (2013-2014)
  • CVPIA Process Phase 2 (2015-2018)
  • CVPIA Process Phase 3 (2019-present), covering the development of the Near-term Restoration Strategy

Why model decisions?

  • Quantitative methods should guide and support decision making
  • Not a replacement for human intuition and subjectivity
  • At least- be an intelligent consumer of models for decision making
  • Collaboration on model development is revealing
  • Models for conservation are not about modeling- they are about conservation

The CVPIA Process: Phase 1

  • Started in 2013
  • Series of workshops
  • Identified objectives, built prototype models (3 Chinook runs, 2 sturgeon)

Decision Context

  • To quickly and efficiently meet the anadromous fish natural production goals of the CVPIA by identifying, restoring, and conserving essential habitats and processes while recognizing and responding to the needs of the public

Fundamental Objectives

  • Naturally reproducing self-sustaining population
  • Optimize the use of project funds

Fundamental Objective Attributes

  • Identify fundamental objective attributes that could be quantified: spatial structure, abundance, adult/juvenile natural production, diversity
  • Quantifying attributes: age structure, return timing, juvenile size distribution, etc.
  • Build model that managed attributes

Spatial Dimensions

  • Step down process: coarse resolution, fine resolution
  • Time step is approximately 2 months

Coarse Resolution Chinook Model

  • Iterative process: Approximately 10 iterations, Predicted the fundamental objective attributes

Model Parameterization

  • Where did we get the information: empirical data, published reports, expert elicitation (a lot)

Phase 1 Model Implementation

  • Calibrated model with estimated production (ChinookProd)
  • Randomly simulated flows
  • Evaluated management alternatives
  • Sensitivity analysis


  • Process for estimating model parameters
  • Minimize difference between model estimated and observed values

What did we learn in Phase 1?

  • The process, data availability, key uncertainties with limited information, who needs to be in the room

The CVPIA Process: Phase 2

  • Science Integration Team (SIT) created
  • Series of workshops, calls
  • Open to everyone, included agency reps, NGOs, consultants, etc.
  • Identified objectives, built new prototype model, refined models, scenario evaluation

SIT Governance Guidelines

  • How do we decide what we decide?
  • Needed guidelines for how to resolve conflicts, disputes

SIT Proposal Process

  • How do we change/improve things?
    • Example: created steelhead models, SIT members disagreed with elements, identified need to provide alternative to move forward in the SDM process
  • Must be science-based
    • Provide References

Fall Chinook Salmon Base Model

  • Why four size classes?
    • To match monitoring data (think adaptive management)
    • Cutoffs based on screwtrap data

Model overview

  • Adult dynamics (see slides for diagrams)
  • Juvenile dynamics (see slides for diagrams)

Model parameterization

  • Where did we get this data?
    • Analyses of empirical data
    • Flows: CalSim (monthly time step)
    • Temps: HecRes/stat. Models
    • Published reports
    • Expert elicitation (very, very little)
    • Lots of help

Model Details

  • Equations, parameter values, sources
  • Listed in supplementary materials, public access
  • Link to SIT website to explore data/data sources/visualizations
  • R Package for running models

Phase 2 Model Implementation

  • Calibrated model with estimated escapement and juvenile catch data
  • Climate type
  • Output at year 5 and 20
  • Management alternatives
  • Increased complexity of management actions, which resulted in SIT evaluating 50,544 outputs (was very difficult)
  • Sensitivity analysis

Phase 2: Addressing more fundamental objectives

  • Developed winter/spring run Chinook model
    • Expert workshops
    • Conceptual models
    • Parameterization
  • Sturgeon and steelhead
    • Expert workshops
    • Conceptual models
    • Roadblocks

Phase 2 (end) More expansions

  • Separated Delta into 13 zones
  • Separate surviving model via South Delta, came from adaptive management group

Scenario Planning in Phase 2

There were many challenges with scenario planning in phase 2

  • Multiple related objectives
    • Correlated outputs (redundant)
    • Mixing fundamental and means
  • Striving for realism in modeling
    • Very complicated models
    • Lack of understanding
  • Imprecisely defined decision alternatives
    • Proportion existing habitat
      • What’s the issue? Lots of habitat, increasing by large amount
    • Proportion theoretical habitat
    • How are actions actually implemented?
      • Is the program able to implement 26 things across all watersheds every year?
  • Results subject to wide interpretation
    • Excessive work
    • Limited participation
    • Disagreement/conflict

Phase 3: Structured Decision Making

  • Phase 3 was about thinking about what really matters (do we really want to look at all those outputs? Do they all really matter?) and identifying realistic alternatives
  • SDM Steps:
    • Identify the decision situation and objectives
    • Identify management alternatives
    • Break down and build a model of the problem
    • Identify the best alternative
    • Evaluate model sensitivity
    • Implement management alternative(s)

Now for the simplifications

  • In revisiting metrics, reviewed what items folks really focused on in previous discussions. Typically it was natural production, juvenile size, and juvenile abundance.
  • Simplified down to a couple objectives: adult production and juvenile biomass with a parity constraint (i.e., doing actions in each diversity group)
  • Landed on the Objective: maximize the number of adult equivalents

Simplifying Management Actions

In looking at all of the actions the SIT previously considered, examined what the actions were actually doing and ended up binning into two categories:

  • Increase habitat
    • Increase minimum flow
    • Remove barriers
    • Physically restore habitats
  • Increase survival
    • Decrease temperatures (flows)
    • Reduce/screen diversion
    • Reduce predation
    • Pulse flows
  • Developed units of effort that corresponded to what people were doing when implementing actions on the landscape

Solving the Decision Model

  • Objective: Use the model to find the optimal restoration strategy
  • Simulate every combo of actions for each tributary to find optimal sequence of actions
  • Assume 4 potential actions per tributary, 20 years

State dependent policy development

  • What is the optimal sequence of actions?
    • e.g., alternate spawning and rearing habitat every year or focus on spawning for multiple years and then focus on rearing for multiple years?
    • Would one or the other help us learn? This is where adaptive management comes into place
  • Task
    • Develop a state-dependent policy for each tributary
    • Solve the decision model
  • Markov Decision Process (MDP)
    • Need: recurrent decision, objective, current system state, decision alternatives, model
    • Process of repeating decisions and observing system states, and then making decisions based on that

What do we really need to know?

  • Over three versions of the Chinook model, sensitivity analysis revealed similar results for the most influential parameters and inputs – see slides for full details

Focus on What Really Matters

  • What tributaries should be treated as different systems?
  • separated tributaries into groupings in acknowledgement that fish are traversing different amounts of habitat to get to the ocean
  • Then looked at what the habitat is doing on average in the groupings. In some groups, juvenile habitat is stable, in others it is decreasing or increasing.

State-Dependent Policy Development

  • Use decision-support model to develop transition probability matrices for each group/tributary type
  • Ran a bunch of simulations for each group and “solved” using stochastic dynamic programming (maximize the cumulative utility value through time)
  • this creates a lookup table to program when running the candidate strategies

Forward Simulation #1

  • What strategy results in the highest cumulative utility? (see slides)
  • Using the policy plots/lookup tables for each of the groups, did a forward simulation to see what patterns emerged
  • Simulated 20 years
  • Five actions implemented per year
  • Evaluated two alternative rulesets: (1) maximize fish returns, and (2) maximize fish returns in each diversity group

Forward Simulation #2

  • Then ran a second simulation with a different ruleset
  • Simulated 20 years
  • Five actions implemented per year
  • Only considered habitat actions in the mainstem sections
  • Rulesets
    • No action
    • Maximize Adults: Baseline
    • Maximize Adults: Work in each diversity group
    • Maximize Adults: Don’t work in hatchery streams
    • Maximize Adults: Only work in hatchery streams
  • Patterns started to emerge in the identified actions in the simulated scenarios (e.g., floodplain and rearing habitat actions in mainstem sections)

Evaluate Candidate Restoration Strategies

  • Based on patterns the SIT saw in solving the decision model, SIT came up with 13 candidate restoration strategies (plus a no action strategy)

Things to remember

  • What are actions doing?
    • Spawning habitat: if limiting, increases no. fry
    • In-channel (perennial) juvenile habitat: if limiting, hold and grow juveniles
    • Floodplain (seasonally inundated) juvenile habitat: if limiting, hold and grow juveniles with higher survival and growth rates over short intervals
    • Increase survival: reduce/screen diversion
  • All strategies have 5 actions per year for 20 years
    • Exception: Strategy 11 has 6 actions per year
  • Winter-run
    • Battle Creek population not included
    • Juveniles rearing in non-natal tributaries not included
  • Spring-run
    • Feather and Yuba fish not included in SIT metrics (they are still included in the model)
      • Lack of segregation between spring and fall run (decided in 2018)

Model Results

  • Utility scores used to rescale results and allow direct comparisons between runs (best = 1, worst =0)
  • Natural production utility scores:
    • Strategy 5 and 6 best for winter-run
    • Strategy 7 best for fall-run
    • Strategy 10 best for spring-run but low for winter and fall-run.
  • Juvenile biomass utility scores:
    • Strategy 2 best for spring-run
    • Strategy 9 and 10 best for winter-run
    • Strategy 10 best for fall-run
  • Combined utility scores: combines utility scores for all runs. Advantage is that if any strategy is zero for one run, it results in a zero score for all runs.
    • Still seeing similar patterns
    • Strategies 5, 6, 9, 10 performing the best
  • Relative Loss scores indicate how much we lose if we select the strategy (best = 0 for no loss, worst = 1)
    • Natural abundance: for strategies 5 and 6, there is some loss for spring run but not much loss for fall and winter. Strategy 9 there is some loss for winter
    • Juvenile biomass: similar patterns
  • SIT Discussion of Results
    • The strategies that had the most benefit with least loss were generally 5, 6, 7, 10.
    • Looking at what these strategies entailed, identified the type of restoration actions and the geographic areas.
    • This conversation generated discussion about overall strategy. Talked about ecological principles, such as connectivity
    • NTRS priorities developed based on interpretation of model output as well as expert knowledge of ecology.

Sensitivity Analysis

Sensitivity analysis is used to identify which uncertainties influence average natural spawner abundance the most as well as which uncertainties change the optimal strategy (based on natural spawner abundance).

  • Common top uncertainties: model parameters (reducible with focused monitoring effort)
    • Juvenile survival
    • Juvenile growth
    • Adult survival
    • Reproduction
    • Juvenile territory size
  • Common top uncertainties: model inputs (reducible through expert meetings; will be refined as new monitoring data submitted)
    • In channel rearing habitat
    • Floodplain habitat
    • Spawning habitat
  • The sensitivity analysis results combined with SIT expert knowledge is what led to the identification of information need priorities

Near-term Restoration Strategy

  • The Strategy includes priority restoration actions for Chinook salmon and priority information needs for Chinook salmon, steelhead, and sturgeon.
  • The Strategy is intended to be implemented over a five-year period so that population-level effects on anadromous fish species can be observed and large-scale restoration efforts can be planned, designed, and implemented.
  • The SIT’s adaptive process enables annual updates if new information improves the decision support models and subsequent prioritization.

What’s Next?

  • At August SIT meeting, will see results of incorporating changes to the Chinook salmon DSMs
  • Review updated Chinook salmon model output
  • Affirm existing Chinook salmon priorities still supported
  • No changes made that would affect priorities for O. mykiss or green and white sturgeon

The Upcoming process

  • Use revised model
  • Recalibration (same)
  • 13 actions + no action
  • 20 years
  • Sensitivity analysis

Outputs for review

SIT can expect to review the following output:

  • Adult returns: valley wide and tributary-specific
  • Juvenile biomass: valley wide and tributary-specific
  • Viability: by diversity group
  • Focus of review and discussion will be on relative differences in these metrics between the candidate restoration strategies the SIT originally evaluated in developing the priorities for the Near-term Restoration Strategy


Julie Leimbach: Interested in conceptual models and policy plot lookup tables.
Megan Cook: conceptual models presented at January SIT meeting and available in those meeting slides ( Intend to post as an independent resource on the SIT website as well.
Jim Peterson: lookup tables will be available

Rod Wittler: what differs between tributaries for policy plots?
Jim Peterson: There are patterns in habitat, for some tributaries, starts high, drops down, some starts lows, goes up, some stay relatively flat

Rod Wittler: How sensitive would the model be to different CalSim inputs?
Jim Peterson: Probably pretty sensitive because they drive habitat availability and water temperature models
Adam Duarte: Capture diversity of years- using one CalSim run

Jason Hassrick: Seems like mainstep Sac habitat-limited, if using CalSim 2, what are constraints in terms of whether habitat is connected or not?
Mark Tompkins: Habitat input is a flow to area curve, that curve is generated by hydrologic modeling and habitat suitability modeling applied to those hydraulics. If it shows up as an input-suitable acre to lifecycle model, would be accessible. Real limit is that because it is a monthly average flow, it means that you’re going to miss some inter-month availability. Some days whether you’re over or under estimating habitat available because CalSim is calculated monthly.

Derek Rupert: Representing Clear Creek- likes that this gives us marching orders to what to go after. If more fine-tuned, for example, if we need juvenile-rearing habitat (further define space/month), would be helpful to define what to provide.
Jim Peterson: That is possible
Adam Duarte: It was an intentional choice not to go sub-tributary and allow the local experts to determine the optimal places to work within a watershed.
Rod Wittler: We should discuss this further. Process has been about doing a coarse resolution model at system scale that could support local watershed groups doing a finer scale modeling within their watershed, but have not yet connected those and that option still remains.