Is there a formal process for initiating consultation on a model with the Ecosystem Restoration Planning Center of Expertise (ECO-PCX)?
Unless a District maintains specific requirements related to model review and certification/approval (e.g., directing requests through a supervisor), planners can reach out directly to Nate Richards or Greg Miller through phone or e-mail to discuss the process for model certification and next steps.
Is every ecological planning model required be certified prior to the tentatively selected plan (TSP) milestone, assuming it has not already been previously certified?
A planning model should always be certified or approved before a project delivery team (PDTs) starts using it to produce data for use in a study, per the USACE planning model policy. While PDTs working on ecosystem restoration studies generally need to start using their models before the TSP milestone, the certification/approval timeline will differ across studies based on numerous factors. However, there may be cases where model certification/approval happens post-TSP – for example, in cases where mitigation planning is required, model use may not come until later in the study process.
How are certified models adjusted when new scientific findings contradict or otherwise change the variables currently used in a certain model?
The ECO-PCX has a formal reevaluation process in place for model re-approval and recertification, which must take place at a minimum of every seven years. In cases where new scientific evidence requires reassessment of a model before the seven year period expires, the ECO-PCX uses the same reevaluation process to consider the original certification rationale, review the new information and determine how it changes the components of the existing model, and re-certify the model, as appropriate.
Is there a requirement to use scientific literature-based sources in model development and documentation, or can model certification rely solely on expert professional judgement?
The basis of evidence for certifying models ranges widely. Models for rare species or habitat types may be primarily based on expert professional judgement and experience in the field given the sparsity or complete lack of literature or data, and therefore are documented to the best of the PCX’s ability. While the citation of at least some peer reviewed literature is preferred for all model certification, it is sometimes impossible. When this is the case, the ECO-PCX documents uncertainties and potential complications related to the lack of data to ensure that these issues are clearly outlined.
Are there model certification requirements under Section 203 of the Water Resources Development Act (WRDA) of 1986, which authorizes non-Federal interests to undertake feasibility studies of proposed water resources development projects for submission to the Secretary of the Army?
Yes, USACE planning model quality guidance applies. Appendix B, Section j(2) of Engineer Regulation 1165-2-209, Studies of Water Resources Development Projects by non-Federal Interests addresses planning model quality assurance in studies undertaken by non-Federal interests.
Is it more difficult or time consuming to approve models developed by state agencies vs. models developed by USACE?
Models developed by state agencies are not necessarily harder to approve than models developed by USACE, but the effort and time required depends on how robust and well-documented the model is when it gets to USACE. Some models with detailed documentation that have already been peer reviewed move fairly quickly through the process, while others that are still in draft form and that lack an active point of contact at the submitting agency are much more difficult to review and approve.
Additionally, some models developed by others have parameters that are not USACE policy compliant. For example, some mitigation models developed by other federal agencies include a discount rate and risk factor. USACE does not discount habitat units per agency policy, and risk is accounted for in our monitoring and adaptive management requirements.
Do Districts typically collaborate with their resource agency partners (i.e., U.S. Fish and Wildlife Service, National Marine Fisheries Service) when developing their ecological planning models?
As best practices continue to be shared widely, the ECO-PCX is seeing increased coordination between Districts and their resource agency partners on model development and certification. The resource agencies are typically enthusiastic about getting involved in the model development process and sharing their expertise. They bring helpful field experience and peer review skills to the table that can improve model quality and make the certification process easier.
How long does it take for a model to be certified?
It depends on the complexity of the model. From the time you have complete model documentation in accordance with EC 1105-2-412, a typical timeline for model review and approval is 3-6 months. This is estimating the time required to scope the review, identify appropriate reviewers, and review process, review time, USACE responses, and the time required for approval for use by the ECO-PCX Technical Director. Areas where the timelines are typically held up include preparation of complete model documentation, failure to seriously address review comments, funding lapses for PDT and/or model proponent.
What happens if I don’t get the model I use certified?
HQ and the ATRT will notify the PDT that you have not complied with the EC’s and you will be directed to stop and conduct model review before you proceed.
What happens if my model review is conducted and the model is not certified?
If the review is conducted and significant issues arise that you cannot resolve through model modifications, you will be asked to assess the significance of the comments on the planning decision and how they effect your selection, and the vertical team will work with the PDT to resolve the issue.
How much does it cost to certify a model?
The costs vary based on the complexity of the model. Costs for model review have been anywhere from $10 to $65k depending on review scope and review method.
What is the difference between approval for single-one time use and model certification?
The review process is basically the same. The principal difference is the degree to which any significant comments have been resolved and the appropriateness for the model to be used on multiple applications.
We developed a Habitat Suitability Index model using HEP. Does it need to be certified?
Yes. Although HEP is an accepted method, individual HSI models must go through the review process.
Why do we need to review the quality of a model that has been published in a peer reviewed journal?
Models published in journal articles do not address policy, planning and forecasting, and applications. They usually only focus on the technical aspects of a model. They also do not review the use of the model in comparison of alternatives. While journal publications contribute to the weight of evidence in support of the use of a model, the referee process does not address the many facets of the model applications related to Corps planning and policy.
When should I engage the PCX on model review requirements?
As soon as possible. In your Review Plan you should describe and cite models you may potentially use in your study. Prior to the AMM you should have a conceptual model of the system and a firm proposal of model(s) intended to be used for the study. At this point you would develop a model review plan and coordinate with the ECO-PCX. The review should be completed prior to using the model in decision-making.
EC 1165-2-412 expired – why should I be concerned with model review?
Guidance for assuring the quality of Planning models contained in EC 1105-2-412 remains in effect per PB 2013-01 until permanent Planning models guidance is issued as an Engineering Regulation.
We used a group of outside technical experts to develop this model – who then is better qualified to review it?
We recommend that when you are developing your model, you reserve a few of the known experts to be available for review of your model as opposed to developing it.
This model is required by a local/state/Federal agency through statute/regulation/etc., why is model certification required for this model?
The group has a vested interest in the utility of the model. Model review will ensure the model is unbiased and able to differentiate among alternatives consistent with Corps planning and policy. If the local model does not meet Corps policy, it may be that you will need to use two models – one to meet the requirements of your locality and one that meets the requirements of quality for Corps planning and policy.