Model credibility

TR&D 1 initially started out as a project to develop tools for scalably aggregating data for modeling, organizing this data for model design, and systematically designing models from this data. This resulted in a number of tools that were developed with our collaborators. Some of these include:

1. Datanator:
Datanator is an integrated database of molecular data designed to facilitate integrative analysis of multiple cellular subsystems. To help investigators overcome the lack of complete data about every molecule and reaction in every organism, Datanator includes tools for building “clouds” of data that include data about chemically similar molecules and reactions in phylogenetically similar organisms.

GitHub site: https://github.com/KarrLab/datanator

2. ObjTables:
ObjTables is a community initiative to make supplementary spreadsheets to journal articles – one of the most common methods for sharing scientific data – more reusable. The long-term goal is to create an ecosystem of reusable data for comparative and integrative research.

In year 2.5, TR&D 1 pivoted to developing the biosimulations portal.

GitHub site: https://github.com/KarrLab/obj_tables

BioSimulations is a free platform for sharing and re-using biomodels, simulations, simulation results, and visualizations of simulation results. BioSimulations supports a wide range of frameworks (e.g., logical, Flux-Balance Analysis (FBA), continuous kinetic, discrete kinetic), simulation algorithms (e.g., FBA, SSA), model formats (e.g., SBML), and tools (e.g., COBRApy, COPASI, tellurium).

Biosimulations may be considered a reproducibility portal for biomedical models. Our current work (August 2022) involves curating over 100 models so that a researcher may search for a given paper on biosimulations.org and reproduce the figures found in that paper.

Moving Forward: With reproducibility of biomedical models technically resolved, we are now switching our efforts to a related topic but even more important, model credibility. Model credibility (or lack of) became more obvious during the covid pandemic when many models of covid spread, and infection came under intense scrutiny. Model credibility is related to many factors, including validation, verification, sources of data, biophysical appropriateness, etc. Each of these factors, as well as others, contributes to an overall measure of credibility or trust in a model.

Although technical resolved, many published models remain non-reproducible and we discovered it is extremely hard to persuade authors and journals to ensure that their models are reproducible. Instead, a measure of a model’s credibility may be of more interest to researchers.