How to use artificial intelligence to strengthen scientific processes and scholarly output

The current artificial-intelligence (AI) landscape offers systems that can, at scale, access, process and synthesize information across the vast, unstructured landscape of scientific literature and rapidly growing repository of datasets. Tools using these models can process the thousands of articles deposited into preprint archives every month, categorizing them and extracting claims and evidence. They are also, more slowly, connecting to and understanding datasets in various neuroscience repositories. These new technologies are changing the conditions of knowledge production at an alarming rate. Most people now have instant access to an assortment of AI systems that can easily answer college test problems and solve math olympiad questions, and that are even helping to iteratively solve some complex questions and mathematical proofs.

However, because these systems rely largely on literature, they inherit many of its problems. Scientific publications often exclude null findings and are often imprecise and full of biases. And these systems only have access to the same limiting context that we do when we read a research article—and in fact have much less context by comparison. AI systems are generally not yet trained through firsthand experience, such as conducting laboratory experiments, evaluating serendipitous discoveries, or participating in spontaneous academic discussions at conferences.

AI systems, with all their strengths and limitations, are now available for every part of a scientific workflow, from ideation, literature review and hypothesis testing to memory storage and communication. To date, the field has largely used AI to automate individual tasks and boost personal productivity. But the greater opportunity lies in using these tools collectively—to strengthen scientific communication and improve the quality of scholarly output. Doing that will require infrastructure that captures not just experimental results but their context and provenance, recorded as the work unfolds and not reconstructed later for publication.

Through these AI-enabled research workflows, we can record and disseminate a richer and more precise context with every data element. We can ensure the entire provenance chain is captured and preserved along the many explorations that scientists carry out, whether successfully or not.

A

While carrying out these tasks, these systems have become really good at tracking interactions and computations that we often find difficult to retain and communicate. We are now capturing not just the digital trace of analyses but also the digital trace of questions and iterations. This strengthens scientific communication and scholarly output by automatically recording every detail, enabling us to preserve acts of exploration—many of which do not ever make their way into a publication—into a shared data and knowledge space. Some of this information is passed on to trainees, but most often it is simply filed into drawers or discarded. These exploratory processes generate useful information for the community but often aren’t deemed important enough to formally communicate. 

By broadening AI tools to record and integrate this type of information, we can create a knowledge landscape that directly connects claims to data and that collects evidence accumulated within and across labs. By capturing the interactive human-AI conversations, we effectively capture how scientists view and interpret the evidence. Such tools would bridge the gap between thought, execution and publication. 

This new knowledge system will not be bias-free, but it will be transparent. And it then functions as a new instrument, validating prior findings, generating and testing hypotheses, and connecting complex nuances into a cohesive framework. It is a co-workspace that transcends labs and disciplines, where everyone can both generate and review knowledge as a connected effort. It is a space where a positive and a negative result can co-exist, and one where we always know the full context that was available at the time of the experiments and all the parameters related to the data collection and analyses.

This new paradigm provides near instant accessibility to what worked and what failed, and the detailed provenance behind it. To that end, it creates much shorter loops in the system, enabling rapid digital experimentation and knowledge to iteratively coalesce into understanding. As this information accumulates, one can also contextualize any entity as a product of its time and the threads of knowledge that led to it. Each finding is connected to a question that led to an analysis, which may in turn be connected to prior conversations or findings in literature. Opening access to information in this way leads to a profound change that reshapes how we learn and how we co-design and co-work with such a new instrument. Building this instrument will be like building Wikipedia, but one in which scientific iteration, its provenance and data connections are integral.  

Ultimately, the goal is not whether such systems can code or plan our next experiment faster, but whether we can use them to build scientific processes that are fundamentally more trustworthy and precise than our current ones.