Iana Atanassova, Ph.D.

We present UnScientify, a system designed to detect scientific uncertainty in scholarly full text. The system utilizes a weakly supervised technique to identify verbally expressed uncertainty in scientific texts and their authorial references. The core methodology of UnScientify is based on a multi-faceted pipeline that integrates span pattern matching, complex sentence analysis and author reference checking. This approach streamlines the labeling and annotation processes essential for identifying scientific uncertainty, covering a variety of uncertainty expression types to support diverse applications including information retrieval, text mining and scientific document processing. The evaluation results highlight the trade-offs between modern large language models (LLMs) and the UnScientify system. UnScientify, which employs more traditional techniques, achieved superior performance in the scientific uncertainty detection task, attaining an accuracy score of 0.808. This finding underscores the continued relevance and efficiency of UnScientify’s simple rule-based and pattern matching strategy for this specific application. The results demonstrate that in scenarios where resource efficiency, interpretability, and domain-specific adaptability are critical, traditional methods can still offer significant advantages.

While science is often portrayed as producing reliable knowledge, scientists tend to express caution about their claims, acknowledging nuances and doubt, all the more so in novel domains of research paved with unknowns. Uncertainty is an intrinsic aspect of scientific inquiry, particularly in recent fields such as astrobiology, which tackles numerous hard questions about the origin, evolution, and distribution of life on Earth and elsewhere. Mapping uncertainty in science matters for achieving a more accurate understanding of scientific knowledge. It also helps identify research domains at the frontiers of knowledge where unknowns are the most salient. In this article, we investigate the presence, distribution and context of uncertainty in the field of astrobiology. We analyze a comprehensive corpus of 3,698 research articles published in three major journals in the domain from 1968 to 2020. We use a linguistically motivated approach to identify expression of uncertainty in article full text. The corpus was further segmented into research topics using Latent Dirichlet Allocation (LDA) to investigate variations in uncertainty across subfields and over time. Our findings show that, while uncertainty has remained relatively stable over the 50 years covered by the corpus, constituting 20–25% of sentences on average, it varies significantly across research fields, highlighting areas where unknowns, doubts and speculations are more prevalent. The analysis also highlights relationships between expression of uncertainty and rhetorical structure. Indeed, higher uncertainty levels were observed in the beginning (introductions) and towards the end (conclusions) of research articles, while middle sections contained less uncertainty. Abstracts also tended to express a slightly higher level of uncertainty compared to main texts, especially with greater variability, suggesting their role in summarizing research and highlighting unknowns. To investigate the context of uncertainty, a lexical analysis was conducted to identify nouns most frequently associated with uncertainty within each topic. Terms such as “life,” “planet,” and “Mars” were found to be strongly associated with uncertainty. Conversely, terms related to experimentation and measurement, such as “sample” and “spectrum,” were linked to an absence of uncertainty, pointing at a dichotomy between speculative and evidence-based lines of inquiry. The findings contribute to a better understanding of the field of astrobiology and exemplify the relevance of the proposed method to identify uncertainty- related concepts in corpora of full text publications. They also offer a foundation for future comparative studies across disciplines.

Scientific uncertainty is an integral part of the research process and inherent to the construction of new knowledge. In this paper, we investigate the ways in which uncertainty is expressed in articles and propose a new interdisciplinary annotation framework to categorize sentences containing uncertainty expressions along five dimensions. We propose a method for the automatic annotation of sentences based on linguistic patterns for identifying the expressions of scientific uncertainty that have been derived from a corpus study. We processed a corpus of 5956 articles from 22 journals in three different discipline groups, which were annotated using our automatic annotation method. We evaluate our annotation method and study the distribution of uncertainty expressions across the different journals and categories. The results show a predominant concentration of the distribution of the scientific uncertainty expressions in the Results and Discussion section (71.4%), followed by 12.5% of expressions in the Background section, and the largest proportion of uncertainty expressions, approximately 70.3%, are formed as author(s) statements. Our research contributes methodological advances and insights into the diverse manifestations of scientific uncertainty across disciplinary domains and provides a basis for ongoing exploration and refinement of the understanding of scientific uncertainty communication.

Purpose – Scientific abstracts reproduce only part of the information and the complexity of argumentation in a scientific article. The purpose of this paper provides a first analysis of the similarity between the text of scientific abstracts and the body of articles, using sentences as the basic textual unit. It contributes to the understanding of the structure of abstracts. Design/methodology/approach – Using sentence-based similarity metrics, the authors quantify the phenomenon of text re-use in abstracts and examine the positions of the sentences that are similar to sentences in abstracts in the introduction, methods, results and discussion structure, using a corpus of over 85,000 research articles published in the seven Public Library of Science journals. Findings – The authors provide evidence that 84 percent of abstract have at least one sentence in common with the body of the paper. Studying the distributions of sentences in the body of the articles that are re-used in abstracts, the authors show that there exists a strong relation between the rhetorical structure of articles and the zones that authors re-use when writing abstracts, with sentences mainly coming from the beginning of the introduction and the end of the conclusion. Originality/value – Scientific abstracts contain what is considered by the author(s) as information that best describe documents’ content. This is a first study that examines the relation between the contents of abstracts and the rhetorical structure of scientific articles. The work might provide new insight for improving automatic abstracting tools as well as information retrieval approaches, in which text organization and structure are important features.

The organization of scientific papers typically follows a standardized pattern, the well-known IMRaD structure (Introduction, Methods, Results, and Discussion). Using the full text of 45,000 papers published in the PLoS series of journals as a case study, this paper investigates, from the viewpoint of bibliometrics, how references are distributed along the structure of scientific papers, as well as the age of these cited references. It shows that, once the sections of articles are realigned to follow the IMRaD sequence, the position of cited references along the text of articles is invariant across all PLoS journals, with the introduction and conclusion accounting for most of the references. It also provides evidence that the age of cited references varies quite a lot by section, with older references being found in the methods and more recent ones in the discussion. On the whole, these results provide insight into the different roles citations have in the scholarly communication process.