Research on Research

Definition

The goal of Research on Research (RoR) is to study research systems using interdisciplinary methods, with the ultimate goal of increasing their productivity, quality, and cost-benefit.  Each one of the components of this definition are discussed in more detail in the following sections.

Components

Systems

Research systems are defined as a number of elements put together to achieve a purpose.  RoR models research systems, these models presenting purposes such as understanding their behavior and predicting their performance.  Systems are categorized within two broad levels:  Micro and macro systems.  Micro systems are defined as teams of researchers working together to accomplish one or more research projects.  Macro systems are defined as research policy makers at the institutional or governmental level coordinating policies that will lead to research projects. 

Interdisciplinarity

Systems are modeled using a combination of methods from multiple disciplines.  Examples of such disciplines include:

• Computational ontologies
• Rhetoric of science
• Cognitive psychology
• History of science
  
• Mathematical modeling
• Philosophy of science
• Computer simulation
• Text mining

Science and operations

RoR has both a scientific and operational focuses.  The scientific focus is achieved by the construction of scientific models, or representations with a specific purpose.  These models are then used to develop applications with practical purposes, usually to improve the systems' quality, productivity, or cost-benefit.  Operationalization is usually achieved through software development and implementation, workflow mapping followed by implementation of standard operating procedures, and needs assessment followed by research training.

Overview

The following picture outlines the general integration among the concepts of systems (micro or researcher level and macro or research policy level), science and operations, and interdisciplinary methods.

Examples in each of the inner quadrants represent the intersection of these four areas:

• Manuscript structure:  The problem approached in this quadrant is that novice researchers (researcher level) have learning how to write manuscripts.  Before we attempt to solve this problem, it is important to understand the underlying structure of scientific manuscripts (science).  This study is conducted through interdisciplinary studies involving linguists, clinical researchers, and computer ontology specialists.  The end-product is a model describing how different sections of the manuscript are usually structured
  
• Training:  Once a good model of the internal structure of manuscripts is developed, then we can translate these models into training courses to novice researchers.  The goal of training course is to ensure that novice researchers can learn how to write better articles in a shorter period of time, thus ultimately increasing their productivity and the quality of their work.
• Text mining of research resources:  The problem approached in this quadrant is that research resources such as databases, experts, or data analysis methods are difficult to obtain from simply scanning the literature.  This creates difficulties since research policy makers do not have a general map of where future investments should occur, creating room for unnecessary duplication.  In order to approach the problem, we create text mining strategies to model the semi-automated extraction of research resource information from millions of full text scientific articles.  These models involve the collaboration of text mining specialists, health service researchers, and statisticians. 
  
• Research mapping: 
  

RoR researchers

RoR researchers are concerned with two primary activities: 

• Science:  RoR researchers build models or research systems using interdisciplinary methods with a well-defined purpose of either explaining how they work or to predict their behavior.  For example, they could use rhetoric of science methods to map clinical trial articles and build visual models with the purpose of understanding how they are usually written and later assist in the training of novice researchers. 
  
• Operations: RoR researchers put these models in operation to see whether they achieve their purpose, for example using the previous models to train novice researchers.  During the operational side, more information is collected about how these models perform, where they work, and where they fail.  For example, a certain model for how clinical trial papers are structured might not work for a field where cluster rather than parallel trials are used, thus requiring a revision of the original model.  This process of building models, validating them in practice, and then collecting data to modify or re-formulate the original models is continuous.

Funding

Funding for RoR usually comes from two sources. 

• Operational RoR projects are usually the main source of funding and involve the use of RoR operational methods to support research teams, departments, national and international research networks, and institutional or government-related activities focused on research policy.  Funds come from both government (grants) as well as industries (contrasts), frequently having RoR researchers in a support position.  Last, training opportunities are also frequent sources of funding.
  
• Research RoR projects are usually through margins from operational RoR projects, many times in projects running in parallel with operational projects.  For example, we will usually run qualitative studies along training courses to evaluate how our methods are performing in practice.
  

Challenges

The main challenges to RoR are the problems inherent to any interdisciplinary effort, namely the lack of a well-defined group of peers and therefore of conferences and peer-reviewed journals.  Other challenges include the lack of well-established metrics to measure performance, since activities are always within the borders of different disciplines.