Meeting Site

Empirical Foundations of Science and Innovation Policy

September 16-17, 2013 - Paris, France

Ministère de l'Enseignement Supérieur et de la Recherche,
25 rue le la Montagne Ste Geneviève
70005 Paris cedex 05

Background and Purpose

There is little systematic evidence about the scientific and economic results of research inputs. One primary difficulty is the lack of a data infrastructure that can trace the empirical relationship between initial research funds, scientists who receive funding and the scientific and economic activities of scientists.

A new approach is being adopted to address this inadequacy based on a successful project in the United States, called STAR METRICS.  Several countries are beginning to develop similar models. In addition, a Sloan Foundation funded project (ETOILE) is using STAR METRICS data to examine the activities of research teams in great detail,  the TRICS projects at Imperial College is using a similar people centred approach to examine research teams, and a team of German researchers is investigating the potential of using Max Planck Institute data to describe German science.

A major challenge to these efforts is the challenge of working with fragmented research data and linking scientists to their activities. But parallel efforts could be leveraged.  There are a wide variety of research management systems and systems like CrossRef, euroCRIS and CASRAI are working to agree on data dictionaries and identifiers.  ORCID is working to create a unique identifier for researchers that could be used to unambiguously identify scientists.   

The purpose  of this workshop is begin to build a community of practice: to bring researchers and selected policy makers involved in each project to:

1. Compare approaches and discuss lessons learned
2. Share code and methods
3. Discuss common data issues, such as metadata documentation
4. Develop approaches for making global comparisons

The particular focus here will be to focus on measuring team structure, topic modeling approaches,  output capture (CV and patents) and approaches to managing research data.

 

Meeting Presentations

The Australian Science, Technology and Research Assessment (ASTRA) Project
​Consortia Advancing Standards in Research Administration Information (CASRAI)
CASRAI New Classification Deck
Evaluating & Monitoring at the European Research Council (ERC)
Examining the Outcomes of Investments in Caltech Entrepreneurs (ETOILE)
HEaLth Investments ObServatory (HELIOS)
Ongoing Development of the CERIF Research Information Metadata Model
Flow of people, flow of ideas? Measuring indirect impacts of research through the allocation of individuals across organizations
Background on ORCID
TRansfer of Knowledge at Imperial College (TRIC)
STAR & UMETRICS
 

Agenda

September 16            
12:00-12:30 Registration
12:30-1:30 Welcome, Motivation and goals; Introductions - Jacques and Julia
1:30-2:00

Measuring Science Investments: 

  • Overview of the emerging U.S. approach - Bruce Weinberg
  • Overview of potential ERC approaches - Reinhilde Veugelers
2:00-2:30

Initial Findings from STAR METRICS data

Presentation by ETOILE team

2:30-3:00 Coffee
3:00-4:00

Group discussion from other teams:

  • What they are doing (or what they hope to do)
  • What they found most useful in their approach (or hope to find most useful)
  • Their major challenges (or what they expect to see as their major challenges
4:00-5:00

Describing the results of research:

  • Output capture (CVs, patents, and people placement) - Markus Perkman, Paul Jensen and Erling Barth
5:15-5:45 Suggestions for practical next steps
5:45-6:30 Drinks
7:00-9:00 Dinner

 

September 17

 
8:30-9:00 Coffee
9:00-9:45

Discussion of key issues:

  • Measuring team structure - Jacques Mairesse and Bruce Weinberg
  • Topic modeling - Ghislaine Fillatreau and Rebecca Rosen
9:45-10:15 General Discussion
10:15-10:45 Coffee
10:45-11:30

Managing all the data:

  • Research management: the view from three practitioners - David Baker, Laure Haak and Ed Simons
11:30-12:00 General discussion
12:00-12:30 Suggestions for practical next steps
12:30

Adjourn

 

Project Summaries

 

Background Reading