-
Notifications
You must be signed in to change notification settings - Fork 3
/
Copy pathindex.qmd
149 lines (105 loc) · 6.57 KB
/
index.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
---
title: "RESQUE: The Research Quality Evaluation scheme for psychological research"
listing:
id: sample-listings
template: includes/news.ejs
contents: news
sort: "date desc"
type: default
max-items: 10
feed: true
date-format: "iso"
image-align: "left"
image-placeholder: "images/OA_icon.png"
format:
html:
grid:
margin-width: 500px
---
::: {.column-margin .news}
# News
::: {.smaller}
See all news [here](news.qmd).
:::
<br>
::: {#sample-listings}
:::
:::
The **Research Quality Evaluation (RESQUE)** framework provides recommendations for a responsible research assessment that does not rely on flawed metrics such as the journal impact factor or the h-index.
In alignment with the principles of [CoARA](https://coara.eu), this approach acknowledges diverse academic contributions, prioritizes the quality of work rather than its volume, and integrates qualitative peer assessment with the responsible use of quantitative indicators. All proposed indicators are open-source and reproducible.
## What is RESQUE? Three modular elements
```{=html}
<div>
<div class="grid" style="--bs-gap: 0.5rem;">
<div class="g-col-6 g-col-md-4">
<div class="content-box">
<h4>1. Evaluated Indicators</h4>
<img src="images/R1.jpg" alt="Indicators Image">
<p>✅ A set of evaluated <b>indicators</b> for aspects of research quality.</p>
<p><strong>👉🏼 "Buffett table" approach:</strong> Escape h-index and JIF by choosing valid indicators that suit your research assessment scenario.</p>
</div>
</div>
<div class="g-col-6 g-col-md-4">
<div class="content-box">
<h4>2. Process Suggestion</h4>
<img src="images/R2.jpg" alt="Process Image">
<p>✅ A suggestion for a <b>two-stage process</b> and weights for indicators.</p>
<p>👉🏼 Change the importance of indicators depending on your use case.</p>
</div>
</div>
<div class="g-col-6 g-col-md-4">
<div class="content-box">
<h4>3. Tools Set</h4>
<img src="images/R3.jpg" alt="Tools Image">
<p>✅ A <b>set of tools</b> that can be used (optionally):</p>
<ul>
<li>A <a href="https://resque-framework.github.io/collector-app/">Collector App</a> where applicants can enter their data</li>
<li>Profile creation for CVs</li>
<li>Interactive comparative overview for committees</li>
</ul>
<p>👉🏼 Open source, free to use, privacy aware. Of course you can also include the selected indicators in your existing recruitment system.</p>
</div>
</div>
</div>
</div>
```
Primarily designed to assist hiring and tenure committees, it emphasizes that the **indicators and algorithmic methods serve as tools to support, not replace, decision-making processes**. By automating the generation of relevant candidate information, this approach enhances the effectiveness of human expertise in evaluating potential hires and tenure candidates.
RESQUE aims to provide objective quality and impact indicators for three types of research outputs:
(a) publications of empirical studies,
(b) published data sets and
(c) research software.
::: {.callout-important}
The RESQUE framework is currently in development and the indicators are subject to change. Currently, only indicators for *publications* are available in a beta version.
:::
The assessment scheme is primarily developed for the field of psychology, but might be easily transferred to neighbouring empirical scientific fields.
## Built by the community for the community
The [SCOPE Framework for Research Evaluation](https://inorms.net/scope-framework-for-research-evaluation/) proposes the principle *"Evaluate with the evaluated"*, advocating for a co-creation process within the communities that are being evaluated. RESQUE is developed in that spirit. We started with four target papers that elicited 40 publihsed commentaries, which led to a major revision. (Many critical commenters now are in the developer team.) We had workshops with most field-specific groups within the DGPs, which also led to substantial revisions and the development of discipline-specific expansion packs.
## Resources
### ☑️ Rating schemes: Core set
With the goal to make data entry for applicants and hiring committees as easy and frictionless as possible, we developed the [RESQUE Collector App](https://resque-framework.github.io/collector-app) that allows to provide information for your best research outputs and to export the data in a JSON format.
We plan to provide three rating schemes for ...
- [**RESQUE-Pubs**](tech_doc/pub_indicators.qmd) (Publications, preprints, book chapters, etc.): ✅ v0.4.0, ready for use (but still likely to change).
- [**RESQUE-Data**](tech_doc/data_indicators.qmd) (Data sets): ❌ under development, not implemented in Collector App yet
- [**RESQUE-Software**](tech_doc/software_indicators.qmd) (Research software): ❌ under development, not implemented in Collector App yet
### ☑︎ Rating schemes: Expansion packs
We aim to develop and collect disciplinary **expansion packs (EPs)** with specific indicators that are relevant for a subfield.
If you plan to provide an expansion pack, please [get in contact](team.qmd) with us (we will help to implement that in the web form).
Such EPs ideally are contributed from a legit academic community and stem from a consensus process. Before including EPs in this project, we do a minimal review for suitability.
The subdivisions of the German Psychological Society have been asked to discuss the need for discipline-specific indicators.
**Available Expansion Packs:**
- [Epistemic Goals](tech_doc/EP-epistemic_goals_indicators.qmd)
- [Multi-/Interdisciplinarity](tech_doc/EP-multidisciplinary_indicators.qmd)
- [Practical Relevance](tech_doc/EP-practical_relevance_indicators.qmd)
### 📊 The RESQUE Research Profile
We provide R scripts that ...
1. load and aggregate multiple json files with RESQUE ratings
2. enrich the data with other sources of information (e.g. citation counts & normalized citation counts from [OpenAlex](https://openalex.org), impact metrics from [BIP!](https://bip.imsi.athenarc.gr))
3. provide candidate profiles that can be requested by hiring and tenure committees, or used by candidates as an attachment to their CV or website.
An interactive dashboard allows to compare multiple candidates and to zoom into the profiles of specific candidates.
For more information, see [here](get_profile.qmd).
## Partners and supporters of RESQUE
::: {layout-ncol=3}
:::