Academic Auspice
- THE SCRIPT CONCORDANCE TEST ASSESSES THE FOLLOWING:
-
· MEDICAL REASONING IN ACTUAL CLINICAL SITUATIONS
· LEVEL OF ORGANIZATION OF COGNITIVE STRUCTURE
· RATIONAL AND EFFECTIVE USE OF KNOWLEDGE IN DECISION MAKING PROCESSES
- Script Concordance Test
- • FROM MEMORIZING TO REASONING
- The Script Concordance Test is a new tool intended to assess mental processes in medical practice. Developed on solid theoretical grounds and supported by pedagogic research on medical education, this test has proven to be a valid, useful and reliable tool to assess reasoning skills in actual clinical situations, specifically in uncertainty scenarios, and to deduce the quality of performance in actual practice. The main goal of the Script Concordance Test is not to measure knowledge levels, but to assess the degree of organization of clinical knowledge into scripts that are revealed as appropriate in professional practice for physicians to carry out adequate contextual reasoning and take effective decisions when faced with the patient. In this way, the Script Concordance Test reveals the scope and depth of the cognitive networks professionals have built during many years of practice, which are the basis of their clinical reasoning and skills. This innovative tool introduces a major change in the traditional health science assessment paradigms that, so far, have been governed by models such as multiple choice questions, open answer questions and oral tests, which mainly measure knowledge and memorization, in many cases in a decontextualized manner.
- DIFFERENCES BETWEEN THE SCRIPT CONCORDANCE TEST AND MULTIPLE CHOICE QUESTIONS
Multiple choice questionsScript Concordance TestThe clinical vignette provides well defined, accurate data, to select a solution.
The clinical vignette is ambiguous, providing vague data; it is aimed at the decision making process in an uncertainty context.
The clinical vignette provides well defined, accurate data, to select a solution.
There is not one single acceptable answer. There are many possible solutions.
Basically, they assess the level of knowledge.
The test assesses medical reasoning in actual clinical situations. Problems cannot be solved by simply memorizing pieces of knowledge.
- THEORETICAL FOUNDATION OF THE SCRIPT CONCORDANCE TEST
- The Script Concordance Test originated in the classic model of hypothetical-deductive reasoning and in the Scripts theory, which is based on the contributions by cognitive psychology and neurosciences during the last decades in connection with human knowledge structuring processes and reflection in action.
The modern view of clinical thinking is supported by the script model acting as the basis for cognitive structures, clinical reasoning and competent clinical practice..
Today we know a physician’s capacity to reason and properly apply knowledge to obtain a competent solution for clinical problems is determined by the level of organization of the physician’s cognitive structure.
The scripts theory is based on the existence of knowledge networks that are well structured in memory, the complexity of which rises as clinical expertise increases. Such knowledge networks become active in professional practice in order to understand and categorize clinical situations and make decisions in an effective manner. According to this theory, expert medical reasoning with each patient goes through a fast sequence of events built by a perception of the key elements within the clinical situation presented, an almost unconscious automatic activation of scripts, an immediate access to the main hypothesis and to alternate hypotheses (abduction), and a verification of symptoms and signs confirming or refuting the hypothesis.
- WHAT IS THE SCRIPT CONCORDANCE TEST ABOUT?
-
Each Script Concordance Test reconstructs an actual clinical situation where the physician has to make decisions regarding diagnosis or conducts to follow in connection with the patient, with the purpose of solving the problem faced. The Script Concordance Test is structured as follows:
1ro. A clinical vignette providing basic and ambiguous information, with several possible diagnoses or solutions.
2do. Several items or decision tables related to the described clinical case, each of which is made up of the following:
• A diagnostic hypothesis, a complementary tests proposal or a treatment option consistent with the clinical scenario.
• New data or relevant finds (signs or symptoms, results from imaging or lab studies, etc.) that are added to the information provided by the clinical vignette, which are useful to solve the problem raised.
• Five possible solutions through a 5-point Likert answer scale where physicians assess the effect or impact of the new information on the hypothesis proposed in such item.
- SCRIPT CONCORDANCE TEST SCORING
-
As indicated by its name, the Script Concordance Test is intended to assess the level of agreement between the scripts of physicians taking the test and the scripts of expert professionals in the appropriate fields of clinical practice, with the purpose of objectively assessing the reasoning process in regular practice.
To achieve this goal, built test questions are answered by a specialty committee made up of 10 to 20 experts, thoroughly selected on the basis of their healthcare and academic profiles.
There is not one agreed single answer in this assessment model: all answers individually chosen by experts are accepted as valid and define the scoring system to be applied after a psychometric validation process.
In this way, the assessment of Script Concordance Test results is based on a comparative assessment of the level of agreement of the answer selected for each item with the answers selected by the reference panel of experts. High scores involve a greater number of experts selecting the same answer and vice versa.
For the purposes of scoring the test, the maximum score is awarded to the answer option matching the answer of the greatest number of experts, and proportional scores are assigned to the other options relative to the number of experts selecting them.
Hypothesis::
New information
Answer profile from 15 experts
A
B
C
D
E
ITEM 1
Number of experts / choices
0
0
2
9
4
Scoring method
0/9
0/9
2/9
9/9
4/9
0
0
0.22
1
0.44
Item Score
0
0
22
100
44
ITEM 2
Number of experts / choices
3
7
5
0
0
Scoring method
3/7
7/7
5/7
0/7
0/7
0,42
1
0,71
0
0
Item Score
42
100
71
0
0
The global test score is the result of the sum of the scores received for each item, divided by the number of items answered. It is expressed in a 100-point scale.