Evidence-based assessment/Templates and Instructions/Creating A New Instrument Page

This is a shell that we use for building new Wikiversity instrument pages on Wikiversity.

Before starting
If you are starting a Wikiversity instrument page using this template, please do the following:
 * 1) We will import all of the content here to a new page.
 * 2) Click on the Edit source tab. Copy all content.
 * 3) Create a new page.
 * 4) Paste it in!
 * 5) Delete these instructions.

Lead section
The lead section gives a quick summary of what the assessment is. Here are some pointers (please do not use bullet points when writing article):


 * 1) Make sure to include a link to the "anchor citation"
 * 2) What are its acronyms?
 * 3) What is its purpose?
 * 4) What population is it intended for? What do the items measure?
 * 5) How long does it take to administer?
 * 6) How many questions are inside? Is it multiple choice?
 * 7) What has been its impact on the clinical world in general?
 * 8) Who uses it? Clinicians? Researchers? What settings?
 * 9) Using the Edit Source function, remove collapse top and collapse bottom curly wurlys to show content.

Steps for evaluating reliability and validity

 * 1) Evaluate the instrument by referring to the rubrics for evaluating reliability and validity (both external Wikiversity pages). For easy reference, open these pages in separate tabs.
 * 2) Reliability rubric
 * 3) Validity rubric
 * 4) Refer to the relevant instrument rubric table. This is the table that you will be editing. Do not confuse this with the external pages on reliability and validity.
 * 5) Instrument rubric table: Reliability
 * 6) Instrument rubric table: Validity
 * 7) Depending on whether instrument was adequate, good, excellent, or too good:
 * 8) Insert your rating.
 * 9) Add the evidence from journal articles that support your evaluation.
 * 10) Provide citations.
 * 11) Refer to the heading for the instrument rubric table ("Rubric for evaluating norms and reliability for the XXX ... indicates new construct or category")
 * 12) Make sure that you change the name of the instrument accordingly.
 * 13) Using the Edit Source function, remove collapse top and collapse bottom curly wurlys to show content.

Instrument rubric table: Reliability
Note: Not all of the different types of reliability apply to the way that questionnaires are typically used. Internal consistency (whether all of the items measure the same construct) is not usually reported in studies of questionnaires; nor is inter-rater reliability (which would measure how similar peoples' responses were if the interviews were repeated again, or different raters listened to the same interview). Therefore, make adjustments as needed.

Reliability
Not all of the different types of reliability apply to the way that questionnaires are typically used. Internal consistency (whether all of the items measure the same construct) is not usually reported in studies of questionnaires; nor is inter-rater reliability (which would measure how similar peoples' responses were if the interviews were repeated again, or different raters listened to the same interview). Therefore, make adjustments as needed.

Reliability refers to whether the scores are reproducible. Unless otherwise specified, the reliability scores and values come from studies done with a United States population sample. Here is the rubric for evaluating the reliability of scores on a measure for the purpose of evidence based assessment.

Validity
Validity describes the evidence that an assessment tool measures what it was supposed to measure. There are many different ways of checking validity. For screening measures, diagnostic accuracy and discriminative validity are probably the most useful ways of looking at validity. Unless otherwise specified, the validity scores and values come from studies done with a United States population sample. Here is a rubric for describing validity of test scores in the context of evidence-based assessment.

Development and history

 * Why was this instrument developed? Why was there a need to do so? What need did it meet?
 * What was the theoretical background behind this assessment? (e.g. addresses importance of 'negative cognitions', such as intrusions, inaccurate, sustained thoughts)
 * How was the scale developed? What was the theoretical background behind it?
 * If there were previous versions, when were they published?
 * Discuss the theoretical ideas behind the changes.

Impact

 * What was the impact of this assessment? How did it affect assessment in psychiatry, psychology and health care professionals?
 * What can the assessment be used for in clinical settings? Can it be used to measure symptoms longitudinally? Developmentally?

Use in other populations

 * How widely has it been used? Has it been translated into different languages? Which languages?

Scoring instructions and syntax
We have syntax in three major languages: R, SPSS, and SAS. All variable names are the same across all three, and all match the CSV shell that we provide as well as the Qualtrics export.

Hand scoring and general instructions


If there are any hand scoring and general administration instructions, it should go here.

CSV shell for sharing

 * 

Here is a shell data file that you could use in your own research. The variable names in the shell corresponds with the scoring code in the code for all three statistical programs.

Note that our CSV includes several demographic variables, which follow current conventions in most developmental and clinical psychology journals. You may want to modify them, depending on where you are working. Also pay attention to the possibility of "deductive identification" -- if we ask personal information in enough detail, then it may be possible to figure out the identity of a participant based on a combination of variables.

When different research projects and groups use the same variable names and syntax, it makes it easier to share the data and work together on integrative data analyses or "mega" analyses (which are different and better than meta-analysis in that they are combining the raw data, versus working with summary descriptive statistics).

R/SPSS/SAS syntax
R code goes here

SPSS code goes here

SAS code goes here

Example page

 * General Behavior Inventory