Can I Write My Own Instrument?

An instrument is a list of questions that, when considered together, are designed to measure a single construct. These questions are indicators of the construct, meaning they each ask about some aspect of the construct. A construct is something that you want to measure that cannot be measured directly. Usually, you will be able to find a published instrument that you can use for free, and this is the best route to take since published instruments have already been tested and found to be at least reliable if not also valid (coming soon!). Other times, if you can’t find the exact one you need you can find one that is close enough that you can adapt it to fit your study. But sometimes you aren’t able to find one that fits your research situation at all, and in that case you will want to develop your own instrument. Though it can be a tedious process, it is doable.

Though each research situation is different, generally speaking the first step is to find out all you can about your construct so you can write good indicator questions. This involves a literature search for relevant research articles where the researchers used the construct as a major player. This is a good way to find out what the typical indicators are of your construct. For example, suppose you want to write an instrument that measures “satisfaction with the counseling relationship” so that you can survey clients to find out how happy they are with the counselor they see every week. In your lit search, you find that counselor variables such as promptness, eye-contact, and understanding are all valued by clients. From these, you might write statements such as “My counselor is always on time for my appointments”, “My counselor looks at me when I am talking”, and “I feel my counselor understands my problems”. You could then use answer choices ranging from 1 to 5 to measure the degree to which these statements are true in the minds of the clients being surveyed. In a setup like this, the scores for each question could then be added up to give an overall satisfaction-with-counselor score for each client.

The next step is to get some feedback from professionals in the counseling field, as well as from non-professionals who are familiar with the counseling process (perhaps friends who have seen counselors). Ask them to read the questions and make a simple judgment as to whether the questions appear to measure what you think they are measuring. This will give you some idea of the instrument’s validity, specifically face validity. The third step is to pilot your new instrument. This is where you gather a small sample of people (10 or so will do) to fill out your new questionnaire. Once you have the data, you can run it through a statistics program for reliability. The resulting reliability coefficient tells you how reliable the instrument is, or how consistent it is likely to be in measuring satisfaction.

The last step is to remove questions from the instrument, or at least alter them, that were found to be unreliable. This unreliability may result from questions that were not written clearly, used too many uncommon words in them, or were “double-barreled” (asked two things in the same question). Once cleaned up, the instrument can then be re-piloted. This process will likely take a few tries before the instrument is ready for use in your actual research, but it is worth the trouble to establish a new instrument.

 

Comments are closed