Post Interview Observations
The surveys, or instruments, completed by QSAN,
now called QSAN, Inc. (Quality Service Assessment Network, Inc.) have
some sheets at the back of the interview for interviewers to complete
about how the interview went, and generally the state of the person
being interviewed. These are called the Post Interview Observation
sheets, and consist of five questions on how the interview went
for the year of 2003. This writer has gone through these sheets
and tallied the results, which we will now report on.
In total, there were 896 Post Interview Observation sheets that
were completed by the interviewers. By far the largest majority,
or 47%, of the interviewers said that the interview went “very
well,” or “okay, or “smoothly,” that there
were no problems with the instrument, and in response to “Suggestions
or learnings to share,” they said “none.” The
remainder of the sheets had to be coded.
There were 263 “short answer” sheets, or Post Interview
Observations that had short comments. From a total of 263, 231,
or 88%, also responded that the interview went anywhere from “okay”
to “very well,” including such answers as “fine,”
“good,” and “smoothly.” Of the Post Interview
Observation sheets that had complex, or long, answers 143 of 190,
or 75%, said that the interviews ranged from “okay”
to “very well.”
This writer, as a data researcher, coded the answers, taking all
the observation responses from both the “short answer”
and “long answer” post interview sheets. By thus coding
the responses, this researcher was able to find the major themes
from the various answers to the five questions posed to the interviewers
as a wrap-up of their interview. There were no large percentages
from these themes, the largest responses representing less than
10%. Therefore, this writer will not attempt to give percentages
to the major themes throughout the responses, but rather give a
count as to how many responses there were. The interviewers were
themselves responsible for filling out the Post Interview Observation
sheets, so the answers will represent their comments on either the
persons they were interviewing, or on the survey, or instrument,
itself.
The largest responses had 30 to nearly 50 people saying the same
thing, and the smallest number answering the question a certain
way was one response only. To be significant for this report, generally
four to six responses are necessary. To start with, we’ll
deal with the survey, or instrument, itself. By far one of the largest
responses, 34 answers, stated that the instrument itself or the
interview ran too long. Some of the comments were that the interviewee
started to get tired, not responding well, towards the end of the
interview. A few responses said the interview was too repetitive,
and along this vein, one of the largest responses to, “Overall,
how did the interview go?” was that it was hard to keep the
interviewee focused. Forty-two comments mentioned this problem for
the interviewer. In response to the above question, twenty-three
times the interviewer said that the consumer was distracted, or
“fidgety/nervous.” Six responses said it was hard for
the interviewee to concentrate on the questions being asked, and
five interviewers said the consumer was over or under medicated.
On the other hand, quite a few replies stated that the person was
eager to do, or in favor, of the survey; a large response, or twenty-five
answers, said that the interviewee was relaxed and answered the
questions easily. Six interviewers reported that the interview went
“slow,” and five times they reported that the consumer
was late for the interview session.
In response to the question, “Any problem areas?” the
largest response, or twenty-four, said it was hard to “draw
out” the consumer being interviewed. For this same question,
thirteen interviewers said that the consumer did not understand
the questions being asked, and six responses indicated that it was
hard to understand the interviewee, or that questions needed to
be rephrased to get a clear response from the person being interviewed.
Sixteen responses said that physical problems of the respondent
interfered with the interview, and a further thirteen replies said
that there was no privacy in which to conduct the survey. In ten
cases, the consumer was upset, and sometimes the interviewer stopped
the interview long enough for the interviewee to regain their composure
and resume the questions. Seven times there were problems with the
“cultural competency” section of the interview, and
a further seven interviewers responded that the interviewee was
concerned for a family member.
Another question on the Post Interview Observations sheet was,
“Anything unusual or concerning about the interview?”
Distractions or interruptions, needing to take lots of breaks was
by far the largest response, representing forty-six answers; a further
ten interviewers reported that they had to take a break for the
interviewee to smoke or eat, or just to take a rest from the survey.
The next largest response to this question was twenty-four interviewers
reporting that the interviewee was very knowledgeable and gave good
responses to the questions being asked.
Nine interviewers stated that the consumer was in denial about
his/her mental illness. This represents a major concern in the mental
health community, as denial is often itself a symptom of the illnesses
of psychiatric disorders or substance abuse. Seven interviewers
said that they would like to have a job, and six interviewers reported
that the consumer had been in the system a long time. Among problems
reported in the interviewing process (although not in significant
numbers), were that the interviewee had been drinking, the interviewer
was concerned for her/his safety two times, and in one instance,
the interviewee made sexual advances to the interviewer.
The question, “Suggestions or learnings to share?”
was a general wrap-up response for the interviewer. By far the largest
response to this area of the post interview sheet was that the interviewer
thought the consumer was “nice,” with thirty-three such
remarks. They also said the interviewee was artistic, intelligent,
or interesting. Further comments were that the consumer was “confused,”
while seven interviewers reported that the respondent was very cooperative.
Housing problems and transportation problems accounted for a total
of fifteen replies regarding the interviewee. Stigma was reported
as a concern six times, and indeed this is a major issue with consumers
and the community as a whole. The fact that the interviewee had
encountered a “rough life” was reported seven times
by the interviewers, while economic problems accounted for four
responses.
Eight of those interviewed thought the survey was a “good
idea,” and six of the consumers stated that they were satisfied
with their mental health services, while only three were dissatisfied.
Ten interviewers reported that the interviewee was “working
on problems” or wanted to improve their lives. The fact that
the consumer had no supports or was alone in the world accounted
for seven responses, while a significant amount of consumers, six
to be precise, were experiencing case management problems. Five
respondents expressed a desire for more drug groups.
The interviewers at QSAN/QSAN, Inc. had many personal comments
about the interviews and the people they interviewed. One positive
aspect of the interviewing process was that several interviewers
reported that they were made aware of services that had previously
been unknown to them. Hence, the personal lives of the interviewers
could be improved by more knowledge of mental health services available
to them. Four interviewers stated that conducting interviews made
them grateful for their own situations.
Six interviewers commented that they would like a sheet listing
all the definitions of services that the survey covers to avoid
confusion for the interviewee. Five responded that a listing of
services available was needed for the consumer and others. Thirteen
interviewers responded that the interview was either very interesting
or a learning experience for them. Eight of our staff reported that
they needed to display patience with the interviewee, and several
of the interviewers said that it is important to stress the confidentiality
aspect of the interview.
Only four interviewers reported that their interview was “very
difficult,” so the majority of interview experiences were
good, which is very important for those of us here at QSAN/QSAN.
A few times interviewers reported that they got lost or were late
because the location was hard to find, and one interviewer said
that she got a parking ticket because the interview went long. A
couple of statements from the interviews said that there was some
confusion about housing services – whether the service existed
within the mental health system or just general housing, so a definition
of the housing services would help clear up any confusion.
Finally, we will report on general comments from our QSAN/QSAN, Inc.
interviewers. By far the largest response from interviewers was
that they encountered adverse physical surroundings when conducting
the interview. They mentioned extremely dirty environments and being
afraid of bugs and cockroaches. Another factor for this category
of response was cramped quarters, and having nowhere to sit to do
the interview. Only three comments pointed out that the home was
very clean and comfortable. Three further observations mentioned
that the interviewee was in AA, or Alcoholics Anonymous, or another
type of drug support group.
Two interviewees suggested mail or phone interviews; the reason
the respondent proposed a mailed survey was that they would be able
to take their time answering the questions. Our sister CQRT (Consumer
Quality Review Team) site, Quality Review Services, Inc. (QRS),
has had some success with mailed provider surveys, but here at QSAN/QSAN, Inc.
we pride ourselves on the fact that our interviews are conducted
on a face-to-face basis. All other coded responses in this general
comment category had only one response, and thus are not significant
enough to comment on.
Summary
We have seen in general that things are going well for QSAN/QSAN, Inc.
in terms of the interviewing process. Interviewers state that the
interviews are a learning experience for them, and they are very
impressed with the people that they interview. Often a strong rapport
is developed between the interviewer and the interviewee. These
Post Interview Observations are a valuable source for us to judge
what problems there might be with the instrument. In this way we
can plan to improve the survey in the future to better serve our
clients. Any feedback from you, our readers, is encouraged and welcome.
Please send an e-mail to gtaylor@qsan.org,
and we will be happy to respond.
|