Validity of a scale for consumer evaluation of mental health service delivery

Article Sidebar

Requires Subscription or Fee PDF (USD 25)
Published: Dec 20, 2011
DOI: https://doi.org/10.5750/ijpcm.v1i4.147

Main Article Content

Grenville Rose
Aftercare
Ivan Beale
University of New South Wales
John Malone
Aftercare
Stephen Kinkead
Judi Higgin

Abstract

Objective: The factor structure of a previously developed consumer constructed scale is based on whether the questions are positively or negatively worded and extant literature suggests that this may reflect the wording rather than the genuine structure of the questionnaire. Given the potential usefulness of this type of scale, further investigation of its structure and applicability to the development of person-centered mental health service organisation and delivery is warranted. Method: The questionnaire, with consumer suggested modifications, was delivered twice across two Australian non-government mental health services. The two data sets were thus independent. The factor structure was investigated using the group 1 data and then applied to both groups using Confirmatory Factor Analysis (CFA) and multiple regression. Results: The factor structure was affected by the phrasing of the questions.  It was not possible to derive a model using the full set of questions and a reduced set was derived.  Using the reduced set, valid models were produced in both the 2 and 1 factor CFAs, but only the model based on 1 factor was robust when applied to an independent data set. Conclusions: The 1 factor model with a reduced question set offers the advantages of being quicker to deliver, not use negative phrasing and produces a more robust CFA.  Further work investigating the delivery of such a questionnaire may contribute usefully to the development of person-centered mental health service organisation and delivery.

Article Details

Issue
Vol. 1 No. 4 (2011): December
Section
Person-centered mental health
Author Biography

Grenville Rose, Aftercare

Research Manager

References

Oades, L.G., Law, J. & Marshall, S.L. (2010). Development of a consumer constructed scale to evaluate mental health service provision. Journal of Evaluation in Clinical Practice17 (6), 1102-1107.

Malins, G., Oades, L., Viney, L. & Aspden, S. (2006). What's in a service? Consumers' views of Australian mental health services. Psychiatric Rehabilitation Journal 29 (3), 197-204.

Williams, B.W.G. (1995). Patient Satisfaction in mental health care: Evaluating an evaluation method. British Journal of Psychiatry 166, 599-662.

Malins, G.M., Morland, K., Strang, J., Dowson, J., T. Hunt, S. Williamson, D. Mason, M. Eman, Y. & Oades L. (2006). A framework for mental health consumers to evaluate service provision.Australasian Psychiatry 14, 277-280.

Slade, M.P., Phelan, M. & Thornicroft, G. (1998). A comparison of needs assessed by staff and by an epidemiologically representative sample of patients with psychosis. Psychological Medicine 28, 543-550.

Ralston, G.B., Beesley, S. & Bogue, J. (1998). User's needs and satisfaction with a community-based mental healh service. Psychiatric Bulletin 22, 473-476.

Slade, M. (1996). Assessing the needs of the severely mentally ill: Cultural and Proessional differences. International Journal of Social Psychiatry 42, 1-9.

Eisen, S.V., Shaul, J.A., Clarridge, B., Nelson, D., Spink, J. & Cleary, P.D. (1999). Development of a consumer survey for behavioral health services. Psychiatric Services 50, 793-798.

Gill, K.J.P., Pratt, C.W. & Libera L.A. (1998). The effects of consumer vs. staff administration on the measurement of consumer satisfaction with psychiatric rehabilitation.Psychiatric Rehabilitation Journal 21, 365-370.

Leese, M.J., Johnson, S., Slade, M., Parkman, S., Kelly, F. & Phelan, M. (1998). User perspective on needs and satisfactioin with psychiatric rehabilitation. British Journal of Psychiatry 173, 409-415.

Williams, S.J.C. & Calnan, M. (1991). Convergence and divergence: Assessing criteria of consumer satisfaction across general practice, dental and hospital care settings. Social Science & Medicine 33, 707-716.

Clients Satisfaction Evaluation. (2000) WHO: Geneva.

Clark, C.C., Scott, E.A., Boydell, K.M. & Goering, P. (1999). Effects of client interviewer on client-reported satisfaction with mental health services. Psychiatric Services 50, 961-963.

Polowczyk, D., Brutus, M., Orvieto, A.A., Vidal, J. & Cipriani, D. (1993). Comparison of patient and staff surveys of consumer satisfaction. Hospital and Community Psychiatry 44, 589-591.

Crawford, M. (2001). Involving users in the development of psychiatric services – no longer an option. Psychiatric Bulletin 25, 84-85.

Everett, B. & Boydell, K. (1994). A Methodology for including consumer’s opinions in mental health evaluation research. Hospital and Community Psychiatry 45, 76-78.

Goodare, H. & Smith, R. (1995). The rights of patients in research. British Medical Journal 310, 1277-1278.

Kaufmann, C.L. (1993). Roles for mental health consumers in self-help group research. Journal of Applied Behavioral Science 29, 257-271.

Perkins, R. (2001). What constitutes success?: The relative priority of service user’s and clinician’s views of mental health services. British Journal of Psychiatry 179, 9-10.

Commonwealth of Australia. (1997). National Standards for Mental Health Services.Canberra: Australian Government Printing Service.

Meehan, T., Bergen, H. & Stedman, T. (2002). Monitoring consumer satisfaction with inpatient service delivery: The inpatient evaluation of service questionnaire. Australian and New Zealand Journal of Psychiatry 36, 807-811.

Lelliott, P., Beevor, A., Hogman, G., Hyslop, J., Lathlean, J. & Ward, M. (2001). Carer's and users expectations of services - user version (CUES-U): A new instrument of measure the experience of users of mental health services. British Journal of Psychiatry 179, 62-72.

Morrell-Bellai, T.L. & Boydell, K.M. (1994). The experience of mental health consumers as researchers. Canadian Journal of Community Mental Health 13, 97-110.

Ross, D. (2001). User’s voices: The perspectives of mental health service users on community and hospital care. London, UK: The Sainsbury Center for Mental Health.

Everett, B. & Steven, L. D. (1989). Working together: A consumer participation research project to develop a new model of high-support housing.Canada's Mental Health 37, 37.

Kopolow, L.E. (1981). Client participation in mental health service delivery. Community Mental Health Journal 17, 46-53.

Lecomte, T., Wilde, J.B. & Wallace, C.J. (1999). Mental health consumers as peer interviewers. Psychiatric Services 50, 693-695.

Sherman, P.S. & Porter, R. (1991). Mental health consumers as case management aides. Hospital and Community Psychiatry 42, 494-498.

Strang, J., Eman, Y., Viney, L., Oades, L., Mcleod, R. & Malins, G. (2001). Consumer evaluation of MHS: Working in partnership to evaluate MHS. Paper presented at Second Annual Conference of the Illawarra Institute for Mental Health. In: Second Annual Conference of the Illawarra Institute for Mental Health. Wollongong, N.S.W., Australia.

Strang, J. & Eman, Y. (2002). Overview of the CEO-MHS project, in Mental Health Services Conference (TheMHS).Sydney, N.S.W., Australia.

Mcleod, R. & Oades, L.G. (2001). Changing roles: Training mental health consumers to evaluate mental health services. , in Second Annual Conference of the Illawarra Institute for Mental Health. Wollongong, N.S.W. Australia.

Lindelow, M.W. & Wagstaff, A. (2003). Health Facility Surveys: An Introduction. World Bank Policy Research Working Paper No. 2953.

Marsh, H. (1996). Positive and Negative Global Self-Esteem: A Substantively Meaningful Distinction or Artifactors? Journal of Personality and Social Psychology 70 (4), 810-819.

Schmitt, N. & Stults, D.M. (1985). Factors defined by negatively keyed items: The result of careless respondents? Applied Psychological Measurement 9 (4), 367-373.

DiStefano, C. & Motl, R.W. (2006). Further investigating method effects associated with negatively worded items on self-report surveys.Structural Equation Modeling 13 (3), 440-464.

Kelloway, E.K. & Barling, J. (1990). Item content versus item wording: Disentangling role conflict and role ambiguity.Journal of Applied Psychology 75 (6), 738-742.

Jorm, A.F., Kelly, C.M. & Morgan, A.J. (2007). Participant distress in psychiatric research: A systematic review.Psychological Medicine: A Journal of Research in Psychiatry and the Allied Sciences 37 (7), 917-926.

Yanos, P.T., Stanley, B.S. & Greene, C.G. (2009). Research Risk for Persons With Psychiatric Disorders: A Decisional Framework to Meet the Ethical Challenge. Psychiatric Services 60 (3), 374-383.

MacCallum, R.C. & Austin, J.T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology 21, 201-226.

Byrne, B. (2009). Structural Equation Modelling with AMOS: Basic concepts, Applications and Programming. 2nd Edition. New York: Routledge.

Cheung, G.W. & Rensvold, R.B. (2002). Evaluating goodness of fit indexes for testing measurement invariance.Structural Equation Modeling: A Multidisciplinary Journal 9, 233-255.

Marsh, H.W. (1986). The bias of negatively worded items in rating scales for young children: A cognitive-developmental phenomenon. Developmental Psychology 22, 37-49.