Second Public Submission to the 
Productivity Commission
Inquiry into a
Lifelong Disability Care and Support Scheme

June, 2011

Response to submission DR958 by the 	NSW Agency for Clinical Innovation (ACI) – NSW Brain Injury Rehabilitation Directorate

Centre for Disability Studies (CDS)
Sydney Medical School, University of Sydney

59 Charles St, Ryde, NSW, Australia
ph: +61-2-8878-0500
fax: +61-2-9807-7053

Contact points for CDS’ submission and for CDS research into support needs assessment:

Samuel Arnold
BA (Psych.), PG Dip (Psych.), MA (Analytical Psych.), PhD Candidate
Research Fellow, Analytical Psychologist, Centre for Disability Studies
Research Affiliate, University of Sydney
sam.arnold@sydney.edu.au
+61-2-8878-0515

and/or

Dr. Vivienne Riches 
BA., Dip.Ed., MA(Hons)., PhD
Senior Research Fellow, Psychologist, Centre for Disability Studies
Clinical Senior Lecturer, University of Sydney
vriches@med.usyd.edu.au
+61-2-8878-0516

The Centre for Disability Studies (CDS) felt it was necessary to provide a response to the critique of the Instrument for the Classification and Assessment of Support Needs (I-CAN) tool (Arnold, Riches, Parmenter, Llewelyn et al., 2009) provided by the NSW Brain Injury Rehabilitation Directorate (NSW BIRD) in their submission to the Productivity Commission.  CDS appreciates the work completed by NSW BIRD in this critique and welcomes further suggestions and feedback about the design and development of the I-CAN.  Feedback about the I-CAN has been of great value over its on-going development and refinement.  Some of the critique provided by the NSW BIRD was based on early research on the first, second and third versions of the I-CAN.  We have now been implementing the fourth version of the I-CAN for the last four years and recently developing and testing a brief version of the I-CAN.  The brief version of the I-CAN is most likely more suited to the specific process of resource allocation than the full version.  We apologise that the latest research for the full and brief versions of the I-CAN has not yet been published or is commercial in confidence and not currently publically available.  Hence aspects of the critique presented by the NSW BIRD may be due to the lack of information on which they were basing their critique.  We hope that our more recent research will be published or made publically available in the near future.

Clarifying specific points raised by the NSW BIRD submission;

•	9.2 Description, p. 32
o	The full I-CAN has 12 domains, 2 of which are optional (My Goals and Health & Support Services), grouped under two ICF (WHO, 2001) based categories of Health & Well-Being and Activities & Participation.
o	Point 4: Our latest studies suggest that in a research context, the full version of the I-CAN takes on average 70 minutes to administer, and the Brief I-CAN takes approximately half this amount of time to administer.
•	9.4 Psychometric properties of the scale, p. 33
o	Point 1: More recent test-retest data have been gathered over a much shorter time period, with promising results.  More recent validation studies have included people with brain injury living semi-independently in the community in the samples.
o	Point 5: More recent inter-rater data have been gathered with a clear process for how the data were gathered and several different raters were involved in gathering these data, with promising results.  This research is not currently publically available.
o	Point 6: The Instrument for Client and Agency Planning (ICAP) (Bruinicks, Hill, Weatherman & Woodcock, 1986) is an assessment tool published in 1986 widely used internationally for both diagnostic processes and to determine resource allocations.  The ICAP, although an adaptive behaviour assessment, does produce a ‘Service Level Score’, hence the ICAP has been used in several research studies, including our own, in comparison to support needs assessments.  As the ICAP is actually an adaptive behaviour not support needs measure, some correlation, though not very high correlations are expected.  There is a current lack of appropriate measures to compare support needs assessments with, and no ‘gold standard’.  We have completed a more recent comparison of the ICAP and I-CAN, with promising results.  This government funded research is not yet publically available.
•	9.4.2 Validity, pg. 34
o	A study looking at the predictive validity of the full I-CAN found high correlations of the I-CAN with clinical judgement (rs = 0.816, n = 186).  These data are already publically available from an international conference presentation that can be downloaded from the I-CAN website (Arnold, Riches, Parmenter & Stancliffe, 2009).  An article detailing this study is being submitted for publication; this study included 22 people (11.8% of the sample) who had sustained a brain injury.  Other validation studies not publically available have included samples involving people with brain injury living in community settings.
•	9.4.3 Responsiveness to measure change, pg. 34
o	We are currently conducting a small study looking at the I-CAN’s ability to measure intervention effectiveness, and to determine minimum clinically significant difference.  
•	9.6 Issues, pg. 34
o	Point 1: Further promising data have been gathered using more recent versions of the I-CAN, but reports are not yet published or publically available.
o	Point 2: Assessment burden mentioned is likely due to similar assessment processes already in place in the Lifetime Care & Support Scheme.
•	9.6.1.1 Overall, pg. 34-35
o	Point 1: A section is marked not applicable if the person does not have support needs related to that domain.  For example, not all people need support with communication, hence the communication domain would be not applicable is some assessments.
o	Point 2: Unsure of the comment that “Confidentiality is overruled when people disclose”.  Any information that the person discloses and is recorded on the I-CAN Individual Support Needs Report is confidential information.  Like any other assessment or personal information collected it is the responsibility of the organisation collecting these data, in collaboration with the person, to maintain its confidentiality, or to take other actions based on the information obtained that may be required by policy or legal requirements.  A further feature is built into the I-CAN which allows for sections that are highly sensitive to the person, e.g. sexuality support needs, to be marked as ‘private’.  In these cases, two versions of the I-CAN report are produced (both of which of course are confidential), one ‘open’ report which is available to all people supporting the person, which does not contain information about sexuality support needs, and a ‘additional privacy’ version of the report which would contain the information about sexuality support needs.  This would only be made available to people who needed to provide the support relating to sexuality.
o	Point 3: The limited jargon used in the I-CAN is discussed in the I-CAN training course, and detailed in the instruction manual.  Some of this jargon comes from the National Minimum Data Set.  Our understanding is that part of the development of the ICF was to use plain, non-pathologising language as much as possible.
o	Point 4: There is a certain amount of overlap between body functions and activities & participation, conceptually we believe this is hard to avoid.  For example, if a person has physical health problems with their bowel or bladder, support related to this is likely to also be reflected in self-care activities.
o	Point 5: The I-CAN groups the assessment items into ‘Health & Well Being’ and ‘Activities & Participation’, this is similar to the grouping presented in the ICF.  Yes, these domains are all inter-related and a certain amount of overlap and interaction is expected.  For example, a body function impairment in balance could flow on to mobility support needs and also community access support needs.  The I-CAN measures support, not functioning, hence not using some of the subscales in the ICF.  
•	9.6.1.2 Tool Specific, pg. 35
o	Point 1: The decision to include diagnoses in support needs (primarily in the Health & Well-Being domains) occurred from findings in the early stages of developing the I-CAN.  It was found that, for example, unless epilepsy was specifically mentioned, the physical health support needs related nervous system structure and function impairments caused by epilepsy would not be recorded.  The inclusion of diagnoses in the support needs items is an aspect of I-CAN that we are still working on and considering, whether we try to add more examples or diagnoses that would relate to particular body structures and functions, or whether we remove diagnoses from these sections altogether.  At the moment, we are adding more example diagnoses to these sections, to ensure that the person’s real support needs are identified and attended to.  We are also considering additional, separate recording of medical conditions and disabilities.
o	Point 2: Not sure of intended meaning of “Relevance to people with brain injury and other diagnoses 9 (sic) not intellectual disabilities”.  As much as possible with our on-going development and testing we have designed the I-CAN tool to be applicable across disability types, and particularly with the Brief version, applicable across ages.
o	Point 3: Diagnosis codes are limited to the classification used in the National Minimum Data Set.  We have considered the functionality that potentially the full I-CAN could operate as a form of e-health record, that could also output ICD and ICF coding, though this would require considerable work and some redesign.  Although we would like to undertake this, we currently do not have the resources to take on a project along these lines.
o	Point 5: To clarify
•	Obesity is recorded under Digestive, Metabolic & Endocrine System functions in the Physical Health domain, along with special dietary needs, ingestion problems, stomach problems etc.  Obesity is considered a physical health problem, and support needs such as encouraging healthy eating and exercise may be recorded here.
•	Eating disorder, compulsive eating, pica etc. is recorded under Eating in the Mental & Emotional Health domain.  These are considered as Mental Health concerns related to eating, and for example support needs related to Anorexia would be recorded here.
•	Eating & Drinking (which you could also describe as feeding) is recorded in the Self Care & Domestic Life domain.  This relates to the specific activities and related supports of physically eating or drinking something.  For example, do you need support to drink from a cup?
o	Point 5 (cont): Although obesity, eating disorder and eating & drinking may all be related, they are also separate functions or health & well-being concerns and separate supports may be required for each.  
o	Point 6: Some changes have been made in the grouping of items under Mental & Emotional Health and Behaviour of Concern in the brief version of the I-CAN.  However, our experience suggests that often people’s behaviour can be labelled as ‘challenging behaviour’ or ‘behaviour of concern’ when actually the behaviour is an indication of an underlying, undiagnosed mental health concern, due to diagnostic overshadowing.
o	Point 7: The Health & Support Services is an optional domain.  It would often be completed in collaboration with a case manager e.g. and in many cases would not be completed by the person with disability alone.  This domain is not used in most scoring algorithms or resource allocation algorithms, though offers another alternative in estimating costs that can be useful for support organisation or in research settings.  We would be interested in working with any interested parties in further developing this domain to be applicable to additional service delivery contexts, for example by giving options for multiple different case managers.  In this domain you only record information on services that are relevant to the person, so if a team leader is not an essential part of the person’s support network, you would not mark this item.  We have designed this domain to be as applicable as possible across a range of supported settings, and there are slight changes in this domain in the brief version that we hope we can incorporate also into the next update of the full I-CAN.  We welcome any suggestions or collaboration in the on-going development and design of the I-CAN.
o	Point 9: the Life Long Learning domain is not specifically from the ICF but we felt it was important to add this domain (added in the fourth version of I-CAN) in order to address and record the supports related to a person’s employment, education or day activities.  A number of people with brain injury have completed this domain, currently labelled ‘life long learning activities’ and this has proven relevant to many people with brain injury who have completed assessments.  Usually, the Life Long Learning domain is not applicable only when an individual is independent in these areas, or perhaps retired and no longer interested in participating in any form of life long learning, which may also include volunteering.
o	Point 10: Stress and safety items are grouped together for brevity and similarities.  Agreed there are various subjectivities involved in the assessment of support needs.  However, if a person has a high level of need, then their need for constant supervision and potential risk of abuse is relatively objective.
o	Point 11: Yes, all domains in the I-CAN, like the ICF, can be interrelated with a certain amount of overlap.  It is important that all of the person’s support needs are addressed, particularly in the full assessment.  Data to date suggest activities and participation support needs are usually relatively stable over time, while physical health, mental and emotional health and behaviour support needs can vary more widely over the short and longer term.  The environment is also addressed dynamically by the I-CAN, by considering what support the person needs in the environment in which they interact of a daily basis (or the environment they plan to interact with, if completing multiple ratings).  
o	Point 12: We would also consider people in a day program to be unemployed.
o	Point 13: If a person can catch a bus to attend a day program, or catch a train independently to get to work, we believe this information would be important for a support service to be aware of.
o	Point 14: Shopping is considered under both the Self Care & Domestic Life and Community, Social & Civic Life domains of the full I-CAN and has a specific item in the Domestic Life domain of the brief version.  Supports for parenting is considered under the Health & Support Services domain.  Acquiring a place to live could be considered under the ‘My Goals’ domain.  Previously there was a specific item for acquiring a place to live in the Brief version of the tool, but this item was dropped as there was a low incidence and frequency of the item being applicable, so did not have a large impact on scoring.
o	Point 15: The Self Care and Domestic Life domains have been separated in the brief version of I-CAN and have specific items for cleaning and household maintenance.
•	9.6.1.3 Clinical Utility pg. 35
o	Point 1 & 2: The brief version takes significantly less time to administer.  Both the brief and full version are automatically scored by the online software.  Other acceptability issues specified appear to be related to clinical load within these specific programs.  The full I-CAN enables support services to review what other processes and documentation can be reduced that are covered by the I-CAN tool.
•	9.6.1.4 Administration, pg. 36
o	Point 1: Administration time for the full version can be long, depending on the context and purpose of the assessment.  Support services may break the assessment process up into several sessions if the person would not be able to attend to the assessment in one sitting.  Review assessments are less time consuming.
o	Point 2: Difficulty in consensus regarding support needs from various people in a support circle can be a significant issue in some cases.  One of the benefits in completing the I-CAN assessment can be the attempt to come to agreement regarding what supports will be provided.  In rare cases where agreement cannot be reached, then ‘multiple ratings’, or side by side assessments can be completed, and the different opposing views are recorded simultaneously.  
o	Point 3: Yes, there can be certain skill required on the part of the assessment facilitator in supporting the person and their support circle in determining appropriate goals, particularly in those cases where the person may lack insight.  This is a clinical skill relevant to goal setting, regardless of the assessment tool or process in which goals are determined and recorded.
•	9.6.1.5 Rating Scale and Scoring, pg. 36
o	Point 1 & 2: the I-CAN attempts to measure direct support need, not functioning.  The I-CAN measures support required in the context of the environment in which a person lives (or will be living in soon).  As in the ICF, we believe that disability and support needs arise as an interaction of the person with the environment in which they function.  Hence, although functionally a person may not be able to cook a meal, there is a difference in the support needed between (a) a person who only needs ‘meals on wheels’ which they can reheat themselves, compared to (b) a person who needs special meals cooked individually for them each day, or (c) a child who eats the same meals that are cooked for the rest of their family.  This example shows a clear difference between what may be measured by an assessment of functioning and what we believe should be measured by an assessment of support needs.
o	Point 3: rating the most frequent or intense support need may under-represent the extent of support need is potentially a valid concern, although we have not yet seen or discovered a practical alternative to this.  An article in preparation for publication, that can be shared on application to the authors does show some support for the validity of scoring the highest support need in a particular domain, as opposed to scoring the average for support needs in a particular domain.
•	9.6.1.6 Psychometric Properties, pg. 36
o	Point 1: Samples in research not currently publically available have included a larger portion of people with multiple types of disability.  The predictive validity study previously mentioned which is publically available included in its sample 22 people (11.8%) with ABI and 5 people (2.7%) with physical disability.  There were also 12 people (6.5%) with both intellectual and physical disability and 22 people (11.8%) recorded as having multiple disabilities.  Plans are to conduct further, larger studies, where there are larger samples of people with ABI or physical disabilities.
o	Point 2: this is a valid critique, to date we have not have sufficient resources, sample size or time available to complete a thorough analysis of the statistical effect of recording multiple ratings.  There is only anecdotal evidence currently in support of the utility of multiple ratings.
o	Point 3: Some additional data not yet publically available have been collected and analysed regarding inter-rater reliability.  This applies to a medium, not large, sample size.
o	Point 4 & 5: To date no Rasch analysis has been conducted and comparisons with other tools have been limited.  We would be interested if people were able to assist us in this process and potentially co-author research articles on the results.
o	Point 6: hopefully more information about more recent research samples will be in the public domain in the near future.
o	Point 7: currently in progress.
o	Point 8: cut off scores is a very interesting concept that we would enjoy exploring.
o	Point 9: the I-CAN Cost Estimation Tool, which is linked to the Health & Support Services domain, is an optional tool that would most likely be used by non-government support services.  This is one way the I-CAN can estimate costs, though for the purposes of resource allocation in an individualised funding context it is more likely that an algorithm would be applied to scores coming from other domains in the I-CAN.
o	Point 10: This relates to clinical judgement and decisions made by the specific support service.
•	9.6.1.7 Information obtained, pg. 36
o	Point 1: Feedback from services would suggest the Individual Support Needs Report can be a very useful document.  The Cost Estimation Tool is designed so that the support service modifies the tool to give as accurate as possible estimate of the cost of providing services from their organisation for the person.
•	9.6.1.8 Ethical consideration, pg. 36
o	Point 1: The I-CAN database is stored on a standalone server in a locked server room.  The I-CAN website uses security similar to websites that process credit cards.  It is possible to develop a local database and scoring software for specific organisations, though of course there is some additional time and cost that this would require.
o	Point 2: Confidentiality issues discussed above.
o	Point 3: Consent issues are applicable to any assessment tool or information recording system, though many do not employ the same level of scrutiny in the need for informed consent as the I-CAN does.
o	Point 4: this seems to be specific to the requirements of a particular program.
•	9.6.1.9 Cost of I-CAN, pg. 37
o	Point 1, 2 & 3: CDS is a not for profit organisation and is not funded to provide training and assessment services free of charge.  The cost of training and assessment services is quite reasonable and the quality of the services provided offers good value for money.
o	Point 4: Ready access to computers and internet is only required at the central office for data entry and scoring.  Although assessments can be conducted online and scored immediately, assessments can also be conducted using pen and paper when internet and laptop are not available or suitable. 
o	Point 5: The I-CAN website when last tested was compatible with all popular internet browsing software, on Windows, Macintosh and Linux based computers.
•	9.7.1 Conceptual issues need to be discussed and addressed:
o	Point (a): As discussed above our current thinking is to include more diagnoses in the Physical Health and Mental & Emotional Health domains, though this is something we are still considering.
o	Point (b): Supplementary specific risk assessments are something that is still required.  The I-CAN does not replace the need to conduct risk assessments.  The I-CAN can be a tool that collates the findings of various assessment and information gathering processes.
o	Point (c): Confidentiality discussed above.
o	Point (d): Parenting and shopping is addressed already in the full I-CAN.  We regularly gather feedback on whether people feel all their relevant needs have been addressed, and are happy to add additional items when gaps are identified.
•	9.7.4 Establish psychometric properties.  Need to ensure tool has been properly evaluated for use in our population, pg. 37-38
o	Points 1 to 8: As discussed, current contractual agreements mean our latest studies are not currently published and / or publically available.  However, plans are to publish as soon as possible and CDS is very interested in conducting further research as time and resources permit.
o	Point 9: This is an interesting question and we would be interested to conduct research and development related to this.
o	Point 10: It would be interesting to conduct a specific study comparing the FIM and I-CAN.
•	Table 1: Possible strategies to address psychometric requirements, pg. 39
o	Further validity testing has been completed though not yet available, however comparisons with specific tools mentioned and Rasch analysis has not yet been conducted.  
o	Responsiveness study in progress, though additional responsiveness study specifically relating to brain injury would be ideal.
o	Clinical utility has been established in other settings.

In conclusion, we appreciate the effort that has been put into the submission coming from NSW BIRD, and apologise that they did not have access to more recent work at the time of writing their submission.  We appreciate some of the ideas and suggestions presented by NSW BIRD, and hope that the on-going development and testing of the I-CAN tools will address many of the points raised.  We would enjoy doing further research looking more specifically at the application of the I-CAN to brain injury populations and compare with existing well established assessment tools used in these settings.  However, it is important to note that many people with brain injury have now participated in I-CAN assessments.

Further, as the draft Productivity Commission report proposed two separate schemes and agencies, the National Injury Insurance Scheme (NIIS) and the National Disability Insurance Scheme (NDIS), and also proposed the development of an assessment toolbox, it is yet to be seen what assessment tools or selection of assessments tools may be applicable for people who have recently acquired brain injury.  The I-CAN has yet to be tested with people with recently acquired brain injury, though this could be interesting research to be completed.

References

Arnold, S. R. C., Riches, V. C., Parmenter, T. R., Llewellyn, G., Chan, J., & Hindmarsh, G. (2009).  I-CAN: Instrument for the Classification and Assessment of Support Needs, Instruction Manual V4.3.  Sydney, Australia: Centre for Disability Studies, Faculty of Medicine, University of Sydney.

Arnold, S. R. C., Riches, V. C., Parmenter, T. R. & Stancliffe, R. J. (2009). I-CAN: Support needed for inclusion and empowerment. IASSID 2nd Asia-Pacific Conference Abstracts, Singapore, 24-27 June. Journal of Policy and Practice in Intellectual Disabilities, 6(2), 93. (Abstract) Link to Presentation

Bruininks R. H., Hill B., Weatherman R. & Woodcock R. (1986). The Inventory for Client and Agency Planning (ICAP). DLM Teaching Resources, Allen, TX.

World Health Organization (WHO) (2001). The International Classification of Functioning, Disability and Health (ICF). Geneva: Author.