Experiences of Care in Aotearoa 2022/2023 - Our Methodology
This section explains how information and data is gathered and analysed to develop the key findings and themes presented in this report.
Effective and meaningful monitoring requires a mix of approaches and the use of quantitative (numbers) data and qualitative (experiences) information.
Our information is from the perspective of and , their , those who care for them, and the professionals who work with them. This helps us understand:
-
what is working well for tamariki and helping them achieve good outcomes in their lives (we call these enablers)
-
what is getting in the way of achieving good outcomes (we call these barriers).
-
Analysing the self-monitoring data from the agencies with tamariki in their custody helps to:
-
look for trends and changes that have implications for tamariki and rangatahi in care
-
understand from the agencies’ data their compliance with the .
-
Gathering data and information
At the centre of our monitoring approach are the stories and lived experiences of tamariki and rangatahi, their whānau, caregivers and community. We also gather information from frontline kaimahi at Oranga Tamariki, Open Home Foundation and Barnardos (the three agencies with custody of tamariki), and Māori partners, care partners, and other government agencies, such as the New Zealand Police, as well as frontline health and education providers. This helps us develop a holistic picture of the experiences of tamariki and rangatahi in care.
Our monitoring teams cover the motu. Our people come from a range of backgrounds and areas of expertise including social work, psychology, education and law. They are trained in listening and speaking with tamariki and rangatahi and have experience in working with different communities, including Māori communities.
We have an assessment framework that includes a set of pātai pākiki (questions) that our monitors use when engaging with people in a community. This framework supports a consistent approach to the (discussions).
The framework incorporates factors that support professionals working with tamariki, rangatahi and whānau; for example, are polices clear, do they have supervision, and do they have the resources they need to do their job. The framework looks for the root cause as to why something is or isn’t happening.
In the 12 months to 30 June 2023, we spoke with over 1,200 people about their experiences.
Who we heard from during 2022/20231
Over three years of reporting, we spoke with almost 3,300 people about their experiences.
Who we heard from during 2020 - 2023
Over the last three years, the ethnicity and gender of the tamariki and rangatahi we spoke to was representative of the ethnicity and gender of the care population. While we heard from a smaller number of tamariki under 10 years of age, the nature of our work means we tend to hear from tamariki and rangatahi who are 10 years of age or older. We try to speak to caregivers of tamariki and rangatahi of all ages to provide insight into the experiences of those younger tamariki.
Around one quarter of tamariki and rangatahi we heard from had an identified disability. This is in line with the proportion of tamariki and rangatahi in care estimated to have a disability.
The whānau we heard from were also broadly representative of the ethnicities of tamariki and rangatahi in care.
We met with both whānau and non-whānau caregivers, however, we heard from twice as many non-whānau caregivers as whānau caregivers. This is not representative of the caregiver population. This may be because connections between non- whānau caregivers made it easier for us to talk with them in groups. We will continue to focus on talking with whānau caregivers in the future, so we hear more of their experiences and perspectives.
| 2020 - 2023 | ||||
|---|---|---|---|---|
| Care agency | Oranga Tamariki | Iwi social service providers & Māori providers | (including Barnardos, Dingwall and Open Home Foundation) | Unknown or Not Recorded |
| Tamariki & Rangatahi | 184 | 34 | 180 | 21 |
Where we visited
Our community visits are organised on a three-yearly cycle, with all Oranga Tamariki regions across the motu visited once during this time. In 2022/2023, we visited Greater Wellington (Wellington, Porirua, Kapiti, and the Hutt Valley), South Auckland, Waikato, Lower South Island (Alexandra, Invercargill, Otago Urban (Dunedin) and Balclutha/Gore), Wairarapa and Hawkes Bay. This completed our three-year schedule around the country.
In 2021/2022 we visited Te Tai Tokerau (Northland), North-West and Central Auckland, Bay of Plenty, Taranaki and Manawatu, upper South Island and Canterbury. In 2020/2021, we visited Kaitaia, South Auckland (Māngere and Otahuhu), Gisborne, Porirua and Paraparaumu, Blenheim and Kaikoura, and the West Coast (Westport, Greymouth and Hokitika).
Our full monitoring schedule is available on our website.
Gathering data
We also request data from Oranga Tamariki, Open Home Foundation and Barnardos - the agencies with custodial responsibilities. The NCS Regulations require these agencies to make their own assessments of how well they are complying with the regulations and provide this information to us. These data requests are available on our website and include areas we have previously identified as needing improvement.
After responses to the data requests are received, we make supplementary requests to Oranga Tamariki and Open Home Foundation where further data, information or clarification is required. Both agencies work closely with us to meet these requests wherever possible.
Analysing information and data
Analysing information
At the end of a monitoring visit, we wānanga (meet and discuss) to understand what we heard from tamariki and rangatahi, their whānau, and caregivers. Quotes are anonymised to protect the identity of the people we spoke with.
We use qualitative analysis methodology and software to capture this collective understanding and develop the findings in this report. For more information on how we wānanga and our qualitative research methodology, see our website.
Analysing data
When we analyse the data from agencies with custodial responsibilities, areas of change are examined to understand how any new initiatives or changes in practice have affected the quality of services the agencies provide to tamariki and rangatahi in their care.
We also look at whether measures are different for tamariki and rangatahi Māori compared to non-Māori, and disabled tamariki and rangatahi compared to those with no identified disabilities.
You can find data tables for Oranga Tamariki and Open Home Foundation showing their performance against the NCS Regulations in [Appendix reference].
Equity analysis
We analysed data from Oranga Tamariki to identify whether compliance with the NCS Regulations is equitable across the population of tamariki and rangatahi in care. We did not find any statistically significant differences in compliance measures between tamariki and rangatahi who are disabled and those who are not.
In most cases, there were no statistically significant differences between measures for Māori and non-Māori either. There was one exception in the measure on opportunities for play and experiences, with tamariki Māori receiving fewer opportunities than non-Māori.
We also analysed the information gathered through our monitoring visits to see if there were any differences among Māori and non-Māori tamariki, rangatahi, whānau and caregivers. Aside from areas relating specifically to (Māori world view), we found that tamariki, rangatahi, whānau and caregivers talked about similar things regardless of ethnicity.
Preparing for publication
Agencies review the report to check for fairness and accuracy
Before publishing this report, each of the three agencies had an opportunity to review the report to:
- check that our analysis and interpretation of the data they provided is accurate, and
- prepare to respond publicly to any comments or findings.
1 Due to Cyclone Gabrielle, our visit to Hawke’s Bay was postponed. A shortened visit took place in August 2023 and the results are included in this report.