Reporting and Accountability Understanding Success Anne Couillard Settlement
Reporting and Accountability Understanding Success Anne Couillard Settlement and Integration Policy Branch IRCC October November 2017 Purpose IRCC perspective on Settlement Program success and desired outcomes Provide an overview of IRCC data collection, and the pros, cons, and gaps Provide some highlights from data analysis Present how this information is used and reported Discuss next steps and ways forward 1 Outline Background IRCCs Settlement Program Focus on Outcomes What is Settlement Program success? How do we know we succeeded? What is IRCC collecting? Examples of findings Annual Project Performance Report (APPR) Clients and services (iCARE)
How is IRCC using this information? Pros, cons, and gaps in data collection Considerations & next steps 2 Background IRCC Settlement Program The Settlement Program aims to assists newcomers to overcome barriers to participation in social, cultural, civic and economic life in Canada and help to foster an integrated society Services are funded through contribution agreements and delivered by third-party providers Funding is allocated by province and territory and centrally managed national and pre-arrival funds. This does not include the transfer to Quebec It delivers services to help newcomers acquire necessary language skills, ensuring their access to information, connecting them to community resources, building their knowledge of life in Canada, and enabling labour market participation To reach these objectives, IRCC also ensures Sector capacity and professional development, standardization of best practices, and building knowledge of the Programs clients Legislative Authorities Canadas Settlement Program fulfills obligations of the
Immigration and Refugee Protection Act to: Promote the successful integration of permanent residents into Canada, while recognizing that integration involves mutual obligations for new immigrants and Canadian society. Integration is a shared role with involvement and engagement of multiple players, whose contributions and levers help to achieve longer-term outcomes and IRCC Settlement Program objectives 3 Background Focus on Outcomes There are a number of factors creating momentum for looking at outcomes, and for review, revision and refinement of data collection and reporting Ministerial Mandate Letter: a rigorous approach to data in order to accurately measure outcomes. Government of Canada and Departmental emphasis on results and delivery, focus on outcomes Settlement Program evaluation Levels Plan - program modernization Ministerial consultations with stakeholders
4 Background Program Design Inputs and Delivery Settlement Program Terms and conditions Funding Logic Model with expected outcomes Settlement Services Outputs Immediate Outcomes Based on priorities for services and programming and directives to manage contribution agreements Needs Assessment Information and Orientation Language Assessment Language Training
Community Connections Indirect Services Intermediate Outcomes Ultimate Outcomes 5 It starts with expected outcomes! What is Settlement Program success? Settlement Program Expected Outcomes Immediate outcomes Intermediate outcomes Ultimate outcomes
Consistent and responsive Settlement Program delivery Access to IRCC- funded settlement services is facilitated Increase understanding of client settlement needs and appropriate linkages to other services Clients increase knowledge of life in Canada Clients improve official language skills Clients acquire knowledge, skills, and connections to prepare for the Canadian labour market Clients increase participation in communities and social networks Partners deliver responsive and coordinated settlement and community services Clients access services that meet their needs Clients make informed decisions about life in Canada Clients use an official language to function in Canadian society Clients participate in the Canadian labour market Clients are connected to communities and institutions Communities foster welcoming environment for immigrants Canada benefits from the settlement and integration of clients Outcomes for clients are a key focus but two-way street important Specific level of outcomes for specific clients groups 6
How do we know we succeeded? Examples of performance measurement indicators Outcomes Measured by: Immediate: Clients increase knowledge of life in Canada Intermediate: Clients participate in the Canadian labour market Ultimate: Canada benefits from the settlement and integration of clients Performance Indicators # and % of clients receiving information by service and topic % of clients receiving information who indicated that their knowledge of life in Canada changed as a result of IRCCfunded services, by topic and overall # and % of clients by employment status as indicated by client (working, have worked but not currently, looking for work) Incidence of employment earnings for clients 1 through 5 years since receiving first service Sense of belonging
Volunteering rates Employment rates 7 What is IRCC collecting? Measuring performance at Projects and Program Level Outputs: Number of clients and their characteristics Number of services and their characteristics Geographic distribution Newcomer Language progress Outcome Survey Delivery: Locations Partnerships & resources iCARE Challenges & opportunities Successes Needs assessment approaches Performance assessment approaches Narrative Reports/APPR GCS Inputs/Resources:
Budgets Targets Operational monitoring External Database Linkages Immediate/intermediate outcomes: Knowledge, behaviour, integration in four areas: Information about Canada Official Language knowledge Labour market Community connections Ultimate Outcomes: Economic, social and cultural integration through iCARE data linkages to IMDB including non-clients for comparison Pilots, experimentation
8 Examples of Findings: Client Needs identified in APPR Employment Connecting with professional networks in the Canadian workplace A desire to find employment within field of expertise Balance between working in a survival job vs need for language training to obtain a better employment, e.g. within field of expertise Language Increase demand for literacy and lower level CLBs due to increase in refugees admissions Need to obtain CLB 4 for citizenship purposes Health and wellness Mental health support for children, youth and adults due to trauma Health and Wellness awareness Support Service Need for child care to facilitate participation in services, particularly language training Translation and interpretation (outside of IRCC services) Transportation required to participate in IRCC and non-IRCC services 9 Examples of Findings: Delivery Challenges from APPR Service Delivery
Constraints Ineligible people / services Administrative issues Lack of support services Increased numbers of clients, waitlists Need for more workshops and training opportunities Canadian citizens, refugee claimants Need for assistance with immigration paperwork Assistance with appointments outside of the SPO Staff turnover Reporting requirements (including iCARE data entry) Lack of space, facilities, funding External factors Local labour market conditions, employment rates impact work placements
Summer, seasonal changes (holidays, school break) Client recruitment and participation Lack of promotion, marketing and outreach Employment as a barrier for participation Client cancellation, turnover, attendance Because of higher numbers of clients, we are trying to do more group orientations in order to reduce the waiting list for some of our programs. 10 Examples of Findings: Clients and Services in iCARE In 2016-17, over 410,000 clients accessed at least one settlement service in Canada, a 2.7% increase in clients served over 2015-16.
11 Measurement Activities at IRCC Inputs and Delivery Audit Monitoring Settlement Program Design Terms and Conditions Objectives Settlement Performance Measurement Services Strategy Services Employment-related Services (ES) Employment-related Services
Research Planning 12 How is IRCC using this information? Planning Identification of newcomer needs (from Needs Assessment and Referrals Services) Design of services e.g.; Community Connections: Volunteer Coordination and Enhancements Analysis of service patterns (e.g.: users profile, language progression, geographic distribution) Project and Financial Monitoring Achievement of project service targets by organizations Service use at various groupings (local, provincial, national) Financial monitoring Performance Measurement Performance reporting to Canadians, reports to Parliament Reporting to SPOs, posting of data (Open Data Portal) Matching with financial data to assess use of funds Annual performance reports on Settlement Program Support for briefings, Cabinet documents, Standing Committees; UN, OECD input Evaluation
Assessment of outcomes, especially for clients but also indirect services Research Emerging needs Factors for social, economic and cultural integration Client vs non-clients analysis 13 Data Collection Pros, Cons, Gaps Pros Wealth of robust quantitative data on program activities and outputs Extensive client profile and settlement services data Data shared with SPOs (monthly reports) and the public (Open Data) Rich sources of qualitative data on project delivery Cons Data entry challenges by service providers (time consuming) Potential overlap in some information collected by various tools (e.g. support services in iCARE, APPR) Variety of information collected can be hard to analyze in a timely manner Gaps Indirect services (e.g. curriculum development)
Data linkages between iCARE and other datasets (e.g. IMDB) Comparable data for ALL newcomers (client vs non-client survey) 14 Considerations Are we collecting the right information? Is it collected at the right time, and at the right frequency? How/Can we alleviate the reporting burden? By combining data? Reducing overlaps? How do we introduce new data elements (addressing gaps) without increasing the overall burden? How do we make reporting more meaningful? Are we providing you information to support decision-making? Project planning? 15 Next Steps Share Settlement Program Evaluation Report and recommendations Implement Evaluation recommendations including review of reporting requirements Review and confirm performance measurement strategy (outcomes, indicators, data sources, reporting strategy) Establish Sector Outcomes Working Group Implement Newcomer Outcomes Survey
Engage with sector on the common settlement vision 16 Thank you! Comments & Questions? Anne Couillard Deputy Director, Performance Measurement Unit Settlement and Integration Policy Branch [email protected]
This is an example text. In these paragraphs, resume the ideas of topic 1. This is an example text. In these paragraphs, resume the ideas of topic 2. This is an example text. In these paragraphs, resume the ideas of...
INTRODUZIONE A MATLAB Sara Poltronieri Archivi Operazioni su files Apertura di un archivio fopen Scrittura di un archivio fprintf Formato flags Field Width and Precision Specifications Conversion Characters Escape Characters Riassumendo...
Tízből négy!5Ennyi fiatal nem védekezik a nem kívánt terhesség ellen, holott aktív szexuális életet él.Miért kockáztatnak? Egy közelmúltban készült felmérés szerint tízből négy fiatal gyakorlatilag nem védekezik, pedig nagy részük aktív szexuális életet él.
Francia festészet Papp Tünde IFO II. nappali 2009.12.09. Francia festészet a XIX. században XIX. sz. első évtizedeitől: Párizs a művészetek fővárosa Jacques-Louis David (1748-1825) (Napóleon udvari festője) neoklasszicizmus Jean August Dominique Ingres (1780-1867) David tanítványa J. L. David: Horatiusok esküje...
The tragic hero is generally a person of importance. The tragic hero exhibits extraordinary abilities and a tragic flaw. Outside forces or people, known as antagonists, may contribute to the hero's downfall. A series of casually related events inevitably leads...
madit-rit: background ICD is highly effective in reducing mortality in high-risk cardiac pts. Despite sophisticated device-detection algorithms, 8-40% of ICD therapies are inappropriate with adverse side effects