| 8:45 AM – 8:50 AM |
Indigenous Recognition |
| 8:50 AM – 9:00 AM |
Welcome & IntroductionsArt Stewart, PPX President & Tom Wileman, Chair, Symposium Committee |
| 9:00 AM – 10:00 AM |
Opening Keynote
|
| 10:00 AM – 10:30 AM |
Networking Break |
| 10:30 AM – 11:30 AM |
Plenary Panel – in room 106HFuture Strategies for Public Audit This panel will be offered in French from 10:30-11:30 and repeated in English from 11:30-12:30
Description: This panel examines the development of strategies for legislative and internal public audit in Canada. Paul Forgues, President of the Canadian Audit and Accountability Foundation will speak about the mandate of the Foundation and its impact on strategic thinking in member audit organizations. Professor Luc Juillet, from the University of Ottawa will present research findings regarding strategic thinking in both forms of public audit. Moderator: Panelists: |
| 10:30 AM – 11:30 AM |
Concurrent 1Click here for PresentationDigitalizing Results Management
Recent research suggests that public managers do not consistently use performance information, and when they do, the data has negligible impact on organizational results. Although scholarly and practitioner debate on the topic of government performance has generated an understanding of the key components of performance measurement systems and of the performance management cycle (van de Walle and van Dooren, 2010), several critical questions emerge, about how managers use performance information (Kroll, 2015a; 2015b; Moynihan and Pandey, 2010; Walker et al., 2018) and the impact of managerial agency (Boyne, 2004; Boyne et al., 2005). Researchers argue that more sophisticated performance information that includes advanced analytics might encourage managers to use performance information more extensively (Ballard, 2020; Kroll, 2015b; 2015a). In addition, organizations could improve the relevance of data by connecting analytics to the organization’s input-to-outcome transformation process (Vogel and Hattke, 2018). This presentation will outline the results of case studies that explored the use of logic models as a touchstone for linking performance information to organizational objectives. The logic model served as a guide to capturing the right data and to creating an analytics strategy. Nevertheless, the case studies (that included private, public, and not-for-profit organizations) reveal common problems: insufficient data, data in the wrong format, and the need for an analytics strategy connected to the organization’s overall objectives. |
| 10:30 AM – 11:30 AM |
Concurrent 2Click here for PresentationPath to Performance Measurement Improvement at Employment and Social John Mitchell, Senior Policy Analyst, Employment and Social Development Canada (ESDC) A presentation detailing how ESDC’s Performance Measurement Division has been investigating the use and utility of performance information in the department. John will look to support the “How will we know we are getting there?” theme by outlining some of the organization’s experiences and findings from research conducted over the last year. In addition, he will provide an overview of the division’s plans to raise the department’s maturity-level with performance measurement and capacity for results-based management over the coming years. |
| 11:30 AM – 12:30 PM |
Concurrent 3Click here for PresentationThe Government of Yukon Performance Measurement & Evaluation Framework: Centering Integration and Concision in Performance System Design Stephanie Hawkins, Director Evaluation, Government of Yukon. Implementation of the Government of Yukon’s new Performance Measurement & Evaluation Framework (the Framework) came into effect on January 1, 2023. The Framework applies to Yukon government programs, policies, and strategies; its scope encompasses evaluation, performance measurement, and the integration of budget, strategy, and results. The Framework is, in many regards, reflective of similar policies such as the Government of Canada’s Policy on Results. The design and implementation of the Performance Measurement & Evaluation Framework, however, makes notable departures from convention in two distinct realms:
|
| 11:30 AM – 12:30 PM |
Concurrent 4Click here for Presentation(NOTE: THIS SESSION WILL BE PRESENTED IN FRENCH) Developing a Framework for Measuring and Monitoring the Performance of Public-Private Partnerships (P3s) in Infrastructure Projects in Canada Cheick Alassane Traore, University of Ottawa The purpose of this study is to highlight the elements in the literature to report on the performance of P3s and to develop a systematic framework for measuring and monitoring this performance from a public sector perspective and the project life cycle. This framework must be operational and accessible to professionals to evaluate the performance of P3s at different stages of the project life cycle and to overcome the ideological debates surrounding P3s. This study makes it possible to establish an original performance measurement framework that combines different aspects of performance and the complete life cycle of P3s. It surpasses the current analysis of the performance of P3s, which is limited to construction, costs and infrastructure quality, and explores issues of governance, competition, sustainability of public funding, etc. Using information from nearly 200 P3s projects in Canada, this study allows for the analysis of a wide range of P3s. It fills a knowledge gap on the performance of P3s and enriches the literature on performance. |
| 11:30 AM – 12:30 PM |
Concurrent 5Click here for PresentationThe Current State of GBA+ Analysis in Federal Government Evaluation: How Close Are We to Support the Result-based Management Practice With a GBA+ Lens? Deen Taposh, Manager Strategic Policy, Employment and Social Development Canada The use of GBA+ analysis in federal government evaluations has increased over the years (Gauthier et al., 2022). The disproportionate impact of the pandemic has also reinstated the importance of GBA+ analysis in the public sector (Cameron and Lindsay, 2023). The question, however, remains to what extent federal government evaluations are using GBA+ analysis. Learning from an environmental scan of a representative sample of recently published and publicly available federal government evaluation reports, the presentation proposes to examine the following questions:
The findings from the environmental scan will be in an aggregate form without stating the underlying causes. Given the content may explore published findings from several federal departments, the views should be solely attributed to the authors. |
| 12:30 PM – 1:30 PM |
LunchSponsored by Carleton University Public Policy and Program Evaluation |
| 1:30 PM – 3:30 PM |
Plenary PanelStrategic Policy (Spending) Review
Professor Evert Lindquist, University of Victoria Don Wilson, Executive Director, Treasury Board of Canada Secretariat |
| 2:30 PM – 3:30 PM |
Case Study PresentationsGita Zareikar, Carleton University What do spending and strategic reviews mean for departments, ministries and agencies? Are the days of free flowing money over? What does it mean to be going into austerity? The pandemic placed many countries, including Canada, into major debt and deficit positions with net debt to GDP levels rising well above 75 percent. In Canada’s case, this figure is closer to 48 percent, which is still significant, with few signs that debt levels will return to pre-pandemic levels anytime soon. The federal government announced that it will be looking to departments and agencies to find $15B in savings over the next five years, which seems insignificant, but is this the real story? It seems fairly certain that budget increases will not be forthcoming, hiring will slow down dramatically, and there will be less reliance placed on outside consultants to support program delivery. This session will divide the time into two main parts. The first will feature a panel of senior public servants, who will discuss the state of readiness in their departments for austerity measures, systems and capability gaps they are observing, and advice on ways to address these gaps in a constructive way. The second will feature some research undertaken that compares the Canadian experience with spending and strategic reviews with other countries that are performing well post-pandemic, including The Netherlands, Ireland, Nordic countries and Japan. The aim is to highlight that there are several models for implementing reviews and how these contribute to not only expenditure management but to facilitating a broader policy agenda of public service reform. |
| 3:30 PM – 4:30 PM |
Concurrent 6Click here for PresentationWho Am I to Evaluate That? How We Aspired to Apply the Values of Diversity and Inclusion in Our Evaluation. Mary Peters, Senior Evaluator, Treasury Board of Canada Secretariat How do you know if your best was good enough? Considering two Treasury Board of Canada Secretariat evaluations, this presentation will discuss our approach and the learning curves inherent in inclusive, diverse, equitable and accessible evaluations. We are all more than we appear – how can we bring this to table? |
| 3:30 PM – 4:30 PM |
Concurrent 7Click here for PresentationRelevance Reconsidered: Exploring Evaluation Criteria for Public Programs and a Forward-Looking Approach in Canada’s Ministry of Natural Resources (NOTE: THIS SESSION WILL BE PRESENTED IN FRENCH) Raoul Tamekou Tsowa, Ph.D, Policy Analyst, Natural Resources Canada The traditional role of evaluators has largely been to analyze past actions or events and assess their outcomes (Rossi, Lipsey and Freeman, 2003). This retrospective analysis provides a vital understanding of what has worked, what has not, and why. Evaluators typically consider a range of criteria, such as effectiveness, efficiency, impact, and sustainability. However, some critics posit that a focus on retrospective analysis might not be enough. Decision-makers often seek guidance for future initiatives, implying that evaluation needs to be more future-oriented. They want to know what actions they should take based on the evaluation findings, and how to apply these learnings to future decisions (Wicker, 2023a). In a few communications, Ted Wicker suggests one way of addressing this concern: by giving more importance to the criterion of ‘relevance’ in evaluations. The concept of relevance in evaluation usually refers to the extent to which the objectives of a project or program are consistent with beneficiaries’ requirements, country needs, global priorities, and partners’ and donors’ policies. By emphasizing relevance, evaluators can help ensure that evaluation findings are not only descriptive of the past but are also prescriptive for future initiatives, reflecting the priorities and needs of the stakeholders involved (2023b). The purpose of this paper is to reflect on evaluation criteria in the practice of evaluating public programs in Canada. More specifically, we question the place of the criterion of relevance, its meaning, and its evolution in program evaluations carried out by the Ministry of Natural Resources Canada since the 2000s. Also, the article emphasizes that while the criterion of relevance proves heuristic from a forward-looking perspective, it is insufficient. Other solutions are considered which aim to more effectively position the practice of evaluation in the future. |
| 3:30 PM – 4:30 PM |
Concurrent 10Click here for PresentationLogic Model and Integrated Reporting Model for Arts Funding
As a public funding agency for the arts, the Council has a mandate to foster and promote the study and enjoyment of, and production of works in, the arts. Many of its programs, services and activities align with the theme of this year’s symposium, as well as its three streams. Some of its key activities and areas of focus that parallel the theme of the symposium include, but are not limited to:
|
| 4:30 PM – 5:00 PM | PPX Membership Presentation
Click here for Presentation
Wayne MacDonald Who Are You? |
| 5:00 PM – 6:00 PM |
Welcome Reception |
2023 – 26th annual PPX symposium
| 8:55 AM – 9:00 AM |
Welcome Day 2Professor Eric Champagne, University of Ottawa and PPX VP |
| 9:00 AM – 10:00 AM |
Opening Keynote
Accountability to Canadians in a Decentralized Federation Jeannine Ritchot, Assistant Deputy Minister Jeannine Ritchot joined Intergovernmental Affairs, Privy Council Office as the Assistant Deputy Minister of Multilateral Relations in April 2022. In this role, she provides advice and thought leadership on matters affecting the evolution of the federation, including on the federal-provincial-territorial dynamics of key social and economic policy files. Prior to this, Jeannine was the Assistant Deputy Minister of the Communications and Portfolio Sector at Natural Resources Canada. She has also held executive positions at Treasury Board Secretariat, Health Canada, and the Public Health Agency of Canada, where she acquired extensive experience developing regulation and programs in collaboration with provinces and territories. In 2020, for her efforts in promoting international and interprovincial regulatory cooperation, she became an inaugural recipient of the Apolitical and World Economic Forum “Agile 50” Award, which recognizes the 50 Most Influential People in the World Revolutionizing Governance. In 2019, Jeannine became the first woman to serve as Chair of the OECD Regulatory Policy Committee. She was also a peer reviewer of governance structures for international regulatory cooperation in the U.K. and Mexico and negotiated the Canada-United States-Mexico Agreement Good Regulatory Practices Chapter. Through her community work, Jeannine is a fierce advocate for the advancement of women’s involvement in sport. She mentors University of Ottawa women athletes and works with organizations in Ottawa to ensure that girls have equal access to sports infrastructure. Jeannine holds a Bachelor of Arts degree in History from the University of Ottawa and a Master of Arts degree in Conflict Studies from St. Paul University / University of Ottawa. |
| 10:00 AM – 10:30 AM |
Networking breakSponsored by Canadian Audit and Accountability Foundation |
| 10:30 AM – 11:30 AM |
Plenary PanelDigital Transformation
Description: What does it mean to have public sector organizations ready to respond to the challenges of today and tomorrow, when service delivery increasingly means digital delivery by default? From government legislative assemblies to the editorials of the day, much talk revolves around the challenges organizations face in transforming service delivery of key government services to citizens. Why can some jurisdictions make it easy to renew drivers’ licenses where others struggle to support passport renewal? The prevailing view is that Canada is failing and falling further behind in meeting its citizens’ needs. Indeed, Canada has plummeted from being ranked 3rd in the world in its digital service delivery to 32nd in little over 10 years. So what’s going on? How do we know what is and is not working well on the digital transformation file? What roles have key functions (e.g., results performance, audit, evaluation, and risk management) played to date in digital transformation. How have they helped – or possibly even hindered – transformation? How can these functions become key partners in public service renewal? Join us for a panel discussion with leaders in digital transformation in Canadian public sectors to hear their thoughts and to ask your questions on these pressing issues. Moderator: Panelists: |
| 11:30 AM – 12:30 AM |
Keynote PanelReview of the TBS Policy on Results
Ken Stephenson, Director, Policy on Results and Engagement, Treasury Board Secretariat of Canada Description: PPX welcomes Kenneth Stephenson’s off er, on behalf of the Secretariat, to provide an update of its ongoing activities in the Policy Review, including the consultations with deputy heads, assistant deputies, and heads of performance measurement and evaluation. |
| 12:30 PM – 1:30 PM |
LunchSponsored by Goss Gilroy Inc. |
| 1:30 PM – 2:30 PM |
Plenary PanelTruth & Reconciliation with Indigenous Peoples
Description: This roundtable brings together Indigenous and non-Indigenous academics who will reflect on their research to incorporate creative and community-engaged approaches to the topic of truth and reconciliation. Themes covered include nation-to-nation/Crown- Chair: |
| 2:30 PM – 3:30 PM |
Concurrent 9Click here for PresentationBreaking Data Barriers and Silos: How Natural Resources Canada is Advancing Disaggregated Data Collection for Gender-Based Analysis Plus (GBA Plus) Allain Barnett, Policy Analyst, Gender Based Analysis Centre of Expertise, Strategic Policy, and Innovation Through sharing data collection experiences in GBA Plus, departments can enhance their capacities to collect and analyze disaggregated data to understand diverse experiences, and to ensure no one is left behind. In this presentation, we share Natural Resources Canada’s practices in GBA Plus data collection. As reflected through data collection information released in Departmental Results Reports (DRR), our GBA Plus Centre of Expertise works to assist NRCan officials build their capacity to collect disaggregated data and measure the impacts of their programs on diverse groups of people. The GBA Plus Centre of Expertise has consulted with NRCan sectors to establish a baseline on NRCan’s capacity to collect disaggregated data, common challenges experienced, and approaches to address these challenges. Overall results indicated that many NRCan’s programs collected disaggregated data through funded project reporting, workforce data, and engagement/participation. Of the programs collecting disaggregated data, most collected data on gender and Indigenous Peoples and/or communities. Data gaps remain for other identity factors, such as racialized individuals, regional diversity, age, and people with disabilities. Several programs have developed data collection approaches to meet these challenges, including:
In this session, you will learn about some of the data challenges, experiences with GBA Plus at NRCan and ways you can fulfill reporting requirements related to disaggregated data collection. This session is your first step in understanding how to take to track your organizations’ policies and programs’ impacts on diverse groups of people. This is also a space for practitioners to exchange on their experiences with designing and implementing GBA Plus research and data collection plans. |
| 2:30 PM – 3:30 PM |
Concurrent 11Click here for PresentationDesigning and Delivering Horizontal Programs and Services in the Federal Government Cristina Drob, Analyst, Treasury Board of Canada Secretariat Horizontal Initiatives in the Government of Canada play a crucial role in addressing complex priority issues by bringing together multiple federal departments and agencies. By utilizing a “whole of government approach,” these initiatives ensure that departments work together to achieve common objectives. During the 2022-23 fiscal year there were 30 active Horizontal Initiatives with $134 billion in total spending. The progress and outcomes of these Horizontal Initiatives are regularly reported through Departmental Plans and Departmental Results Reports. These reports not only ensure that Canadians stay informed about the government’s work but also provide a mechanism for tracking the government’s performance and making necessary adjustments to better serve Canadians. The presentation will provide insights into the strengths and challenges departments face in managing and reporting on Horizontal Initiatives. By sharing best practices and lessons learned with the performance and planning community, the Results Division at TBS aims to support federal departments to better plan and implement Horizontal Initiatives. |
| 3:30 PM – 4:30 PM |
Practical Ways to improve the efficiency and effectiveness of your Procurement activitiesClick here for Presentation
Carolyn Montague, Partner, Performance Management Network Inc and President, Canadian Institute for Procurement and Materiel Management (Volunteer) In the current climate of cost cutting pressure will be on all public servants to justify the outcome of their procurement activities. Craig Szelestowski of Lean Agility and Carolyn Montague of the Canadian Institute for Procurement and Materiel Management will share with delegates a number of practical ways Project Authorities can improve the efficiency and effectiveness of their procurement activities. |
| 4:30 PM – 4:40 PM |
Closing RemarksArt Stewart, PPX President |
Past PPX Webinars
2019 SYMPOSIUM PRESENTATIONS
Understanding the take-up and use of information may be the most important measurement focus for government, not-for-profit and private initiatives in any time. It has even more importance in times of world crisis such as now. Over the last several years a uniquely tailored tool for assessing the uptake + use of information and knowledge across many domains has been developed in Canada. (The latest evidence available suggests that this Canadian developed approach has the potential to be a unique world leader in this kind of assessment.) The Knowledge Uptake and Use Tool (KUUT), originally developed by Dr Kelly Skinner of Waterloo to assess health promotion initiatives and since tried and evolved in a number of Canadian contexts, has shown itself to be an excellent means to gauge the reach and influence of information products, services and events. It tracks important responses based on a research based ‘influence’ pattern, and if used correctly can provide key insights on how context and conditions can affect information and knowledge use.
This one-hour webinar will provide an update on the initiative and will highlight one recent application of the tool which has been adapted for COVID 19 times and online information sessions, ‘fairs’ and conferences.
This session will highlight one of several on-going uses / users. The project is essentially a creative commons effort – so most applications and practices are shareable at this time. The key presenters for this session will be Kelly Skinner and Jasmin Bhawra of Waterloo University followed by Mark Shepherd of the Canadian Wood Council and Wood Works who will share their experiences using a streamlined version of the tool to gauge the influence of information products, solutions fairs, webinar sessions and even networking events. The session will be facilitated by Steve Montague of PPX and CES – and a keen user and secondary developer of versions of the approach. Brief presentations and a ‘show and tell’ component of the actual survey will be followed by a Q &A session. This event should be of keen interest to evaluators, performance measurement specialists, and / or anyone interested in understanding the reach and influence of the information they generate and produce from reports, sessions or conferences. A version of the tool will be used to gauge the value of this session – so registrants will get first-hand knowledge of the approach.
Panelists
Steve Montague, Moderator
Kelly Skinner, Waterloo University
Jasmin Bhawra, Waterloo University
Mark Shepherd, Canadian Wood Council and Wood Works
SPEAKERS
Steve Montague, partner of Performance Management Network Inc. is a credentialed evaluator, a Fellow of the Canadian Evaluation Society and has 35 years of experience as a practitioner in performance measurement and program evaluation as a management consultant, an adjunct professor and as an evaluation manager in a major Canadian federal government department. Mr. Montague has managed major and minor projects analyzing a wide variety of programs for Canadian federal, provincial, United States, Scottish and Australian governments, as well as conducting work for the OECD, the World Bank, the European Union and the United Nations. He has published articles on measurement and evaluation in journals in Canada, the US and Europe and facilitated numerous presentations, panels and workshops on evaluation, performance management and information management. Steve is also a founding member and three times president of the Performance and Planning Exchange.
Kelly Skinner is an Assistant Professor in the School of Public Health and Health Systems at the University of Waterloo where she leads the new online Master of Health Evaluation program. Kelly’s research and evaluation interests focus in two areas: (1) the health and well-being of First Nations populations and northern communities including the evaluation of health programs and food systems approaches; and (2) using evaluation theory and conceptual approaches to operationalize and build evaluation systems, with a continued interest in measuring knowledge uptake and utilization. Kelly has experience with stakeholder engagement, working with communities as an evaluation consultant, and currently teaches several courses she developed to online graduate students on evaluation theory, applications, practice, and management.
Jasmin Bhawra is a PhD Candidate and CIHR Health System Impact Fellow in the School of Public Health and Health Systems at the University of Waterloo. She is a population health researcher and program evaluator whose research interests include chronic disease prevention, health program planning and evaluation. Jasmin has worked with public health units, non-profit organizations, and governmental agencies coordinating research and leading health program evaluations.
Mark Shepherd has been working with the Canadian Wood Council for nearly 20 years as an advisor to Wood WORKS!, the association’s industry-and-government funded national market development program. In this role, he has worked with a diverse team of professionals to increase the use of wood in non-residential and multi-family construction projects. He also helped launch a similar program in the U.S. Mark is passionate about the Wood WORKS! program mission to ensure the built environment maximizes the use of wood-based solutions for the domestic wood industry and has developed strategies which leverage advocacy, education, collaboration, and communication. With the team, Mark has developed numerous national and regional strategic and operating plans and progress reports. He helped create and implement the program’s initial logic model, exit survey and methods for demonstrating funder “return on investment”.
Mark has a B.A. from the University of Waterloo (co-op) and M.A. from the University of Victoria.
GOVERNING THROUGH PERFORMANCE: LEARNING, AND UNLEARNING, IN THE UK
Dr. Colin Talbot, Professor of Government at the University of Manchester, Manchester, UK
LESSONS LEARNED FROM AROUND THE WORLD
Sir Michael Barber
IMPLEMENTING THE GOVERNMENT’S RESULTS AND DELIVERY AGENDA: THE CHIEF RESULTS AND DELIVERY OFFICER’S (CRDO) PERSPECTIVE
Moderators:
Francis Bilodeau, Assistant Secretary To The Cabinet, Results And Delivery, Privy Council Office
Tolga Yalkin, Acting Executive Director, Program Performance and Evaluation Division, Expenditure Management, Treasury Board of Canada Secretariat.
CRDO Panel:
Neil Bouwer, Formerly Assistant Deputy Minister, Strategic Policy And Results Sector, Natural Resources Canada, now with Treasury Board Secretariat
Cheryl Grant, Director General, Policy Co-ordination And Planning Directorate, Health Canada
Sheryl Groeneweg, Director General, Results Delivery Unit, Innovation, Science, And Economic Development Canada
CANADA’S NEW INFRASTRUCTURE PROGRAM – FOLLOWING THE MONEY
Peter Weltman, Sr. Director, Costing and Program Analysis, Office of the Parliamentary Budget Officer (PBO)
2021 LEARNING EVENTS
Session 2: Going into the Future
Discussion on the future of evaluation and other review functions post-pandemic with Alexandra Dagger, Director General, Audit and Evaluation at the National Research Council (NRC) and Yves Gingras, Director General, Evaluation Directorate at Employment and Social Development Canada (ESDC).
The discussion will consider what is needed to ensure that evaluation and other review functions remain relevant and well positioned in the post-COVID context. It will be followed by a Q&A and further discussion among attendees.
Thursday, March 17th 2021
12:00 – 1:00 pm EST
Session 1: Lessons from the Pandemic
Lessons learned for evaluation and performance measurement from the pandemic, a presentation by Pascale Poirier. Pascale is an Evaluation Manager at the Public Health Agency of Canada (PHAC) and headed the review of lessons learned in PHAC’s response to the first wave of COVID-19. The short presentation will be followed by a Q&A and attendee discussions on learnings from the pandemic.
Thursday, March 4th 2021
12:00 – 1:00 pm EST
RISK MANAGEMENT:THE OBSCURE HERO OF STRATEGIC PLANNING?
Risk management is currently receiving renewed attention within public administrations in the context of the pandemic, geopolitical tensions, historical deficits, etc. How can public sector organizations plan and operate in a highly unpredictable and complex environment? Risk management aims to mitigate this uncertainty by identifying and analyzing the causes that could prevent an organization from fulfilling its mandate and achieving its objectives. Thus, risk management should be one of the driving forces behind the strategic planning process and, by extension, sound results-based management practices. In this presentation, our guest speakers ague that it is imperative that risks and their mitigation strategies be at the heart of the organizational priority-setting process as well as upstream of any investment and budget reallocation decisions. This webinar jointly organized by the Performance and Planning Exchange (PPX), the Institute of Public Administration of Canada (IPAC) and the University of Ottawa, aims to present the benefits of sound risk management in the context of strategic planning and the broader context of results-based management. The case of Innovation, Science and Economic Development Canada will serve as a backdrop to this presentation.
Presentation in English and Q and A in bilingual format.
Presentation for DownloadView Recording
Moderated by Christine Minas
Director, Results Division, Expenditure Management Sector, Treasury Board of Canada Secretariat (and PPX Board member)
Marc Morin
Director General at Innovation, Science and Economic Development Canada, (formerly Chief Risk Officer at ISED)
Olivier Choinière
Affiliate Researcher at the University of Ottawa’s Centre on Governance and Consultant, GGA (formerly Director of Performance and Risk Management at ISED)
Jointly organised by:
Performance and Planning Exchange (PPX)
Institute of Public Administration of Canada (National Capital Region)
University of Ottawa (Centre on Governance)
February 25, 2021
Noon – 1:00 pm EST
2020 LEARNING EVENTS
Knowledge Uptake and Use An Emerging Key Measurement for Our Time
Understanding the take-up and use of information may be the most important measurement focus for government, not-for-profit and private initiatives in any time. It has even more importance in times of world crisis such as now. Over the last several years a uniquely tailored tool for assessing the uptake + use of information and knowledge across many domains has been developed in Canada. (The latest evidence available suggests that this Canadian developed approach has the potential to be a unique world leader in this kind of assessment.) The Knowledge Uptake and Use Tool (KUUT), originally developed by Dr Kelly Skinner of Waterloo to assess health promotion initiatives and since tried and evolved in a number of Canadian contexts, has shown itself to be an excellent means to gauge the reach and influence of information products, services and events. It tracks important responses based on a research based ‘influence’ pattern, and if used correctly can provide key insights on how context and conditions can affect information and knowledge use.
This one-hour webinar will provide an update on the initiative and will highlight one recent application of the tool which has been adapted for COVID 19 times and online information sessions, ‘fairs’ and conferences.
This session will highlight one of several on-going uses / users. The project is essentially a creative commons effort – so most applications and practices are shareable at this time. The key presenters for this session will be Kelly Skinner and Jasmin Bhawra of Waterloo University followed by Mark Shepherd of the Canadian Wood Council and Wood Works who will share their experiences using a streamlined version of the tool to gauge the influence of information products, solutions fairs, webinar sessions and even networking events. The session will be facilitated by Steve Montague of PPX and CES – and a keen user and secondary developer of versions of the approach. Brief presentations and a ‘show and tell’ component of the actual survey will be followed by a Q &A session. This event should be of keen interest to evaluators, performance measurement specialists, and / or anyone interested in understanding the reach and influence of the information they generate and produce from reports, sessions or conferences. A version of the tool will be used to gauge the value of this session – so registrants will get first-hand knowledge of the approach.
Steve Montague
partner of Performance Management Network Inc. is a credentialed evaluator, a Fellow of the Canadian Evaluation Society and has 35 years of experience as a practitioner in performance measurement and program evaluation as a management consultant, an adjunct professor and as an evaluation manager in a major Canadian federal government department. Mr. Montague has managed major and minor projects analyzing a wide variety of programs for Canadian federal, provincial, United States, Scottish and Australian governments, as well as conducting work for the OECD, the World Bank, the European Union and the United Nations. He has published articles on measurement and evaluation in journals in Canada, the US and Europe and facilitated numerous presentations, panels and workshops on evaluation, performance management and information management. Steve is also a founding member and three times president of the Performance and Planning Exchange.
Kelly Skinner
is an Assistant Professor in the School of Public Health and Health Systems at the University of Waterloo where she leads the new online Master of Health Evaluation program. Kelly’s research and evaluation interests focus in two areas: (1) the health and well-being of First Nations populations and northern communities including the evaluation of health programs and food systems approaches; and (2) using evaluation theory and conceptual approaches to operationalize and build evaluation systems, with a continued interest in measuring knowledge uptake and utilization. Kelly has experience with stakeholder engagement, working with communities as an evaluation consultant, and currently teaches several courses she developed to online graduate students on evaluation theory, applications, practice, and management.
Jasmin Bhawra
is a PhD Candidate and CIHR Health System Impact Fellow in the School of Public Health and Health Systems at the University of Waterloo. She is a population health researcher and program evaluator whose research interests include chronic disease prevention, health program planning and evaluation. Jasmin has worked with public health units, non-profit organizations, and governmental agencies coordinating research and leading health program evaluations.
Mark Shepherd
has been working with the Canadian Wood Council for nearly 20 years as an advisor to Wood WORKS!, the association’s industry-and-government funded national market development program. In this role, he has worked with a diverse team of professionals to increase the use of wood in non-residential and multi-family construction projects. He also helped launch a similar program in the U.S. Mark is passionate about the Wood WORKS! program mission to ensure the built environment maximizes the use of wood-based solutions for the domestic wood industry and has developed strategies which leverage advocacy, education, collaboration, and communication. With the team, Mark has developed numerous national and regional strategic and operating plans and progress reports. He helped create and implement the program’s initial logic model, exit survey and methods for demonstrating funder “return on investment”.
November 25, 2020
Noon – 1:00 pm EST
2019 LEARNING EVENTS
Challenges in implementing new methodologies and managing data
The next learning event will be panel presentations and a discussion on the Challenges in Implementing New Methodologies and Managing Data. Come hear approaches being developed and lessons learned by federal science Departments to implement new performance measurement methodologies and effectively manage results data. The purpose of the event is to share best practices in data management and methodology implementation related to the Policy on Results. The event is targeted to Science-based Organizations but anyone interested in sharing best practices in performance measurement and evaluation are welcome.
Pierre Pérusse
Director, Strategic Management
Agriculture and Agri-Food Canada
Juan Andres Leon
Senior Advisor, Office of Strategic Policy and Planning
Public Health Agency of Canada
Susan Morris
Director, Evaluation
Natural Sciences and Engineering Research Council
Louise-Michelle Verrier
Deputy Director, Corporate Strategy & Performance Division
Social Sciences and Humanities Research Council
Wednesday, April 10, 2019
RCAF Officers’ Mess
158 Gloucester Avenue
Ottawa, ON
9:00 am – 11:30 am (Registration starting at 8:30 am)
Registration Fee $60 + HST
2018 LEARNING EVENTS
Informing-Decision Making Following the Implementation of the Policy on Results
April 18, 2018
9:00 am – 12:00 pm
Registration begins at 8:00 am
Registration Fee $60.00 + HST
RCAF Officers’ Mess
158 Gloucester Street
Ottawa, ON
Registration Fee $60.00 + HST
CLICK HERE TO DOWNLOAD THE PRESENTATIONS ON THE EXCHANGEOVERVIEW:
The next learning event will be panel presentations and a discussion on the Use of Performance Measurement and Evaluation Results in decision-making. Come hear the Department of Fisheries and Oceans, Natural Resources Canada, Health Canada and the Canadian Space Agency talk about the challenges they have identified, approaches being considered to support the use of results and reporting processes and tools being implemented to leverage performance measurement and evaluation results in decision processes.
The purpose of the event is to share best practices and lessons learned in implementing the requirements of the Policy on Results, including the updated MAF lines of evidence. The event is targeted to Science-based Organizations but anyone interested in sharing best practices in performance measurement are welcome.
Panelists include:
Susan Cole, Director, Results Transformation, Department of Fisheries and Oceans
Marc Desjardins, Director General, Planning and Corporate Management Practices, Health Canada
Anne Routhier, Senior Director, Strategic Planning, Reporting and Results, Natural Resources Canada
Melanie Winzer, Executive Director, Programs and Integrated Planning, Canadian Space Agency
Moderated by:
Nadine Cyr, Senior Officer at the National Research Council
Hope to see you there!
Learning Event on Lessons Learned and Meaningful Reporting (focused on Science-Based Organizations but all are welcome)
March 22, 2018
9:00 am – 12:00 pm
Registration begins at 8:00 am
Registration Fee $60.00 + HST
RCAF Officers’ Mess
158 Gloucester Street
Ottawa, ON[/su_spacer]
Registration Fee $60.00 + HST
CLICK HERE TO DOWNLOAD THE PRESENTATIONS ON THE EXCHANGEOVERVIEW:
The next learning event will be a Panel Discussion on the Implementation of the Departmental Results Framework, Meaningful Reporting and sharing of best practices. Come hear multiple departments and agencies talk about the challenges they experienced, the reporting processes and tools being used and gain an insight on lessons learned from Treasury Board of Canada – Secretariat. The purpose of the event is to share best practices and give people a chance to ask the questions that have been on their minds since the implementation of the Departmental Results Framework.
Agenda:
8 am – 9 am – Registration and Networking Breakfast
9:00 am – 9:05 am – Opening Remarks by Melanie Winzer, Executive Director Programs and Integrated Planning, Canadian Space Agency
9:05 – 10 am – Panel Presentations and Questions
- Chris Boughton, A/ Director, Results Division, Expenditure Management Sector, Treasury Board of Canada Secretariat (10 min.)
- Sheryl Lattimore, Director, Corporate Planning and Governance, Innovation, Science and Economic Development Canada (10 min.)
- Joe Faragone, Manager, Results-based Management, Public Health Agency of Canada (10 min.)
- Nadine Cyr, Senior Analyst, Planning and Reporting, National Research Council Canada (10 min.)
10:00 am – 10:15 am – Networking Break
10:15 am – 11:00 am – Panel Questions and Sharing of Best Practices by Attendees
11 am – 12 pm – Table top group exercise on what meaningful reporting is, using recent extractions from recent Departmental Reports yet to be tabled.
2017 LEARNING EVENTS
Deliverology: Principles, Fundamentals and Practical Knowledge
A 1-1/2 day Learning Event
Day 1 – Thursday, February 16, 2017 1:00 to 4:00 pm
Day 2 – Friday February 17, 2017 8:30 am to 4:00 pm
Ottawa Conference and Event Centre
200 Coventry Road, Ottawa, Ontario
OVERVIEW:
You’ve heard and read so much about Deliverology in the federal and provincial governments and internationally. The Government of Canada has embedded the concepts into the new Policy on Results. It’s time to fully understand and gain practical knowledge in the fundamentals of this approach to creating high-performing organizations, from one of the co-authors of the Deliverology books and creators of the method who has worked internationally with many public sector organizations.
In addition, we will have Francis Bilodeau, Assistant Secretary to the Cabinet, Results and Delivery, Privy Council Office, to offer the Government of Canada perspective on this approach and how it applies to departments. At the end of the session we will be holding a discussion with Chief Results and Delivery Officers (CRDOs) to get their perspective on the practical implementation of these ideas in the Canadian Federal Government context.
SESSION LEADER:
Nick Rodriguez is one of the world’s leading experts in the theory and practice of Results and Delivery. He is Sir Michael Barber’s co-author of “Deliverology 101” and its successor, “Deliverology in Practice,” which have helped to institutionalize Results and Delivery as a discipline that can be taught, learned, and replicated. He has advised governments at every level on this discipline in the U.S., Europe, the Middle East, and Africa. Most recently, he co-founded and served as a Director for the U.S. Education Delivery Institute, a non-profit dedicated to bringing the Results and Delivery approach to the American education sector. Based in Washington, D.C. with Delivery Associates, he focuses his work on governments in Africa and the Americas.
Registration fee: $795 + HST
Included with the course registration:
- All learning materials
- A copy of the book “How to Run a Government: So that Citizens Benefit and Taxpayers Don’t Go Crazy” by Sir Michael Barber
- Light lunch and coffee / tea breaks both days
TECHNOLOGY ENABLEMENT FOR ENHANCED PERFORMANCE AND PLANNING
AGENDA:
8:00 Registration and Breakfast
8:30 Welcome and Update on 2017 PPX Symposium
8:40 Centralized Reporting and IBP
9:40 Better Data for Smarter Spending
10:15 Coffee and Networking
10:30 Panel Discussion
11:00 Close
CLICK HERE TO DOWNLOAD THE PRESENTATIONS ON THE EXCHANGE
OVERVIEW:
The recent Policy on Results strengthens the federal governments focus on the quality of data and the ability to monitor and report on performance across the enterprise. There are a variety of enabling technologies such as dashboards, scorecarding, and general Business Intelligence (BI) analytics that are being designed or implemented in numerous departments. In addition, many departments and agencies are now focusing on how technology can help streamline both the data collection and reporting of completely integrated business planning whereby strategic, operational, and financial planning can be brought together for more holistic decision-making.
In our morning program we’ll hear from Treasury Board Secretariat on the general direction of analytics and how departmental information is and will be used in tools such as Infobase, the searchable online database for providing both financial and HR information on government operations. The session will conclude with an open panel discussion to address questions from participants on the how best to initiate, implement, and sustain technology solutions for improving performance and planning. Please detailed presentations info below:
Presentation #1: Implementing Centralized Reporting at TBS and Exploring Shared Integrated Business Planning
ABSTRACT
In 2014, the Treasury Board of Canada (TBS) began a multi-year program to implement a centralized reporting infrastructure using the SAP Business Warehouse application and the SAP BusinessObjects Business Intelligence platform. Following SAP’s analytics road map for simplification and best practices, TBS developed a robust solution that provides improved data access to thousands of public sector workers. Now TBS has begun expanding on this strong base to move forward with integrated business planning.
Speaker Bio
Jonathan Andrews, Technical Director for Financial Management Transformation
Currently, managing the technical components of a project that will be creating a centralized financial and material management system for 50+ departments using S4HANA as well as managing the technical components of an Enterprise Information Reporting and Analytic system.
Previous Departments / Experience
- Managed the Central Online Reporting Platform for TBS
- Managed the deployment & maintenance of a SAP ECC 6.0 solution with a complex set of business functionality to support the budgetary process of the Government of Canada
- 10+ Years’ experience working on SAP projects as Functional or Reporting & Business Intelligence team lead
Ghislain Cardinal, Stream Lead for Financial Planning, Budgeting and Forecasting for TBS GCFM project
Currently managing the development of a project aimed at providing an integrated solution for Financial Planning, Budgeting, Forecasting and reporting for 50+ departments using BW and BPC powered by HANA.
Previous Departments / Experience
- Managed SAP BI and BPC for the Agriculture and Agri-Food Canada SAP cluster (AAFC/CFIA/NRCan/EC)
- Managed a Financial Management Advisory team providing services to multiple branches at Environment Canada
- Senior Financial Analyst in a corporate financial Resources Management unit at Environment Canada
- 15+ year experience in the financial management stream in the Federal Government
Presentation #2: Better Data for Smarter Spending
ABSTRACT
Our government is committed to improving how resources are allocated to eliminate poorly targeted and wasteful spending and provide clearer reporting on results. These commitments rely on better data and the capacity to fully exploit it. The TBS InfoBase has a proven track record to deliver linked, dynamic, clear and reliable reporting at a low cost. Created in 2013 as a government response to Parliament’s request for an integrated presentation of the spending they approve, the InfoBase is now an important pillar in the government’s results agenda. Moving forward, the InfoBase will be the central place where performance data can be found and analysed. We will present the tool as is, discuss some of the feedback we’ve received form users and provide a sneak-peak of upcoming functionality.
Speakers Bio
Andrew Gibson / Julie Van Audenrode
Andrew and Julie both work in the Expenditure Analysis group of TBS’ Expenditure Management Sector (EMS) and are responsible for the TBS InfoBase. Julie has over five years’ experience working at TBS; previous to EMS, she worked in Priorities and Planning where she helped coordinate the Management Accountability Framework. Julie has a background in Economics and International Affairs. Andrew has worked in EMS for over ten years in various positions and has a background in Engineering.