PEBC Toolkit

The Guideline Endorsement Protocol provides a process for OH (CCO) to use guidelines from other jurisdictions/guideline developers in order to provide guidance for use in Ontario. This strategy will enable access to high quality relevant guidance in a shorter time frame and using fewer resources than would be expected from de novo guideline development. OH(CCO) teams are encouraged to use the Endorsement Protocol whenever possible. Once a candidate guideline has been identified and assessed for quality, the developers of the original guidance should be contacted before starting an endorsement process to determine if there are any licensing, intellectual property or costs associated with the use of materials.

PEBC staff should refer to the Guideline Endorsement section on the WIKI Document Templates page for the PEBC guideline endorsement template and additional instructions.

The endorsement strategy is appropriate for use when either (a) a known guideline has been released by another organization and there is interest by a OH(CCO) team to immediately endorse it (a priori endorsement situation) OR (b) in starting a new guideline-related project, a OH(CCO) guidelines team identifies an appropriate document from another group during the scoping of the project or the initial literature search (post hoc endorsement situation). It is strongly recommended that Clinical Practice Guidelines that are being considered for endorsement should be assessed using the AGREE II Instrument. AGREE scores should be reported in the final endorsement document.

Regardless of circumstances, the following resources are required:

• A staff person (PEBC or non-PEBC) available and capable of executing the endorsement protocol and drafting the necessary documents as work proceeds. The staff person does not need to have the full methodological experience and training of a PEBC health research methodologist, but the individual must be able to execute the steps in the protocol, as described below.
• Clinical experts on the topics in question are available and willing to participate in the process, as described below

From initiation to the publication of the endorsement on the OH(CCO) web site will take between 3 and 8 months. The exact timeframe will vary depending on: the size of the guideline to be endorsed; the complexity of the topic; whether or not professional consultation is sought; availability of volunteer clinical experts for calls/meetings, etc. In contrast, de novo guideline development takes between 18 months and 30 months to complete.

• Working Group: A small group (3-6 members) of individuals who will do the work of drafting the endorsement document. This includes a staff person (PEBC or non-PEBC, referred to in this protocol as the Endorsement Coordinator) who will be responsible for the day to day execution of the protocol, and several clinicians with relevant clinical expertise. It is strongly recommended that OH(CCO) Ontario Cancer Leads or Program Leads in areas relevant to the endorsement topic be part of the initial discussions and the initial assessment of the targeted guideline for endorsement (Steps 1 and 2, described below).
• Expert Panel: A larger group (usually at least 10 members) of clinical/content experts who will review the draft endorsement document, provide feedback, and approve the final version.
• Conflicts of Interest should be managed according to relevant protocols at the PEBC (see PEBCConflictInterestPolicy.pdf) or at OH(CCO).

The process proceeds in the following stages. Each stage is described in detail below.

1. Initial assessment of the candidate guideline by the Working Group
2. Project Planning
3. Assessment of Recommendations by Working Group
4. Draft Endorsement Document
5. Expert Panel Approval
6. Professional Consultation
7. Publication

During the initial assessment, the goal is to determine whether the potential guidelines are appropriate candidates for endorsement. If they are not, the process stops and the OH(CCO) program area or Guideline Development Group will need to consider what other options are available (doing nothing, developing a de novo guideline, updating an existing guideline, etc.) The initial assessment would likely take place as a conference call, moderated by the Endorsement Coordinator.

The initial assessment should consider the following questions.

SCOPE, RELEVANCE, AND TIMELINESS

1) Do the guideline’s questions/objectives address all clinically and jurisdictionally relevant aspects of the topic for which guidance is being sought in Ontario?

2) Is it unlikely that new recommendation-changing evidence has been published since the guideline was developed? (In general, a guideline more than 3 years old should be considered skeptically.)

QUALITY AND METHODS

3) Was the guideline based upon a systematic review of the available evidence, that is, it is not strictly a consensus/expert opinion-based guideline?

4) Is the systematic review of the evidence available for review?

5) Is the link between the guideline’s recommendations and the evidence, as well as the guideline panel’s reasoning and justification of the recommendations clearly described?

OH(CCO) RESOURCE AVAILABILITY

6) Would extensive effort be necessary on the part of OH(CCO) to replicate or improve on the guideline?

ACCEPTABILITY

7) Is it believed that the guideline recommendations would be generally acceptable without substantial controversy in Ontario?

There are no strict thresholds (e.g. number of “yes” answers) at which the Working Group should definitely consider or definitely reject endorsement. If all of these questions are answered “Yes”, then the guideline(s) are likely excellent candidates for endorsement. If all were answered “No”, the guideline(s) are very poor candidates for endorsement. The more “yes” answers, the more endorsement will likely be successful.

Once a decision is made to endorse a guideline, participants can help to ensure quicker progress along with better quality and value in the final product by applying these principles:

• Change only what must be changed
• Add only what must be added

The ability to accept the candidate guideline “as is” streamlines the process tremendously. Teams are encouraged to approach the candidate guideline as being one that is “good enough”.

Once the decision has been made to continue with endorsement, a project plan should be created. The plan should have the following elements.

• Participants: the Working Group and Expert Panel should be named. The Endorsement Coordinator will invite these participants, and should collect potential conflict of interest declarations (using the PEBC CoI policy or an applicable OH(CCO) policy). An Endorsement Lead should be chosen at this time to chair the Working Group and Expert Panel.
• Professional Consultation: a decision will need to be made about whether the Endorsement should be externally reviewed through professional consultation. Professional consultation is an attempt to get the draft endorsement document in the hands of as many of the Ontario stakeholders as possible before it is finalized, to solicit their feedback and as a dissemination tool. Most endorsement documents should go through professional consultation.
• Work Timelines: a Timeline for the work should be prepared. In general, the process will take between 1 and 3 conference call/meetings of the Working Group, and then one or two additional conference call/meetings of the Expert Panel. The timeline will be contingent on how quickly these calls/meetings can be arranged for the participants. Also, professional consultation can be expected to add at least 6 and as many as 10 weeks to the timeline. All members of the Working Group should approve the project plan.

During this phase, the Working Group will assess the individual recommendations of the guideline.

The Working Group considers the following issues for each recommendation:

• Interpretation and Justification: Does the Working group agree with the interpretation of the evidence and the justification of the recommendation in the original guideline?
• Ontario Applicability/Relevance: Do any of the recommendations need modification to the Ontario context? Is it applicable at all to Ontario?
• Likelihood of New Evidence: For each specific recommendation, is it likely there is new, unidentified evidence that would call it into question?
• Qualifications/Clarifications: Would additional statements of qualification/clarification be valuable in Ontario?

For each recommendation, a final determination will be made as to whether the recommendation is:

• Endorsed Unchanged
• Endorsed with some clarification/modification
• Rejected

An example of a table that can be used by the Working Group to work through the recommendations can be found here: Example Recommendations Assessment Table

If the Working Groups’s initial assessment and decision was accurate, and if the Working Group is able to apply the 'good enough' endorsement principles noted above, it should be expected that the majority of the recommendations will be endorsed unchanged, a small proportion may need some clarification or modification, and very few or none will be rejected.

In rare cases, the Working Group may consider drafting new recommendations, or replacing rejected recommendations. However, the Working Group should not do so unless they have access to the same description of the underlying evidence that would be available for a new guideline, and without the same level of attention to quality/certainty of the evidence, balancing of benefits against harms, and other activities that would be considered in a new guideline. This may add substantially to the time and effort it takes to complete the endorsement, so should be considered carefully.

Upon completion of the assessment of recommendations, the Endorsement Coordinator will draft the document. A template for an endorsement document is available here: pebc_guideline_endorsement_template.dotm NOTE: If you are unable to access this template file, contact Sheila McNair at mcnairs@mcmaster.ca, who can arrange for you to access it. PEBC staff should refer to the WIKI Document Templates page for the PEBC guideline endorsement template.

If the guideline recommendations were endorsed with no changes or clarifications, than the endorsement document may simply point the reader to the original guideline. But it there were any changes or clarifications, the full recommendations should be reprinted from the original guideline to maximize usability; permission should be sought from the original guideline developer as necessary. Once the document is drafted, the Working Group should approve it as ready to go the Expert Panel.

The draft endorsement document should be presented to the Expert Panel for their consideration, feedback, and approval. In the case of longer, more complicated guidelines, this is likely best done in the form of a face-to-face meeting.

The Expert Panel should achieve consensus on the endorsement document. This is intended to be a streamlined, non-controversial process. If there is difficulty achieving consensus in the Expert Panel, assuming the Panel has adopted the attitude of endorsement described above, this is a sign that the initial assessment that the guideline was a candidate for endorsement may have been in error.

If the planning decision was made to have the endorsement document reviewed through Professional Consultation, this should be conducted after Expert Panel approval. The PEBC has resources and capacity to conduct professional consultation and a full description of that process is outside the context of the Endorsement Protocol. The feedback from professional consultation should be considered by the Working Group, and a response to each major area of feedback formulated. The response would either be a change to the text, or no change but written description of why the text was maintained.

Once the endorsement is completed, it should be published in an appropriate location on the OH(CCO) website, and then disseminated in an appropriate manner (especially if no professional consultation was conducted).

An endorsement should be reconsidered each year by the OH(CCO) Program Area responsible for the endorsed document. The program area should consider the following issues:

• Has the original guideline been updated?
• Has a new guideline on the same topic been published?
• Is the Program Area aware of new evidence that may change the recommendations?
• Is the endorsement more than three years old?

If any of those questions is answered “yes”, the Program Area should consider…

• Implementing the Endorsement process again on the newer guidelines.
• Withdrawing the endorsement from the OH(CCO) website

Any endorsement older than three years that remains on the OH(CCO) website should probably be updated with additional language to indicate that it has been reviewed and is considered still valid, with the date of the last consideration.