Multi-Source Feedback

Colleague Feedback is a process in which colleagues give you feedback on how they perceive you work with them. It looks at a number of attributes applicable to all doctors rather than your individual clinical skills and knowledge. The feedback is usually obtained using an anonymous questionnaire. Colleague Feedback is often referred to as Multi-Source Feedback (MSF) or 360 degree feedback.

The purpose of Colleague Feedback (MSF) is to provide you with feedback that gives you an opportunity to reflect on the feedback received and to compare it with your own self-assessment. It is hoped that this will reinforce good practice and provide opportunities for development. The feedback can provide information about important qualities you demonstrate as a doctor, such as integrity, respect and the ability to communicate and work as a team member, and how these are perceived by those you work with.

The doctor seeking feedback (you) invites colleagues to complete a short questionnaire. Colleagues are selected from those with whom you work on a regular basis and from across your practice. For example other doctors, nursing colleagues, administrative staff, managerial staff, allied health professionals, other health professionals and reception staff.

In most systems, including the NES MSF and the GMC Colleague Questionnaire, the doctor also completes a self assessment questionnaire. In most systems the questionnaire includes questions with a scale and space for free text comments.

The results are then collated and a summary report is provided to the doctor. (The NES MSF web based system does this electronically). You are encouraged to reflect on:

• the report;
• any differences in the scores you gave yourself and the average scores given by others;
• any areas highlighted as particular strengths; and
• any areas where development or change might be required.

Research suggests that if enough colleagues participate, then the scores will be consistent across different "raters". This applies whether you selected the raters or the raters are selected for you.

However, individual people may see you differently and may score you differently. Some may work with you in different settings. Some raters will work closely with you in a clinical setting; others may only see you at the beginning and end of sessions or clinics to debrief on the administration. Free text comments are one individual's view and this should be borne in mind when looking at the results.

Most Colleague Feedback systems recommend that you discuss your results formally with someone within a few days of receiving them. This could be a trusted colleague, an appraiser or any other trained facilitator. The reasons for this are that it can be very helpful to talk through the results; fresh insights and understanding may come from this. The feedback may also generate powerful feelings in the recipient which he/she may benefit from exploring with another person.

If you are accessing MSF via SOAR for the first time, you may be asked at some stage to "Connect to WASP". The NES MSF tool was developed by a company called WASP, so when you come across "WASP" or "WASP MSF" - that is what it is referring to.

You may have already done your MSF and your report is currently held on WASP, or you may be just getting started on the MSF process.  Regardless, you need to complete the first steps below to link your SOAR and WASP accounts first.

1. Login to SOAR (as Appraisee)
2. Click on the "MSF" link in the menu (located underneath "My Details")
3. On the MSF page, answer the questions as if you are just beginning the MSF process (even if you already have an MSF report waiting on WASP)
4. Click "Begin MSF"
5. When the page refreshes, scroll to the bottom and click on "Connect your WASP account" button
6. You will next be asked to give permission for SOAR to access your WASP account - please click Yes
7. Now your SOAR and WASP accounts are linked.  Two things will happen at this stage:
   • You will then be redirected back to SOAR, and
   • A pop-up page should open, taking you to WASP (If you have a pop-up blocker, please adjust your browser settings accordingly)
8. If you are looking to proceed with the MSF process, or you wish to check on your MSF progress, continue and follow the instructions in the pop-up window.
   • If you accidentally closed the pop-up window, return to the page on SOAR and click on "Access WASP"
9. If you are looking to download an MSF report already completed, return to the SOAR page and click on "Download your latest data".  When clicked, if there is an MSF report on WASP, a copy of the report will be downloaded to your "My Documents" (link on menu).  Additionally:
   • If you have NOT yet submitted your appraisal forms to the interview, the MSF report will also be added to your draft Form 3, so that this will be made available to the appraiser when submitted;
   • If you HAVE already submitted your appraisal forms to the interview, the MSF report will be inserted to the submitted forms (saves you from having to resubmit), and an email will be sent to the appraiser notifying them of this.
10. Once the MSF report is downloaded to SOAR from WASP, the WASP version of the report is archived after the interview date, and the version on SOAR will be the main copy (i.e. don't delete it from your "My Documents"!).

[To begin the MSF process, you need to login to SOAR and connect your SOAR and WASP accounts, see "Overview of accessing MSF via SOAR" above for more information.]

For all MSF processes, we ask that you proceed at least 4 weeks before your scheduled appraisal meeting.  The MSF process takes 3 weeks to conclude (waiting for your nominated raters' responses).  The collated report will be downloaded automatically to SOAR a week before your appraisal interview.

If you are initiating your MSF process and your appraisal meeting is less than 4 weeks away, the automated MSF report download function will NOT work for you.  If this is the case, please email SOAR@nes.scot.nhs.uk and let us know so we can manually close off your report earlier than the process runs for.  We need to know your GMC number and the date of your appraisal interview. This ensures you have time to review and reflect on the report.

For new MSF processes

When you have successfully logged into WASP, you will be asked to complete a couple of registration / equal opportunity questions (such as your profession, age, ethnic group etc).  When completed, you will then be asked to complete your self assessment questionnaire.

Self assessment questionnaire:

In your self assessment, you will be asked questions on Communication, Respect, Manner, Team, Knowledge, Support, Insight and Overall.

We have included screenshots of what questions you will be asked in each section below.

Once you have completed your self assessment, you then need to enter email addresses of your nominated raters.

Please note the self assessment questions are the same that are asked of your nominated raters, so the wording of the question will refer to you in the third person.  (e.g. "is the doctor approachable" = "are you approachable" etc)

Nominated raters:

You are advised to enter at least 15 raters for your MSF process.  The system requires at least 8 raters' email addresses entered into the system.

8 raters' responses are also the minimum requirement for an MSF report to be generated.  The more people you nominate, the better chance you have of reaching the minimum 8 rater responses required.

MSF Report:

If you have at least 8 raters responded to your MSF, and if your appraisal interview is on SOAR, then your MSF report will automatically be downloaded to your interview on SOAR a week before your interview, allowing you time to reflect on your report and prepare for discussions with your appraiser.

MSF can be initiated at any time throughout the year.

Each 5 year revalidation cycle should include one MSF report.

It is our recommendation that appraisees try and time it around 6-8 weeks before their appraisals to tie in with the automated processes.

MSF on WASP takes 3 weeks (to allow raters time to provide feedback). if the report has enough rater responses and the appraisee has done their self assessment, then it closes automatically after 3 weeks.

If Interview is on SOAR, then 7 days before interview date, SOAR goes in and downloads report automatically and sends out an auto email confirmation when completed.

If Interview is not on SOAR, it simply sits on WASP until either the appraisee or an administrator go in and manually download it.

Link your WASP and SOAR accounts first, then manually download the MSF report.

If for whatever reason, when you click on the "Download latest data" button and you are told "No MSF Data available", email us at SOAR@nes.scot.nhs.uk with your GMC number and we will investigate this further for you.

If you receive the "Error 500" page, it means your SOAR and WASP accounts have not been able to link up, or the link is broken.  The reason for this is because the email address you have on SOAR and WASP are different (maybe you updated the email address on SOAR?).

To resolve this, please email SOAR@nes.scot.nhs.uk, enter "Error 500" in the subject title, and in the email itself, tell us your GMC number and your preferred email address to use, and we will then manually make the changes for you.

Simply upload your MSF report via the MSF page:

1. Login to SOAR
2. Click on MSF from the menu
3. Scroll to the bottom and use the file upload feature to select your MSF report from your computer and click upload.

Do not forget to upload your reflections!

Alternatively you can also upload directly into Form 3 (Domain 3), and submit it when all your forms have been completed.

For those users who are beginning the MSF process from SOAR, their MSF reports will be downloaded directly to SOAR a week prior to the appraisal interview. You must ensure your appraisal interview is added to SOAR. If it's not there, liaise with your appraiser to get this added first.

For each MSF report held on WASP, there is a "Report due date" associated with it. As long as 8 raters have responded and the self-assessment completed, the MSF report will be generated and sent from WASP to SOAR the next morning after the "Report due date" (a little after 9am). For example, if the "Report due date" was on 21/02/2013, then the MSF report will be generated and sent to SOAR on 22/02/2013 around 9am.

No.

Submit your appraisal forms when you have finished completing them.  When the MSF report is ready, it will automatically be downloaded to your "My Documents" on SOAR, and your submitted appraisal forms.  The system will also email the appraiser alerting them to this, so there is literally no action required from you.

If your MSF report is ready before you submit your appraisal forms, it will still be downloaded to your "My Documents" - a copy of it will be inserted into your draft Form 3 Domain 3 as well, so when you submit your appraisal forms it will automatically be there for the appraiser to review.

Due to the anonymised nature of the MSF process, you are not meant to find out who has/hasn't responded to the MSF requests, just numbers.

If you wish to contact the non-responders, simply trigger the reminder email for all - the WASP system will not email those who have already completed the requested feedback.

1. Login to SOAR
2. Click on MSF from the menu on left
3. Click on "Access WASP" button
4. You are now redirected to the WASP website, click on "Nominate Raters" from menu on top
5. Click "Resend Email" button in bottom right of page.

If you to to add more raters to the list, simply follow steps 1-4 above, and then add more raters just as you did so when you first nominated your raters.

Because you are only required to do one MSF over a 5 year (revalidation) cycle, the WASP system was not configured to do MSF in continuous years. However, as with most things, we do have a workaround if this is a requirement for you.

We just need to ensure you definitely have a copy of the recent MSF report, and then we will go into WASP and delete this MSF so that you can proceed with a fresh one.

Not ideal but it's the next best thing.

If you wish to proceed with this, please submit a helpdesk ticket for further assistance.

Our apologies for the confusion, but the email you received will be triggered if your MSF report is due and you have not received enough rater responses, OR you have not completed the required self assessment questionnaire (most common scenario).  To do this:

1. Login to SOAR 
2. Click on MSF from menu 
3. Click on "Access WASP" button which will redirect you to WASP system 
4. Click on "Complete Questionnaire" from menu along top 
5. Follow the onscreen instructions and complete the self assessment, finish by clicking on "Finish" on last page (important)

A lot of times it's a case of users not clicking on the "Finish" button on the last page.