The following is a collection of Frequently Asked Questions by doctors working in Scotland, regarding Multi-Source Feedback.
Please note the MSF tool on SOAR is NOT designed for Trainee doctors usage. If you are a Trainee doctor and need to carry out an MSF, please contact your Deanery regional team for further assistance.
If you have any queries that don't appear below, please contact the SOAR Helpdesk for further assistance.
Based on research and background work done by Drs Murray Lough and Diane Kelly, NES developed an MSF tool, designed to be used by all doctors working in Scotland. The MSF tool was built into an online system by a company called WASP.
Funding was provided by the Scottish Government to link SOAR with this MSF tool, so that users can access the WASP MSF system via SOAR, and more importantly, the MSF reports can be automatically sent to SOAR near the time of the appraisal meeting.
This link project was completed and deployed to SOAR on 19th February 2013.
Colleague Feedback is a process in which colleagues give you feedback on how they perceive you work with them. It looks at a number of attributes applicable to all doctors rather than your individual clinical skills and knowledge. The feedback is usually obtained using an anonymous questionnaire. Colleague Feedback is often referred to as Multi-Source Feedback (MSF) or 360 degree feedback.
The purpose of Colleague Feedback (MSF) is to provide you with feedback that gives you an opportunity to reflect on the feedback received and to compare it with your own self-assessment. It is hoped that this will reinforce good practice and provide opportunities for development. The feedback can provide information about important qualities you demonstrate as a doctor, such as integrity, respect and the ability to communicate and work as a team member, and how these are perceived by those you work with.
The doctor seeking feedback (you) invites colleagues to complete a short questionnaire. Colleagues are selected from those with whom you work on a regular basis and from across your practice. For example other doctors, nursing colleagues, administrative staff, managerial staff, allied health professionals, other health professionals and reception staff.
In most systems, including the NES MSF and the GMC Colleague Questionnaire, the doctor also completes a self assessment questionnaire. In most systems the questionnaire includes questions with a scale and space for free text comments.
The results are then collated and a summary report is provided to the doctor. (The NES MSF web based system does this electronically). You are encouraged to reflect on:
Research suggests that if enough colleagues participate, then the scores will be consistent across different "raters". This applies whether you selected the raters or the raters are selected for you.
However, individual people may see you differently and may score you differently. Some may work with you in different settings. Some raters will work closely with you in a clinical setting; others may only see you at the beginning and end of sessions or clinics to debrief on the administration. Free text comments are one individual's view and this should be borne in mind when looking at the results.
Most Colleague Feedback systems recommend that you discuss your results formally with someone within a few days of receiving them. This could be a trusted colleague, an appraiser or any other trained facilitator. The reasons for this are that it can be very helpful to talk through the results; fresh insights and understanding may come from this. The feedback may also generate powerful feelings in the recipient which he/she may benefit from exploring with another person.
If you are accessing MSF via SOAR for the first time, you may be asked at some stage to "Connect to WASP". The NES MSF tool was developed by a company called WASP, so when you come across "WASP" or "WASP MSF" - that is what it is referring to.
You may have already done your MSF and your report is currently held on WASP, or you may be just getting started on the MSF process. Regardless, you need to complete the first steps below to link your SOAR and WASP accounts first.
[To begin the MSF process, you need to login to SOAR and connect your SOAR and WASP accounts, see "Overview of accessing MSF via SOAR" above for more information.]
For all MSF processes, we ask that you proceed at least 4 weeks before your scheduled appraisal meeting. The MSF process takes 3 weeks to conclude (waiting for your nominated raters' responses). The collated report will be downloaded automatically to SOAR a week before your appraisal interview.
If you are initiating your MSF process and your appraisal meeting is less than 4 weeks away, the automated MSF report download function will NOT work for you. If this is the case, please email SOAR@nes.scot.nhs.uk and let us know so we can manually close off your report earlier than the process runs for. We need to know your GMC number and the date of your appraisal interview. This ensures you have time to review and reflect on the report.
When you have successfully logged into WASP, you will be asked to complete a couple of registration / equal opportunity questions (such as your profession, age, ethnic group etc). When completed, you will then be asked to complete your self assessment questionnaire.
Self assessment questionnaire:
In your self assessment, you will be asked questions on Communication, Respect, Manner, Team, Knowledge, Support, Insight and Overall.
We have included screenshots of what questions you will be asked in each section below.
Once you have completed your self assessment, you then need to enter email addresses of your nominated raters.
Please note the self assessment questions are the same that are asked of your nominated raters, so the wording of the question will refer to you in the third person. (e.g. "is the doctor approachable" = "are you approachable" etc)
You are advised to enter at least 15 raters for your MSF process. The system requires at least 8 raters' email addresses entered into the system.
8 raters' responses are also the minimum requirement for an MSF report to be generated. The more people you nominate, the better chance you have of reaching the minimum 8 rater responses required.
If you have at least 8 raters responded to your MSF, and if your appraisal interview is on SOAR, then your MSF report will automatically be downloaded to your interview on SOAR a week before your interview, allowing you time to reflect on your report and prepare for discussions with your appraiser.
MSF can be initiated at any time throughout the year.
Each 5 year revalidation cycle should include one MSF report.
It is our recommendation that appraisees try and time it around 6-8 weeks before their appraisals to tie in with the automated processes.
MSF on WASP takes 3 weeks (to allow raters time to provide feedback). if the report has enough rater responses and the appraisee has done their self assessment, then it closes automatically after 3 weeks.
If Interview is on SOAR, then 7 days before interview date, SOAR goes in and downloads report automatically and sends out an auto email confirmation when completed.
If Interview is not on SOAR, it simply sits on WASP until either the appraisee or an administrator go in and manually download it.
Link your WASP and SOAR accounts first, then manually download the MSF report.
If for whatever reason, when you click on the "Download latest data" button and you are told "No MSF Data available", email us at SOAR@nes.scot.nhs.uk with your GMC number and we will investigate this further for you.
If you receive the "Error 500" page, it means your SOAR and WASP accounts have not been able to link up, or the link is broken. The reason for this is because the email address you have on SOAR and WASP are different (maybe you updated the email address on SOAR?).
To resolve this, please email SOAR@nes.scot.nhs.uk, enter "Error 500" in the subject title, and in the email itself, tell us your GMC number and your preferred email address to use, and we will then manually make the changes for you.
Simply upload your MSF report via the MSF page:
Do not forget to upload your reflections!
Alternatively you can also upload directly into Form 3 (Domain 3), and submit it when all your forms have been completed.
For those users who are beginning the MSF process from SOAR, their MSF reports will be downloaded directly to SOAR a week prior to the appraisal interview. You must ensure your appraisal interview is added to SOAR. If it's not there, liaise with your appraiser to get this added first.
For each MSF report held on WASP, there is a "Report due date" associated with it. As long as 8 raters have responded and the self-assessment completed, the MSF report will be generated and sent from WASP to SOAR the next morning after the "Report due date" (a little after 9am). For example, if the "Report due date" was on 21/02/2013, then the MSF report will be generated and sent to SOAR on 22/02/2013 around 9am.
Submit your appraisal forms when you have finished completing them. When the MSF report is ready, it will automatically be downloaded to your "My Documents" on SOAR, and your submitted appraisal forms. The system will also email the appraiser alerting them to this, so there is literally no action required from you.
If your MSF report is ready before you submit your appraisal forms, it will still be downloaded to your "My Documents" - a copy of it will be inserted into your draft Form 3 Domain 3 as well, so when you submit your appraisal forms it will automatically be there for the appraiser to review.
Due to the anonymised nature of the MSF process, you are not meant to find out who has/hasn't responded to the MSF requests, just numbers.
If you wish to contact the non-responders, simply trigger the reminder email for all - the WASP system will not email those who have already completed the requested feedback.
If you to to add more raters to the list, simply follow steps 1-4 above, and then add more raters just as you did so when you first nominated your raters.
Because you are only required to do one MSF over a 5 year (revalidation) cycle, the WASP system was not configured to do MSF in continuous years. However, as with most things, we do have a workaround if this is a requirement for you.
We just need to ensure you definitely have a copy of the recent MSF report, and then we will go into WASP and delete this MSF so that you can proceed with a fresh one.
Not ideal but it's the next best thing.
If you wish to proceed with this, please submit a helpdesk ticket for further assistance.
Our apologies for the confusion, but the email you received will be triggered if your MSF report is due and you have not received enough rater responses, OR you have not completed the required self assessment questionnaire (most common scenario). To do this:
A lot of times it's a case of users not clicking on the "Finish" button on the last page.
This page was last updated on: 11/03/2022