|Year : 2015 | Volume
| Issue : 1 | Page : 12-17
Improving medical education: Need for educational research
Kurien Thomas1, Aneesh Basheer1, Ashok Kumar Das1, Sheela Kuruvilla2, N Nagaraj1, Thomas Alexander3, Georgi Abraham1, Bhirappa Shriman1
1 Department of General Medicine, Pondicherry Institute of Medical Sciences, Ganapathichettikulam, Kalapet, Puducherry, India
2 Department of Dermatology, Pondicherry Institute of Medical Sciences, Ganapathichettikulam, Kalapet, Puducherry, India
3 Department of Medical Education, Pondicherry Institute of Medical Sciences, Ganapathichettikulam, Kalapet, Puducherry, India
|Date of Submission||14-Sep-2015|
|Date of Acceptance||08-Oct-2015|
|Date of Web Publication||9-Nov-2015|
Department of General Medicine, Pondicherry Institute of Medical Sciences, Ganapathichettikulam, Kalapet, Puducherry - 605 014
Source of Support: None, Conflict of Interest: None
Background: A formal evaluation of the Postgraduate (PG) training program has not been previously carried out at Pondicherry Institute of Medical Sciences. An evidence-based participatory approach was considered important to identify activities and to improve the training program.
Population: Important stake holders to the PG training program including PG students, faculty in the Department of General Medicine, Faculty of Specialty Training the PGs, PG program administrator, and representative from the educational unit of the institution participated in the program evaluation (PE).
Methods: The strategic planning (SP) exercise consisted of three activities. (1) A rapid assessment of present status by an anonymous cross-sectional survey of all identified stake holders (2) presentation of the results of the survey and discussion on different dimensions of PG training (3) a situational analysis of strengths, weakness, opportunities, and threats (SWOT) to develop an evidence-based recommendations on the definite steps to be under taken to improve the program.
Results: There were 25/28 (89%) identified stake holders participated in the exercise. Through the exercise, consensus was obtained to broaden the training objectives to include seven competencies necessary for PG student. The overall satisfaction with training was 80.5% (95% confidence interval [CI]: 74.3–86.7). However, there were five responders (20% 95% CI: 9–39) who scored <80% for overall score. It identified 15 definite steps to strengthen the program in the dimensions of (1) educational resources and inputs, (2) educational interactions and postings, and (3) student evaluation.
Conclusions: Participatory research, including PE and SWOT analysis are valid tools, which can be used to understand the present status and identify activities as for evidence-based SP in academic medical institutions.
Keywords: Educational research, strategic planning, SWOT analysis
|How to cite this article:|
Thomas K, Basheer A, Das AK, Kuruvilla S, Nagaraj N, Alexander T, Abraham G, Shriman B. Improving medical education: Need for educational research. J Curr Res Sci Med 2015;1:12-7
|How to cite this URL:|
Thomas K, Basheer A, Das AK, Kuruvilla S, Nagaraj N, Alexander T, Abraham G, Shriman B. Improving medical education: Need for educational research. J Curr Res Sci Med [serial online] 2015 [cited 2019 Jan 18];1:12-7. Available from: http://www.jcrsmed.org/text.asp?2015/1/1/12/168922
| Introduction|| |
The standards for Medical Education in India are set by the Medical Council of India (MCI). MCI has the responsibility to set minimum standards for certification for both Undergraduate (UG) and Postgraduate (PG) programs in the recognized medical colleges in India. However, individual institutions have the freedom to improve on the educational interactions and toward that end; every medical college has the requirement of having a Department/Unit of Medical Education. Program evaluation (PE), and strategic planning (SP) are essential steps in understanding the quality and status of any program and are also tools used in educational research. The present study describes a method and outcome of PE and SP activity in the Department of General Medicine carried out at the Pondicherry Institute of Medical Sciences (PIMS) for improving the PG training program.
The primary objective
To evaluate the ongoing PG teaching program to assess stake holder satisfaction with:
- Educational inputs and resources
- Educational interactions and postings.
The secondary objectives
- To identify the strengths, weaknesses, opportunities, and threats (SWOT) to the ongoing PG training program
- To develop consensus on the way forward to improve the program and meet the training objectives specified.
| Methods|| |
A cross-sectional survey method was used to evaluate the level of satisfaction with the existing PG teaching programs and postings. The following stake holders at PIMS were eligible to participate in the survey.
- All PG students in the Department of General Medicine
- All faculty in the Department of General Medicine
- Faculty from Specialty Departments (Neurology, Cardiology, and Gastroenterology) taking part in the PG teaching programs.
The questionnaire [Appendix 1 [Additional file 1] ] consisted of 21 questions on the following dimensions of interest. There were 17 questions assessing satisfaction with quantitative responses scored on a 10 point Likert scale and 4 open-ended qualitative questions. The self-administered questionnaire was completed anonymously by the participants of the survey, and the administrator had no idea of who the responders were. The quantitative components of the questionnaire covered the following dimensions:
- Overall satisfaction with the teaching program (1 question)
- Educational inputs and resources (5 questions)
- Educational interactions and postings (10 questions)
- Assessment and evaluation (1 question).
Educational inputs and resources were rated with five questions. These were related to (1) content of the teaching training program, (2) regularity and punctuality of teaching sessions, (3) availability of faculty, (4) availability of patients, and (5) availability of other learning resources, for example, E-learning.
Educational interactions and postings were assessed with 10 questions. These included (1) morning audit, (2) paper reading, (3) journal Club, (4) ward rounds, (5) Intensive Care Unit (ICU) posting, (6) specialty posting, (7) afternoon clinics, (8) clinical meeting, (9) institutional coordinated market economies, and (10) mortality meeting.
Time allocation for these activities was scored with 1 question, as was the method of evaluation and feedback.
There were four open-ended questions related to:
(a) Strengths, (b) weakness, (c) suggestions for improvement, and (d) type of responder.
Meeting of stake holders
A meeting of all the stakeholders was held and the following activities were included in this meeting. The faculty participated in all the four sessions below. However, the PG students participated only in the discussions on session 2 (results of the questionnaire).
- Analysis of laid down aims and objectives of the training program
- Discussions on the result of the questionnaire
- Discussion on the SWOT as perceived by the faculty
- Suggestions and the way forward was finalized based on the inputs received in the meeting.
| Results|| |
The stakeholders who participated included 13/15 faculty of the Department of General Medicine, 10/12 PG students from the department general medicine, 4/5 faculty from specialty departments, and 1 faculty from the Medical Education Unit.
A total of 25 participants responded to the survey questionnaire. The distribution is given in [Table 1].
The mean overall score regarding satisfaction was 80.5% (95% confidence interval [CI]: 74.3–86.7). Data are given in [Table 2]. The median scores given by the three groups of responders were 80%, 85%, and 80% for PG Students, General Medicine Faculty, and Specialty Faculty, respectively. This was statistically not different (P = 0.9) [Figure 1]. There were 5/25 (20% 95% CI: 9–39) responders who marked <80% for the overall score.
Educational inputs and resources
The mean score for five items assessing inputs and resources was 74% (95% CI: 68–80%). The median was 76% with the minimum score being 40% and maximum 100%. The mean score of five sub items for the educational inputs varied from 67% for availability of patients to 81% for regularity and punctuality. The individual scores obtained in the five items are given in [Table 3].
The following items were scored <75%:
- Availability of patients
- Availability of faculty.
Educational interactions and postings
The mean score for interactions and postings was 76% (95% CI: 68–84%). The median was 81% with a minimum of 32% and maximum of 98%. The mean score for 10 sub items for interactions and postings varied from 69% for specialty posting to 85% for journal club. Individual scores for the 10 questions are given in [Table 4].
The following items were scored <75%:
- ICU posting
- Specialty posting
- Clinical meeting.
Time allocation for teaching learning interactions
The mean score for time allocation was 72% (95% CI: 58–87%). The minimum was 10% and maximum was 100%.
Student assessment and evaluation
The mean score obtained for student evaluation was 77% (95% CI: 66–88%). The minimum was 40% and maximum was 100%.
Open-ended questions and strengths, weakness, opportunities, and threat analysis
All the 25 participants gave inputs on strengths, weakness, and suggestions on the way forward. The strengths and weakness are summarized in [Table 5] and [Table 6], respectively. During the meeting of stakeholders based on the strengths and weakness identified through the questionnaire, the faculty listed the potential opportunities and threats.
Strengths, weakness, opportunities, and threat analysis
A total of 22 responders answered this question. The following categories of responses were deemed as important factors contributing to the strength of the PIMS training program:
- Number of responders who listed the academic programs = 11
- Number of responders who listed the faculty = 8
- Number of responders who listed both faculty and programs = 2.
A total of 20 responders listed the perceived weaknesses of the program. The main categories were:
- Number of responders who listed student and motivation = 9
- Number of responders who listed time = 6
- Number of responders who listed other causes = 5.
An analysis of opportunities to improve training centered on the following:
Dimensions of improving patient load
- Conducting health camps
- Developing city clinics.
Dimension of teaching and training
- Potential linkages with established training institutes nearby such as the Jawaharlal Institute of PG Medical Education and Research (JIPMER)
- Linkage with Madras Medical Mission (MMM) in Chennai for short training sessions.
International Clinical Epidemiology Network and IndiaCLEN network linkages for improving collaborative research.
An analysis of potential threats to improve the training program was presented at the meeting. These included the following:
- MCI inspection short falls
- Increased Faculty turn over
- Inability to develop specialty training programs
- Inability to attract academically sound PG students.
Recommendations and the way forward
From session-1 – discussions on objectives
- PG evaluation and feedback needs to incorporate the seven dimensions (medical expert, scholar, professional, health advocate, communicator, collaborator, and manager) at regular intervals
- More detailing of the specifics of how these dimensions are to be incorporated in different academic sessions and interactions is required.
From session-2 – survey results
- Availability of patients for training needs strengthening. The most important factor which will contribute to an increased patient load is to ensure that high quality medical care is delivered to all patients seeking treatment at PIMS. The following steps were identified:
- Ensure appointment system and provide continuity of care
- Ensure supervision of all serious cases in emergency by Assistant Professors
- Regular ward rounds by Consultants and improve patient interactions
- Enhance private patient clinics and identify cases from medical camps.
- Modify the PG program to reduce the program load on final year PG students by including 1st year PGs to present at least one paper on basic science topics, particularly neuroanatomy
- Improve specialty program by ensuring specific training objectives and road maps for each of their posting in specialty departments
- Discuss with ICU faculty and administration on methods to strengthen ICU posting and supervision of PGs during ICU posting
- Improve and strengthen teaching rounds and increase academic discussion of cases
- Devote some time to the relevant basic science issues while discussing cases during ward rounds and clinics
- Strengthen teaching rounds, which should be conducted by the entire unit faculty in rotation, on relatively free days
- Improve the existing examination pattern. However, this is a matter beyond our control at present. As a small initial step, the faculty involved as examiners may insist on conducting examinations as per university norms and regulations
- Involve PG students in UG teaching, especially clinical classes. The faculty may guide them on the content of these classes.
From session-strengths, weakness, opportunities, and threats analysis
- Develop linkages with JIPMER and MMM for short training modules
- Ensure compliance with MCI regulations
- Strengthen outpatient department services and outreach programs to improve patient load
- Improve facilities for junior faculty to reduce faculty turn over.
| Discussion|| |
The three main areas of activity for an academic medical doctor are patient care, teaching, and research. Till date, academic promotions were not based on evaluation of any of the above activities. MCI has recently made minimum number of research publications mandatory for academic promotions. There are misgivings in many quarters that "Research" is perhaps overdone and does not stand on the same level for faculty assessment in medical colleges as "Teaching" and "Patient care." This is perhaps the result of misunderstanding about "Research" and its role. Research is not an esoteric activity undertaken by a few for their self-gain. It is essentially the only method to assess given situations and find optimum answers to questions which challenge the academic community in all three dimensions of "Patient care," "Teaching," and "Research" itself. Educational research has gained momentum over the last 10 years  as a tool for evidence-based policy change. The present educational research was undertaken as part of PE and SWOT analysis intended primarily to improve PG medical training in the institution by evidence-based approach. PE and SWOT analysis are well-established techniques to assess and improve the quality of ongoing programs., The participatory rapid assessment carried out in this study was able to gather consensus on the way forward by an evidence-based approach. PE needs to assess whether the teaching training activities of the department are consistent with the laid down expanded objectives of training program. PG training objectives traditionally have focused only expertise in knowledge, skill, and attitudes related to the management of sick patients. These are reflected in the content of medical expert and academic scholar. There is very little emphasis on the social dimensions of a consultant in the traditional training program. Ethical behavior, professionalism, administrative management training, leadership, the capacity to work as a team player, and being a health advocate have been identified as additional essential components of a good PG medical professional. The Department of Medicine at PIMS, has expanded the basic curriculum of MCI and included Canadian Medical Council (CANMED Competencies), to make the objectives comprehensive.
In the dimension of program inputs and resources
Two main issues came up namely availability of patients and availability of faculty. Specific strategies have to be developed to improve the patient load in our setting. Conducting medical camps and running a city clinic were identified as potential ways to improve patient load. There was a general agreement among participants that the best way to improve patient load was to provide high quality care to all those who seek treatment at PIMS, since they are the best ambassadors of the institution. Nonavailability of faculty was also found to be an important area of concern and dissatisfaction. There is a need to improve the staffing pattern particularly in many specialty departments and also reduce the turnover of medicine faculty in the institution.
Dimension of postings and interactions
Most of the academic sessions were scored very high. The highest score was obtained for "Journal Club sessions" which are the essential evidence-based training sessions of the PG training program. There was concern and dissatisfaction with specialty posting and ICU posting. In depth discussion revealed that these primarily reflect the staffing problems in these departments. SWOT analysis helps to build on the "Strengths," convert "Weakness" to "Opportunities" and identify and neutralize the "Threats" by taking adequate precautions. The analysis provided important points in each of these areas which have formed the basis of the 15 summary recommendations to move forward. In any training program, feedback and evaluation of the trainees is an integral component of the program. It is important that different dimensions of the objectives can be incorporated in a systematic and regular way in the feedback system. This was well brought out during the discussions. Though the respondents gave a mean score of 77% with regard to the system of evaluation of students, during stakeholder discussions, there was general dissatisfaction with the existing system of evaluation. Lack of professionalism from the external examiners was pointed out as an important weakness of the system. All faculties were in agreement that there is much to improve on the prevailing examination system. Lack of evidence-based approach by the external examiners was also considered a third problem in PG student evaluation. It was pointed out that there is a dire need for a more structured examination system to be introduced after discussion with the University. One of the important weaknesses identified by the students was lack of time. While 55% of the PG students rated this as an important weakness of the program, only 10% of the faculty felt that this was an issue. During discussion in the stakeholder meeting, this was thought to be a problem with student motivation rather than a genuine lack of time, considering the lack of pressure from patient care activities. Feedback and evaluation if properly conducted can be an important motivating factor for the students. Terminal evaluation at the MD final examinations is a "gate" that assesses the minimum required standards for qualification and certification as a specialist often this standard is lowered, during a second attempt by the PG. There is a need to reassess this "silent" policy to improve student motivation.
One limitation of this study may be related to the internal and external validity considering the relatively small number of participants. This study was undertaken to address how to move forward in a special situation. One must accept that there is very little generalizability to the results of this study to other medical colleges or institutions. However, the internal validity is not compromised for the following reasons. There is no sampling involved. Instead, the whole population of PGs and faculty of general medicine from the institution were eligible for inclusion in the study. In this setting, validity is not likely to be compromised unlike a sample survey that can be influenced by sampling method and sample size. Small number of subjects in the survey can, however, influence the confidence level of the inferences and the interpretation of results.
The SP session by the Department of Medicine which incorporated PE and SWOT analysis was able to identify the important action points to improve the PG education program in the institution. This effort was both participatory and evidence-based. The method enabled the department to obtain a good consensus on the way forward from all the involved stakeholders.
| Conclusions|| |
There is a need for expanding educational research activities to improve the quality of training programs in academic medical institutions. Participatory research, including PE and SWOT analysis are valid tools, which can be used to understand the current status and identify the activities for evidence-based SP in such institutions.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Bauman A, Nutbeam D. Evaluation in a Nutshell: A Practical Guide to the Evaluation of Health Promotion Programs. Sydney, AU: McGraw Hill; 2013.
Vassar M, Wheeler DL, Davison M, Franklin J. Program evaluation in medical education: An overview of the utilization-focused approach. J Educ Eval Health Prof 2010;7:1.
Gordon J, Hazlett C, Ten Cate O, Mann K, Kilminster S, Prince K, et al.
Strategic planning in medical education: Enhancing the learning environment for students in clinical settings. Med Educ 2000;34:841-50.
Norman G. Research in medical education: Three decades of progress. BMJ 2002;324:1560-2.
Dixit H, Marahatta SB. Medical education and training in Nepal: SWOT analysis. Kathmandu Univ Med J (KUMJ) 2008;6:412-20.
Skinner K, Hanning RM, Sutherland C, Edwards-Wheesk R, Tsuji LJ. Using a SWOT analysis to inform healthy eating and physical activity strategies for a remote first Nations community in Canada. Am J Health Promot 2012;26:e159-70.
Hassan IS, Kuriry H, Ansari LA, Al-Khathami A, Qahtani MA, Anazi TA, et al.
Competency-structured case discussion in the morning meeting: Enhancing CanMEDS integration in daily practice. Adv Med Educ Pract 2015;6:353-8.
Kelly K, McCarthy A, McLean L. Distributed learning or medical tourism? A Canadian residency program's experience in global health. J Surg Educ 2015;72:e33-45.
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6]