Resident as Teacher: JGME-ALiEM Scholarship Hot Topic in Medical Education

Resident as Teacher: JGME-ALiEM Scholarship Hot Topic in Medical Education



hi everyone and welcome to the inaugural academic life and emergency medicine journal of Graduate Medical Education hot topics in medical education this is an experiment for both of our organizations and I think a really cool initiative the next 30 minutes we have a the principal investigator of a really neat study on the resident as teacher we have a leading medical educator from the University of Chicago and we have you who will be able to share questions and comments that you want to hear about this topic of what makes a great resident teacher with a 30 minutes together um first some quick introductions my name is Jonathan sherba no I'm a clinician educator and emergency medicine physician at the at McMaster University and with me I have the lead author Lindsey Melvin say hi Lindsey hi I'm ready a little bit a bunch of stuff quickly I'm a pgy for general internal medicine fellow at the University of Toronto and I did my core internal medicine at McMaster University see again and we have a beard from say hi Barrett hi gang I'm Barrett fro me I'm a pediatric hospitalist from the university of chicago and i run our rat program here then hang up the background we have the editor-in-chief of academic life in emergency medicine michelle lynn and one of the co-editors nikki joshi they'll be kind of coordinating behind-the-scenes stuff and if there's ideas or comments that come in via the internet they will direct them to us I want to set up our talk by just giving a brief overview of how the study went this was a survey of medical students on what they thought made great qualities in great teaching characteristics for resident Teachers it was done by designing a survey based on the literature what the literature would suggest are good qualities and then deliver to a series of medical students representing all the Ontario medical schools after they attended an internal medicine course that ran over the over the period of a weekend so I was hoping maybe Lindsay based on that you can walk us through some of the key findings in terms of what are great qualities and what are great teaching techniques used by resident teachers and then contrast with things you thought were surprising turn zero comes sure absolutely so what we found when we surveyed the students is that when it comes to resident teaching techniques the students really value the use of clinical example is the repetition of sort of key points and and demonstration of techniques really things that help define the key teaching points um the teaching techniques that were less valued which were going to talk about in today's chat a little bit it's feedback the students really didn't rank feedback very highly compared to some of the other teaching techniques that we surveyed them on and when it came to teacher qualities we found that students really value educators who are aware of the students level who are able to tailor teaching to the level of a student and who obviously have a strong knowledge base and things that weren't value not so surprisingly were serious teachers or actually teachers who asked challenging probing questions so maybe we can touch on that a little bit and see what everyone thinks about that finding I think that some you've certainly provoked us a little bit with some of your findings maybe not intuitively the results that we'd have anticipated let me bring in Barrett from kind of our expert analyst running color play-by-play for us Barrett what do you think about these findings there are there some global comments that you want to make around the results that Lindsay and her group discovered yeah so I first of all think wednesday great job on this study because this is an area I love articles on rat mainly which we have to come up with a better name for mainly because everything we've done to develop residence has teacher programs really isn't based on a faculty model I think you will leave that to that your your article and we don't know if that's what read need to learn compared to what a faculty member so any piece of information we can get on really what is better to train our residents to be teachers is valuable I think it's fascinating that you guys got that feedback although it's interesting you know feedback was rated as important it's just the lowest of the ones you had listed so it's not like they said it's horrible it sucks they said it's just comparatively not as important so I don't want everyone to think that feedback was please don't ever God give us feedback but any it is a fascinating piece when you comparatively talk about what we trained residents to do when we train as you do I also like that you looked at techniques versus qualities what is the quality of a great educator because that has been studied in faculty for a very long time in fact I did it in hostel lyst pediatric hospital as educators and I think there's some similarities with what we found before but the techniques are really the unique aspect and I think a lot of what they list are great educate what you've listed are great educational techniques when talking about making it relevant it's learning theory basically some of the stuff you listed so that's why I found it feedback to be missing is if it interesting I know we're going to discuss about an interesting point but I think this definitely adds to the to the small dialog that really is the doma literature on specifically resident teaching great um we've been talking online over the last four days the the a Liam blog the Twitter and there's been a lot of questions that have been raised I want to walk us through three questions that I think are most important and give up an opportunity for people to kind of jump in as well so first is let's dig it a little bit more into feedback the finding that feedback wasn't ranked highly by medical students as a quality for resident Teachers a little controversial doesn't align with what we'd assume would be best practices as medical educators and maybe a bit incoherent with some of the other literature here Barrett you've given us a little bit of a taste as to what you think explanation is but Lindsay I'm interested to hear your take first and then we'll maybe Lincoln or loop in some of the comments that we've heard previously this week yeah absolutely i think you know Barrett Barrett touched on it a little bit it's not that it was ranked really poorly it it comes out is that when you look at how everything else ranked quite a bit higher but when you look at how the breakdown was still a good number of students ranked feedback is highly useful and when I went back and compared sort of the proportion of senior medical students versus more junior all of our medical student attendees were either just about to enter clerkship or senior and some of them were more senior than others the breakdown actually mirrors we see with who responded about feedback and we didn't track individual respondents and we didn't do a subgroup analysis but the the proportion is roughly similar so in my mind I wonder if some of the more senior medical students are the ones that responded as feedback being really highly useful and if some of the more junior medical students either misinterpreted what's meant by feedback and I think the lack of explanations around some of our descriptors may have had something to do with that or don't yet know the value of feedback from residents and I think that that's when I look at that the finding that we had I don't think that feedback is you know should be thrown out anything like that but I think that what we're seeing is maybe some of the more senior medical Rep made of medical students saw that that was more valuable than maybe some of the juniors Barrett one of the the themes I've come from the online discussion is that it be residents or residents don't label their feedback as feedback maybe medical students don't know feedback when they see it is it a question of labeling I think it's two things i love that discussion on the blog this is not specific to residents there's data that says faculty don't label it and when faculty don't label it then they don't know they're getting it so we are all if you go to a lot of teaching seminars we're told to say let me give you some feedback and it's that statement that triggers it similarly we're supposed to say let me teach you something or let me make a teaching point and that tells you we're teaching I also thought what was interesting and I think Lindsay gets at it with her though there wasn't a subgroup analysis the potential I noticed Lindsay correct me if I'm wrong but ten percent of the answer to that question said not applicable which yep I felt like it was almost people saying it's not applicable residents don't give feedback right I wondered if there is an essence in this group that had gone on clerkships yet and some people just thinking they don't think it's a resident bowl and because of that don't rank it highly when maybe some good know going in or more of the senior ones I think it's too full I think possibly it's within the group of the study and how much do they understand the world resident and what they are expecting them to do and also that we label I think the number one thing any educator can do is label what they're doing whether it's teaching or whether it's a giving feedback those are just easy sentences that start I just had a meeting where we talked about how chairman should label let me give you feedback to their faculty it's the same on every level so I think that I think there's an understanding of what feedback is and I think Lindsey you hinted at this to feedback versus evaluation they may think residents don't give feedback because residents don't fill out evaluations necessarily and if there are two totally different things but I think our learners and even faculty don't understand the difference sometimes so that's the nursing ID dad I guess what were you're suggesting is if I use the word let me give you some feedback I'm halfway through to winning a Teaching Award is that about right oh your hat your three-quarters wafer yeah and then a lot and your smile by visa um so Lindsey I have a question for you one of the other takes on their finding is perhaps we use feedback to in an internal way to validate ourselves we want feedback that says you know what you're doing okay you're doing well but there's a disk enslave your feedback that says you're not next necessarily meeting the mark you're an interesting position you're a senior resident you do lots of teaching but you're also receiving a lot of feedback regularly from faculty what do you think on what's your thought on that tape I think very much so I think when you have a medical student population they're very type-a they're very eager to please and everyone wants to be the best and when it comes to feedback sometimes that can not be as perfect and the goal of feedback is really to improve on a skill or improve on some sort of observed hopefully encounter and I think with medical students they really value when it comes from faculty because that's the be-all and end-all and that those are the people who matter and I think sometimes in in this in this way you wonder if the medical students don't want or don't value the feedback from residents or you know what's highly because it maybe doesn't matter as much to them because the resident is and who the medical student is trying to impress although I you know I've met a number of medical students and certainly from the Twitter discussion we see that medical students certainly do value the help and the coaching they get from residents and there's a you know a bit of a language clarification I think that sometimes students tend to evaluate tintu wait feedback with evaluation and when it's coming from a resident then they're not sure what to do with it or don't want to hear it if the residents not responsible for their value or valuation of the end great all the transition us from this question of feedback to a question of how do you balance the needs when you design curriculum so if you're going to design the perfect resident as teacher curriculum Lindsay you've identified the teaching techniques that the users would say are important historically we've always looked at well we're not as interested in what the users want we're interested in designing based on what we think the education theory has so what's that balance between the perceived needs of the user and the observed needs of the so-called experts how do you mix those two components together I think it's an interesting it's an interesting dilemma you face because I don't think medical students necessarily know what they need and I know when I and that's very personal on my account when I look back it's sure you know you want to be in situations that are comfortable with approachable teachers but it's the moments when you're actually being challenged and then sometimes when you're a little bit scared in a safe way that you learn the most and I think that as educators we have a responsibility to design curriculum a curricula that addresses all facets of Education and that's not always going to make someone happy that's not always going to be you know the easiest way to do things or necessarily meet exactly what a student wants but i think the literature just speak very well to this that students don't always know what they want and i'm sure residents are the same group residents don't always know what they want and so when you're designing a resident is teacher curricula you have to match what we need with making the end user happy which you take barrett on the same question I liked Lindsay's comment we always say don't let the inmates from the asylum we would never build just like student said hey what do you want is if you'd asked me that when I was a medical student I would say tell me only pediatric things don't tell me anything else you can't let that happen I think a model of framing this is there's a there's a book called Kern's six steps for medical education curriculum development something like that and one of the steps is identifying the problem and identify your stakeholders I think thinking of stakeholders is essential in this rather than thinking of X it's versus residents or students its stakeholders students are stakeholders obviously so it's important what they think they're in there on the ground they've seen what works they see what have it hasn't and that's a key piece residents are essential right because a student could say I want residents to teach me my lecture for five hours a day and we'd all laugh because that's not going to happen so how do residents think it's best they've also had the experience of being students but certainly the faculty who have taught and taught teaching and done research in this area are valuable one could even look to patients as a stakeholder if you wanted to to see you know how they how teaching can interfere or aid in their hospital stay or in their encounter I think of clinics that's one place where teaching can occur so I think thinking framing this as stakeholders thinking of the stakeholders in the stakeholder needs and whether that's through more studies like this or focus groups or interviews or anything like that that will I think definitely feeder ability to set your goals objectives for here curriculum and then move forward into a resonance st2 curriculum that's ideal I want to spin in this question a little bit to the left so we have the current generation or Millennials as a culture are there any features that make them as stakeholders unique in that you know as a Gen X member do I need to be aware of some of the my perceptions or my perceived biases either of you have a thought on that I definitely think it does i think the generation that we see coming through right now i think where we have to engage them a little bit differently than even i was engaged as a medical student and to me that is predominantly on the technology front I think there's so much to be done in medicine and technology and I think using Twitter and this is a great example but engaging students on all fronts because that's where they're going that's their that's their resources that's what they're used on a daily basis and so when we talk about teaching residents and teaching students I think that using technology is probably the biggest gap when we talk about generations that came before them and I think there's a also a piece too I won't call it the Millennials it's almost the Millennials in medical education in the sense that you're seeing a lot of changes within residency programs about how we love how we grew our conferences rather than every day there's a noon conference now we're doing you know academic half days those those changes not necessarily with the individuals but with how the individuals are getting their education also products to be taken into account when you plan these things so not only the individual generational differences but almost the education generational differences of how we're doing education all come into play and how you deliver this interesting the third question I want to move us to is in a perfect world with all the money that you had Lindsay any wishes about how you might design your study differently or the way to ask the same question is every study has a limitation there is issues of feasibility all the time what are some of the issues that you identified as a limitation to your study and and what are the next steps that you would like to take this whole research question forward with I definitely learned a lot from doing this and one of the big things I learned about study design is the limitations of doing something after the fact and certainly we did this after its conference and the conference featured a lot of residences teachers and while that was great and the conference itself was fantastic and having that many medical students in one place from all different institutions was sort of why we capitalized on it and to me it really limited our generalizability and the results because we were constantly questioning you know are we are we getting these answers because it's a post-conference situation and so in future I would have you know I'd like to repeat the study in a more prospective way looking at maybe more generalized group of students who aren't only there because they're interested in internal medicine though I think the findings probably would generalize across I think it's worth repeating in a more broad sense and I think that was one of the big things that I took away from doing the study and the big things I learned from it so Barrett it's easy for us to be armchair quarterbacks we didn't have to do the hard legwork of getting the data what do you think about these issues of her own recall bias or opportunistic sampling that limit the generalizability of the findings I think it's interesting i love i love it you had a captive audience and you took advantage of that which is fantastic and that's one of the biggest issues and research is getting people to participate and cooperate do what you want them to do but they're obviously it'll is yard out there some issues with recall bias there's a really interesting study from Bhutan II I think it last year august 2013 for two years ago a medical teacher where they did just pediatrics just in patient teaching and what they actually did was they did focus groups at I think the outset of a clerkship and at the end of the clerkship as well as resident focus groups and so they took that into account say you're coming in what do you think now you're leaving what do you think and so there are ways to get around that there's no let's be honest there's no one way to do this right there's no one way to skin a cat there is survey design versus qualitative focus groups and interviews there's doing it a situation like this where you've got a lot of people from across Ontario in different schools in that area versus the National versus one site with a little bit more robust multiple multiple measures so I think I think it's a good study in that we did what we could but there's definitely this could be followed up with a qualitative piece because we don't understand what they meant by feedback and that would also add to some of the some of the depth of what you have you know you see a lot of mixed method studies not you don't see a lot but you're seeing more mixed method studies for just that reason so I think there is there are some issues to armchair it but I think getting the preliminary data now I always think of my first study is the one that launches the second one that makes it even more robust so Lindsay where were you going from here how are you taking your hypothesis generation that's come out of your study and what would you like assuming that you have all the time in the world and you're not busy trying to finish up your own residency training I'm where would you take this whole next step in your study of what makes a great present teacher i think i'd like to refine the quad steps a little bit more and refine more about how how we execute us how residences teachers execute this on a day-to-day basis and look at transitioning that into clinical practice and finding those teaching moments and and and I've been involved with some some endeavors at Mac that that in work with residents as teachers and trying to optimize the daily moments that we have into teaching moments and translating it into the resident as teacher role and that's something i'm really looking at in the future for not just students in internal medicine but students in general and at the resonant level to turn the residence into into more comfortable teachers because I think that's one of the things we find is residents just become teachers and so part of this work is to inform them as to what students are looking for and what they can do and working on that really well thanks Lindy I'm not sure you know that what the first rule of the Internet's is but anything you say here you have to do and live up to no matter what with no kind of change though lots of interesting opportunities for you to move forward with um I think the final comment is rather than a question maybe a comment there maybe you could tell us about some of the resources that you're working on one of the challenges I think we have as an education community is they have this perpetual desire to reinvent the wheel we were very much in silos and we don't share as broadly as we can I think with this new age of social media we have that ability to kind of interact and touch different communities by sharing what we're using so that we don't have to develop our own idiosyncratic resident as teacher curriculum and then replicate that across the country and around the world I'm interested to hear a little bit about what's already being built and maybe you can tell us what the project you're working on sure I agree with you I think I've been doing this for residents a seizure farley for 14 years and I built it originally off of a three-part workshop handbook that the academic or America cadmea Pediatrics put out and then I just evolved from there but we're all there's so many amazing things that are out there and a lot of martin published because you just build them and you do them and maybe you don't get your outcome so you're not going to publish them in a paper i think some resources always i just actually looked last night if you go to the double AMC meded portal has a handful of resin says teacher curricula that are on there that will have teachers guides I think I saw four or five off the just off a quick search I collaborative is also on the double AMC if you don't know the difference meta at portal is peer review takes a little while to get published it has a little bit of work to do it I collaborative as long as there's not a copyright issue you can post any piece of curriculum you have up there and people who grabbed it down the problem with that is it's sometimes hard to find what you want so actually knowing that we want people to be able to just grab there are a lot of small programs there are a lot of programs that have young people who don't know how to build this but want to do it I think it's important that we all share and right now I'm working with the academic pediatric Association to build a website that is a repository for rat curricula it is not specific to Pediatrics any curricula can go we started by pulling a handle we poached a paper recently on the rat curriculum that are published and we rated them for reproducibility could you just grab it and do it and for level of Kirkpatrick outcome how high was the level of outcome that you had gotten and we found a select group that we're going to be able to post on to that website with their the office permission we're actually looking for anybody to submit to our website what it's going to be is a little bit easier than med portal to submit because it's not going to be quite the thickness of peer review but it's going to be a little bit better than I collaborative is be very specific to residences teacher that's the only kind of curricula will have their we're going to assign score so you if you're the user can go say oh this one had really good outcomes but it's and it's partly reproducible without making a call to the author and we're also going to classify them by dt2 workshop do you need a half-day module do you need a one-hour lecture are you a small program a large program are you looking for Pediatrics are you looking for internal medicine and so we're really excited that academic pediatric Association has given us the platform to do that and we should be going live in the next i would say eight weeks so if anyone does have curricula that they would like to submit i'm taking donations currently will also have a wet way by we're going to give submitters annual data on how many people have pulled down there their data so it will help with teaching portfolios but i think anything we now have the tech on websites to be able to do this a little bit morrison years ago probably 15 years ago had a website that anyone could post but it wasn't it's not like the tech we have now and so we can classify we can sort we can search we can do a lot of different things and I think that's the next step for / at okay so uh for everyone who's listening and following along at home I all these details and how you can contribute to this kind of free open access medical education repository you can find the details at the alien blog additionally there is a number of other crowd-sourced resources that you might want to find there and more importantly if you have something you think is amazing don't keep it just yourself when you share it so that its impact can move beyond your home program and have an influence more broadly within the health professional education community we're almost at our time limit here Lindsay I'm gonna give you final thoughts of anything you want to kind of share any final message that you think it's important that's come from your research I think we've chatted about all that you know interesting findings I just wanted to thank everyone out there who read the paper and who got engaged with it and has been talking about it on twitter i think when we started this study and we went through the publication process as any anybody can attest to but as a resident it was really daunting and i had great support hamilton and when we actually did this the hope was that people would read it but i don't think we ever imagined that this many people would be reading it and and talking about it so we really appreciate it and I anybody who wants to chat about it more please tweet at me I try to answer as soon as I can and my twitter handle is attached to any number of these and we'll make sure it's available after this Barrett yeah the one thing I would say that I think about coming out of this as it as you said it takes us out of our silos and we're talking Pediatrics internal medicine emergency medicine through things like this uh meet collaborators because you know Lindsay did it one place but now maybe Lindsay and I talked and we do something across you know country service in that sense across specialties find your collaborators nationally because that is going to make everything more generalizable and clearly by this article and by looking at ease elixir we need more information to understand and change the model of training residents to teach that then we've seen before I want to think about both of you for joining us I think this has been a really important and insightful chat that we've had today I'm for everyone else I want to encourage you to comment on what you've seen here we will keep the blog open for one more day and then we're going to curate it I hope you will all check out Lindsey's publication in one of the spring issues of the Journal of graduate medical education as a companion piece we are collating this virtual discussion that we've had over the last week identifying some of the key themes some of the resources and some of the important points as a react or reflective commentary based on her work so also look for that as an accompanying manuscript in these one of the spring issues of the Journal of Graduate Medical Education a direct you back to the academic life in emergency medicine com blog where you can share comments insight insights or if you want to reach us via Twitter it's hashtag j GME scholar I want to thank Nikki Joshi who's been providing a lot of questions to us from the internets into michelle lynn the editor-in-chief at a Liam for kind of coordinating everything I'm Jonathan Sherman oh and thanks for listening