Jul 09, 2020 Civil Service Commission on 2020-07-09 10:30 AM

All right. I'd like to call the July 9th meeting of the Civil Service Commission to order at 1037. First order of business is introduction of guests. Basically, everyone's a part of the agenda. So, does anyone feel like they're a real guest? Seeing none. We actually introduced Mark Fliniken. He is on your video. He will be the part of our Community Services Officer. So, he is actually replacing Jason Pettyman. Okay. Welcome, Mark. Glad to have you. I guess we'll be seeing some of you. Okay. Next order of business, public comments. I don't know of any. I've gotten no notification of any. Next order of business is approval of the minutes of the June 10th and June 15th meetings. We got those in our packet. Commissioner Roddo, did you see any? No. I can move that we approve the minutes. Okay. I'll second. John can jump in then. So, all in favor, say aye. Aye. Opposed? All right. Minutes for both of the meetings are approved. Next order of business is the Fire Recruit POA. So, I will go to who's going to do it. Colin, are you doing it or? I'll take care of it. Fire's doing it. Who's doing it? I'll take care of it, Chair. So, since Captain Fliniken's new, I'm going to kind of walk through the process. So, this is what you have before you on the documentation is the notice of some sort of examination scheduled for Saturday, January 9, 2021. This is the annual entrance exam that's administered by the fire department. Everything is pretty standard, with the exception if you go down and look under training and experience. Because in the past, as you know, we have hired fully certified individuals. So, you had to possess your fire and your paramedic certification to be hired. That didn't decrease our recruitment pool, but it did assist on listing the amount of funds that were required to bring an interloved person in and actually get them implemented in the fire department. So, as we discussed this, we actually kind of thought we'd try to open the phone and try to create a more diverse group of opportunities. So, we are backing off having both certifications and now we're asking that a person only possess one. You either have your PCFP fire certification or you have your paramedic certification. With understanding knowing that now when we bring them in, we very well make sense in school for upwards of six months so we can utilize them. Our recruitment efforts over the past two to three years, even though we increased the pool has dropped dramatically. This last test we had only under 30 people show up to take the test. So, we've had great concerns over that number dwindling. This is actually the market. It's not nothing we're doing wrong. We're seeing other departments open up to where you have to have no certifications. I know they did a survey and I think the city of Irving has it open to no certifications. They only had 100 people show up. All you had to be is age 18 and drugs license. So, the market is very strong if you're a recruit out there. So, we're going to step back this one year to do a test, see if it increases our numbers. Of course, it will incur additional costs to the city, but we feel at this point this is a good call to try to once again open that recruitment fund up and see if we can get more people interested in the city. Questions? So, with this, if the city is paying for part of their certification, is there an amount of time that they have to commit to the city to working for them? Yes, we put a two year stipulation on that. Okay, thank you. Any other questions? I was trying to read through the form here. It's just the standard civil service test. There's not anything additional that they need to study, right? Correct. It's a standard 100 question multiple choice test. Any other questions? I would entertain a motion that we approve the fire recruit POA. I motion that we accept the POA. I'll second. All right. Is there any further discussion? If not, all in favor of approval of the POA say aye. Aye. Opposed? Unanimous. Next. The next item is kind of a discussion of the whole testing procedure and the appeals. So, Michael, do you want to start us off or do you mean with your insights or Stephanie or? I think it's really, I know last time we had a discussion about it and we just put this as a placeholder that if there was any further discussion that the commissioners felt they needed to have. So I really kind of defer to you folks if you think you need to have any other questions or want to discuss it further. Well, firstly, ladies and gentlemen, I'd like to say that I apologize for being late. I've not had internet or email since Monday at 10 o'clock. So we've been flying in the dark. I have not done any work. I'm not taking care of my responsibilities. I was doing tax returns when you called and I was shocked. I just lost. So please forgive me. It wasn't intentional. And thank you Crystal Babcock for making the phone call. I do apologize that I was disoriented, but not out of my own wishes. I wanted to say that when it came to having this discussion, I think kind of some of the conversation we've had in the last few meetings helped me to clarify how I need to approach the exams and my job as the commissioner. And then I thank the Stephanie and Crystal and Michael for having an individual meeting with me to once again clarify some points. And I feel more comfortable in my role. And I feel that I can do a better job of being a commissioner with the task at hand. I realize there's still some room for improvement and we'll look to provide input to you when and where we can. Thank you, John. Yeah. I mean, again, I feel like we each had a meeting with Stephanie and with Michael. If the fire at this particular time, I'm going to pick on fire since they're the ones that are here, but also police will take a little more care in looking at in the questions themselves, so that we don't get conflicting information within approved materials. The problem as I see it is it's pure memorization. And what I would like as a citizen, what I would like to see is or know is that the firemen are going to make the right decisions in the heat of battle, so to speak, as opposed to thinking about, okay, well, I'm paid such and such and such a book that said do this. And I mean, the application of the knowledge is really the important component in my mind, which means concept. So if we're going to go with this idea of it's what's written in the book, then we really have very little to do as a commission, because there's no argument. You're right. It's word for word out of the book. But neither do the appellants have any chance to appeal. And none of us is perfect. And so I think we need to serve that role as a commission. But we got to make sure that we don't have that we have the information and that we don't have conflicting information. So mission, I'm just curious, maybe chief hedges can address this from a practical standpoint is what is done in the field. Very diametrically opposed to what's in these books. I mean, are we having to memorize and study stuff that really isn't the way it's done in the field? Well, you'll never be able to test individuals and apply to build application. I think while we do include our standard operating procedures, procedures are more guidelines. I know last meeting there was discussion about many lots of your fire fighters, more critical thinking skills. I have absolutely 100% faith they're able to do that out on the fire ground and EMS calls. Everybody takes the service test understand it is anomaly. It is memorization over here and not as much can be applied in the field. There are information you can take the path of an aerial ladder above a building. Yes, it's three foot above or six foot above the roof. It's based on the situation. Every situation is different. The book cannot identify what we're going today versus they go in possibly tomorrow as well. So I don't know how best to describe it. It is a very slice of time a picture that taking this test on what they need those information, what they apply in the field is totally different. And they understand that they know that going into it. If you're a 1.3 department, that's just a fact of it. They know 1.3 inside and out. This is well by administration does well. So sometimes not said people put a lot of effort studying sometimes put more effort in the appeals process because they have learned over the history that they can actually make more progress in a short amount of time, but kind of working the appeals process. So everything that our association has done with the city has been trying to limit that. I don't want to use the term manipulation, but it's not. It's just utilizing the appeals process to increase your score on the test versus actually applying the time to memorize. Unfortunately, that's what it is. 1.3 is from my perspective. One way around and for the commissioners benefit, state law says that promotional exams can only be based on a written test. It can't include any interview component. That is something that the city and the police and fire associations can try to negotiate a change to through the meet and confer process. But it is established in state law. It's based solely your placement on the list and the test itself is based solely on a written examination and you have to identify the source materials that are going to be used in that test at least 90 days in advance. So the departments are kind of limited as to what the test can do unless it's a negotiated change through the meet and confer process. Go ahead. I would just like to add also we have negotiated with the fire association to add some minimum requirements for promotion. So in the meet and confer Article 5 motions for promotion of the ranks of captain and battalion chief, you have to have the normal two years in rank, which is what 1.3 states, but you also have to have the Texas Commission on Fire Protection Fire Officer 2 certification, which that's an advanced certification. You have to go through test and be awarded that. In the beginning in 2021 for meet and confer, we have an internal company officer task clear that they have to go through certain skills and get checked off. So we were able to negotiate outside 1.3 to add additional requirements to the officer in this department as well. Are those requirements for them to be eligible to take the test or are those points tacked on after they take the test eligible to even sit for the exam? I think when I was talking to Michael and Stephanie, the way that the way that I think Mr. Konig put it, he said everybody has to everybody has different experiences, right? So when we asked them to talk between application and theory, I guess, what these authors are writing, they can't all take from the same experience. So they have to be able to take the information from the same source, which is the source material you guys are saying. I just I believe I'm sure that, you know, in your department, you guys talk and it's understood that all of these questions are only going to come from the source material. I wonder if if somewhere it's written down that says questions on this examination are only coming from source material, they're not coming from practical application, or anyone's experience. Does that make sense? So that we can kind of get away from that. So when they're coming in, they're saying, okay, I'm not I'm putting away my what I actually do. I'm, you know, I'm focusing only on what I would be able to do in the perfect scenario. Theory. Yeah. Is that is that an understood thing with all the officers? Or is that written somewhere? I know that you're I know that Mr. Kroneck said, you have to give them source material 90 days before, so they know they have to study these 90. But did they understand that it's only information coming from these books, and nothing else? It is to me, it is communicated, and then obviously, and we have limited to the court is source material, we've tried everything we can to reinforce that interior that we have commission approved, that is your source material, that is where the test bank is coming from. So it's understanding. But then again, sometimes you still have appellants come in, and they want to bring in the personal experiences from outside that source material, even outside book that was for that one specific test question. Because it's been this for 27 years, it's hard to get a good list of source material that don't conflict on each other. Every author has their own opinion. You can see you can West Coast firefighting versus East Coast, it's totally different on the type of structure. We're kind of central. So unless you just pick authors from the Texas area, you're gonna have conflict information on how tasks that are not applied in Texas, in North Texas, specifically as well. Okay, but I'm your you. What was the one you said Article Three, the income for it says limited to only source material questions are only limited. Yeah, article five. And then section seven scope of the fields. He'll be limited to quoted source material attributable to the reference examination question. I'm sorry, you're cutting out Can you read it one more time? I have my microphones not the best appeals to the commission under section 143.034. The text of the code shall be limited to the quoted source material, attributable to the reference examination question. Okay, but that's talking about appeals. Is there anything that says test questions will come only from the source materials? That's, that's, that's, that's again, that's the reason why the source materials are approved by you as a commissioner. Okay, but you're saying it's a given, I'm asking if it's written anywhere. I don't, I can't say this written, written anywhere. We don't have a policy of examination that also 143. I think that's the part that puts us in a situation because if it's not written, I understand that in department, it's, it's a given, everyone should know, right, it's, it's passed on, that's the way it's always been. But when it's not written, and they're bringing in, they're making a solid argument, I think that's the part that's kind of, at least for me, that's the part that kind of puts me in a situation because it's not written. What is written is when the source materials are posted, it says the examination questions will be taken from these, these materials. But I don't think there's anything that says, and we won't be testing on any of your actual experience. I don't think it says that last part, but it does say, and I don't have one in front of me. This is what there's this, these are the books that the questions are going to be taken from. So they know that, but yeah, there's no express saying and we won't test you on practical applications. I don't think it says that, but I think it's, I think it's understood because it's the way it's been done for years and years. Well, but recently, I've, I've seen a little bit more of taking what you were just saying, Michael, instead of, I mean, that sort of says that the tests are coming from all of the books that we approve, but they've, it is now gone to, well, no, it's that book and that page. And if it's in another book or if there's conflicting information in another book that's still on the approved list, that's not a valid appeal because that was the court of material. I mean, so now we're asking them not only to memorize the material of what looks like 10,000 pages worth of stuff. Now they've got to memorize which page it's on because they got to make sure that they get the right page. And if they remember that something's in another book that's approved, then I think that, then the idea that it's, if that, then that's a valid appeal in my, in my estimation. I want to show y'all the actual test real quick to show you what it says to the test taker. Sure. You're going to share your screen? Yes, sir. Okay. Can y'all see that? The test. So right there at the top, it says the following questions are coming from this book and then they have questions one, two, three, four, yada, yada, yada. It doesn't give you an exact page number on these questions. It just says these questions are referencing this book. And then, you know, when they get down and they're done with that book, it says you're going to, these following questions are going to reference this book. And it goes on. This, this is the cover page. Wait one second. I'm sorry. I can share this other one. This is the cover page here on the very front of the test, of the exam. And this is what it says in the instructions. There is one and only one best answer to each question. Mark the best answer for the separate, on that separate answer sheet. No credit will be given for anything written in this textbook. So the best answer and the reference book, and it doesn't say the page. So if they pull, if they pull an appeal from a different page in that same book, then that's a valid appeal. Well, if their point is strong enough. But, so that's, that's exactly what they see when they come in to take their test. Well, how do we get around this conflicting information between books other than having to memorize by book? I mean, if there are, if there is conflicting information, that's a, that's a problem, I think. Well, just from my perspective as the training officer and really coming up through the process, I should know when a book conflicts with another book. I should know this author's take on this kind of predicament and then this author's take on predicament. Because any situation that we encounter in the field or whatever, it's, you know, there's not going to be just one specific answer. There's going to be differences of opinions every time. So basically you're learning, hey, there's an option A and this author makes his points and there's an option B and this author makes his points. I know I have both of those. I have that information in my head now and based off of the situation, I can make a decision that fits best fits that situation. But I think all of us coming up through the ranks when we've studied to the extent that we study, I could tell you this book A says this about this topic and book B says this about this topic and they are not the same. People who don't study that much, they're the ones that come into the appeal and start pulling things out of different books. And once and I'm in my opinion, and Chief might hate me for saying this, once a commissioner decides to or the commission says, okay, we're going to accept book B, then the guy who did all that extra work to know the difference and answer the question properly gets punished for the guy who just spent two hours reading through all his books after the test. If that makes sense. It makes sense. I think if that explanation was given, you know, there's been some times where the explanation from the department just says, that's what's written in the book. You know, if the explanation like if you came out and said, I understand what he would have gotten confused, but because the this question went with this book, this is the book he should have gotten it from, you know what I mean? And you can say because all these obviously, you know, so you would say, there are different, you know, opinions or whatnot. But we asked them to reference this book, then that doesn't give us much wiggle room as a commission to entertain, you know, to be persuaded or manipulated. But, but whenever it just says on the on the, you know, it's just like what I told the officers whenever they come in the appeal, I said, if you put this little effort, and all you're telling me is, you should accept C2, that doesn't give me much, you know, but it's the same with the department just says, that's the that's what's verbatim in the book, you know. And I think that's the part because we're we're trying to be fair, but we're really at y'all's mercy, because you guys are the ones that know the information. We're just trying to, to think and to, and to be as fair as possible and to not only compensate good memorization, but to also, you know, acknowledge these people that are thinking and looking at all these different parts. And you know what I mean? So even if it was like even if they're taking two hours to try to make a strong appeal, they're doing it there, you made them think you made them, you know, take more time and look at the material and, and look at different scenarios in different ways. And I have one more thing that I'll I'll wait until later. Let me just let me go right into that pretty quickly. But I'll certainly agree with that. And I've talked with, well, Chief Skipper, I know we talked about the fact that I would like to know why an appeal is not valid, rather than it's the other one is written in the book. Why is that appeal wrong? A little more information about that part of the of the appeal process will help us as being the outsiders, the overseers, but the outsiders in that sense. So why is it why is something wrong when something else, when it appears to be right? So. So can I ask you this, if I were to say, he has a valid point, if I use the information from this book that this question didn't reference, but this book question was referenced out of the with the question was referenced out of this book. So, you know, both situations, you would have to take that into account. But the question references this book, and that's how this book answered it. Would that be a more appropriate response from FIRE? Because I can concede that, like I said, there's two there's situations and there's a maybe two correct answers, but the most or best answer would be the one that was referenced that the question referenced that book me that would make a huge difference because you're acknowledging their point, but you're but you're still they still have to be able to follow directions. And if the right direction was to reference this particular source material, then that's where they should have gotten it. And like you said, they should know the difference, right? If they get confused, well, then that's then they got the question wrong, you know, but if but if you can say that, if you said something like that, according to this, that would have been correct, but they failed to notice this one detail that they were supposed to get the sort the information from this book. And that's why the appeal is not solid. And that would be huge. And I think that that's an acknowledgement for them too, because you're you're saying you are correct. You know, it's like, yeah, I mean, that would make a huge difference for me. I also like I'm teetering a line or at least I felt like I was this year. Y'all, the commissioners know that three battalion chiefs look at these questions to determine the department response. And I'm, I'm the one voice that gets to present the department response. If I go against what three but like if I say, hey, I said something like that, and I sway you to make a choice that was against the department response, then that puts me in a bad position, not only with the appellants, but with the department response. Does that make sense? What I'm saying? So I'm trying to I'm trying to sway your decision making as little as possible, and just be as black and white as I possibly can. Why would why would that why would our decision go against what your three battalion chiefs would say because they're using the same source material that you're quoting, isn't it? Well, to your point, y'all had seven questions on the captain's exam that y'all ruled. And I think three or four of them were against the department recommendation. So if I say something than what the department different than what the department recommendation is, then I have personally influenced your decision against or for the department recommendation, but your department recommendation is already written. But that's what I'm saying. So if I say something more than that, when you ask me, you want to you want to know chief skippers personal opinion about the topic, if I say something more than that, then I've influenced you beyond what the department has recommended. And then go against what the department if you if you rule against the department recommendation, based off of something I've said, now I'm I'm in a bad position. I think I was gonna say chief skipper, the I'm on I'm on a zoom call. Okay, thank you. The you just told me something I haven't heard in three years. I thought the department position was you. Three other people involved. So to me, that's that's news. And that's good. I appreciate that there is more oversight than one voice, not that we discount your voice. But you know, you got other people looking at it. And so when that when you say the department's position is this, that gives me a new perspective. You know, however, I think one of the issues that we were dealing with was on one page, it may say one thing, we're staying in the same book, but on another page, that's also an equally correct answer. You know, I think that's what we get into the issue. But when you're talking about going from one book to another book, to me, I think you've clarified that issue, as long as that presentation is given to all of the test takers, so that they know both before the exam, and after the exam, that they're supposed to answer based on that book, and not in a kind of a comprehensive way. I would say con if the if we expand the department response to acknowledge and explain why an appeal isn't the best answer, then that takes you off the hook. But that means that that you guys have to work maybe work a little bit harder than just simply it's a direct quote from the book. You might have to work a little harder. But I think that would help us with understanding the situation going in and help us to understand better how we need to make our decisions. So I think more information from your three battalion chiefs will help us not put you on the spot. They can help you but they've got to do something more than just well, that's what's quoted in the book. I was gonna say the same thing if they if they if the department response, you know, has already been looked at by three individuals that three collectively with no more than you know, one person, and the argument should already be made there, you wouldn't have to change your stance after that department response has already been written and presented. But we do kind of put them on the spot in one set when an appeal does come up. I mean, I, and over the years, that's happened. What happens in reality kinds of things. But again, I think if the if the response is more directed toward the appeal itself, and why that appeal is not a valid appeal, then I think we have even less reason to put the the the sacrificial lamb on the line there when when the time comes. So I, I would agree, like, I absolutely agree, we need to do more vetting of questions. And then a little bit better explanation on the the appeal, our responses to the appeals. I do get the feeling sometimes. And this is just a personal opinion that the default mode is almost to give the appeal as opposed to stand with the departmental response. And I don't know that that's that's just that's a subjective feeling on my part, as opposed to anything I could say this right here, I don't have a good example of that. So eight, nine, 10, 11, 12, 13, 14, 15, 16, 17 of the probably 25 appeals were sustained the key. Okay, yeah, here's here was the thing I was thinking earlier, like the driver's exam. The driver's exam is very much a cognitive fact based like, hey, you're going to raise a ladder, it's going to be this it's information that you can remember, a captain or battalion chief exam, we get much more into leadership philosophy. And everyone has an opinion about leadership philosophy. So when we're talking about different books or different opinions about a certain subject, that's where we get a little because I've had in the past, you know, when Commissioner Abbott was on he, he straight up said to me, and one of my appeals, he didn't agree with what the book said. And I was like, I can't, I can't take a test based off a commissioners opinion of the subject material, I can only take a test off of the subject material. I'll tell you, Colin, that when I first came on, and my understanding, I was, I was because I'm, I'm resistant and I'm still I understand what Michael saying that and you know, 143, it says that it has to be a written examination. I understand all of that. I hope that later, which is going to now be another three years, whenever you guys do the meet and confer again and negotiations that, you know, I know police does the lieutenant and they have an interview process in there, but maybe y'all could even introduce scenario questions, or, you know, some kind of comprehensive essay or something that allows, you know, your people's critical thinking to shine instead of these memorization, right. But that's beside the point. But at the beginning, I was resistant to the memorization. So if someone came in and they showed me a strong argument, and they did their work and they provided critical thinking and they provided information from different parts of the text, or even another text, I was persuaded to advocate for the appellate. Now that you're telling me, you know, and all of this has been kind of clear that you're saying it is on top of the question, take it from this source material. And then now I'm like, okay, I'm not going to look at other books. I'm not looking at your application. I recognize what you're saying, but you can't use that, right? You can only use this book. But now, and if it's in the book, but on page one, it says, I don't know, there was a question last time about wording. And I remember, I even looked up the definitions of the words, something about leadership. And then on page 30, it said something different. You know, then that to me is still a strong appeal, because it's in the same source material, it's in the same, you know what I mean? But for all others, I'm going to sustain the key as long as the department's giving me enough information to say, you know, to say this is from this book, this is what this is saying, he's talking about that book that has nothing to do with this question right now. And I think Steve Hedges, you said it's a snapshot, which I think that's important, too. I'm not asking you of what you would do nine times out of 10. It's a snapshot question of right now, this book, and this paragraph or whatever, you know what I mean? And so that's more clear for me now. Now I can stand more with what the department is saying and still feel that I'm being fair because they've already been told it's this book, you know, it's not anything else. It's this book. It's not, you know, your experience. Does that make sense? But I mean, I do wish that that we would allow them to do, you know, these multiple choice. I know that's the easiest grade. I know it's the fastest. But we would allow maybe these three Italian chiefs can read two, two essays, you know, of what would happen or something that would give them an opportunity to take all the experience and all the years that they had and to put it on paper. It's kind of my understanding that the way civil service is written and the fact that it is just a test is to take the subjective part of it, the good old boy system that the past that Fire to Heart ran off to make it so objective that I couldn't go in there. Like to your example, if I create an essay for people to answer and I was grading it, I might grade my buddy really high because I look at it with a certain bias as opposed to the guy I don't like. He might actually have better answers, but I'm already going into that with a certain bias and I grade him really low. Yeah, I mean, there's ways to go around that. You guys can set up rubrics. You know, you can have rubrics that are that are subjective and they can help. I mean, I know that I got an MPA at UNT and we had we sat in that comprehensive exam for eight hours and all essays, you know, I mean, there are ways to get around that. I completely agree. I think that that's good in the right steps, but I think I think also it just makes it too black and white and you're rewarding people who are good test takers, who are who can memorize, but they're not. But I'm not but like, you know, to Commissioner Ramsey's point at the beginning, not just with fire, but even with police. I'm worried with the people on the street. I'm worried how how we're going to interact, the decision you're going to make on the spot, you know what I mean? For the protection of the rest of the citizens and for their own protection. So I think that we absolutely should move to something like that personally, but that's going to take contract negotiations and all this stuff that Chief Hedges was talking about earlier with the system we have now. I just that I believe that we can vet questions better. We can provide better input as far as why we sustain the key or accept an appeal. And to me, the stronger appellant is the one that says question the answer a was wrong. My answer b was better because I picked the best answer that we're saying. This is the best answer. Not that both answers are correct, because both answers might be correct to an extent, but there's one that's better. You know, it's just like being a paramedic. If you're having a heart attack, there are 100 different things I could do that will probably be correct. But eventually there's a best thing that I could do that's going to be more helpful to you than the other thing I could have done. In your fire response to the appeal is where we need to know what that best, why the second answer isn't as good as the best answer. Because that very likely is what we don't know. I remember the questions of medications. I had no idea when we tested the EMTs. I had no idea what the medications were to know whether one was better than the other. That's where your expertise, your and the other bat chiefs need to help us be able to make the right decision that also is right for the firemen that are doing the appeals, the folks that are doing the appeals. Kenneth, I would suggest that you might think about especially at the bat chief level, putting some kind of an interview process similar to what police does, because there are so much, all of the leadership stuff is really important, but really subjective, much more so than the driver test. It's not very subjective, but when you get the leadership books, you might think very seriously about throwing that out on the table and seeing where it goes in the next meet and confer agreement. This is kind of a step process. What we negotiated with fire sociologists in the last year, 2019, was one more step towards some type of a subjective assessment. They have assessment centers out there in the fire industry where, yes, you have to write essays, interviews on leadership traits, scenario-based questions as well. I've sat through a few with other cities and I don't fully agree. I think there's definitely hybrid systems we can implement, but that's outside 143. So we're covered by 143. We can go outside that, so it's going to be obviously a negotiation with our fire sociologists. But we definitely start at the higher level, with the battalions and captains. That's where more of the leadership world comes into play. That's what meet and confer was all about, or is all about, is supplementing the 143 rigidness. Can I say that? I guess I just did. I would also say, Colin, I know in the past, I think the first few meetings that I attended, sometimes it was brought up that out of the 45 test takers, 40 of them got the question correct. To me, that made a strong point. I can't remember what other meeting I asked about that and someone said, "Oh, we're not looking at that." But I think for me, especially in a department response, if you guys use that as a justification for saying this isn't a fluke, this isn't a bad question, I mean, 95% of the test takers knew what we were talking about, then to me that would be a stronger justification to hold the key. I agree. It speaks to the validity of the question, right? If that could be included, that would be in the department response, that would be great. Colin, do you guys go back and look at reliability and validity? I don't say that we do because we hired, this is all outsourced. We hire a professional company to write all the tests and stuff. We just give them the books. So they don't capture the information, the results, and try to... No, I'd have to go physically do it. I have done it in my training, but it's been a long time. You need a book, right? On page 20. No, no, never mind. I think that would help at least because if one person's appealing a question and everyone else got it right, it's like man, it speaks to something. I mean, you looked at it a different way than everybody else, but you looked at it wrong. Is there anything else? Lauren, did we get to your second point? We were in the middle of one. Did we get to your second? You said you had two issues. Yeah, no, it was that. It was the suggestion of the next time the meet and confer come around that supplemental factors are taken into consideration on just a written test. I think that that's important. They already talked about it a little bit. Okay. Well, I just want to make sure because I knew you had that second issue, so I wanted to make sure you got that. Anything else? Hearing none, seeing none, there's nothing else. Next meeting, Crystal, do we need to... If we would like to meet in August, we do need to set a meeting date today because I have to get that to the city secretary's office. We're very limited on dates in August. We're down to either August 17th, 26th, or 27th. Those are the only three dates that the room is that it's available. What do we have coming forward? What days of the week are those, Crystal? It looks like, I think, August 17th is a Monday and then the 26th is a Wednesday. The 27th is Thursday. I don't know of anything coming forward right now. We've already done the POAs for PD and Fire. There are no more appeals right now. PD might have... I don't know. I don't know of anything that's pressing that anybody has right now. Should we maybe go ahead and schedule the room and then we can always cancel it? You can always sell it to someone else for a fee? We could, but I won't charge them. I'm not going to say that. I would prefer not to do a Monday. Other than that, I'm okay. Okay. That leaves August 26th and 27th. I am going back to school, I think. Yeah, me too. I think. So later afternoons would be better for me. Okay. Thursdays... Well, I can do it after 10, after 10 on Wednesdays. My class is from 9 to 10. So I can do it Wednesday, mid-morning, or we can do it after school. School will be back. Maybe, depending on Friday, maybe school will be back in... Well, probably by then. I think the new start date that they're tossing around is the 26th. So that'd be the second day of school. Maybe mid to late afternoon, 3.30, 4 o'clock. I'm good with either of those. I have enough advance notice, but Mondays are usually hard on me, so... Okay. Williams, 3.30? Yeah. 3.30 is fine. 3.30 or 4 is fine. All right. You can do 3.30 on that Wednesday, the 26th or 7th, but 6th. The Wednesday is the 26th, yes. Okay. That's first day of school. Is that okay, Lillian? Yeah. I was saying if that roused a reaction, so... No, I'm good. Okay. Okay. So we will tentatively schedule that. And we can always... If we don't have anything, there's no reason to meet other than just to see each other, so... Right. Okay. Hey, I just wanted to tell Colin and kind of thank you for the information, because to me, having that background information is really helpful. I came into this position as a volunteer, and I want to do right not only by the department, but by the officers as well. So anytime that you guys explain it, I know to you it's probably redundant, repetitive, because you live it every day. It's already... You already know it. But for me, any kind of background information you can give me, then that just helps me feel more confident and feel good about the time that we spend volunteering here. I think that last time I might've gotten a little bit frustrated because I was like, "Well, then what's the point? If it just says it's in the book, then what's the point? Why am I spending hours on this?" And so I think that that's helpful. So I appreciate y'all's time. And Michael, you too, just for all the explanation and looking at things from a different perspective. So thank you to all of y'all. Thanks, Mark. Mark, you too, and Crystal, you too. I would like to say thank you for your service on this committee. I mean, I know your scope seems limited, but it's a bigger role than what you actually imagine. There are certain people that they spend three, four, five months studying for one exam here in New York as well. So a lot of times, but then just I can speak in personal experiences with people that put in time, they're so invested in the pills and it can go so many different variations. So as we go through this, we're trying to narrow the ability of people to, like I said, not study and then swing and appeal a certain way as well. And it is definitely a work in progress. Thank you. I would like to keep working at it. I'm sorry, John. Go ahead, John. I'd certainly like to say cool bye, y'all. Thank you. I see Mark is over there chilling, asking for a change of positions after the first day. All right. If there's nothing else but formality for the good of the order, I would entertain a motion that we adjourn at 1126. So moved. Seconded. All in favor say aye. Aye. Opposed. All right. Thank you all. Stay safe. Stay cool.