Manager, Exams - Program Outcome Assessment Professional III (College of Medicine-Phoenix)

Manage administrative processes for all NBME diagnostic and shelf examinations:

• Design and implement expanded assessment opportunities within ExamSoft such as short answer, essay, and reflective writing while liaising with course directors to map to their course needs in the assessment domain.
• Design and implement, along with course directors, end of Year 1 NBME diagnostic examination; explore and make recommendations to ADCA&PE and CC about additional use of NBME examination.
• Design and implement expanded student-specific ExamSoft reports for student development and student use to improve performance across entire curricular spectrum.
• Develop and oversee all financial aspects of ExamSoft and NBME exam animations and other resources necessary to carry out assessment strategies across Years 1-4.
• Liaise with ExamSoft vendor to provide client feedback and lobby for platform enhancements to support the COM needs.

Manage data for student performance metrics:

• Conduct data mining and provide longitudinal curriculum exam data performance measures; surveil and present data on medical knowledge for curricular block and per academic year; provide data to Office of Assessment and Evaluation.
• Lead all exam review team meetings, ensuring that all examination items are mapped and tagged for best practices, including Blooms level, session learning objectives, CCT tags, discipline tags, and author tags.
• Liaise with the Curriculum Committee (CC), providing in depth data analyses on all examinations including cohort data of mean scores, Blooms levels, discrimination indices, and other psychometric data streams; make recommendations to course directors and CC concerning best examination item writing to ensure optimal student assessment and performance outcomes.
• Monitors and resources the post-Exam Review sessions.

Committee, Policy, and Tracking:

• Liaise with other members of OAE, Student Affairs, and resident biostatisticians to provide ongoing prognosticating rubrics to track student success and struggles to ensure optimal student performance a rose years 1-4; to include prognosticators for Step exam success as well as influences, if any, of behavioral competencies in such success/struggles.
• Review current processes and recommend policy or procedural changes to improve high-stakes exam operations.
• Provides technical and resource assistance to Block Exam Review Team subcommittees, which includes editing, proofing, and tagging of exam questions.
• Actively participate in appropriate committees such as Assessment and Evaluation Subcommittee, CC, Clinical Curriculum Subcommittee, and Block, Course, Theme Subcommittee as lead resources for ExamSoft and NBME assessment strategies.
• Liaise with NBME and inform College of Medicine-Phoenix leadership of any changes in Step, diagnostic, and shelf examination initiatives.

Manage Exam Proctoring Activities: 

• Supervise all exam proctoring and serve as the main administrator of medical examinations for years 1-4, ensuring adequate proctors, venues, IT requirements, and special student accommodating.
• Serve as Chief Proctor of Examinations, train and supervise proctoring staff, troubleshoot issues, and act as back up proctor as needed.
• Manage and coordinate administrative exam support services, testing facilities, technology resource planning, test security, scores distribution, student record management, and other office support services.

Administrative Duties:

• Responsible for the recruitment, training, and supervising activities for testing and exams.
• Cross train other employees in the office of Assessment & Evaluation on the use of ExamSoft.
• Communicate discrepancies and issues with senior management.
• Additional duties as assigned.

Knowledge, Skills & Abilities:

• Knowledge of processes to conduct needs assessments.
• Skill in designing materials and presentations to meet needs assessed.
• Ability to implement training, session reviews, and follow-up procedures.
• Skill in planning, budgeting, implementing, and evaluating projects.