download Chapter 4 Faculty Assessment of the Effectiveness of Academic Technology
When Lab staff work with faculty or promote new teaching tools, they encourage faculty members to measure the results to discern whether their new approaches are having the desired effect on student learning. In general, faculty use instructional technologies more effectively when they are attentive to the effectiveness of these technologies. However, because faculty often do not have the time or resources to conduct rigorous research into the effectiveness of their teaching approaches, our campus’s approach has been to cultivate an attitude of assessment in which faculty at a minimum seek informal feedback on how academic technologies are working in their classrooms and make modifications to their approaches as needed.
According to our survey, a majority of faculty have adopted this attitude. Most (60%) report that they conduct some form of assessment of the effectiveness of the technology they use. A small number (7%) report conducting formal assessment (e.g., educational research, scholarship of teaching, etc.) but most (53%) take a more informal approach to assessment. Of the 106 who offered comments, the following were typical responses: questions on end-of-semester evaluations (28); use of mid-semester surveys (26); informal conversations with students (26); observing impact on grades or other forms of student performance (25); paying attention to whether changes are working well or saving time (17); use of others’ research or expert opinion on the effectiveness of a tool or approach (5); tracking Blackboard usage statistics (5); and focus groups/TAs/exit interviews (4). View all faculty comments.
Student responses support the faculty’s perspectives on this topic. 69% report that all (7%) or some (62%) of their instructors assess the effectiveness of the technology. Many said they are frequently asked directly for feedback on the technology both informally during class, in mid-semester surveys, and on end-of-semester evaluations. Some report that the faculty use exam questions or assignments that reveal the effectiveness of the mode of presentation. Others feel that faculty are simply paying attention and are well qualified to judge what’s working and what’s not. View all student comments
Among the 21 faculty respondents who offered comments as to why they did not conduct such assessments, reasons offered included a lack of time (8); a lack of expertise in how to assess (6); a view that the technology they use in their classes has simply become standard (5); and the difficulty in determining whether technology is the cause of improvements they see (2).
By combining the responses regarding usage and assessment, one can see that faculty are, for the most part, inviting feedback on their approaches to technology. They are willing to try new approaches that look promising, they put technology into place when they see a fit, and will abandon technologies that aren’t helping them meet their course objectives. As a campus, we recognize the value of assessment and plan to continue to encourage all faculty to be attentive to the effects of their instructional practices. We also plan to offer more tools and guidance for those faculty who want to assess but lack the time or experience to develop these tools on their own.
While the overall picture of faculty use of technology to enhance teaching and learning is positive, there is a small number of faculty who have expressed concerns about technology and its role on our campus. Those who were neutral (10%) or disagreed (2%) that technology was enhancing their teaching expressed a view that traditional tools and approaches are still sometimes better and that technology may be having an adverse effect on students learning. A few examples of this view follow:
“Students are far too willing to take the easy way out of research projects by relying heavily or exclusively on web content.”
“The main downside of 'technology' use seems to be that students read less and, less importantly, expect to be entertained.”
“The pace of chalkboard work seems to jive better with soaking in ideas.”
“Blackboard is clumsy, time consuming and does not allow some of the most obvious types of student collaborations in writing projects.[…] I just want an intuitive application that works easily and effectively.”
And one professor offered a reminder that computer-related technologies are not the only ones to be considered:
“I have used pencils (one of the greatest technological advancements in the history of civilization) to instruct students in a wide range of artistic expression. I also use all types of paints (oils, acrylics, enamels, etc.) to enhance my instruction in painting and expression.”
A small number of students believe their learning (1%) or their instructors’ teaching (4%) has not been enhanced by technology. The following reasons were following the most commonly cited: technology is used ineffectively by faculty, especially PowerPoint (13); problems with the technology itself (8); professors do not use technology/ it is not relevant to their major (4); inconsistent usage—some professors use it while others do not (3); better to have small classes and engage students in traditional ways (2); students are not taught how to use it (1).
While we are pleased that a relatively low number of faculty and students find that technology is not enhancing the educational process, we recognize that such concerns need to be addressed. In some cases, the solution may be to continue to offer support and development opportunities so that when technology is used, it is used effectively. However, we are also cognizant of the fact that technology is simply not an appropriate solution in some areas. For this reason, we again recommend that Lehigh continue avoiding a ‘one-size-fits-all’ approach to usage. More, we encourage the campus community to continue promoting a culture in which faculty do not feel pressured to use technology when their professional judgment sees such deployment as counterproductive. In short, the Lab should continue to follow its core values of promoting high quality teaching and research, never promoting technology for its own sake.