North Dakota State University

Project 1.
Breaking the cycle: Understanding how visual representations impact student understanding of biogeochemical cycling

Jennifer Momsen, Mentor and Project Lead
Erika Offerdahl, Associated Faculty
Amy Williams, Undergraduate Research Assistant

Visual representations, including qualitative models of biological systems and processes, are essential components of teaching and learning in undergraduate biology. However, many common representations present biological information in ways that can confuse or mislead the learner and contribute to or reinforce common misconceptions. For example, a model of the carbon cycle portraying decomposition as occurring near plant roots may perpetuate the well-documented misconception that plants absorb carbon through their roots. This project focuses on representations of biogeochemical cycling routinely used in introductory biology and seeks to:

• Characterize how introductory biology textbooks represent carbon and nitrogen cycling at multiple levels of biological organization.
• Investigate the link between textbook figures and common student misconceptions.

At the end of this research experience, students will be able to:

• Qualitatively describe and compare visual representations of carbon and nitrogen cycling,
• Initiate and maintain data collection,
• Complete basic data analysis using R, 
• Synthesize literature, and
• Present findings to a broad scientific audience. 

Project 2.
Draw something: Student-generated drawings in Human Anatomy and Physiology courses

Lisa Montplaisir, Mentor and Project Lead
Jennifer Momsen, Associated Faculty
Tara Slominski, Graduate Research Assistant

Gathering information about students’ understandings of different kinds of scientific phenomena has been studied for decades. Despite the richness and variety of the methods used by science educators, most of these methods rely on students’ talking or writing about science. In-class drawings convey information as well as measure student understanding. Because of this dual utility, drawings can be powerful instructional tools. This research will characterize student-generated in-class drawings of human anatomy and physiology.  Students will participate in analyzing a rich collection of student-generated drawings and will work as part of a team that includes a graduate student and faculty member.  This research is guided by the hypothesis that there is a relationship between students actively organizing ideas through in-class drawings and subsequent test performance. Through this research experience, students will

• Develop skills in data analysis such as use of scoring rubrics for coding and related statistical analyses, and
• Gain a rich understanding of Human Anatomy and Physiology content for their particular project. Some familiarity of Anatomy and Physiology is preferred.

Project 3.
A picture is worth a single word?  An analysis of textbook visualizations in the molecular life sciences

Erika G. Offerdahl, Mentor and Project Lead
Jennifer Momsen, Associated Faculty
Jessie Arneson, Graduate Research Assistant
Jan Ohm, Undergraduate Research Assistant

Visualizations (e.g., graphs, diagrams, figures) are ubiquitous in science. As scientists, we use them to communicate complex data sets, processes, and relationships to one another, our students, and the public. National calls to transform science instruction underscore the need to develop students’ understanding of the tools and disciplinary practices of scientists.  Visual thinking – the ability to interpret and make sense of visualizations – is one such disciplinary practice that, to date, is seldom an explicit learning outcome of undergraduate science curricula.  Moreover, while science instruction makes extensive use of visualizations in textbooks, simulations, and lecture slides, it is unclear the degree to which such visualizations support development of students’ visual thinking skills.  The goal of this study is to characterize the nature of textbook visualizations to determine the role of visualization and representation in supporting visual literacy in the molecular life sciences.

Members of the visualization team will benefit from the experience by:

• Deepening their conceptual understanding of molecular biology and biochemistry,
• Systematically categorizing or “coding” textbook visualizations using an established coding scheme,
• Developing skills in creating and querying Access databases, and
• Applying basic descriptive statistics to characterize textbook visualizations.

Members of the visualization team will be expected to:

• Work independently and as part of a team to analyze figures from textbooks and scientific papers,
• Read research articles on visualization and participate in weekly discussion of the research,
• Present their research progress in lab group meeting at least twice during the summer,
• Synthesize research findings in the form of a scientific poster to be presented at the conclusion of the program, and
• Attend a national conference (SABER) in Minneapolis in July 2013. 

Project 4.
Spatial reasoning, a potential roadblock to conceptual understanding of Physics

Mila Kryjevskaia, Mentor and Project Lead

Research suggests that even after targeted instruction on many physics topics some students still are not able to systematically analyze complex unfamiliar situations.  As researchers, curriculum developers, and physics instructors, we feel that one of the weaknesses in our current understanding of student learning is an inability to differentiate clearly between those student difficulties that stem from inadequate understanding of subject matter and those that are due to other factors.  This distinction is critical to efforts to improve student learning of physics.  We hypothesize that poor performance on some physics tasks may be attributable to student difficulties in visualizing and reasoning spatially.  Therefore, we will probe the extent to which student understanding of some phenomena hinges on visualization skills by examining the degree of association between student performance on many tasks and on a spatial visualization test (paper folding test).  We will attempt to distinguish between student difficulties that are due to deficiencies with conceptual understanding and those that are due to lack of visualization skills.  

Research assistants will:

• Become familiar with the literature on the impact of visualization skills on student abilities to succeed in science courses,
• Analyze student responses to examination tasks,
• Perform statistical analysis in order to determine the degree of association between student understanding (as measured by an exam performance) and the level of spatial reasoning skills (as measured by the paper folding test), and
• Discuss implications for instruction.

Project 5.
Will this be on the test? Characterizing the cognitive skills, conceptual understanding, and disciplinary practices routinely assessed in biology

Jennifer Momsen, Mentor and Project Lead
Lisa Montplaisir, Associated Faculty
Elizabeth Anderson, Undergraduate Research Assistant 

Assessment drives learning is a refrain that echoes through many colleges and universities, wherein students selectively study and learn the content and skills they believe critical to passing an exam. It follows that to understand the cognitive skills and content understanding students develop as a result of their undergraduate science major, we must turn our attention to assessments and the nature of the tasks we routinely use to assess student learning. This project will use established coding frames and coding rubrics created de novo to characterize assessments drawn from across the biology major’s curriculum. Specific questions may include:

• What cognitive levels are routinely assessed in introductory versus upper division courses?
• What biological skills are students expected to become proficient with?
• What is the role of visual representations in assessing student learning?

At the end of this research experience, students will be able to:

• Use multiple coding schemes to describe assessment items,
• Initiate and maintain data collection,
• Complete basic data analysis using R,
• Synthesize relevant literature, and
• Present findings to a broad scientific audience. 

Project 6.
How did I miss that? An analysis of introductory chemistry and biochemistry assessment items 

Erika G. Offerdahl, Mentor and Project Lead
James Nyachwaya, Mila Kryjevskaia, Jennifer Momsen, Associated Faculty
Nate Grosz, Graduate Research Assistant

Assessments such as exams and quizzes are routine in undergraduate chemistry courses. Understanding the numerous possible factors impacting students’ performance on these assessments is a complex problem.  Prior research suggests that the difficulty of any exam/quiz question is dependent not only on the cognitive skills required to answer the question but also other factors such as surface features of the problem, context (both of the content assessed as well as the course in which the student is enrolled), and linguistic complexity.  The goal of this study is to understand the relationship between student performance on assessments in introductory chemistry and biochemistry and mediating factors such as cognitive level, linguistic complexity, and degree of abstractness of the content assessed.

Members of this research team will benefit from the experience by:

• Deepening their conceptual understanding of chemistry and biochemistry,
• Systematically categorizing or “code” exam and quiz questions using Bloom’s taxonomy,
• Develop a coding rubric to characterize the abstract nature of chemistry content presented in questions,
• Developing skills in creating and querying Access databases, and
• Applying basic descriptive statistics and inferential statistics.

Members of this team will be expected to:

• Work independently and as part of a team,
• Read research articles on assessment and student misconceptions in chemistry/biochemistry and participate in weekly discussion of the research,
• Present their research progress in lab group meeting at least twice during the summer, and
• Synthesize research findings in the form of a scientific poster to be presented at the conclusion of the program. 

Project 7.
Scaling Up: Adapting Physics Tutorials for large-enrollment courses

Warren Christensen, Mentor and Project Lead
Mila Kryjevskaia, Associated faculty

One of the most successful research-based curricula for introductory physics are the Tutorials in Introductory Physics by the Physics Education Group at the University of Washington. These materials rely on interactive environments with a 12:1 student to teacher ratio, to engage students in challenging tasks that force them to confront and resolve inconsistencies in their elicited conceptual reasoning. During the past three years a number of these tutorials were adapted for use in a large lecture course at NDSU. Student multiple-choice and written responses were collected before and after instruction with these adapted materials. 

A student in this project will analyze these pre- and post-instruction conceptual questions, as well as exam questions to determine students learning of concepts involving buoyancy, the ideal gas law, and the first and second law of thermodynamics. The study aims to identify how different pedagogical approaches to adapting these materials, such as giving students paper copies of the tutorial or relying solely on electronic slides, may impact student performance.

Research questions include: 

• To what extent can curriculum developed for use in a small classroom setting be effectively scaled to a large lecture setting?
• How do variations in pedagogical approaches using these tutorial-based lecture activities impact student performance?

The researcher on this project will:

• Develop skills in analyzing and coding student responses,
• Test hypotheses via developing statistical skills,
• Gain insights into research-based curriculum development, and
• Increase content understanding for the topics they are researching. 

Project 8.
The right moves? How student feedback informs faculty teaching decisions

Warren Christensen, Mentor and Project Lead
Jennifer Momsen, Lisa Montplaisir, Associated Faculty

Henderson and Dancy have done seminal work on factors that motivate and influence faculties’ pedagogical decisions in physics. Their enormous data corpus features dozens of faculty participants and artifacts including interviews, course syllabi, conceptual survey data, student evaluations and sample exams over multiple semesters.  A summer researcher will investigate how the feedback a faculty member receives from students (both their comments and performance) affects the faculty’s pedagogical choices during a semester and in subsequent classes.

Investigating the culture of faculty and their responses to the feedback they receive from their students makes this an ethnographic study. The study will rely on a mixture of qualitative and quantitative methods.

Research Questions include:

• How does feedback from students influence an instructor’s pedagogical decisions in classroom?
  To what extent are student concerns reflective of the learning that occurs in the classroom? 

A student researcher will:

• Analyze student performance data and student evaluations,
• Learn to code and analyze interview data,
• Search for artifacts of evolving pedagogical strategies, and
• Engage in hypothesis development and testing. 

Project 9.
Using portfolios of learning to measure student learning in biology

Wendy Reed, Co-Mentor and Project Lead
Jennifer Momsen, Co-Mentor

Assessments are more than a means to a grade and can facilitate a meaningful learning process. Portfolios of learning are somewhat novel assessments in STEM education, representing a purposeful collection of evidence documenting a learner’s achievement, progress, and growth. Through the process of assembling the portfolio, learners must reflect deeply on their understanding and are required go through a process of review and revision of their ideas. This project will investigate the utility of portfolios of learning in assessing student learning in an upper division biology course.

Potential research questions inclu

• Do portfolios of learning align with the stated learning goals of the course and national standards (i.e., Vision and Change in Undergraduate Biology Education)?
• In what ways do portfolios of learning capture student learning?
• Does the evidence of learning captured from portfolios in align with evidence of learning measured through open-ended assessments?

The student researcher will:

• Learn to code and transform student-generated narratives,
• Create and maintain data collection,
• Complete basic data analysis using R,
• Synthesize relevant literature, and
• Present findings to a broad scientific audience.

Project 10.
Digital exhaust: Predicting performance from online interactions 

Jeff Boyer, Co-mentor and Project Lead
Warren Christensen, Co-mentor

When students interact with online tools, such as a learning management system (e.g., Blackboard, Moodle, D2L, etc.), they leave behind a digital "exhaust" that is collected in access logs.  Thus, it is possible to summarize and analyze patterns of student behavior based on these documented timestamps.  It is likely that successful students exhibit different access patterns than non-successful students.  Instructors may benefit from understanding how and when students interact online.  In essence, a pattern of interaction with an online resource or tool may provide information to instructors that indicate a need to intervene with students who are not performing adequately within a course. Thus, the purpose of this research is to identify students' online interaction patterns and to determine if these can be used as an intervention tool for struggling students (i.e., an early warning system).  The research questions that guide this work a

• To what degree does interaction with online resources predict course performance?
• Could interaction with online resources serve as an early warning system for students who need intervention?

At the end of this research experience, students will be able to:

• "Clean" collected data in preparation for data analysis
• Analyze and summarize data using R and other statistical tools
• Test hypotheses against data sets
• Develop predictive models for student performance
• Present findings based on available evidence