Learning with Text and Image: The Relationship of Text-Image

Learning with Text and Image: The Relationship of Text-Image

Hello, I’m Matthew Peterson, a Graphic Design Assistant Professor. Let me share some insights about the integration of text and image in media.

I developed a keen interest in this topic through my fascination with science textbooks. My background in graphic design provided a unique angle to potentially enhance science instruction. Though psychology offers rich insights into how we process text and image separately with unique but complementary resources, I noticed that many experimental studies simplified the designs to isolate a single image and caption. This didn’t reflect the holistic approach that designers usually employ, making it challenging to apply these findings in practical design settings.

In my exploration, I wanted to develop strategies that designers could employ in real-world applications. I hypothesized there was an implicit strategy underlying any design decision, often based on assumptions about text and image integration and the typical production models of textbooks, which frequently prioritize text before adding images.

Design Strategies and Comprehension

To study the effects of design on comprehension and interest levels, I categorized design strategies representing a range of text-image integration:

1. Prose Primary: This is common in most illustrated books. A central prose column dictates a serial reading path—images, if present, require readers to leave the main text.

2. Images as Entry Points: This middle-ground strategy uses multiple images to guide readers into different text parts, providing several access points.

3. Fully Integrated: Text and image are interwoven, allowing readers to enter at any point and follow various paths. This parallel system is presumed easier for information retrieval but might seem confusing initially.

Here's a brief look at these strategies:

1. Prose Primary: There’s a central prose column with occasional images that require readers to leave the text.

2. Images as Entry Points: Contains multiple entry points into the text with images serving critical functions.

3. Fully Integrated: Text and image are seamlessly combined, offering non-linear reading paths and easier information retrieval.

To test these strategies, I had to reverse-engineer an existing textbook designed with a Prose Primary model, converting some text to images (e.g., showing a fish at the bottom of a lake instead of just describing it).

Study Methods and Findings

I conducted open-book tests where students used these different textbook designs to complete factual and conceptual items. While fully integrated designs showed the highest comprehension scores, the differences were often subtle, and only in one treatment condition did the fully integrated design significantly outperform prose primary.

However, situational interest yielded more striking results. After the tests, I showed the students individual pages from the three design versions and asked which they found most and least interesting. Prose Primary scored very low in interest, while more integrated designs scored higher, indicating a strong preference for integrated media.

Summary and Implications

Though comprehension benefits from integrated designs were inconclusive, there was clear evidence that younger students found these more engaging. This suggests educators might consider more integrated designs to spark interest, even if comprehension gains are modest.

Keywords

• Text-Image Integration
• Graphic Design
• Science Textbooks
• Prose Primary
• Fully Integrated Design
• Situational Interest
• Comprehension

FAQ

Q: What inspired your interest in text and image integration? A: My interest was sparked by a desire to positively impact science teaching through my background in graphic design.

Q: What were the design strategies you studied? A: The strategies included Prose Primary, Images as Entry Points, and Fully Integrated designs, representing a spectrum from text-dominated to fully integrated text and image layouts.

Q: How did you measure comprehension in your study? A: I conducted open-book tests where students answered factual and conceptual questions using textbooks designed with different integration strategies.

Q: What were the main findings regarding situational interest? A: Designs with higher text-image integration significantly increased student interest, showing a preference for more visually engaging designs.

Q: Did integrated designs improve comprehension? A: While fully integrated designs showed slightly higher comprehension scores, the differences were generally subtle and not always significant.