Judging Books by Their Covers: The Impact of Text and Image Features on the Aesthetic Evaluation and Memorability of Italian Novels

Abstract

Book covers are often the first component seen before a reader engages with a book’s contents; therefore, careful consideration is given to the text and image features that constitute their design. This study investigates the effects of the presentation of verbal (text) and visual (image) features on memorability and aesthetic evaluation in the context of book covers. To this aim, 50 participants took part in a memory recognition task in which the same book cover information was encoded in a learning phase, and either text or image features from the book covers acted as an informational cue for memory recognition and aesthetic evaluations. Our results revealed that image features significantly aided memory performance more than text features. Image features that were rated more beautiful were not better recognized as a result. However, differences in memory performance were found in relation to familiarity and, in a non-linear fashion, the extent to which the book’s contents could be inferred from the image’s informational content. Additionally, reading behavior was not found to influence memory performance. These results are discussed with regard to the interplay of text and image informational cues on book cover perception and provide implications for future studies.

1. Foreword

Empirical studies of literature often concern the cognitive effects of literary texts. Literary texts, also referred to narrative texts, form a large body of research, especially pertaining to the reading experience. However, a broader, often overlooked, concern is the extent to which a literary text can ever independently impact or influence a reader’s cognition, without being linked to other attenuating factors. One such factor is paratextual information, which no doubt has a bearing on the reading experience (Altmann et al. 2014). Whilst it is not the purpose of this paper to dispute/explore all possible attenuating factors, in an effort to explore the everyday reading experience from a cognitive science perspective, the focus here is strictly on paratextual information. This concept is explored in the form of book covers and brief interactions with paratext (see Appel et al. 2021), which can be characterized as ‘micro-reading’, and perhaps lexical processing even more akin to image viewing (Bauer and Mayer 2008).

2. Introduction

Despite its overuse, the idiom ‘Don’t judge a book by its cover’ alludes to the moral decision regarding how assumptions are made based on ‘outward appearance’. While the phrase refers to objects and people more generally, in a literal sense, book covers themselves are indeed often judged—and form our first impression of the book’s contents. Hence, as opposed to the literary text itself, we focus on the informational content of the book cover. As the first thing that we see, the book cover and its design set the expectations a reader has of its contents. This design comprises several features that can be categorized as either text- or image-based and are processed in distinct ways that influence cognitive and psychological processes differently (Reed 2022). Despite their combined use in design, the role that text and image features play in shaping book cover perception has received little attention. At the same time, recent studies have suggested the potential of the interplay of text and images and their collective impact, specifically on memorability and aesthetic judgment, which again remains understudied. As such, this study seeks to address this research gap by investigating how visual and textual factors of book covers contribute to the perception, memorability, and evaluation of books.

2.1. The Function of a Book Cover

A book cover can serve many functions, including acting as a teaser, a highlight, or even an attention grab. In some cases, the cover offers seemingly random glimpses of the mood of the book or simply suggests a mode of reading (as in the case of book covers without pictures). In other cases, the cover offers a spectrum of interpretations. For example, the different covers of Dumas’ The Count of Monte Cristo depict a couple, suggesting a sad love story, a chained young man, suggesting the story of a tragic hero, or even a masked, uncanny man with wild eyes, suggesting the story of a madman (Dumas et al. 1923). These book cover variations each imply different interpretations of the type of story the reader is about to embark on and may very well even trigger genre expectations. In some sense, these depictions could insinuate that a book cover acts as a ‘visual summary’ (Sonzogni 2011) with regard to the brief précis these pictorial aspects allow for, in the interpretation and evaluation of the book as a whole.

As Sonzogni (2011) highlights, the book’s cover may serve three functions: (1) present visual content that allows potential readers to decide whether to read a book or not (within a brief span of a few seconds); (2) impart information to the reader by displaying the title and author, as well as by summarizing the text through images and words; and (3) prompt the reader’s memory regarding their existing knowledge of the text. Because it conveys visual information about the book, the cover can be considered an aesthetic stimulus with regard to the way the illustrations, author’s name, title, and any other information (e.g., awards/prizes, endorsements/testimonials, and taglines) are integrated into a pleasing design (Sonzogni 2011). By taking the function of the cover into account, we can then explore how words and images are integrated within a design, as well as how this affects the subsequent evaluations that are formed.

2.2. Text and Image Information in Book Cover Perception/Evaluation

Classically aesthetic objects1 often pair imagery with text, such as a painting being displayed with a caption or title (Darda and Chatterjee 2023; Leder et al. 2006; Mastandrea and Umiltà 2016; Millis 2001; Mullennix and Robinet 2018) or street art incorporating text within the depicted images (Chamberlin 2018). Additionally, in album covers, which are arguably a closer parallel to book covers, we also see the pairing of the album title and music artist with the album cover’s graphics (Le 2020; Venkatesan et al. 2022)2. As Sonzogni (2011) points out, in book covers, the pairing of captions or titles with art goes beyond identification and facilitates the meaning making process (Franklin 1988; Leder et al. 2004, 2006; Russell 2003; Russell and Milne 1997). Moreover, the additional elaboration that such text provides in turn increases the aesthetic response (Millis 2001; Venkatesan et al. 2022), with this effect being more apparent for shorter viewing times (Leder et al. 2006).

Similar to art and music, the way that literature is consumed has also changed with the increased use of digital platforms, streaming, and online interfaces (Murray 2018; Pianzola 2021; Spjeldnæs and Karlsen 2022). Given that album artwork no longer actively encourages music purchases (Le 2020), in the same light, the way book covers are presented/perceived may have varying effects on consumer purchase. Research has focused on how consumers select books, especially given the shorter time-span of viewing linked with the presentation of thumbnail previews or image tiles on digital online interfaces (Gudinavičius and Šuminas 2017; McKay et al. 2012; Osinska and Osinsk 2020; Rollins 2014). Pictorial and textual elements featured on book covers therefore evidently determine choice (Ross 2000; see also Wang and Song 2020). For example, Childers et al. (1985); Childers and Jass (2002) highlight typeface as one factor to consider in influencing consumer perceptions. Specifically, this alludes to a preference for visual information over verbal information3 and, consequently, suggests that the image-related properties of typeface (see Melmer et al. 2013) are more influential than textual (semantic) information.

It is noted that these studies focus on the verbal/visual features of text, i.e., the effects of semantic information, rather than the image-related properties of the text, such as typeface. The current paper, however, does not place its focus on the processing of visual and verbal properties of text but rather on the effects of verbal (titles/words) and nonverbal (pictorial/illustrative) information. The subsequent sections will elaborate on the interplay of this verbal/visual information (i.e., text and image information) in the context of book covers.

2.3. Effects of Text and Image Information on Book Cover Memorability

When considering book covers and similar stimuli, one could explore how verbal and nonverbal information is processed, especially given that the main underlying intention of text is to preserve information (Jean 1992; Sonzogni 2011). Investigating the types of information that are remembered from a book cover and the strength of the memory trace therefore provides insights into which parts of this information are important. While text (i.e., words in the written form) is the preferred method of communication (Dewan 2015), images are easier to recognize, process, and remember (Kulhavy et al. 1993; Levie and Lentz 1982; Robinson et al. 2003; Verdi et al. 1997), with images and words being processed by different neural networks (Feng et al. 2021). This is known as the Picture Superiority Effect (Paivio and Csapo 1973; see also Mintzer and Snodgrass 1999). A possible reason for this increased memorability of pictures/images is that in our mental representations, images are coded both visually and verbally (i.e., dually encoded), which means that both visual and verbal mental codes can be used for memory retrieval (Paivio 1990; Sadoski and Paivio 2013).

Language is fundamentally built on visual thinking (Reed 2022), with metaphoric language being especially intertwined with visual imagery. Building a clear mental model is essential for understanding a text, such as a book (Kintsch 2004). As mentioned above, using verbal and visual information can contribute to this process. However, there is evidence to suggest that visual information is more robustly encoded, and therefore, when a recognition task involves images, the stimuli are recognized more easily and quickly compared to their word counterparts (Nelson et al. 1976).

Book covers, as stimuli, contain both image and word information, which, according to the aforementioned theories, are processed and encoded differently, making use of both verbal and visual codes. This dual-coding process thus strengthens the memory trace and likelihood of recall (Harp and Mayer 1997; Sadoski and Paivio 2013), since multiple retrieval codes are created for the visual (image) and linguistic (word) information. This suggests that to effectively remember something such as a book, a combination of words and images should be used during encoding. With regard to memorability, given that visual and verbal information use separate memory stores, recall is improved when either one can be accessed (Kosslyn 2007). This notion of cross-modal recognition taps into the levels of processing theory (Lockhart and Craik 1990), which asserts that deeper encoding leads to better memory retention. When combined text–image stimuli are present during encoding, a more elaborate processing of the words and image are encouraged, thus strengthening the associations with visual and verbal memory traces and subsequently improving the retrieval of information from memory.

Research suggests that readers’ capabilities also affect the construction of a mental model. Stronger readers4, for example, tend to have better memory recall (e.g., author recognition: Moore and Gordon 2015; Stanovich and West 1989), largely due to their ability to build robust mental models while reading. Enhanced vocabulary, comprehension skills, and memory capacity help these readers make more meaningful connections between new information and what they already know, supporting a deeper understanding and retention of text (Kintsch 2004; Calvo et al. 2003). In contrast, weaker readers may benefit more from illustrations in written texts than stronger readers, as illustrations can help compensate for difficulties in processing and integrating textual information (Cooney and Swanson 1987; Levie and Lentz 1982; Mastropieri and Scruggs 1989; Schnotz 2014; Mendhakar et al. 2024).

In the context of book covers, combining text and imagery can activate this memory process, providing visual and verbal cues that engage readers more deeply. For skilled readers, the interaction between these elements may facilitate stronger connections with the content and enhance memory retention, as the interplay of words and images reinforces memory pathways (De Leeuw et al. 2016). Conversely, Schnotz (2014) reports that the cognitive effort invested in processing text may decrease when images are added, potentially resulting in less thorough textual processing and lower memory retention for the text itself (Mayer and Gallini 1990; Schnotz and Bannert 1999). These findings raise the question of whether reading ability influences how effectively readers integrate text and imagery to form memorable impressions of book covers. This study aims to address this question by exploring whether stronger readers are better able to process and, more importantly, retain information from book covers.

2.4. Influence of Aesthetic Evaluations on Memorability

The memorability of a book cover could also be enhanced by its aesthetic evaluation, engaging both cognitive and emotional processes through text and image elements (Chana et al. 2023). Text on a book cover, such as the title, can trigger mental representations and emotional responses by evoking background knowledge and generating inferences, which, in turn, strengthen memory encoding (Jacobs 2015, 2017). This aligns with Jacobs’ findings on the role of language in evoking emotional reactions and mental imagery, suggesting that the more evocative or emotionally charged the text, the stronger the memorability. On the other hand, image elements on the cover—such as the illustrations, colors, and design—can immediately activate visual–spatial processing, creating vivid mental images and emotional associations. According to his research on mental imagery, visual elements are processed differently from text, leading to an immediate emotional response that reinforces memory retention through sensory experiences (Jacobs 2017). The combination of these text and image elements taps into dual-processing systems, which suggests that verbal and visual stimuli are processed through separate mental systems, strengthening recall by engaging both verbal and visual mental codes (Kosslyn 2007). While text engages deeper cognitive processing through language, images perhaps evoke more immediate emotional reactions and mental representations (Alpium and Ehrenberg 2023; Bradley et al. 2014; Jacobs 2015; Leder et al. 2022). When they are effectively integrated, these elements work together to activate both memory systems, leading to more durable and accessible recall. In a similar vein, Leder et al. (2006) found that text influences how viewers interpret and emotionally engage with visual art, enhancing their appreciation and overall memorability. Therefore, one can posit that a well-designed book cover that blends evocative text with compelling imagery can create a stronger, more lasting impression on the reader, improving the likelihood of the book being remembered.

Research indicates that a well-aligned combination of textual and visual elements on a book cover enhances comprehension and emotional engagement, leading to a more cohesive and memorable impression (Yonelinas 1994). Recognition memory theories, such as the dual-process model, highlight the interplay between recollection, a slower process of retrieving contextual details, and familiarity, a faster mechanism that relies on memory strength without specific details (Yonelinas 1994). Additionally, signal detection theory posits that recognition is influenced by the strength of the memory trace and decision criteria. While reading, verbal and semantic attributes of text are typically prioritized, with visual/physical properties assumed to be transient. However, studies have shown that these physical attributes persist in memory for 10–25 s, far exceeding the time required for word identification (Kroll et al. 1970; Warrington and Shallice 1969). Visual properties, or rather typographic features, such as typeface and letter case, influence verbal processing in the early stages, potentially activating independent semantic representations (Posner 1970; Tinker 1963; Webster and Tinker 1935).

Building on this foundation, the present study isolates the text and image features of book covers to test their respective contributions to aesthetic evaluations and recognition memory. This approach allows us to disentangle the cognitive and emotional impacts of these elements, providing insight into how design features influence memorability and appeal.

2.5. Aims and Objectives

Previous research has explored the effects of visual and verbal information on memory and perception. This study focuses on the text vs. image content present in Italian book covers, as well as determining which can be used as a better cue for remembering the book. Furthermore, the role of aesthetic quality, particularly how it influences memorability and perception, has been investigated in art, advertising, and other areas but remains unclear with regard to book cover design. By dissecting these elements and accounting for individual differences in reading behavior, this study aims to provide a comprehensive understanding of how book covers are perceived, remembered, and evaluated.

Importantly, we note here that the purpose of this study is not to alter the text/image information, since we use pre-existing stimuli that should have already been designed professionally with this intent. Instead, the primary focus is to gain a better understanding of the interplay between visual and verbal factors. With this aim, we intend to investigate how much memorability and aesthetic value is driven by visual (image) vs. verbal (text) information. We achieved this with the following research questions:

• Do we exhibit better memory recognition of a book based on the presentation of the book cover image or its text information?
• What are the effects of book cover information that differs in aesthetic quality on memory performance?
• Does reading behavior influence the memorability of book covers?

We aimed to assess our research questions by performing a memory recognition task, in which the same book cover information is encoded during a learning phase. During the testing phase, either the text or image content from the book cover acts as an informational cue for memory recognition. This allows one to see the different contributions of verbal and visual information to memory retrieval, indicating which type can be used as a better informational cue for remembering a book. The details of the experimental procedure and conditions are outlined in further detail in Section 3.

The following hypotheses correspond to the research questions outlined above:

• Memory recognition will be greater when presented with the book cover image (visual information) compared to the book title (verbal information);
• Memory recognition will be greater for information with higher aesthetic value;
• Stronger readers will demonstrate better memory performance.

3. Materials and Methods

3.1. Study Design

The book cover information type (text/image condition) was set as the between-subjects factor of the experiment. Hence, while all participants saw the same book cover stimuli during the learning phase, during the testing phase, half of the participants saw the book covers with text only, and the other half saw them with only the image for memory retrieval (see Procedure). Following this, their recognition and aesthetics evaluations were assessed, and the results were compared between the two conditions.

3.2. Measurement

3.2.1. Memorability Measurements

Memorability was measured through a recognition task. Recognition memory is often assessed with forced choice or yes–no test formats (Khoe et al. 2000; Stanislaw and Todorov 1999). The present study adopts the latter, whereby target and distractor stimuli are displayed in single trials following a study/learning phase, and individuals are asked to indicate whether they had seen the stimuli (see Procedure for further detail). This process allows us to analyze the data within the framework of signal detection theory (Green and Swets 1974).

3.2.2. Aesthetic Evaluation Measurements

We are interested in assessing whether participants’ aesthetic evaluation of a book cover influences the way they process the information it conveys, thus affecting their memory performance. Importantly, this evaluation can operate on multiple levels: readers may assess the cover purely as a visual artifact or consider how effectively it communicates the essence of the book. We assumed that these levels could contribute differently to readers’ aesthetic evaluation and therefore decided to target them separately.

With regard to the aesthetic value of the book cover itself, we asked participants to rate, on a scale from 1 to 7, both how beautiful and how interesting they found it. Beauty was selected over other rating scales because it has been previously connected to other hedonic assessments, such as liking and interest (see Chana et al. 2023; Jacobsen 2006), and thus serves as a broad indicator of aesthetic appreciation. Interest was included given the debate on how closely related beauty and interest are in this line of research (Mastandrea and Umiltà 2016; Millis 2001). Concerning the level of the literary work, we first asked participants to quantify the degree to which they believed they were able to infer the content of the book based on its cover. Following Salgaro et al. (2018; see also Heydebrand and Winko 1996), we also assessed literary value through both the social prestige and economic value of the work. We asked participants to rate how well they thought the book was received by critics and to guess its price (on a range from EUR 5 to EUR 35).

Finally, we included a question (yes/no) to check whether participants were familiar with the book in question. Although the stimuli had been pre-selected for unfamiliarity, this item was included to control for the possible effects of familiarity, given its strong influence on recognition tasks (McKenzie and Tiberghien 2004).

3.3. Participants

This study includes a final opportunistic sample of 50 participants (39 female, 10 male, and 1 undisclosed gender; M_age = 22.32, SD_age = 2.34, range: 19–33 years). Twenty-five participants were included in either one of the experimental conditions. All participants included in the analysis were proficient in the Italian language. Participants were all university students at the University of Verona and took part in the study as a course requirement. Anonymity was guaranteed by the use of personal identifiers. This study adhered to the ethical guidelines set by the University of Verona.

3.4. Stimuli

As familiarity and recollection have been reported to affect recognition performance (Bastin and Van der Linden 2003; Chana et al. 2023), we aimed to collect book covers that are generally unfamiliar to avoid familiarity effects on memorability and to allow recognition performance to be considered an effect of the recognition test format. To select the stimuli, the most sold Italian books per month were ranked and scored based on a series of bestseller lists https://www.giornaledellalibreria.it/categorie/narrativa-italiana-22.html (accessed on 6 October 2023). Books that scored the lowest were used as a proxy for unfamiliarity (n = 474). Similar to Gudinavičius and Šuminas (2017), we excluded books deemed as bestsellers and those written by widely recognized authors. Any classics were also excluded from the sample. The selected books were all fictional novels that were written in the Italian language by Italian authors. From this corpus, only the lowest scoring books (n = 160) were selected for a pre-test of familiarity.

One hundred and sixty books selected from a larger corpus were used in a pre-test to assess the familiarity of the novels. Fifty-seven students at the University of Verona were asked to indicate their familiarity with the book titles and authors of a series of Italian novels by marking one of three boxes (read, know the book, and know the author). At the same time, participants provided an approximation of how many books they had read in their spare time over the past year as an indicator of their reading habits (see Supplementary Figure S1; OSF https://osf.io/jxumc/ accessed on 29 January 2025).

For each book, if a participant knew the author, the book was given a score of 0.25; if they knew the book title, the book received a score of 0.5; and if they read the book, it received a score of 1. This means that for each participant, the total score per book was a maximum of 1. There were 27 books that had a score of 0, meaning that none of the surveyed participants were familiar with the title or author, nor had they read the book.

The final sample included the 80 books that had the lowest familiarity scores (see Supplementary Table S1).

3.5. Procedure

Data collection took place over two days in December 2023 (4 December 2023; 6 December 2023). On average, participation time was 41.56 min (SD_time = 8.55; range: 22–64 min). Participants were provided with a link to the study through the Labvanced online platform (Finger et al. 2017) and were randomly assigned to one of the two between-subjects conditions. Once they had given informed consent, participants provided demographic information and completed the learning phase.

The experiment consisted of three phases: a learning phase, an interference phase, and a testing phase. The learning phase required participants to view 40 book cover tiles that were each presented on the screen for three seconds at a time in a randomized order (a total of two minutes).

Following the learning phase, participants completed an interference phase, in which they answered questions about their reading habits and use of media and filled out the Interest section of the Vienna Art Interest and Art Knowledge questionnaire (VAIAK; Specker et al. 2020) (see Supplementary Table S2). They also indicated which Italian publishing houses (out of a list of 43) they were familiar with (see Supplementary Table S3).

Following the interference tasks, participants performed memory and rating tasks in the testing phase. This phase involved the 40 book covers presented in the learning phase (target) and 40 additional book covers that were not presented in the learning phase (distractor) (see Supplementary Table S1 for the target and distractor stimuli). According to the between-subjects condition that participants were assigned to, the book cover stimuli were either presented based on the image content (with the text removed) or text content (with the image removed and in a neutral typeface) (see Figure 1). Based on the stimuli presented on the screen, participants had to respond whether they had seen each book cover in the learning phase (yes/no). At the same time, participants reported if they were familiar with the book and rated the stimuli on a series of 7-point scales (beauty, interest, contents, importance to critics, and cost estimate).

At the end of the experiment, participants were debriefed and thanked for their participation.

3.6. Data Preparation

The data were analyzed with R (Version 1.1.423. R Development Core Team: Vienna, Austria), using the dplyr package (Wickham 2016). All the analyses reported in the Results Section 4 were performed in the same R environment.

Signal detection theory (SDT) is an established mathematical model that can be used in memory recognition (Green and Swets 1974). In the memory recognition task, a person must decide on a response based on the evidence they are presented with, i.e., whether the target vs. distractor stimulus was shown in the learning phase, by indicating yes (same; 1) or no (different; 0) as a response. After completion, the responses were marked using signal detection analysis (see

Table 1. Signal detection theory paradigm.

	Different Response (0)	Same Response (1)	Total Number per Condition
Different Stimuli (0)	Hit	Miss	40
Same Stimuli (1)	False Alarm (FA)	Correct Rejection (CR)	40

), and the number of ‘Hits’, ‘False Alarms’, ‘Misses’, and ‘Correct Rejections’ for each participant were computed.

SDT then accounts for the assumption that there is no random variation and that responses are biased—in other words, it controls for the tendency to respond yes in the task. The recognition task responses were converted into d’ prime (d′) scores. The statistic d′ is a measure of sensitivity, reflecting the participants’ ability to distinguish between target stimuli (signal) and distractor stimuli (noise), controlling for the inclination to indicate that the stimulus was present when it was not (response bias, β) (for a more extensive explanation, see Green and Swets 1974 or Stanislaw and Todorov 1999).

The SDT analyses were performed using the psycho package in R (Makowski 2018).

4. Results

Before the main analyses, we first conducted a series of manipulation checks to ensure that there were no pre-existing differences between the groups. All of the reported data and code used for the analysis are available on the Open Scientific Framework (OSF): https://osf.io/xaduy/ (accessed on 22 January 2025).

First, in addition to choosing the least familiar books in the stimuli selection using a pre-study (see Section 3), the familiarity of the book cover stimuli was also assessed by participants in both the text and image groups. For the text condition, participants reported that the book covers were familiar 2.95% of the time. For the image condition, participants reported that the book covers were familiar 4.55% of the time. This suggests that the book cover stimuli were not familiar to the participants, and therefore, the results henceforth are not driven by familiarity.

Second, we compared general reading behavior and art interest across the text and image groups, as these aspects can be confounding factors for both recognition and evaluation performance. The number of books read per year, as a proxy of the participants’ general reading behavior, was compared across the two groups. In the image group, an average of 10.6 books were read per year (SD = 17.0, 95% CI [3.62, 17.7]), while in the text group, an average of 11.4 books were read per year (SD = 13.4, 95% CI [5.86, 16.9]). As can be seen in the ranges of the 95% confidence intervals, there were no significant differences in reading behavior between the two groups. Participants’ art interest, as measured using the Art Interest section of the VAIAK questionnaire (Specker et al. 2020), also showed no significant differences between the two groups (text condition group: M = 45.7, SD = 12.4, 95% CI [40.6, 50.8]; image condition group: M = 41.0, SD = 11.9, 95% CI [36.1, 46.0]).

Overall, the two condition groups exhibited low familiarity with the presented book cover stimuli and showed similar reading behaviors and art interest, with moderate variability in both measures. Hence, any observed differences can be more confidently attributed to experimental manipulation (i.e., the text vs. image conditions), rather than pre-existing differences between the groups.

Furthermore, we also computed the descriptive statistics for the evaluations of stimuli per condition.

Table 2. Descriptive statistics for book cover evaluations for text/image conditions.

	Condition
	Text			Image
	M	SD	95% CI 1	M	SD	95% CI 1
Beauty	2.35	1.40	[2.29, 2.41]	3.75	1.79	[3.67, 3.83]
Interest	2.53	1.50	[2.46, 2.59]	3.62	1.80	[3.54, 3.70]
Contents	2.77	1.61	[2.70, 2.84]	2.94	1.51	[2.88, 3.01]
Cost	3.12	0.92	[3.08, 3.16]	3.10	0.944	[3.06, 3.14]
Critic Importance	2.68	1.22	[2.62, 2.73]	2.81	1.26	[2.75, 2.87]

¹ 95% CI = 95% confidence interval.

shows the means, standard deviations (SD), and 95% confidence intervals (CI) for each scale by condition. As can be seen in the ranges of the 95% confidence intervals, there were significant differences between the ratings in the two conditions.

4.1. Memory Performance

Our primary research question addressed whether memory performance differed according to the condition (text/image) in the recognition task, predicting better memory recognition for image content than text content. First, participants’ responses to the recognition task were categorized as hits, correct rejections (CR), false alarms (FA) and misses. Overall, hits accounted for the highest percentage of responses (44.64%), followed by CRs (26.29%), FAs (23.39%), and misses (5.68%) (see

Table 3. Descriptive statistics for the average frequency of each response type in the recognition task.

Response Type	Group 1	M	SD	95% CI
CR	(Overall)	20.9	7.55	[18.8, 23.1]
	Text	20.2	8.19	[16.8, 23.6]
	Image	21.7	6.94	[18.8, 24.5]
FA	(Overall)	19.1	7.55	[16.9, 21.2]
	Text	19.8	8.19	[16.4, 23.2]
	Image	18.3	6.94	[15.5, 21.2]
Hit	(Overall)	35.5	7.19	[33.4, 37.5]
	Text	32.8	9.34	[28.9, 36.7]
	Image	38.2	1.89	[37.4, 38.9]
Miss	(Overall)	4.52	7.19	[2.48, 6.56]
	Text	7.2	9.34	[3.35, 11.1]
	Image	1.84	1.89	[1.06, 2.62]

¹ n = 50; n = 25 (text condition), n = 25 (image condition).

). This distribution indicates that participants were generally accurate, with a substantial proportion of correct responses (hits and CRs combined = 70.93%).

When considering performance by group, as shown in Table 3, the text condition group exhibited a similar proportion of hits (20.5%) compared to the image condition group (23.85%). Similarly, CRs were marginally lower in the text condition (12.62%) than in the image condition (13.55%). False alarms were marginally higher in the text condition (12.38%) compared to the image condition (11.45%), while misses were more frequent in the text condition (4.50%) than the image condition (1.15%). These findings suggest differences in memory accuracy and response patterns between text and image conditions, with participants generally exhibiting higher memory accuracy for image-based stimuli. Figure 2 shows the distribution of these responses across the two conditions for each response type, with individual data points, boxplots and violin plots.

To statistically assess the impact of the two conditions on memory performance, we compared the two memory performance measures retrieved from signal detection theory (SDT), d′ and β. D′ measures the sensitivity, or the ability to discriminate between the target (i.e., hits) and distractors (i.e., false alarms), of the recognition task. It is computed by subtracting the z value of the false alarm from that of the hit. Higher d′ values indicate better sensitivity and more accurate recognition. β measures the response bias or the tendency to say yes or no regardless of the type of stimulus (target or distractor). A neutral unbiased response would be reflected in a β value of 1.0, meaning that there is an equal likelihood of responding yes or no to both targets and distractors. A β value less than 1.0 indicates a tendency to respond ‘yes’ (i.e., leading to more hits and false alarms), while a β value above 1.0 indicates a bias toward saying ‘no’ (i.e., leading to more correct rejections and misses). First, we computed the d′ and β for each participant in the two condition groups and then performed Welch two-sample t tests, using each measure as a dependent variable across the two conditions.

A Welch two-sample t test using d′ scores revealed a significant difference between the text and image conditions (t (45.89) = −3.66, p = 0.0006, 95% CI [−1.04, −0.30]). According to the descriptive statistics, participants in the image condition group showed higher memory sensitivity (M = 1.82, SD = 0.71, 95% CI [1.52, 2.11]) compared to participants in the text condition group (M = 1.15, SD = 0.57, 95% CI [0.91, 1.38). The Cohen’s d for this difference was 1.04 (95% CI [0.44, 1.62]), indicating a large effect.

A Welch two-sample t test using β scores revealed a significant difference between the text and image conditions (t (46.64) = 2.45, p = 0.018). Participants in the text condition group (M = 0.54, SD = 0.33, 95% CI [0.41, 0.68]) had higher β values compared to those in the image group (M = 0.33, SD = 0.28, 95% CI [0.22, 0.45]). The Cohen’s d for this difference was 0.69 (95% CI [0.12, 1.26]), indicating a moderate effect. In both conditions, β was less than 1.0, meaning that participants were more liberal in their responses, tending to respond ‘yes’ to both the targets and distractors. The lower β values in the image condition group indicate an increased response bias. In spite of this, it remains unclear from these results whether the response can be attributed to the condition rather than a response bias in one of the conditions. Further elaboration on this point can be found in the discussion Section 5.

4.2. Memory Performance x Aesthetic Evaluation

In this section, the relationship between memory performance and hedonic evaluations (i.e., beauty rating) for the presented cue in the recognition task will be assessed. The question addressed here assesses whether the memorability of a book cover is influenced by its perceived beauty. Given our experimental design, SDT measures, such as d′ and β, are not the most appropriate metrics for addressing this question. First, such measures are obtained by aggregating responses over the entire set of stimuli, while beauty ratings are attributed to individual stimuli. Secondly, they are computed using the participants’ responses for both targets and distractors. As we aim to assess whether participants remember the target stimuli better after evaluating them as more beautiful in this section, the responses for the distractors are not our interest.

For these reasons, the responses for the distractors were excluded from the following analysis. By doing so, we ensured that responses could be taken as a proxy of the book cover’s memorability. Since all the stimuli were presented during the learning phase, we can assume that the higher the memorability, the higher the chances of a positive response.

Furthermore, although participants were asked to rate all stimuli in the same way in both the image and text condition groups (to ensure consistency in the design), when it comes to the analysis, it would not make much sense to compare the ratings attributed to book cover images to those attributed to plain text. For this reason, the present analysis was limited to the target stimuli in the image condition group.

To assess the effect of the beauty ratings for the book covers on memorability, we included the beauty ratings as the independent variables in a logistic regression model, with recognition response (0 or 1) as the dependent variable. To control for its effect on memorability, other ratings, such as familiarity (whether the participant knew the book), contents (whether they could guess the contents of the book from the book cover), cost (how much they thought the book costs), and importance to the critic (whether they thought that literary critics deemed the book important), were also included as predictors in the model. Before running the model, we checked Spearman’s correlation coefficients between the variables with 7-point Likert scales (i.e., beauty, contents, cost, importance for the critic, and interest). The beauty and interest scores were strongly correlated (r = 0.82); hence, the interest score was not included as the controlling variable to avoid issues, such as multicollinearity. The predictors and model outputs are shown in

Table 4. Descriptive statistics for the average frequency of each response type in the recognition task.

Fixed Factors	Estimate	95% CI 1		z Value	Pr (>|t|)
		Lower	Upper
(Intercept)	−0.43	−0.93	0.060	−1.71	0.086
Beauty	−0.021	−0.10	0.058	−0.52	0.60
Contents	0.16	0.075	0.25	3.67	<0.001
Cost	0.13	−0.029	0.29	1.59	0.11
Critic	−0.09	−0.21	0.035	−1.40	0.16
Familiarity	1.20	0.49	2.01	3.13	0.0017

¹ 95% CI = 95% confidence interval. Significant effects are highlighted in bold.

.

As shown in Table 4, familiarity significantly influenced the model’s performance. The only other significant effect was due to the item measuring how much participants could infer the book’s contents based on the cover itself (Rating_Content). However, a point biserial correlation test between this item and the model’s target returned a coefficient that, albeit significant (p < 0.001), was negligible (r = 0.01). This suggests that the effect of the variable Rating_Content on recognition might be non-linear. Indeed, looking at Figure 3, we note that the ratio of recognized stimuli to non-recognized stimuli does not increase consistently with the rating. Specifically, book covers rated 4 or higher tend to be remembered considerably better than those rated 3 or lower, suggesting a threshold effect. In other words, it appears that a certain level of perceived content informativeness is necessary to significantly enhance recognition performance, which is not influenced by variations below this threshold.

4.3. Memory Performance x Reading Behavior

The third aim of this study was to examine whether reading behavior influences the memorability of book covers. To this aim, we assessed whether participants’ reading habits impacted the memory task performances, specifically the d′ and β scores. The number of books participants read in a year, their reading frequency, and their reading speed were used as proxies for their general reading habits. They were also used as variables to predict memory task performance in multiple linear regression models. As there are multiple predictors, we applied the model selection process using the MuMIn package in R to select the models with the best fit.

Two global models, implemented separately for d′ and β, included the main effects of the number of books participants read in a year, their reading frequency scores, and their reading speed scores, all coded continuously. Model selection was performed using the dredge function in the MuMIn package. Here, all possible subsets of the global model structure are assessed. Akaike’s Information Criterion (AIC) was used to evaluate the model performance. All model performance results for both d′ and β are reported in the Supplementary Materials (see Table S4). Below, we only report the result of the best fitting models.

Regarding d′, the best model is the one that only includes the intercept (β_Intercept = 1.47, t = 14.49, p < 0.001, 95% CI [1.28, 1.69], AIC = 112.7). Hence, the addition of any reading habits did not improve the model performance and did not significantly impact the d′ score. Regarding β, the best model includes the intercept and the reading frequency score (intercept: β = 0.82, t = 5.76, p < 0.001, 95% CI [0.54, 1.11], reading frequency: β = −0.08, t = −2.82, p = 0.006, 95% CI [−0.13, −0.02]). According to the model coefficient, the result suggests that the more often participants read, the smaller the β. Hence, people who read more often tend to say yes more often in the recognition task.

5. Discussion

This study considered the differences in the sensitivity of memory recognition and aesthetic evaluations according to the presentation of different book cover information. The primary aim was to determine whether memory recognition performance was better with image or text informational cues from book covers. Memory performance was analyzed using signal detection theory, which posits that recognition is dependent on the strength of the memory trace related to a decision criterion. In line with our hypothesis, differences were found in d′ in accordance with information type, with greater recognition found for the image features than the text features, suggesting that manipulating which book cover information acted as a cue influenced memory sensitivity. Moreover, image features appear to strongly aid in the recognition of book covers, which supports previous findings that visual information strengthens memorability (Kulhavy et al. 1993; Levie and Lentz 1982; Robinson et al. 2003; Verdi et al. 1997).

Furthermore, we also assessed β to examine response bias to prior presentation, independent of memory performance. As with the d′ sensitivity measure, β scores were also significantly different in accordance with information type. As a whole, participants responded liberally and were more likely to answer ‘yes’, irrespective of prior presentation (i.e., despite whether the book cover was a target or distractor stimulus). This response bias was significantly greater for the image features than for the text features, as indicated by the lower β scores; however, from this study, it is not possible to determine if this result is attributable to the image condition itself or not. This is because the information type (text/image) served as a between-subjects factor, and the same individuals did not participate in both the text and image condition groups. To ascertain whether there is a stronger response bias toward image features of book covers, it is suggested that future studies include information type as a within-subjects factor to account for individual participants’ response tendencies.

The second aim of this study was to address the effects of aesthetic quality on memory performance. To this aim, we looked at the relationship between hedonic responses (i.e., pertaining to beauty) and image content in the target stimuli. The results did not verify the hypothesis that more aesthetically valued book covers would be more recognizable, with no significant effect on memorability in accordance with beauty ratings. The recognition of book covers in the image condition group did not improve as a result of increased beauty; however, a significant effect was found for familiarity and book contents. This indicates that the ability to correctly identify target stimuli is not impacted by how beautiful they are perceived to be. Instead, differences were found in memory performance based on the presentation of image features in relation to familiarity and the extent to which the book’s contents could be inferred from the book cover image. Although we controlled for familiarity in our stimuli selection, in line with past research, it is still plausible that familiarity influenced recognition performance (Bastin and Van der Linden 2003; Chana et al. 2023). Most interestingly, the effect of the book’s contents was non-linear, suggesting that a certain threshold of inferred information needs to be reached in order to improve memory. This finding is elaborated further in the subsequent sections of the discussion.

The lack of a hedonic effect might have been because the ratings were insufficient to create a stronger memory trace. While it was supposed that beauty would drive memorability, book cover information that was rated less beautiful (i.e., ugly) was also well recognized. Accordingly, book cover information that evoked aversion might also be remembered, as arousal—regardless of positive or negative valence—may influence memorability (Chana et al. 2023). Moreover, these assessments were made based on the image features (text features were assessed in the same way for consistency in the study design, but only the image condition group was used for this analysis) as opposed to the ratings of the book cover as a whole; therefore, it is unclear to what extent the visual features or word content affected participants’ ratings and memory. This study highlights that the way that aesthetic evaluations of text and image information are made does indeed differ. Because this means that the aesthetic evaluations of the two conditions are not directly comparable, it draws attention to how book covers are assessed when both information types are presented together. Thus, if the entire book cover is evaluated (i.e., with both text and image information present), one can question what features are driving these evaluations of a book cover and how, given that the quality assessment of verbal and visual information is fundamentally different.

The third aim of this study was to address the effects of reading behavior on memory performance. The results did not verify the hypothesis that stronger readers would exhibit better recognition. No statistical differences were found in d′ in accordance with reading behaviors. The recognition of book covers did not improve as a result of reading behavior. This indicates that the ability to correctly identify target stimuli is not impacted by individual reading habits, and the differences in memory performance according to the presentation of text or image features of the book cover is consistent regardless of reading behavior.

Again, we also assessed β to examine response bias toward reading behavior. β scores were significantly impacted in accordance with the reading frequency score. As a whole, participants responded liberally and were more likely to answer ‘yes’ in relation to reading frequency, with this response bias being significantly greater for people who read more.

5.1. The Interplay of Text and Image Features

While this research has explored the effects of visual and verbal information/text and image elements on memory and perception independently, the combined impact of these modalities in book covers remains unclear. Therefore, in addition to analyzing the effects of text and images in book cover design separately, a more cohesive understanding could be gained by further examining the effects of text and image in combination. Exploring how text and images work together to influence cognitive processing would address the notion of congruence between the features of a book cover (i.e., the extent to which these features impart the same information or meaning) and how this congruency impacts memory and perception.

The Integrated Text–Picture Comprehension (ITPC) model suggests that the effectiveness of combining text and images in creating a shared mental model depends on the semantic relationship between the two elements (Schnotz 2014). For both text and images to effectively contribute to the mental model, they must share related meaning, facilitating a seamless integration. The extent of this alignment could influence how the elements of a book cover interact, with a more complementary relationship between text and images enhancing overall cognitive processing.

Alternatively, and contrary to the dual-coding theory, which asserts that adding pictures to text always leads to better learning by providing two distinct memory codes, the ITPC model also offers a different perspective. It suggests that combining text and images can sometimes have negative effects, particularly when the viewer has significant prior knowledge. In such cases, the multimedia effect may be reduced or even suspended, as the added visual input may not offer new or useful information. Moreover, the ITPC model indicates that presenting both text and images simultaneously can divide the viewer’s attention. The eye must switch between the two sources of information, resulting in split attention, which can disrupt cognitive processing and affect memory retention.

Processing fluency theory, on the other hand, suggests that stimuli that are easier to process are evaluated more positively, due to a preference for simplicity and clarity (Reber et al. 2004). When applied to book cover design, this theory could explain how the visual and textual components of the cover influence perception and memory. If the text and image are semantically related and presented in a way that allows for efficient processing in working memory, processing fluency can be enhanced. However, as outlined by the ITPC model, this requires both the text and image to be simultaneously available in working memory, which is necessary for effective integration (Schnotz 2014). The interaction between the text and image features of a cover could either enhance or hinder overall processing fluency and memory.

Therefore, as a development of this current study, future research should examine the integration of text and image information together in memory retrieval. This may, in turn, lead to stark differences in accordance with reading behavior or even a more pronounced effect of aesthetic value, especially in response to the entire stimulus (i.e., visual/verbal information collectively). Understanding how this combination impacts cognitive ease and memory retention could provide further valuable insights into how the design of book covers—through their textual and visual elements—affects both immediate perception and long-term memory, ultimately shaping aesthetic judgments.

5.2. Evaluations of Text and Image Informational Content

One of the aims of this study was to consider how aesthetic evaluations influence the memorability of book covers. In doing so, we highlight that responses differ based on the text and image features of books. Indeed, as addressed in the conducted analysis, images and text are not evaluated in the same way—the way the beauty of an image is assessed cannot be likened to the assessment of the beauty of plain text. For this reason, it does not make sense to compare both conditions in this way. This finding highlights how assessments of the aesthetics of literary materials differ when the focus is placed on verbal or visual information (see also Chamberlain et al. 2022; Chana et al. 2023). As such, this could also lead to further reflection on what makes text and images beautiful, given that verbal and visual thinking appear to be different processes. Therefore, a next step is then to explore how much of the way we perceive and respond to book covers is driven by visual beauty or semantic beauty. This could involve further investigations into the content depicted on book covers, both in terms of images and text. For example, whether the image content is in the form of artwork, an illustration, or a photograph or depicts everyday objects, scenes, or people, it could be seen as more or less beautiful. Furthermore, many studies have looked at verbal and visual information in text (e.g., semantics vs. typeface, see Childers et al. 1985; Childers and Jass 2002; Melmer et al. 2013; Schnotz 2014), which was not the focus of this paper.

Moreover, in everyday evaluations of books, the phrase ‘judging a book by its cover’, refers not only to the front cover of the book but also to the evaluations of the book’s content, which together form a judgment of the book as a whole. This was not the focus of this study since the book covers were regarded as aesthetic objects rather than as literary devices, and we considered how evaluations would be made solely using this information. This choice was made to reflect the way in which books are often presented as consumer products, with the book cover tile on display. However, this does not reflect further interactions that people have with physical copies of books and other expectations, such as genre. Some examples inlcude flicking through the pages of a book in a bookstore as part of the decision-making process, gauging the book’s thickness as an indicator of the number of words, and reading the back cover content, including the blurb and price. Perhaps further investigations should take this into consideration, seeing as our study shows that the contents influence memorability in a non-linear fashion. This suggests the need to analyze the semantic information inferred from text and image features and how they work together in our aesthetic responses to book stimuli.

6. Conclusions

This study reveals how responses are impacted by text and image informational cues, resulting in differences in memorability in terms of both sensitivity and response biases. While we see that image and text features are recognized differently, with better memory performance for image/visual information, it is important to note that the roles of text and images in book cover design were examined separately during the testing phase of this memory task. This provides insight into the differences in how features of book cover design are perceived and how they aid in memory retrieval; however, we are unable to draw conclusions about how the interplay of text and image influences subsequent memorability and evaluations.

From a broader perspective, this research provides insights into how literature is initially perceived when presented in the form of a book, prior to reading the literary text itself. Perceiving a literary book as a visual object in this way shapes subsequent evaluations. The different features that contribute to its design, namely, visual and verbal information, are presented harmoniously for aesthetic appeal while also acting as informational cues regarding the book’s content and quality. This study is a step toward understanding how the combination of text and image information in the design of a book cover influences how it is perceived, evaluated, and remembered.