Final Version

Kyle Prior BA(Hons) Illustration 2017/18 Title: How is Technology Changing Storytelling in Children’s Publishing?

5422 words

Contents

Chapter

Page Number

Introduction

3

Chapter 1 – Context & Themes

5

Chapter 2 – Case Studies

12

Chapter 3 – Reflective Practice

17

Conclusion

20

Images

22

Bibliography

28

Appendices

31

Kyle Prior

2

Introduction Humans have always told stories. We do so for a variety of reasons: to teach behaviours, to discuss morals and values, or to satisfy curiosity. (Eisner, 2008) Although the way in which stories are told, and the tools that are used to tell them are ever evolving. While the reason for telling stories remains mostly the same, the delivery systems used to ‘teach behaviours’ or ‘satisfy curiosity’ change with the times and the tools available. From the introduction of the printing press, to the development of motion pictures, technology has, and always will play a part in storytelling.

The purpose of this study is to explore how contemporary technology is changing storytelling and how it might continue to do so in the future. The focus area will be how it affects storytelling within children’s publishing, primarily those with a target audience of 5-11 year olds, however other relevant sources and other target audiences will still be assessed as the core applications are still relevant. The investigation will be primarily qualitative and will be carried out by triangulating theories from journals, articles, books and interviews carried out solely for use in this essay. Relevant, pre-existing quantitative data will also be used where polls and statistics are needed to back up theories relating to usage amongst children and in education.

Discovering the impact of using technology as a storytelling tool is vital at this early stage of its implementation in books, as well as finding out how the technology can be used in a way which is advantageous to either the creator or reader. This will all be key to the development of my own creative practice and will help form my understanding of the potential applications in my own work.

This study will concern itself with contemporary technologies including: digital books or e-books, apps, traditional books enhanced with augmented reality, and virtual reality counterparts to books. The impact of the advancements within printing Kyle Prior

3

technologies, or other technologies that have impacted the production or propagation of traditional books will not be addressed within this study. However, there will be comparisons drawn between the effectiveness of traditional, printed books, and their technological counterparts.

The research carried out for this study can be broken down into multiple focal points.

These include: the uses of storytelling, how technology impacts storytelling within entertainment, interactivity and immersion, how technology effects storytelling within education, and finally, how it might successfully be used to enhance and change storytelling.

Kyle Prior

4

Chapter 1 - Context & Themes To understand the way in which technology is impacting and changing storytelling, it is necessary to understand the specific function or purpose of the story. When discussing the role of a storyteller in primitive times, Will Eisner states:

“[…] the teller of stories in a clan or tribe served as entertainer, teacher and historian. Storytelling preserved knowledge by passing it from generation to generation. This mission has continued into modern time.” (Eisner, 2008).

In practically all forms of storytelling, this statement is still applicable today, especially when children’s publishing is concerned. Children’s books function as either a form of entertainment or as an educational tool, and in a lot of cases it is a combination of the two.

Classic fairy tales serve as a prominent example of this union of entertainment and education, combining an entertaining narrative full of colourful characters and comedic moments, yet with an underlying moral lesson highlighted by said narrative (The importance of hard work in ‘The Three Little Pigs’, not trusting strangers in ‘Little Red Riding Hood’, not judging a book by its cover in ‘The Beauty and The Beast’, etc.).

The combination of entertainment and education within books is ultimately at the core of children’s publishing and is ever present even in contemporary children’s books. However, for the purposes of this research essay, the functions of entertainment and education will be assessed separately, as technology changes and impacts the two in different ways.

Technologically enhanced storytelling within entertainment Creatives and publishers have been attempting to use contemporary technology to tell stories for as long as it has possible to do so. There are examples of novels

Kyle Prior

5

being published onto floppy disks from as early as 1993, and the digitisation of books

by Project Gutenberg started even earlier in 1971. (Flood, 2014)

Digital books or e-books have played a large part in the publishing industry in the last few decades. The reasons for the increasing popularity of the e-book are many.

They are cheap to make and even cheaper for the consumers, there’s no limitations to the number of pages or physical format, and there’s no reason for e-books to be linear. (Salisbury and Styles 2012)

While there are many benefits, there are also many downfalls, some of which are a result of how publishers have been trying to use the platform. In the magazine industry publishers hoped that they could simply port the same content across a variety of platforms; a printed version, an online version, a mobile phone version, etc.

Ultimately, they failed to consider how people consumed each medium and what kind of content worked best on each. (Moggridge, 2010) Without realising, they were producing inferior iterations of their printed content, full of images and content that were optimised for a print based platform, ignoring the additional capabilities of the medium they were porting to.

This idea that when the delivery system is not being used to its full potential, digital publications are inferior to traditional books, is something that is echoed by practitioners, industry professionals and creatives across the board. For technology to successfully be used within publishing the tools need to be used to their full potential. Simply re-distributing the same content digitally will result in a static, visual representation of what was once a book.

Rather than trying to re-invent the way in which stories are told using these contemporary platforms, we have created digital books that act as simulations of printed books, limiting the diverse toolset provided when creating content for the internet, mobile devices or augmented reality. It’s necessary to rethink what can be done with these mediums and the additional features they offer, pushing beyond

Kyle Prior

6

what a book typically is. (Piper, 2013)

Printed books have their own strengths and characteristics. They’re a physical, tactile experience that can be shared, savoured and revisited. “The eBook needs to exploit its own particular characteristics and strengths to evolve as similarly special but distinct experience.” (Salisbury & Styles, 2012) But what are those characteristics? What are the benefits of telling a story in a digital format?

As made clear above, each platform should be considered individually as to take advantage of the unique traits and strengths that they offer, however, there are traits that are shared by e-books, augmented reality and virtual reality alike. When asked what elements should be considered when attempting to successfully tell a story digitally, Jonathan Skuse suggested the following

“Technology provides unique opportunities for us to allow children to never have the same experience twice, to offer autonomy, to synthesise with the real world (cameras, gps etc.), to personalise […]”. (Skuse 2017)

Essentially, potential benefits of technological storytelling include dynamic experiences, interactivity, immersion and increased engagement.

Interactivity & Immersion Interactivity is one of the key characteristics that is synonymous with digital books and mixed reality storytelling. There are already many examples of content creators integrating it into their digital publications, particularly within recreational tablet and mobile apps that allow the child to "play" with the content (Peck, 2017). ‘Choose your own path’ style books that offer non-linear narratives offer a level of interactivity that sits on the boundary between book and video game. Mini games are introduced that tie in to the story and often have an educational purpose, and sound design and animation are often triggered and moved along by the reader performing specific actions and interacting with their device. These elements are being used by

Kyle Prior

7

publishers as an attempt to increase engagement and provide a new narrative experience.

This integration of interactive elements is partly driven by the format and by the publishers attempting to create new narrative experiences, however, it is also driven by a demand for this type of media. Today, children grow up surrounded by screens, using them recreationally and educationally in their day to day lives, every generation more competent and familiar with it than the last. Interactivity within media is becoming the norm, and children desire to be in control of their environment, therefore in control of their books. (Peck, 2017)

How though, does interactivity affect the narrative? With the technology still being relatively young, many wonder whether it is simply a gimmick or whether it is truly beneficial to storytelling. One aspect in which interactive media is excelling, when used in the correct way, is in reader engagement. If the interactive features are unrelated to the story it can have a negative impact, distracting the child from the narrative (Takacs, Swart and Bus, 2015). However, when these elements relate to and support the story they can be beneficial. In a study to explore vocabulary training in kindergarten children with special needs, Eliane Segers found that interactive games could be used to explain vocabulary within a story and that doing so was more beneficial to the vocabulary development of special needs children than an adult reading a story to them was (Segers, Nooijen and de Moor, 2006).

Interactivity within narratives can also impact the way in which a reader connects with a story and its characters. In reference to his interactive re-imagining of ‘Frankenstein’, author and game designer Dave Morris stated that “Interactive storytelling emphasizes a personal connection with the characters. It is a powerful tool that can draw you so deeply into the world of a story that you lose sight of it as a story. You think you are there - at least, if it is done right.” (Morris, 2012). This connection with the characters and the world which Morris refers to is one that is synonymous with interactive media, allowing readers to not only read a story, but to become part of the narrative itself, often even becoming the protagonist.

Kyle Prior

8

Interactivity of this sort becomes even more prevalent within augmented and virtual reality (VR) based storytelling. While still in a very developmental stage, there are already examples of virtual reality being used to tell stories, both as standalone narratives and experiences that accompany printed books. The level of interactivity that virtual reality allows for far exceeds that of a book, immersing the viewer directly into a narrative, and potentially crossing the boundary from book to game.

‘Immersion’ is a term that is often mentioned in relation to the interactivity introduced when dealing with the first-person perspective that virtual reality offers. It is a term that is used extensively in video game design, and in this context, it refers to the viewer having a sense of spatial presence within a virtual environment (Wissmath, Weibel and Groner, 2009).

This sense of presence is of course possible for a viewer to experience through many mediums including even text and audiobooks, but this can rely on the viewers’ ability to relate to an idea of a space or situation and have the imagination to immerse themselves into it. A person viewing a scene through a virtual reality headset on the other hand, is instantly immersed and ‘transported’ to it, viewing the virtual world from the same level as the characters within a story, or even being one of the characters themselves. (Ryan, 1999) An example of this within children’s publishing would be ‘Tara’s Locket’, a virtual reality app that allows children to step into the story and meet the main characters face to face, follow the interactive story and help guide the protagonist through the fictional landscape.

While extremely powerful, one of the downsides of virtual reality is the cost of the equipment needed for consumers to view content. It’s still in its developmental stages and that not many people have access to, especially in the case of children’s publishing, the expense makes it much less viable as a delivery system. This makes it less commercially viable, especially in comparison to alternative platforms such as augmented reality (Gilbert, 2016).

Kyle Prior

9

On the other hand, augmented reality (AR) has the advantage of having an extremely wide consumer base that already possess the equipment needed to view it (Gilbert, 2016). As it is a technology that most modern smartphones and tablets are capable of supporting, it is a delivery system that most parents and their children have access to and are accustomed to using.

Storytelling using AR differs from VR in that the viewer isn’t immersed in a virtual world, but instead, virtual elements are superimposed onto the real world, allowing for a narrative that takes place in a familiar environment but that can contain fictional characters, objects and scenery.

How technology affects storytelling within education Educational storytelling has an entirely distinct set of functions that need to be fulfilled for it to be successful. Therefore, it is extremely important to make sure that these functions are still met or are improved when creating a digital narrative experience. These functions include, but are not limited to, literacy development, narrative and storyline comprehension and vocabulary development.

On the topic of literacy and comprehension in particular, a meta-analysis put together by the Institute of Education and Child Studies of Leiden University (Takacs, Swart and Bus, 2014), reviews the way in which adult guidance is necessary in order to understand a narrative story and learn words when dealing with traditional storytelling. To discover whether the same level of learning is possible using multimedia technology instead of adult guidance, their team compiled and assessed 29 studies with a total of 1272 children. They concluded that multimedia features such as animated illustrations, background music and sound effects provided a similar, if not superior, level of scaffolding for story comprehension and word learning in comparison to adult guidance.

Kyle Prior

10

However, they also found that some of the interactive features that are often present in digital books for children were in fact detrimental to story comprehension. (Takacs, Swart and Bus, 2014) Due to interactive elements such as hotspots and mini-games often being unrelated to the narrative, these features were found to be distracting and decreased learning potential. Another meta-analysis from the same team, looked at a range of e-books and found that 75% of them included hotspots and 65% included game-like activities, but only 20-25% of these were related to the story. This lack of relation between the story and the interactive elements meant that the children were switching between listening and playing, affecting story comprehension and learning due to cognitive overload (Bus, Takacs, & Kegel, 2015).

Another potential benefit of this in educational applications is the improvement of engagement. A study carried out by Juan Cristobal Garcia-Sanchez found that, augmented reality in particular, had a positive impact on literacy and learning effectiveness through cognitive attainment.

“Augmented technology contributes to increasing engagement, invites participation, and develops appreciation of the context. Augmented books are proposed to incentivize curiosity, facilitate the interpretation of text and illustrations, and provide a learning tool that relates to the reader [‌]â€? (GarciaSanchez, J, 2017)

Fundamentally, it can be concluded that in an educational setting, technology can be used to help keep children engaged, but has also been found to improve literacy, vocabulary and story comprehension, in particular when reading without adult guidance. However, this is only the case when the e-book or application has been optimised for said functions, and does not implement unnecessary interactive elements that can detract from the learning experience.

Kyle Prior

11

Chapter 2 - Case Studies To best apply the theories discussed in Chapter 1 and assess some of the real-world applications within children’s publishing, three digital publications have been chosen for their differing applications of the technology: ‘Axel Scheffler’s Flip Flap’ series which is a series of digital adaptations of traditional interactive children’s books; ‘The Hungry Caterpillar AR’ which is an augmented reality iteration of the iconic picture book; and ‘Tara’s Locket’, an original VR story experience for children. These three apps cover a wide range of technologies and potential uses within publishing and should allow for a full exploration of the benefits and pitfalls of each.

Axel Scheffler’s Flip Flap Series The Flip Flap series consists of numerous apps that are based on Axel Scheffler’s picture books of the same name. The original books [fig. 1] are aimed at 2-6 year olds and feature interactive flip book style pages that allow the reader to combine animals to create new creatures. This makes for an extremely tactile narrative experience that helps beginner readers while also keeping them engaged.

The question is whether the app versions of the flip flap books use the technology effectively to make reader experience comparative to the original?

When comparing the app to the book, there are very few differences. The bold colours and distinctive art style remain the same, as does the concept of the book itself, the reader is still flipping the pages back and forth to create creature combinations [Fig. 2], although on screen it is a much less tactile experience.

The biggest differences are in the layout of the pages and the ‘read along’ option that the app features [Fig. 3]. The layout in the app is portrait instead of landscape as seen in the book, this seems like a somewhat small change to make, however, it shows that they have clearly attempted to optimise for the platform, considering how the content would be viewed best and how the reader would interact with it.

Kyle Prior

12

Ultimately, they have avoided the pitfall of attempting to simply port over the same static content across all their platforms (Moggridge, 2010).

As stated, the only other big difference between the book and the app is the ‘read along’ option. This is one of the more beneficial features that’s implemented into a lot of digital books for children, and with good reason. As stated by the Institute of Education and Child Studies of Leiden University (Takacs, Swart and Bus, 2014), features such as ‘read along” are extremely beneficial to a child reading without adult guidance and can in fact improve story and vocabulary comprehension.

Overall this digital adaptation is an extremely simple one which has adapted to the platform only where it’s needed to, which, when considering the simplicity of the original book, is all it needed to do. There are no unnecessary features, games or hotspots that attempt to show off the technology, which avoids unnecessary distractions (Bus, Takacs, & Kegel, 2015), and the one feature that has been added is there to assist reading and learning.

The Hungry Caterpillar AR The Hungry Caterpillar Augmented Reality app is on the opposite end of the scale to the adaptation of Axel Scheffler’s books in that the AR app is an entirely separate experience to the original Hungry Caterpillar book. The main character, the underlying narrative and some of the art style draw from the original, however, almost everything else about the storytelling experience has changed

The original book, designed, written and illustrated by Eric Carle is one of the most iconic children’s books of all time, has won numerous children’s literature awards and has sold over 30 million copies (Brockes, 2009). The book is of a simple, linear narrative that follows the life of a caterpillar, yet it is the combination of the bright mixed media imagery, bold and playful illustration and distinctive ‘hole punched’ tactile elements [Fig. 4] that have made it the household name it is today.

Kyle Prior

13

The app uses augmented reality to combine the real world with the fictional, using the camera to take the caterpillar character and other story elements, and placing them into a real-life environment. This can be an extremely immersive technique, allowing the reader to connect the narrative to familiar surroundings.

As stated, the look of the original picture book is similar in some regards: the bold colours, the collaged textures and the characters all remain present. However, a lot of the character introduced by Eric Carle’s style has been lost when being transposed from paper to 3D.

One of the main issues of the app is the functionality and how the story is progressed. It takes the idea from the original, but moves it from being a naturally progressing story to a mini game in which the reader is required to fill a ‘food meter’ before they can progress to the next day [Fig. 5], this then repeats five times before the caterpillar cocoons and becomes a butterfly. Ultimately, this is the opposite of the dilemma proposed by Moggridge in which he mentions publishers falling into the trap of porting static content to digital devices and ignoring the additional capabilities.

Instead, the source material has been dissected and the app has been driven by the technology instead of the storytelling.

Another difference between the book and the app is the addition of interactive elements that are unrelated to the story and the readers progression through it. [Fig. 6] This could be seen as both a negative and a positive. The interactivity is fun, but also has the potential to increase engagement, invite participation, and incentivise curiosity (Garcia-Sanchez, J, 2017), however, as discussed earlier, interactivity that is unrelated to the narrative has been found to potentially decrease learning potential, be detrimental to story comprehension, and cognitively overload the child reader. (Takacs, Swart and Bus, 2014)

Kyle Prior

14

As a result of all of this, it could be said that the Hungry Caterpillar AR app is a less successful attempt at storytelling when comparing it to the original or even to other digital publications and apps. The spontaneous collaged images, the simplicity of the narrative and the tactile interactivity is what makes The Very Hungry Caterpillar a successful picture book, yet these elements have not been successfully transposed to the digital version.

Tara’s Locket The Tora’s Locket app is VR storybook created and developed by the experience design studio Big Motive. In contrast to the two previous case studies, this digital experience is an original one, not an adaptation of an existing book. This is a key distinction to make as this means that the narrative, characters, illustrations and the core idea have been developed with the platform in mind, allowing Big Motive to create a story which uses every aspect of the platform to its advantage.

Due to this digital story being developed in this way, as an original experience rather than an adaptation, Big Motive have created something that uses many of the positive elements discussed throughout the entirety of this discussion, while avoiding many of the potentially detrimental aspects. The pitfall of porting content that is not optimised for the platform (Moggridge, 2010) is instantly avoided as the content is only being created with this platform in mind. There is no limitation on the pages as it adopts an entirely new way of telling a story which requires the reader to progress the story by interacting with their surrounding (Salisbury and Styles 2012).

The app re-imagines the way in which we tell stories [Fig. 7], creating a brand-new format and pushing beyond what a book typically is (Piper, 2013). The use of VR allows for interactivity and immersion, granting the reader a sense of spatial presence and a connection to the story and characters not possible in other platforms (Wissmath, Weibel and Groner, 2009). The interactivity featured throughout Tara’s Locket is intertwined with the story, combining traditional storytelling and reading, with something more akin to a game [Fig. 8]. As discussed,

Kyle Prior

15

this sort of interactivity can provide scaffolding for story comprehension and word learning (Takacs, Swart and Bus, 2014), as well as increasing engagement, inviting participation, and developing appreciation of the context (Garcia-Sanchez, J, 2017).

One of the concerns of VR is the cost of the equipment needed to use it (Charara, 2017), with this app having a target audience of children, this is even more of a concern, however it is available on the Apple App Store and the Google Play Store, which opens up the potential audience to any child who may have access to a smart phone. The only equipment needed to view VR of this sort is an inexpensive mobile VR headset such as the Google Cardboard (Google.com, 2017)

There is a clear conclusion that can be drawn from these case studies: the key to creating a successful digital storybook is to develop it with the platform in mind from the beginning. With Axel Scheffler’s Flip Flap series, this meant staying true to the original book while taking into account how technology could be used to enhance it subtly yet effectively. The Hungry Caterpillar AR app has revealed how technology can distract creators from telling stories. Whilst Tara’s locket has provided the perfect example of the way in which creating an original narrative experience specifically for a digital delivery system is potentially better than attempting to adapt a traditional book. All insights that have been, and will be, extremely useful in the practical element of this research project.

Kyle Prior

16

Chapter 3 - Reflective Practice For the practical element of this research project I have taken the theories surrounding technology within children’s publishing and applied them to my own practice. Exploring what the creative process entails when developing a narrative for digital platforms, while also considering what works when attempting to enhance the audience experience.

To do so, I selected a scene from the fable ‘The Emperors New Clothes’ and have recreated it in four different formats: a traditional 2D printed version, an augmented reality version, an eBook or digital storybook and a virtual reality version. Doing this has provided me with a way to see how various delivery systems affect a story, while also allowing me to develop my illustration and digital skills.

After carrying out the initial research into the story, developing the characters, and testing the various pieces of software that I would be using to create the eBook and AR iterations, I started creating the scene I would carry across all four mediums.

Despite beginning with the traditional printed version, I wanted to make sure that I was considering the format and potential of all the platforms while doing so, for example, the dimensions of the image allow it to easily be printed onto an A4 sheet of paper for both a book or the AR version [Fig. 9], but it also works when viewed in landscape on an iPad or mobile device [Fig. 10]. I found myself considering these things throughout the development. When drawing I would be thinking about where a page fold would fall and simultaneously, in which way certain elements of the scene might work in AR. I think that this once again exemplifies the importance of developing with one, or numerous platforms in mind from the beginning to avoid not effectively optimising the content (Moggridge, 2010).

To fully allow myself to explore the medium, I created two variations of the eBook, or digital storybook. The first features an extremely simple example of interactivity in the form of a “Emperor view” button that switches between how the crowd sees the Emperor and how the Emperor perceives himself and is dressed in his finest clothes Kyle Prior

17

[Fig. 11]. This button is fundamentally related to the narrative, thus avoiding the pitfall of unrelated interactivity distracting from story comprehension and instead inviting participation and increasing engagement. (Takacs, Swart and Bus, 2014). I also integrated sound into the app, using the platform to add a layer of immersion not possible in traditional books. The murmur of the crowd, the child laughing at the emperor and the sound of royal trumpets when the “Emperor view” button is pressed; all of which add to the narrative and suggest an ambience. A prime example of using the characteristics of the digital platform to create a distinct experience (Salisbury & Styles, 2012).

The second variation of the digital storybook adds another simple feature using the unique traits of the delivery system, which in this case is a tablet device or smart phone. Once again, I have used sound, but this time to create a “read to me” feature.

By pressing the button, the text is narrated, and the words are highlighted along with it. This correlates with the study carried out by Institute of Education and Child Studies of Leiden University, that states that multimedia features of these sorts provide the same level of scaffolding for story comprehension and word learning as adult guidance does.

Developing the augmented reality version of the scene seemed to be the next logical step in my research. Using the same scene, I created a “portal” that allows the reader to immerse themselves into the image using a smart phone or tablet, adding depth and sound to the printed image [Fig. 12]. From my perspective as a creative and illustrator, this was the most technical and difficult version to create, as a result, the end product has potentially become more about showing off the technology than the story, as seemed to be the case with The Hungry Caterpillar AR. Seeing the obstacles that augmented reality puts in place for the creator makes me question whether it is the technology which is not ready, or simply not right for narrative-based experiences.

The virtual reality based experience which I created, was the version that I found the most interesting. As a creator, it is a completely new experience. Not only have I Kyle Prior

18

been able to create a scene in which the “reader� can be actively present in through full 360-degree immersion, but while doing so, I have been spatially immersed in the scene during the creation process.

While the scene I have created is extremely surface level in terms of what can be created with VR, it is extremely easy to see its potential within storytelling. The ability to walk around a single scene and see it from every point of view provides the narrative with endless potential. Even in my simple example, the scene could be viewed from the eyes of one of the crowd, the small boy, or the Emperor, each time gaining an entirely new meaning (Prior, 2018. Virtual Reality).

When comparing VR with the other digital delivery systems, I believe that it is virtual reality that holds the most potential for narrative experiences. One of the primary reasons for this being that the toolset for creatives is an entirely new one, ultimately forcing them to re-invent and re-think the way a story is told with this new platform.

Kyle Prior

19

Conclusion In response to the primary question, ‘How can technology be used to enhance storytelling in children’s publishing?’, this essay identifies that multimedia delivery systems like e-books, and technologies such as AR and VR, can, and are being used to enhance storytelling in several different ways, depending on the goal.

Storytelling with the purpose of entertainment and leisure can be enhanced by transforming the very static format of a print book into something much more dynamic, fluid, and most importantly, interactive. Potentially sitting in-between books and videogames, the interactivity introduced by technological delivery systems allows for an additional layer of engagement, immersing the reader fully in the story, allowing them to participate actively. In some cases, when looking towards VR in particular, this takes the reader from being a third-party party observer of a story, to being a first-person participant, potentially even the protagonist.

Storytelling with an educational purpose has an entirely distinct set of requirements.

When considering how storytelling can be enhanced for education specifically, it was necessary to consider the purpose of reading within the learning environment.

Several of the studies and articles mentioned in Chapter One show that, while its use within education is still controversial, technology can assist the learning process when it comes to story comprehension and vocabulary development. The ability to keep distracted students continuously engaged is also to be commended.

Regarding the impact that this has on the practice of illustrating and creating children’s books, it is vital to note the importance of how content is created for these platforms. It is evident that every individual delivery system has differing characteristics that can be used to the advantage of the narrative with varying success.

Kyle Prior

20

Attempting to replicate a traditional picture book in a digital setting is something that rarely seems to benefit the reader experience. The ideal would be to create a storytelling experience that is more than a “digital book�. Something that uses the full potential of the technology, and which sits in-between being a book, a videogame and a movie; an entirely new storytelling experience.

Creatives and developers will, without a doubt, continue to explore the potential applications of technology within children’s publishing. The way in which digital editions and narratives enhance the storytelling experience is becoming clearer as the platforms evolve. The potential for it to introduce interactivity, immersion, improve engagement and enhance cognitive development are already key examples of how the technology is changing the industry, and it is only a matter of time before further benefits become apparent as creatives adapt to the tools and develop entirely new ways to tell stories.

Kyle Prior

21

Images

Fig. 1. Scheffler, A. (2016). Axel Scheffler’s Flip Flap Pets. [image] Available at: https://nosycrow.com/wp-content/uploads/2016/07/axel-schefflers-flip-flap-petsstickish.png [Accessed 19 Dec. 2017].

Fig. 2. Scheffler, A. (2013). Axel Scheffler's Flip Flap Farm. [image] Available at: https://i.ytimg.com/vi/PpTJj9oT-zg/maxresdefault.jpg [Accessed 21 Dec. 2017].

Kyle Prior

22

Fig. 3. Scheffler, A. (n.d.). Axel Scheffler's Flip Flap Safari. [image] Available at: https://static-s.aacdn.net/img/ios/1223355850/3712d7af8174bc7d78167dd7dbbf4c20 [Accessed 25 Dec. 2017].

Fig. 4. Carle, E. (1969). The Very Hungry Caterpillar. [image] Available at: https://cmaynardgraphics.files.wordpress.com/2016/02/35.jpg?w=648 [Accessed 18 Nov. 2017].

Kyle Prior

23

Fig. 5. My Very Hungry Caterpillar AR. (2017). StoryToys Entertainment Limited.

Fig. 6. My Very Hungry Caterpillar AR. (2017). StoryToys Entertainment Limited.

Fig. 7. Big Motive (2017). Tara's Locket. [image] Available at: https://medium.com/@big_motive/taras-story-a-children-s-book-brought-to-life-in-vrdf62ae6519ed [Accessed 5 Nov. 2017].

Kyle Prior

24

Fig. 8. Big Motive (2017). Tara's Locket. [image] Available at: https://cdn-images1.medium.com/max/2000/1*FiwK8UC7U1J_Y5-R_qpCRQ.jpeg [Accessed 5 Nov. 2017].

Fig. 9. Prior, K (2017). The Emperors New Clothes. [Digital illustration, Photoshop]

Kyle Prior

25

Fig. 10. Prior, K (2017). The Emperors New Clothes, app. [Digital illustration, Photoshop]

Fig. 11. Prior, K (2017). The Emperors New Clothes, app. [Digital illustration, Photoshop]

Kyle Prior

26

Fig. 12. Prior, K (2017). The Emperors New Clothes AR. [Digital illustration, Photoshop & Zappar]

Kyle Prior

27

Bibliography

Books Eisner, W. (2008), Graphic storytelling and visual narrative : principles and practices from the legendary cartoonist, New York, Norton & Company.

Moggridge, B. (2010). Designing media. Cambridge, Mass. [u.a.]: MIT Press.

Salisbury, M. Styles, M. and Skuse, J (2012). Children's picturebooks. London: Laurence King Publishing.

Email Prior, K. Skuse, J (2017). Technology in Children's Publishing. [email].

Prior, K. Bergs, A (2017). Innovative Storytelling. [email].

Journals Garcia-Sanchez, J. (2017). Augmenting Reality in Books: A Tool for Enhancing Reading Skills in Mexico. Publishing Research Quarterly, [online] 33(1), pp.19-27.

Available at: https://link.springer.com/article/10.1007/s12109-017-9499-2 [Accessed 12 May 2017].

Piper, A. (2013). Beyond the e-Book: The New World of Electronic Reading. World Literature Today, [online] 87(6), p.29. Available at: http://www.jstor.org/stable/10.7588/worllitetoda.87.6.0029 [Accessed 18 Oct. 2017].

Ryan, M. (1999). Immersion vs. Interactivity: Virtual Reality and Literary Theory.

SubStance, [online] 28(2), p.110. Available at: http://www.jstor.org/stable/3685793 [Accessed 24 Nov. 2017].

Segers, E., Nooijen, M. and de Moor, J. (2006). Computer vocabulary training in kindergarten children with special needs. International Journal of Rehabilitation Research, [online] 29(4), pp.343-345. Available at: https://www.ncbi.nlm.nih.gov/pubmed/17106354/ [Accessed 19 Nov. 2017].

Kyle Prior

28

Takacs, Z., Swart, E. and Bus, A. (2014). Can the computer replace the adult for storybook reading? A meta-analysis on the effects of multimedia stories as compared to sharing print stories with an adult. Frontiers in Psychology, [online] 5.

Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4253734/ [Accessed 2 Dec. 2017].

Takacs, Z., Swart, E. and Bus, A. (2015). Benefits and Pitfalls of Multimedia and Interactive Features in Technology-Enhanced Storybooks: A Meta-Analysis. Review of Educational Research, [online] 85(4), pp.698-739. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4647204/#bibr87-0034654314566989 [Accessed 15 Nov. 2017].

Wissmath, B., Weibel, D. and Groner, R. (2009). Dubbing or Subtitling? Effects on Spatial Presence, Transportation, Flow, and Enjoyment. Journal of Media Psychology, 21(3).

Videos Prior, K. (2018). "Emperor View" eBook. [video] Available at: https://vimeo.com/250358509 Prior, K. (2018). "Read To Me" eBook. [video] Available at: https://vimeo.com/250358649 Prior, K. (2018). Augmented Reality. [video] Available at: https://vimeo.com/250358705 Prior, K. (2018). Virtual Reality. [video] Available at: http://kylebrianprior.co.uk/virtualreality/ Web Brockes, E. (2009). This one's got legs. [Blog] The Guardian. Available at: https://www.theguardian.com/books/2009/mar/14/eric-carle-author [Accessed 6 Dec. 2017].

Charara, S. (2017). Virtual reality v augmented reality: Which is the future?. [Blog] Wareable. Available at: https://www.wareable.com/vr/virtual-reality-vs-augmentedreality-which-is-the-future [Accessed 16 Dec. 2017].

Kyle Prior

29

Flood, A. (2014). Where did the story of ebooks begin?. [Blog] The Guardian.

Available at: https://www.theguardian.com/books/2014/mar/12/ebooks-beginmedium-reading-peter-james [Accessed 6 Nov. 2017].

Gilbert, D. (2016). The next big thing in tech: Why augmented reality will dominate 2017. [Blog] IB Times. Available at: http://www.ibtimes.co.uk/next-big-thing-techwhy-augmented-reality-will-dominate-2017-1598708 [Accessed 2 Dec. 2017].

Google.com. (2018). Google Cardboard – Google VR. [online] Available at: https://vr.google.com/cardboard/ [Accessed 19 Dec. 2017].

Morris, D. (2012). The Divided Self: Remaking Frankenstein as an Interactive Novel.

[Blog] Huffington Post. Available at: http://www.huffingtonpost.co.uk/davemorris/frankenstien-interactive-novel-divided-self_b_1456960.html [Accessed 24 Nov. 2017].

Peck, G. (2017). Making a connection with interactive children's books. [Blog] BookBusiness. Available at: http://www.bookbusinessmag.com/article/makingconnection-with-interactive-childrens-books/all/ [Accessed 24 Nov. 2017].

Sweney, M. (2017). 'Screen fatigue' sees UK ebook sales plunge 17% as readers return to print. [Blog] Guardian. Available at: https://www.theguardian.com/books/2017/apr/27/screen-fatigue-sees-uk-ebooksales-plunge-17-as-readers-return-to-print [Accessed 3 Nov. 2017].

Kyle Prior

30

Appendices

URLs of Videos Showing Practical Resolutions “Emperor View” eBook: https://vimeo.com/250358509 “Read To Me” eBook: https://vimeo.com/250358649 Augmented Reality: https://vimeo.com/250358705 Virtual Reality: http://kylebrianprior.co.uk/virtual-reality/

Email Interview with Jonathan Skuse 1. Since writing about the need to create distinct experiences within digital books, you've worked on projects like 'Teach Your Monster to Read'. With this experience in mind, what elements do you consider to be key to successfully make use of the technology?

I think the key to working in any medium is to try and understand its nature and then to produce work which carefully makes the best use of its strengths and minimises any weaknesses. Pop up books are extremely low-tech but provide a better experience (or at the very least more *tangible*) than many Augmented Reality applications with vast funding behind them. Choose Your Own Adventure books are largely weaker than their digital equivalent (although that said, there's an interesting game design lesson to be taken from benefits that come with the ease of cheating / subverting the rules in the paper version, limitations often bring interesting nuance, but that's a whole other story). Fixed format ebooks (eg. picturebooks) are a terrible experience when compared to the original book exactly because books are REALLY hard to beat when it comes to telling linear stories. They're a bad use of technology because they're a lesser *emulation* of something else. So, I think, the key is exactly that: to make *use* of technology.

Technology provides unique opportunities for us to allow children to eg. never have the same experience twice, to offer autonomy, to synthesise with the real world (cameras, GPS etc.) to personalise, it's crazy to ignore all that and instead

Kyle Prior

31

try to mimic another format badly.

2. How do you think technology changes storytelling?

Digital stories are less wedded to being narrative in nature. See Minecraft and so on, they're huge factories of stories and every kid who plays can regale you with tales of their exploits. These stories are emergent and created in *conjunction* with the user, not dictated to them. The Toca apps are obviously also excellent for this. There's an alchemy there which creates unique, personal stories which reflect the player's choices and whims. That's awesome. This doesn't just apply to children's media/games; games like "Everything" or "Journey" are fascinating in this respect. To a greater or lesser extent, digital storytellers ultimately create worlds and set users to lose in them and as time goes on we're coming round to the idea that the less you try and dictate what they *do* in these worlds, the better. The other thing I think that technology changes about storytelling are that it increases *reach*. Teach Monster has been played about 27 million times and was primarily designed and built by two and a half of us (admittedly there's more infrastructure on our publisher's side), or I look at the massive numbers of interesting zines and comics being produced, and the legions of indie game makers. I think tech removes gatekeepers and allows for greater pluralism in the stories being told.

3. Regarding the use of technology in publishing, how do you see it being used in the future?

Predicting the future is a fools game. Hopefully, it'll be used *better*, but almost certainly in ways that are unforeseeable right now. For example, I met someone today who is working on holographic technology that supposedly blows VR out of the water and reckons it'll be in homes in 5 years (I'm skeptical, but that's the point). Erm, maybe it's easier to guess what the *impact* will be? Having worked with both giant publishers and tiny upstarts, I think the big beasts will increasingly retreat back to their core competence of "books" (be they physical or ebooks) and younger companies will carve out their own space rethinking the ideas that have Kyle Prior

32

been so dominant for so long. On the flip-side, I think physical books will actually come out of this better crafted, designed etc. because they need to create enough added value to compete with the convenience/cost advantages ebooks have. Much like with the resurgence of vinyl, I think books are going to get better and better, even as technology marches on.

Email Interview with André Bergs 1. You've seem to have been exploring ways in which technology can be used to tell stories, so how has technology changed visual storytelling for you?

Technology and experimenting with it has always been a part of my own creative experience. So in essence nothing has changed in the process. If you look back into history, I believe technology has always been a part of storytelling, perhaps even going back to the invention of fire giving us campfire stories. For me the computer specifically has been a part of creating ever since I was young.

2. There seems to be an interesting crossover between video games and storytelling at the moment, even Protanopia has some interesting layers of interactivity. How important do you think interactivity is within storytelling?

Honestly, I don’t think it is. It’s an old discussion by now and I gladly admit that there has been interesting crossover work done. But I do believe that in essence storytelling and gameplay are two separate things, appealing to two different modes in the audience/user. So I feel that if you want to tell the best story, you should focus on that only. That’s why I limited the control you have to just camera control. You as an audience have no influence on the scene, only on how you see it.

3. Digital books, along with other storytelling technologies such as VR and AR are relatively new, particularly within the publishing industry? What direction do you see it going in?

Kyle Prior

33

I hope that there will be more self-published works as we move forward. That more authors find their audience on their own and we get to see more interesting works appear. It’s been great to see the game industry develop a healthy indie scene parallel to the giant industry for example. I’m hoping that we will see a similar development in narrative works, whatever shape or form they will take. I personally can’t wait for AR glasses to become as widely spread as smartphones are now and see all the things that will be made and stories that will be told!

4. What elements do you consider to be important for a digital storybook to successfully make use of the technology?

In the ending the only thing that really matters is that you make the audience feel something. And to do that your book needs to transcend itself. The reader should become unaware of the technology and all the details, but just follow the story. This is the same for any good experience. So I feel that the specific technology is not so important, it’s how they work together and are balanced that maters the most.

Kyle Prior

34