Mark Chavez wrote:
FUTURE CINEMA
FUTURE CINEMA: SIMULATION IN ANIMATION
by Mark Chavez
Abstract
As society changes around us so does the means of communication and representation that we use. Our times are marked by an extraordinary and ongoing surge of activity affecting the ways the moving image is created and presented. These changes are evident in the progress of technical development in the Creative Industries by means of conception, presentation, and distribution; these forms are offering the possibility of innovative new aesthetic experiences. As events have created the possibility to steer the course of future development, an understanding of the relationships that exist between varying disciplines has emerged and with it a structure for technical and creative innovation. In the past 15 years alone emergent animation techniques have presented new methods that allow the artist/ film-maker to create new and innovative content. This has and will continue to change the way content is developed in cinematic animation.
This paper will discuss in a historical context contemporary Trends in Simulation as pertaining to film-making: How Digital Content Creation (DCC) technologies effect artistic innovation; Trends in Interactive Storytelling discussing influence in cinematic construction; and lastly project Directions that may be plausible for the Art of Animation in the near future: How these developments are significant to cinematic entertainment in both feature and hybrid animation (e.g. movies with live action visual effects, mobile entertainment and other forms) and specifically how these approaches are creating new possibilities. In addition this paper will discuss the influence of technology on animation creation, how scientific uses of animation in the realm of simulation have come back to effect entertainment animation bringing with it new possibilities.
Introduction
Animation has changed over the past 100 plus years that it has been used as an entertainment form. Animation’s inception is in innovation. As a form it has changed in many ways both in use and in purpose since its beginning. At the turn of the 19th and 20th centuries Charles-Émile Reynaud inventiveness and Georges Méliès cinemagical visual effects experiments anticipated contemporary animation’s fascination with the possibilities of the form. With Windsor McCay’s, “The Sinking of the Lusitania” animation was well established as a tool for conveying emotionally charged messages. J.R. Bray’s use of CEL animation enabled mass production of animation allowing for more conventional entertainment works to emerge. [1]
More recently with the advent of computers strong in graphics processing powers, simulation has become a major force in presenting new opportunities for media innovation.
Trends in Simulation
Contemporary techniques involving animation as aided by insights offered through other disciplines such as computer vision, cognitive science, learning theory and linguistics are developing techniques that will allow film-makers to extend beyond the current key-framed animation studio approach towards what can currently be defined as an experimental or procedural approach. Motion capture, long the whipping dog of die-hard key-frame animators, holds promise become something other than a motion digitization tool. It’s current use to quickly and often carelessly generate output for character motion thereby creating motion that often does not match the character. If left alone it causes an uncanny feeling with the action not quite serving the story and not working in a way that supports animations ability to focus all design elements. Computer vision offers ways of quickly digitizing sculpted character dimensions, though labor for the most part must be spent optimizing these volumes, algorithms for automated optimization are in development. Additionally methods for quickly deriving digital models from 2D drawings are and have been in development for some time. [2] Though a difficult task, it is a problem that will be solved.
Jung, B., H. B. Amor, et al describe in their paper titled “Action Capture: A VR-Based Method for Character Animation” their attempt to teach their characters to “learn to imitate the actor’s goal-directed behavior while interacting with the virtual scene.” [3] This system uses motion capture techniques that go beyond current key-framed approaches where every frame of an animated sequence is digitized to a method that captures gestures. In their paper “A Bayesian Interactive Optimization Approach to Procedural Animation Design” researchers at the Unversity of British Columbia discuss a method that employs the Baysian technique of bringing in “prior” belief based on previous runs of the system and/or expert knowledge, to assist users in finding good parameter settings in as few steps as possible. [4] The work of Talton et al. [5] suggests a system that draws from the literature on design and human cognition to better understand the design processes, a self-reinforcing system that becomes easier to use as more people participate. Brochu et al, uses a “principled probabilistic decision making approach to model the design process, which can be studied using the existing tools of machine learning and optimization, and adapted to a variety of design scenarios.”
Trends in Interactive Cinema
Numerous scholars and professionals have explored the topic of interactive narratives or interactive storytelling with observations taken on various interfaces. These are seen to be comprised of two varieties: content and form. But due to the nature of interactive narratives and its significant difference from linear cinematic storytelling, how to tell a story attracts far more attention than what the story actually is.
According to Tomlinson, there are five major differences between linear and interactive animation. [6] These are Intelligence, Emotional Expressiveness, Navigation Collision Avoidance, Transitions and Multi-character Interaction. Among these specific differences there are two unsolved problems from aspects of Transitions and Multi-Character Interaction, which are: 1) characters may have discontinuous motion if animated cycles do not complete through motion action and transition, and 2) if there is more than one character in a scene, characters are not enabled to robustly engage in expressive close contact. In interactive storytelling, characters’ behavior, intelligence and emotional expressiveness is not predictable due to the broad range of possibilities available to the user, while all these factors can be decided and by the film-maker in linear animation they are difficult to resolve within the interactive animation sphere. Current limitations of image quality and animation granularity (the quality and number of frames used for each individual action and transition) in real-time graphics is a problem that can be overcome with quicker processing power, computer memory and programming. Synchronizing a game character to the scene based on events transpiring both to the character and within the game is difficult for a system to resolve and detract from cinematic believability. For example a character that walks through a puddle of water in a game system makes no adaptation to the way s/he walks; there are no changes in the costume resulting from the contact with water and no consequences to the action.
In comparison to a character’s action, emotion is more intrinsic and hard to precisely synchronize through real-time on screen representation. Human emotional expressiveness can be observed through verbal and non-verbal forms. These can be mimicked by the animator to equivalent comparison in a linear narrative, cinematic approach. Voice (tone, speed, pitch, etc.) and words (literal meaning, implication, indication, etc.) comprise verbal expression, while scene color, audio ambiance, character facial expression and gesture are major channels of non-verbal expression. Research done by Justine Cassell and colleagues at the University of Pennsylvania probed into this area, specifically on the topic of ‘Facial, Gesture & Spoken Intonation for Multiple Conversational Agents’. This study has built up the synchrony of gesture, facial movement and speech by facial control system, while gesture is separately generated by the dialogue planner based on ‘a database of facts describing the way the world works.’ [7] This SIGGRAPH 1994 paper proposed a possibility and vision of integrating emotional expression and storytelling as interactive storytelling. At the same time, authors also believed that it could be used in non-real-time animation production as a more precise database of facial and gesture movement what they determined as guess work, an unfortunate aside that continues to alienate practitioners of the art of character animation from researchers who leverage their expertise on scientific skill sets. Their project emphatically aimed at the construction of database, information retrieval and synchrony methodology, not surprisingly the quality of animation and storytelling was ignored, resulting in a lack of entertainment appeal for the audience.
Another debate also lies in the topic of tension between the coherence and the control of story-telling.[8] [9] [10] Compared to other concrete and technical aspects of presentation, balancing these fundamental narrative relationships ultimately shapes the intensity of interactivity of the final work. In the game domain what audiences demand in an interactive experience is reduced by the author’s control over the story and characters. In the game design, developers have to carefully limit the options that users can choose and thus move users through a pre-defined set of branching actions. In fact the user’s choices are confined to a small set of possible actions. A technique called narrative mediation achieves a certain degree of balance between coherence and control. It provides more options for audience than in game play while keeping the coherence of the narrative by giving ‘a centralized author agent control over a central characters actions.’ [10] Nelson et al (2006) also presented a method using drama manager to guide the user through the story and modify the experience based on user’s choices. [11]
In “Cinematics and Narratives: an exploitation of real-time animation” the authors Chavez et al writes about a system that is able to visually adapt to the viewer extracting biometric cues (facial expressions, body posture and position) and size of audience from a camera based interface. Basing their assumptions on studies made on archetypal forms in abstract imagery they create a system they coin as Active Cinema, dynamic movie system. “Active Cinema is designed to be a specific experience that changes it’s playback to suit the audience response in a manner that manipulates the audience from a preplanned emotional story beat.” [12]
àRagdoll techniques for animationß
Directions
Many contemporary youth spend hours and hours living out virtual lives in the Metaverse. Within this space, since the 1990’s, Machinima [13] techniques, toolsets built from video game technologies, have enabled amateur film-makers to make significant works that have international impact. In November of 2005, Alex Chan under the internet pseudonym Koulamata created “The French Democracy.” It was a movie about situations that led to riots that broke out in the Parisian suburb if Clichy-sous-Bois in October of 2005. The movie proved that Machinima film-makers can make a significant contribution to public discourse regarding current events. [14]
What this says goes beyond the artifacts present in movies created from assets developed using game technology. The look and style of motion, the design of characters and the fluidity of visual film language is significant however impactful story-telling always wins over poorly scripted narrative. In the case of Machinima the remixing of existing media used to create new forms of cinematic entertainment is significant. Regardless of the quality of image and sound, if the story is compelling and pertinent to contemporary life people will find the movie fascinating to watch.
The Machinima technique also speaks to the co-production aspect of online toolsets. This group development approach of online film-making points to possible larger networks of cinematic toolsets designed to optimize the users experience in authoring content and suggests a way to understand how users manipulate story to form archetypal narrative. Although tools specifically created to enable film-makers to create in an online environment are available in Machinima environments like Second Life [15], the quality of the output is not sufficiently attractive enough to compete with toolsets that studios use. The significant difference is that one is created for real-time game play and the other disregards computational limitations and pursues the final rendered image to be expressed in the most visually descriptive manner.
In his article “Ending the Tradeoff of Time vs. Quality when Creating 3D Computer Graphics Content” Alex Herrera speaks of “The Render Gap,” the disparity between two current options for final delivery of imagery: real-time render quality vs. offline render quality. [16] “Rendering high-quality 3D computer graphics (CG) is traditionally a very time-consuming process, and one that requires the use of multiple computer systems and resources. Rendering, though taxing on production workflow, is a necessary and integral part of digital content creation (DCC); and, it is a process that DCC professionals must undertake at various intervals, pre-production and post-production. A CG project will undergo a wealth of changes, both minute and monumental, from start to finish.” He writes that the solution is in the development of render software that exploits current and emergent graphics hardware configurations and streamlines “complex elements that are notorious for slowing rendering times...” He goes on to describe the creation of high-definition imagery comprised of 1.98 million polygons rendered at more than 2 hours using current production rendering software. Taking the same dataset and settings and using a render package optimized to use GPU stream processor arrays this same task is reduced to 12 seconds.
Previsualization in Feature Films
Future Trends ‘All that is solid melts into air’ Live action pipeline: Virtual Production Animation pipeline: Virtual Production - A completely non-linear workflow
- Film-makers design final CG imagery in real-time
|
"Previsualization has been around in various forms for decades. Before shooting the original Star Wars, George Lucas worked out the timing for space battles by cutting together footage of World War II fighter-plane dogfights. On the third Star Wars film, Return of the Jedi, the speeder-bike sequence in the forest was tested out using action figures that were shot with lipstick cameras. “We could figure out movements and work out a little, dynamic piece of action, then cut it together and see if the shots we’d planned were going to work well,” says Neil Krepela, ASC, a cinematographer and visual-effects supervisor who worked on Jedi.
In the definition crafted by the Previsualization Committee, previs generally comprises computer-generated imagery created in a 3-D modeling-and-animation application and then edited together to demonstrate the potential execution of a scene or sequence." http://www.theasc.com/ac_magazine/June2009/Previs/page1.php
These approaches allow the user to more easily tune their animation without going through tedious experimentation or years of unrelated programming study. Taking this concept further this approach will allow an artist to bringing expert knowledge outside of his own experience to a creative problem, making a difficult task easy. Imagine a collaborative system that uses the data from a body of users to learn varying parameters. In the same way a dancer learn motions in a choreographed dance, an animation system constructed in this way would learn from a group of active participants what the film-maker desires to create from his creative actions and provide solutions to the problems s/he presents. Following this line of research to its inevitable conclusion suggests the creation of a tool where film-making becomes a participatory act of creation where memory and perception become intertwined.
Conclusion
As these technologies gain acceptance, and in fact become commonplace, the opportunity to narrow the act of conception to delivery will reduce labor costs and create an efficient though a considerably different animation production work flow. Currently each step along the production process is staffed by highly skilled specialists who painstakingly niggle tiny details to better the final product. This work flow will inevitably be influenced by innovations being developed as research for use in scientific simulation, interactive entertainment and directly being developed for big budget motion picture film-making. Computer games and their not so distant cousin, interactive storytelling present opportunities for innovative film-makers to make high-quality animated visuals and stories. These practices are enhancing the budgeting process creating a situation where entire movies are conceived in a previsualized state. These visuals will be used in hybrid animation (live action cinema mixed with animated visual effects). Emergent gesture based animation techniques being developed by research teams internationally could reduce the time that animators take to develop character behavior. Sophisticated group based narrative development,
Acknowledgement
The research project Cinematics and Narratives: creating stories within real-time visual toolsets (NRF2008IDM-IDM003) is fully funded by National Research Foundation, Singapore.
References
1. Callahan, D., Cel Animation: Mass Production and Marginalization in the Animated Film Industry. Film History, 1988. 2(3): p. 223-228.
2. Liu, Y. and Y.T. Lee, 3D reconstruction of freeform shapes from 2D line drawings, in Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry. 2010, ACM: Seoul, South Korea. p. 303-310.
3. Jung, B., et al., Action Capture: A VR-Based Method for Character Animation, in Virtual Realities, G. Brunnett, S. Coquillart, and G. Welch, Editors. 2011, Springer Vienna. p. 97-122.
4. Brochu, E., T. Brochu, and N. de Freitas. A Bayesian Interactive Optimization Approach to Procedural Animation Design. 2010: Citeseer.
5. Talton, J.O., et al., Exploratory modeling with collaborative design spaces. ACM Transactions on Graphics (TOG), 2009. 28(5): p. 1-10.
6. Tomlinson, B., From linear to interactive animation: how autonomous characters change the process and product of animating. Comput. Entertain., 2005. 3(1): p. 5-5.
7. Cassell, J., et al., Animated conversation: rule-based generation of facial expression, gesture \& spoken intonation for multiple conversational agents, in Proceedings of the 21st annual conference on Computer graphics and interactive techniques. 1994, ACM. p. 413-420.
8. Riedl, M., C.J. Saretto, and R.M. Young, Managing interaction between users and agents in a multi-agent storytelling environment, in Proceedings of the second international joint conference on Autonomous agents and multiagent systems. 2003, ACM: Melbourne, Australia. p. 741-748.
9. Steiner, K.E. and J. Tomkins, Narrative event adaptation in virtual environments, in Proceedings of the 9th international conference on Intelligent user interfaces. 2004, ACM: Funchal, Madeira, Portugal. p. 46-53.
10. Riedl, M.O. and R.M. Young, From linear story generation to branching story graphs. Computer Graphics and Applications, IEEE, 2006. 26(3): p. 23-31.
11. Nelson, M.J., et al., Declarative optimization-based drama management in interactive fiction. Computer Graphics and Applications, IEEE, 2006. 26(3): p. 32-41.
12. Chavez, M. and Y. Liu Lin. Cinematics and Narratives: An Exploitation of Real-Time Animation. in Cyberworlds (CW), 2010 International Conference on. 2010.
13. Marino, P., 3D game-based filmmaking: The art of machinima. 2004: Paraglyph Pr.
14. Lowood, H., Found technology: Players as Innovators in the making of machinima. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning, 2007: p. 165-196.
15. Rymaszewski, M., W.J. Au, and M. Wallace, Second life: The official guide. 2007: Sybex.
16. Herrera, A., Ending the tradeoff of time vs. quality when creating 3D computer graphics content: stepping up the production workflow with real-time rendering software. SIGGRAPH Comput. Graph., 2010. 44(1): p. 1-12.