Dynamic Perception of Plastic Movements: Biomechanics and Digital Artifacts ^†

Abstract

The article offers some reflections on the body-space relationship through images, graphic translations and visual studies that converge into a contemporary digital illuminated avant-garde of the fourth virtual dimension. Starting from the study of the Soviet coreutics of the 1920s and the biomechanics of multimedia performances, it moves on the cinesthetic suggestions and to the dance theorization integrated with the musical score. Finally, some contemporary forms of digital image production are taken into account by addressing a number of issues regarding the exploration of the object-image and its experiential re-elaboration through the production of new graphic knowledge.

1. The hyle of the Image and the Cinesthetic Perceptions

In 1910 the Swiss musicologist Émile Jacques Dalcroze, harmony professor at the Conservatory of Geneva, taught in Hellerau’s garden city. He made a radical experiment, considering music not as an end in itself, but as a means to increase happiness and creativity through the harmony of the body with the neuronal activity of the brain [1] (pp. 4–5). His goal was to make the body “a wonderful tool of beauty and harmony when vibrating in tune with artistic imagination and collaborating with creative thinking” [2] (p. 21). Exploiting the eurhythmic principles, he thought that his own body rhythm awareness had to be applied to other arts as well. We can consider this idea of the body as “tool of beauty and harmony” such as the specific intentionality of the soul, which, through new hyletic contents, constitutes the matter (hyle) giving image to the object. Let us consider the hyle a set of elements referred to the sensitivity that Husserl borrowed from Aristotle’s matter, and Sartre, in the world-image infrastructure, conceived as an object for the consciousness, aiming at the real thing in the world of perception. The consciousness of image needs for this psychic content, this materiality of image represented by Sartre, acting as an analogon, an equivalent of perception [3] (p. 30).

Talking about the relationships with the world, Martin Heidegger conceived human beings working within ontological fields of certain moods and emotions, but also with particular skills through which understanding and dealing with things.

According to the German philosopher, we learn things in the world not as a separate entity known by labels or words, but as “means” (Zeug) defined by their “manipulability” (Handlichkeit) or practicality [4] (p. 205). Therefore, considering the image as “medium” through its “manipulability” and chance to interpret its perceptions and declinations in relation to ontological fields, moods and emotions, therefore the passage from the medium-1 (not yet manipulated) to the medium-3 (transformed by the interpretation of its visual acquisition) defines a constraint with the observer. It is an intermediate level (means-2), corresponding to the image acquisition and its mental re-elaboration; this level of knowledge graphically determines the consequential expression of medium-1 into the medium-3.

In this process of image-body acquisition, a meaning transposition takes place through the linguistic visual channel able to trigger dynamic sensations schemes that, according to Evan Thompson, create a series of multi-layered “casual paths” (genetic, cellular, social and cultural) [5] (p. 62). This paradigm structures an action-perception-emotion network based on the temporal flow that places the mental dimension in the world by involving the body through its representations of reality.

Later, James Gibson’s ecological psychology carried out the reflection on the distance between the world and the body according to the studies of the ‘Embodied Cognition’, and in reference to the so-called affordances, those physical qualities of objects suggesting to the human being the appropriate actions to manipulate it. Following this repositioning of the body in relation to the perception and reconfiguration of space, Gibson redefined the visual perception in the involvement of the whole organism within the environment as its proprioception. According to this theory, both the environment and the body are complementary in their relationship and perception is an establishment of configurations involving the self-perception [6]. Bergson later attributed to the natural perception process of the object and image production, a necessary bond with the thing representing the “dynamic scheme” to preserve its mental dimension, as in a rethought model that transforms a multitude of details into signs. In the descriptions of consciousness, Bergson stated that the scheme is a becoming whose elements intersect each other, denying the “closed contours” of the image. Thinking of the art of cinema that with its technique “returns the flow of life [...] cinema is not only movement, but it is movement because it represents the consciousness life in its flow” [7] (p. XXIX).

“The flow of things like an organized symphony” [8] that Bergson traced in the rhythm of cinema, is also in the development of visual education through the improvement of illustrations in textbooks, films and television that, according to Arnheim, constitute an important factor in conceptual processing and in the development of visual thinking. “A visual artwork, in other words, is not an illustration of its operator’s thoughts, but rather the final manifestation of that same thinking” [9] (p. 4).

This visual image elaboration is possible when the properties of the object configured in mind are made visible to the eyes through images.

Therefore, perception means also to grasp abstract elements from readable information that Arnheim identifies in the correct relationship between the figure-background, the contour and the contrast referring to the processes of spontaneous perception, which consist in the capturing of abstract features such as “triangularity”, while individual differences are recognized later.

Through a sorting process of the figurative elements that constitute the image structure, visual thinking aims at the clarity that makes the meaning visible. It is to establish a relationship between the speakable and visible that, in 1963, Foucault delineated in the Birth of the clinic through “the Eye that would speak [...] this eye that speaks would be the servant of things and the master of truth” [10] (p. 39). A dichotomy between language and vision that, to know the truth, tends to the construction of speech.

Referring to visual supports for conceptual exposition of ideas, such as diagrams, schemes, symbols for word replacements, one of Antony Hill’s research about the ‘structure’ in the field of semiotics, syntax and semantics points to the use of the visual artist’s paradigms as a method of communicating ideas or assertions carried out in logic and geometry. Such as in Leibniz’s isomorphism that identified analogies for logic relationships, in fact, according to Antony Hill it is possible to find ‘structures’ in such relationships as lines and points. In 1936, psychologist Kurt Lewin stated, “the figures on the board illustrating some problems to a group of psychologists could be non-simple illustrations, but representation of real concepts [...]. We are not dealing with the representation of the dependence of certain classes of events from others, or about pedagogical tools for visualization, but about the conceptual determination of the dynamic properties of concrete situations. The fact we generally illustrate this conceptual representation through a figure is of secondary importance. We must emphasize that the figures lead to a misunderstanding, if the viewer looks at them in terms of the usual metric geometry instead of the topology. The chart on paper is in fact only a representation of certain topological structures, which should serve as a conceptual representation of psychological facts” [11] (p. 6).

Nevertheless, what are the connections existing between these thought’s representations by symbols and diagrams and the meaning of the image? Moreover, how can perception initiate analytical processes for the understanding of the sign? The mathematician George D. Birkhoff in his research on aesthetic forms, published between 1928 and 1932, focused on the concepts of ‘aesthetic measure’ [12] dealing with the perception of a group of objects characterized by ‘aesthetic sensitivity’ establishing a numerical scale [13]. His solution consisted of three elements of analysis, the first relating to the comprehension of the aesthetic object that, in order to be perceived with the senses, requires an effort of the observer’s sensory activity proportionate to a particular object’s property in relation to its complexity. Therefore, a final description in physiological terms corresponding to a ‘brain nerve fiber network’.

As a measure of complexity, Birkhoff defined the number of signs constituting a certain object; in the object perception, the observer exits this repertoire of signs trying to trace a texture of images, rhythms, periods, and information that are synthesized through a subjective channel.

Another perceptual element concerns the pleasure of the aesthetic object that compensates for the primary activity of the perceptual experience. Lastly, the mathematician referred to an ‘order’ inherent in the elements that may concern the graphic properties, musical compositions or ‘density’ measured according to an ‘order measure’ in relation to the aesthetic pleasure.

At this point, we want to bring these reflections on the representations arising from the imaginative act, the relationship between motion and image. J.-P. Sartre in his text “The Imaginary” reveals interesting research carried out by the French psychologist Henri Piéron on the study of imaginative consciousness developed on to the kinesthetic sensations who believed that eye movements reproduce lines accompanying hands with synergistic movements [14] (pp. 113–114). Later, in the wake of the cinesthetic sensations representation through images, spatiality, plastic arts and architecture, visual arts, twentieth century coreuthics, Soviet sound-gestural hybridizations, biomechanics of multimedia performances, theories of dance integrated into musical score and scenic space consisting of mobile platforms and screens took place.

2. Soviet Coreuthic and Biomechanics

The 20th century’s avant-garde represents a moment of interaction between plastic arts and image configurations that moves within a kinetic dimension through which understanding the continuation of the forms in the movement’s abstractness. In this context, an unprecedented phenomenon, the performance, sees the avant-garde artist covering a number of roles, including the costume designer, the dancer, the choreographer and the set-designer, a total expression of the image configured in the third dimension of space.

The avant-garde recognized in the idea of total-theater, dance-theater and multimedia show a series of interventions on the formal language of dance in relation to the artistic, cultural and technological innovations of time, synchronizing dance performances and plastic arts. In 1917, the presence of the Ballets Russes in Italy facilitated the launch of the ‘Manifesto of futuristic dance’ by declining a series of samples of artistic modernity that form a neuralgic node in the circuit of the theoretical-figurative approach to the mechanical expression of the line. They were imitations of machines through gestures, materials, movements and customs that assimilate biomorphic kinetics to the mechanical one [15], as prefigured in the Balla typographic machine.

The combination between the aesthetic mechanical evolution and the compositional attitude theorized in painting and sculpture was a product of the postwar period. Two forms of artistic expression that led to many scenography and choreographic experiences like the reinterpretation of the puppet before the 1920s. It was the transition element to the mechanized-body of the performer concealed by molded silhouettes, costumes and embedded within a kinetic space characterized by metallic structures, plastic surfaces and dynamic sets. The robot-costume was the natural evolution through which the human body disappeared and constructed mechanical trajectories by containing its actions in space and sliding strength lines through the configuration of the mechanized movement. This dehumanization of the body transcended the envelope of the skin while continuing to use the gesture tension referring to a new form of kinematic aesthetics. The ‘Futuristic mechanical dance’ by Paladini and Pannaggi of 1922 [16] represented the expression of this artistic form through which an asymmetrical structure of broken planes and straight edge volumes give the body an aggressive physiognomy linked to the idea of modern knight.

Mask, backdrop and gesture helped create the new figure of the artist in the field of the plastic arts and right this mechanical scene generated the Magnetic Theater project by Prampolini presented at the Exposition Internationale des Arts Décoratifs in Paris in 1925. Two years later Pannaggi presented in Rome an experimental work with the creation of a robot-costume worn by Mikailov for the ‘Anguish of machines’ by Ruggero Vasari characterized by an orthogonal stereometric structure for a performance where stylized gesture was accompanied by the chromatic-bright dynamism of the lights in scene. An ambiguous vision of the body in which the denial of plastic humanization and the exaltation of the machine-model, constantly oscillating, recover an ancestral ex-centrism inverted into the expressive code of the rhythm.

The influence of industrial Taylorism encoded in the theater of the set designer and choreographer Nikolai Foregger is the theoretical value of the mechanical dances conceptually linked to the rhythmic gesture of moving pistons, connecting rods and rocker arms: inside this theatrical car, the bodies of the dancers shaped an industrial moving plant. Marinetti exalted the energetic dynamism of this theatricality of the show inside the show that characterized shows and films of F.E.K.S. (Factory of eccentric actor) [17] and led to the unpredictability of the avant-garde and the jammed mechanics of syncopated movements.

Artists such as El Lissitzky who, through a process of depersonalization of anthropomorphism, assembled electromechanical structures into his characters, performed the transposition of such kinetic wheels into visual representation. This is how he worked for the futuristic work ‘Victory over the sun’ by Kroutchenykh, Matiuchine and Khlebnikov, or for the short film Ballet Mécanique by Fernand Léger and Gérald Murphy. Moments of reflection on the reconfiguration of the symbolic image celebrated within new values of urban modernity no longer accepting the human figure at the center of the scene but in an eccentric position, as the eccentric and nervous asymmetry of the avant-garde imago urbis interpreted by El Lissitzky, Schlemmer, Prampolini, Depero, Balla, Malevich, Exter, Ermolaeva, Kudriashev, Moholy-Nagy.

Experimental research for the transformation of the human figure according to a process of mechanical values redefinition in conjunction with a mechanization of the human plastic body and after figurative one is carried out. Schlemmer used the physical qualities of materials such as lightness, transparency, metal brightness, linearity of wooden rods, rotating expressivity of large rings involving a strong perceptive component. He referred to the futurism and constructivism philosophy that, with their dynamic-rhythmic progression, suppressed the human figure enhancing the sense of dance in its spatial and temporal development for the realization of total art. In 1925 along with Klee and Moholy-Nagy, Kandinsky manifested a similar form of holistic art through a series of linear and arabesque schemes, for the description of a performance by Gret Palucca in Dresden, by his visual language: mechanical aesthetic expression of lights and kinematic automatisms.

In this research on bodily kinematism in repositioning within the world, the artist expressed a kind of egocentric mythology corresponding to the urgency of establishing a direct creation process with the spectator in the pantomime style of the mask into the scene [18].

The protagonist of avant-garde dance Akarova [19] elaborated a gestural expression system through individual performances in an absolute dimension in which the costumes designed by the cube-futurist painter Marcel Baugniet provided her for a musical lexicon subordinate to the music in the composition of scenographic geometries enriched by bright chromatic projections, metallic costumes and masks. The visual synthesis of the play was an elementary poetics of the dynamic form as an expression of mechanical and bright will. Even Alexandra Exter, in her work for the Tairov Theater of 1917, Salomè used “sliding self-propelled scenes consisting of colored surfaces operated by electricity, whose movements have to fit perfectly with the dynamics of the drama. The importance of colored surfaces is due to the emotional value of these color combinations. Exploiting the light variations is also possible, to allow that light, both in the hall and in the scene, changes color and intensity according to the drama trend. The hall and the scene are united in a single atmosphere that accentuates the dramatic intensity of the representation” [20] (p. 81). Similar studies on light in the film’s performance produced by Exter concerned her work on the film Aelita, by Jakov Protazanov in 1924.

The Russian artist conceived transparent plastic costumes approaching to the scenic project of Rabinovic and the rotating bright structures, privileging the emotional state of the spectator stimulated by the actor’s abstract action and the dynamism of lights and colors on stage. Later, the same celluloid-mixed plastic inspired Naum Gabo and Pevsner to create the scenes and costumes for La Chatte performed by Ballets Russes in Montecarlo in 1927. The dancer’s body produced the rhythm of his inner drama structuring in space transparent trajectories, in a figure-background relationship with the scenic system, made essential by the functioning of the kinematic scene.

Larionov’s theater of 1913 developed similar implications of light and color on the gesture scene of the mechanized bodies of dancers. He created moving sets, real theatrical experiments in his fantastic theater project of the future “Futu” [21] obtaining multiple viewpoints for spectators.

The motion pictures characterizing the biomechanics of multimedia performances by Vsevolod Meyerhold, through a research on the plastic arts values into the chromatic rhythmicity of the space, gave rise to technological experiments which often used multiple projection screens for the mounting documentary material like texts, images and complex sound structures [22] (p. 24). The work of Gropius, Schlemmer and Moholy-Nagy at Bauhaus, together with the structuring of the actor’s movement into the stage, focused on the aesthetic principles of perception as well. They gave birth to avant-garde theatrical productions through which conceiving the mise-en-scène as a total artwork in which various forms of expression came together, including music, dance, painting, sculpture and word [23] (p. 85).

Fedor Lopukhov’s suggestions, involving theories on dance integrated into the musical score, focused on the values of the dynamic image. In fact, he was heavily influenced by the Constructivist movement, and imagined a borrowing of the machine’s aesthetic into the movements of the dancer using the scenic space in an unprecedented way, developing his choreographies on and around platforms and mobile screens [24].

In this artistic and cultural context, image synthesized from the observation of the movement became device through which making the movement; a diverse number of disciplines contributed to the construction of the scene, returning that process of self-perception theorized by Gibson.

3. Digital Avant-Garde on the Drawing in Motion

Digital artifacts today translate the dynamic image through technological possibilities of the virtual environment. Through its effect on our cognitive structures, this dimension creates new ‘prosthesis’ of the imaginative process, that radically change our way of reading and conceiving formal modulation of the surrounding space. In this way, a profound change comes also in the way of thinking on moving or fixed objects in relation to a predetermined space that becomes infinite in the virtual dimension.

The work of Walter Ruttmann, German director and prominent exponent of the film avant-garde, is one of the first experiments of the Motion Graphic and abstract cinematography, forerunner of the current digital exploration. Through his abstract short films (Lichtspiel I, Lichtspiel II, Ruttmann Opus III and Ruttmann Opus IV) he manipulated lights, sounds and graphic movements by generating a cinema with images in a rhythmic relationship with sound [25].

In the early twentieth century, Oskar Fischinger began to produce synchronization processes with images and the music of his time, as in An Optical Poem of 1938. It was an ante-litteram video project conceived through the progression of geometric shapes, concentric converging circles and rotating rhombus around coinciding vertexes. Geometric elements move by changing fading colors and transforming in symbiosis with the “Hungarian Rhapsody No. 2” by Franz Liszt. Fischinger worked on his abstract compositions thinking of an infinite space giving a physiognomy at a perception level of the image as meaning of itself and latent of sinesthetic values, using colored liquids, filters, slides and wax, anticipating the ‘light show’ of the Sixties [26] (pp. 76–79).

Looking at the contemporary concept of digital exploration there are similar interactive graphics experiments as the graphic-digital project by the Russian artist Dmitry Zakharov ‘Inside me’. It is based on the use of a 3D scanner that allows reproducing a transformed body image into a 3D object through a software, which makes possible to see this object from the inside, generating abstract shapes whose color is obtained from the real image elaborated by the scanner.

The video highlights the interaction between the experience of reality and its perception via the digital tool, into an unexplored body space readable through the digitization and manipulation of data obtained from 3D scanning. In this way, the artist obtains the creation and deformation of the image elaborated by the computer. According to an investigation about the implications of body movement on the visual arts, the two artists Quayola and Memo Akten, through audiovisual artifacts and multimedia installations, study the strength, tension and body balances in an abstract digital translation of the matter in motion controlled and returned in the form of an altered and liquid representation.

Their field of investigation deals with that level of knowledge in which the disciplines of electronic and engineering science interact with the artistic expressions of the digital tool.

Current applications of the animation technique known as ‘rotoscoping’ are increasingly approaching the representation of moving bodies that define digital traces through their kinetic transformations. The integration of cognitive science studies led the EUPHRATES research group under the direction of Masahiko Sato from Tokyo, along with the graphic designers Masaya Ishikawa and Akio Tokita to develop educational methods of communication and animation in the recognition of moving figures. The research group synthesized these figures through dots, lines, and letters, following a sound rhythm, to lead the gaze towards a figurative imitation of the symbols. This is a graphic method for reproducing the movements and the appearance of the main lines of the human figure. Sequence and serialized figures organize the video through the repetition of geometric patterns, which make the image always recognizable.

4. Conclusions

Observing those moving images, we produce a large number of figurative wills that participate in the definition of a final image, converging into a system of signs that interacts on consciousness. According to Sartre, the draughtsman “stimulates that action manipulating the black lines. We should not believe that those lines immediately affect the perception as pure and simple lines, with the attitude of the image as if they were elements of a representation. In the perception itself, the lines manifest themselves as representative meanings” [27] (pp. 57–58). As in the past, more and more accurate tools assimilate and rework the image through the acquisition of the real object explored at infinitesimal scale. Nevertheless, the process of learning the experience and knowledge of the obtained data moves within that uncontaminated field of transcription from A to B which is the cause and expression of the form and that determines a configuration borrowed from the lens of its reproducibility, out of the object itself, its imitation altered in time, interpreted and rewritten.

This quality of the representation belongs to the signs, but it is a knowledge in itself. The awareness of an image is consciousness of itself in a real position belonging to the perception, the pure hyle of the image. The Drawing of the movement constitutes the imaginative structure of the new meaning.