SIGNS, SPACE, AND TIME IN HCI - EVIDENCE FOR A SEMIOTIC RELATIVITY

         When using signs, symbols, icons, emoticons, and colours (i.e., when creating new sites), we rely on the mutual intelligibility of the output. Yet, every user brings along with him the “ semiotic landscape ” (Kress, 1996 :15) he has grown up with, as well as the semiotic practices which are strongly coloured by affective factors. We speak of connected communities, but we forget that every community has its “ semiotic unity ”. (McHoul, 1996 : 48) which might be unintelligible to other communities. In the same way as Sapir and Whorf considered language to act as a shaper of ideas, we deduce that the semiotic practices of each culture and community influence the cognition as well as the world view of every user. Research in this area would give way to a theory of semiotic relativity, in the sense of linguistic relativity. We use this thought experiment to explore what it means to be member of a certain community, that is to say to be ‘prisoner’ of that community’s semiotic practices and frames.

         The question that imposes itself when analysing semiotic communication via the WWW is the following : apart from the purely descriptive work of analysis of the WWW as an ever-evolving semiotic space, how can we enhance mutual intelligibility when creating sites or virtual worlds ? Up to now, we are investigating up to which point signs, as opposed to natural language, can act as a substitute and ersatz of natural language: we consider the computer and the WWW as a sign-processing system and “ self-organizing system ” (Andersen, 1996) in that it can have internal states which allow the processing of signs.

         Up to now, linguistics and semiotics have been trapped in the dilemma of a purely descriptive analysis. The speed of information flow on the WWW, as well as its large distribution, offer for the first time, a somewhat creative possibility for every WWW user to be part of the creation of a new semiotic space: a completely new generation of meaning structures triggered by sign expressions make their way via the WWW every day and will force us to revisit the intelligibility, actionability and historicity of semiotic systems, their meaning structures and their relationship to the experience of the world.

         The theoretical objective of our research work is twofold : firstly, we want to gain new insights in the human inference processes via multi-canal communication and interaction ; secondly , we want to use these insights in order to develop a creative space of symbolic and pictographic and iconic communication within the virtual world of AMUSEMENT. Every day, hundreds of people from different countries meet in AMUSEMENT and will experience what is actually communicated by means of images, symbols, icons and visual design. Yet, instead of simply experiencing the visual, they will not only be able to positively create meaning, but also negatively experience the limitations of meaning via visual aids.

         This does not mean that natural language cannot be a vital part of interaction within this space. Nevertheless, we believe that ‘meaning-in-use’ can be experienced to a large extent via non-linguistic signs, in the Hallian sense of a ‘silent language’. It will be very interesting to see how far we can go without the interference of natural language.

         Within the framework of AMUSEMENT, we investigate which parts of information conveyance are being taken over (and to what extent) by other meaning structures. Presently, we are witnessing a changing role of distribution between language and icons on the WWW, where visual aids are more capable of communicating feelings, emotions or affection. Interestingly enough, we have noticed a wide range of cognitive divergences due to language differences between English speaking, French speaking or German speaking users. Thus, analysis must not only be community-based, but language-orientated, in that language apparently influences perception of space and time. Thus, we noticed an interesting divergence concerning spatial and temporal cognition of the information space between Anglo-Saxon users and French users. This accounted also for cognitive differences between Anglo-Saxon users and German users, to a smaller extent, however.

         These findings will impose a theoretical basis for a combination of linguistic relativity and semiotic relativity

        The outcome of our investigations will hopefully help to connect communities by enhancing mutual understanding via new channels of communication, information conveyance and positive recognition of cognitive differences between different users and communities within the WWW and virtual spaces.

         Références :

         Andersen, P.B. (1992). Computer semiotics. Scandinavian Journal of Information systems. Vol. 4: 1992. pp. 3-30.

        Andersen, P.B. (1998). WWW as a self-organizing system. In: Cybernetics & Human Knowing, 1998, 5/2, 5 - 42.

        Kress, G. & van Leeuwen, T.(1996). Reading images. The Grammar of Visual Design. London : Routledge.

        McHoul, A. (1996). Semiotic Investigations. Towards an Effective Semiotics. Lincoln, Ne. : University of Nebraska Press.

       Stefik, M. (1997). Internet Dreams. Archetypes, Myths, and Metaphors. Cambridge, Mass. : MIT Press.

        Whorf, B.L. (1956). Language, Thought and Reality. Cambridge, Mass. :MIT Press. Hall, E.T. (1959). The silent language. New York : Doubleday & Company