Dagmar Schmauks

Abstract

Referent identification in human conversation is performed both by describing the objects in question and by pointing at them. Up till now, only the linguistic component could be simulated in dialog systems. But recently, technical innovations have made it possible to 'point' at the objects on a display as well. The paper has two intentions. First, it investigates natural pointing in more detail and offers some possibilities to classify the great variety of pointing actions. Then, it tries to clarify the extent to which pointing by technical means (especially mouse-clicks) can be regarded as a simulation of natural pointing or as a functional equivalent. Furthermore, some steps towards even more accurate simulation are briefly mentioned.

1. Introduction

1.1 Terminological remarks

The term 'deixis' denotes those referential devices whose interpretation requires a consideration of the situation of utterance. "Local deixis' means the specification of directions, places and objects relative to the speaker's actual orientation. The closely related topics 'anaphora' and 'text-deixis' are not treated in this paper. One component of deictic actions are linguistic expressions, mainly demonstrative pronouns ('this', 'that') and adverbs ('here', 'there'), the other being extralinguistic means, especially pointing gestures. In this paper, the latter are represented by the sign '/'.

"Natural pointing' denotes pointing actions occurring during interhuman dialog. This includes the use of easily available aids like pencils or pointers. "Simulated pohuing' means the use of technical pointing devices during man-computer dialog.

Following the terminology of Clark, Schreuder, and Buttrick (1983), the object pointed at is called the 'demonstratum', and the descriptive part of the accompanying noun phrase (if there is one) is called the 'descriptor'. The 'referent" is the object to which the whole pointing act is intended to refer.

1.2 Motivation

In face-to-face interaction, pointing gestures are used frequently and efficiently. Although their referential power is beyond any doubt, they have, up till now, hardly ever been treated in more detail. The disciplines concerned with them are mainly semiotics, linguistics and psychology.

Recently, the investigation of natural pointing has also become interesting for the area of Artificial Intelligence. In dialog systems developed to date, objects could be referred to by more or less complex verbal descriptions or unique artificial identifiers only. Technical innovations (e.g., high-resolution graphic displays, touch-sensitive screens, pointing devices such as mice or joysticks) have made it possible to simulate pointing gestures to various degrees as well. Multimodal input is both more comfortable from the user's point of view, and a more natural simulation of interhuman communication.

Therefore, several systems have been developed recently which allow the combination of verbal descriptions and pointing gestures for referent identification (see section 5.2). One example is the dialog system XTRA (a natural-language access system for expert systems) which is currently under development at the University of Saarbriicken. Its current application domain is to assist the user in filling out a tax form which is visible on the screen. In section 5.3, XTRA's deictic component TACTILUS is shortly presented. The term "form deixis" shall henceforth denote all those pointing actions which are performed in order to specify regions or entries of a form.

An adequate simulation of pointing gestures presupposes a thorough investigation of the regularities which underlie natural pointing. Therefore, the next three sections investigate natural pointing in more detail. Section 2 shows that pointing actions (although functionally similar) are not a uniform phenomenon but differ with respect to various aspects. Semiotics, linguistics and psychology (study of nonverbal behavior) are concerned with these investigations. The interdependency of describing and pointing is the topic of section 3. This relationship is relevant with regard to tan-guage processing, because natural and simulated dialog have a lot of problems in common. More details on the issues discussed in section 2 and 3 are to be found in Schmauks (1986b). Section 4 treats the peculiarities of form deixis, which is the special type of deixis occuring in the XTRA system. Section 5 tries to clarify the extent to which technical pointing devices already in existence can be regarded as a simulation of natural pointing, or as a functional equivalent. In section 6, some steps towards even more accurate simulation are briefly mentioned. (Thus, the last two sections are interesting from a cognitive science point of view as well.)

2. Essential features of natural pointing

All efforts to simulate natural pointing have to take into account that pointing is not a uniform phenomenon. This section shows that the goal 'pointing at something' is achieved by a great variety of body movements. Up till now, only a small part of these can be simulated (see sections 5 and 6).

2.1 The variety of pointing actions

Pointing actions are those body movements which are performed by a speaker to direct the hearer's attention to some part of the shared visual field. In the normal case, both for their encoding and their reception by the hearer no other means than the human body are involved. Successful reference by pointing requires that the addressee pays attention visually to the person who is pointing. One may suppose, therefore, that linguistic material such as demonstrative pronouns or deictic adverbs serve as a request to turn one's face to the speaker.

Pointing can be performed by various body movements, mainly gestures. The most frequent one is the 'finger point', by which the index finger is extended in the direction of the object or place indicated. A much more vague gesture is pointing with the thumb over one's shoulder. Other extralinguistic reference devices are head movements and line of sight.

All these actions are only interpretable as 'Look there!' if the speaker uses a body movement which belongs to the stock of signs s/he shares with the hearer. For example, the African 'mouth point' (Kirk, Burton 1981) will not cause the intended reaction on the part of a European hearer.

2.2 'Visual' and 'tactile' pointing

Sometimes it is possible not only to point to an object, but also to touch an object within reach. In these cases, pointing becomes much more precise, because some of the ambiguities of natural pointing are dropped (see sections 3.3 and 4.1). If there is a physical contact between finger (or pencil etc.) and the indicated object, the action in question is called "tactile pointing' as opposed to 'visual pointing' where there is no such contact.

So far. only a small subset of naturally occurring pointing gestures can be simulated on a terminal screen, namely certain kinds of tactile pointing gestures. The emphasis of the remainder of this paper will therefore rely upon this type of deictic gesture and its relation to verbal descriptions. However, many observations to follow will also hold for pointing gestures in general.

Tactile pointing gestures can be classified according to various aspects of their observable appearance. Some distinguishing characteristics are:

• body parts involved in execution of the gesture, i.e. number and position of fingers,
• presence or absence of visual guidance,
• use of aids (pencil, pointer . . . . ),
• complexity of movement (singular, repeated, multiple pointing), and
• duration and intensity of gesture.

An adequate simulation of tactile pointing has to take into account at least some of these features.

2.3 The relationship between pointing gesture and demonstratum

One open problem is whether there are correlations between the physical features of pointing gestures and the objects thereby indicated. Up till now, it cannot be taken for granted that different persons point in an identical manner at objects of a specific size, location, shape, depth of embedding etc. Empirical investigations are currently being carried out in the XTRA project to answer these questions.

Pointing is called 'punctual', if the movement of the arm reaches only one apex and thus indicates one single point in space. This gesture is only adequate if the demonstratum is relatively small and motionless. During non-punctual pointing actions, the apex itself performs a complex motion which corresponds in various ways to the object in question, e.g. follows its motion, gives its shape or indicates the part of space the object is supposed to be in.

Furthermore, pointing gestures differ in accuracy. Pointing with a pencil, pointer etc. can be more precise than pointing with a finger or the whole hand.

3. The interdependency of describing and pointing

In face-to-face interaction, objects are frequendy referred to by gestures and speech in parallel. Simulation of this multimodal process presupposes the investigation of the specific limitations of each component and the advantages of their combination. This is done in the following section.

There exist both functional and temporal, relations between gestures and phrases. Gestures can substitute, repeat, contradict, modify or amplify the vocal output (Scherer 1979). Pointing gestures usually amplify deictic expressions and therefore belong to the kind of gestures called 'illustrators' (Ekman, Friesen 1969). Norreally, pointing gestures and their correlated phrases are produced simultaneously (Levelt, Richardson, and La Heij 1985).

3.1 Obligatory and optional pointing gestures

Some deictic expressions must be accompanied by a pointing action (or a linguistic equivalent, Sennholz 1985). These include:

• demonstrative pronouns: 'this book',
• heterodeictic lncal adverbs: 'the tree there',
• personal pronouns with deictic function: 'he did it', and
• 'such'-constructions: 'I hke such flowers'.

Syntactically, obligatory pointing gestures are embedded in noun phrases or adverbial phrases. In the former case, they amplify a linguistic attribute. Within its corresponding phrase, the location of the pointing gesture is arbitrary. Usually, it will accompany the most emphasized expression.

A lot of expressions can be accompanied by pointing gestures, in principle all those which refer to visible objects, events etc. OptionM pointing gestures have various functions, e.g. to mark whether the speaker uses adverbs deictically or relative to another orientation system.

3.2 Pointing simplifies describing

The use of purely verbal descriptions can fail for various~.reasons. For example, some descriptions may not completely specify their referents: They can be wrong, inconsistent or too subjective. But even adequate descriptions can cause misinterpretations. One extreme would include descriptions with little intension and therefore too wide an extension, such 'whatsit' or 'thingamajig' (generally used if one doesn't know a more precise descriptor). The other extreme includes very detailed and complex descriptions which are difficult to process (e.g., 'the small red book on the left side in the second shelf from the top'). A closely related problem is that of technical terms used in conversation with non-specialists: Although the description may be totally adequate, the hearer is not able to understand it.

Therefore, verbal description alone may be too general or too specific. Within this range, the speaker has the task of specifying the referent in enough detail without constructing a verbal expression which is too complex.

One frequent solution is the use of pointing gestures. They allow successful reference without the need of totally specified verbal descriptions (Pechmann, Deutsch 1982). The use of pointing shortens the accompanying descriptor and the loss of intension is compensated by the gesture. General nouns amplified by pointing gestures can substitute for more specific nouns (e.g., 'I like cornflowers' is replacable by 'I like these / flowers'). Thus, additional pointing allows unambiguous (or at least relatively precise) referent specification even if one doesn't know an exact descriptor. The process of referent identification is speeded up, because the orientation to the object's direction and the processing of the verbal description are performed simultaneously.

3.3 Describing disambiguates pointing

One essential drawback of pointing gestures is their inevitable dependency on the here-and-now. Furthermore, pointing without describing the referent is fundamentaUy ambiguous (Wittgenstein 1958). Referent identification involves the following three steps: First, one has to recognize the direction indicated. This requires facing the speaker and following his/her gesture with gaze and eventually a body turn. Thus, the deictic spaces of both participants are co-oriented by physical means and not by mental acts (e.g., transformation of 'left' into 'right' and vice versa, see Klein 1978).

The second task is the identification of the object indicated. Usually, there is more than one object situated in any one direction. Problems arise if possible demonstrata are:

• next to each other,
• behind each other, or
• embedded in one another.

In these cases, unambiguous reference requires the naming or describing of the demonstratum.

Thirdly, one has to decide what aspect of the object is being referred to. Like the second step, this is usually done by consideration of the descriptor. For example, pointing at a moving car can refer to its colour ('Nice green 7 , isn't it?') or its kind of motion ('This speed .1" causes lots of accidents') etc. Pointing at sets of objects can even refer to aspects of higher degree such as number ('I'd like to have that man), ,I books').

4. Form deixis

Pointing at two-dimensional objects (forms, diagrams, maps, pictures etc.) differs in various aspects from pointing at objects within the entire visual field. This offers a definite advantage from a linguistic point of view: Some problems of local deixis are reduced in complexity without the communicative setting having to become unnatural (Schmauks 1986a). Furthermore, this domain is interesting from an artificial intelligence point of view, since some of the pointing actions with regard to forms can now be simulated on a terminal screen.

4.1 Reduction of problems

Following Bfihler's terminology (1982), form deixis belongs to the kind of deixis called 'demonstratio ad oculos', because all objects pointed at are visible. Furthermore, it represents an example of the 'canonical situation of utterance' (Lyons 1977): All the participants are co-present and can thus mutually perceive their (pointing) gestures etc. Form deixis is relatively precise, because tactile pointing is always possible. Precise pointing at small objects (e.g. single words) is frequently performed by using a pencil etc., larger areas by encircling them. The ambiguity with regard to objects behind each other does not occur, because the deictic space is only two-dimensional. If speaker and hearer are situated side by side, their deictic fields are co-oriented. Therefore, this position makes cooperation easier, and thus is the most advantageous one.

4.2 Remaining problems

Although form deixis implies a reduction of problems, referent identification has not at all become a trivial task. It cannot be taken for granted that demonstratum and referent are identical. This might be due to the fact that the speaker has mistakenly pointed at a wrong place because s/he doesn't know the referent's actual location or misses the target by accident. Other divergencies emerge intentionally: The speaker doesn't want to cover the referent and therefore points a bit lower.

Other essential problems arise because there exist subset relations among form regions. For example, the demonstratum can be a part of the referent - this is referred to as 'pars-pro-toto deixis'. In those cases, one must take into account the verbal description to resolve the ambiguity.

Furthermore, pointing at one form region can (depending on linguistic context) refer to three different entities:

1. The form region itself: 'What is to be entered here?
2. The actual entry: 'I want to increase this sum'
3. Correlated concepts: 'Are these expenses to be verified?'

5. Simulated pointing

This section investigates the extent to which some features of natural pointing can already be simulated in dialog systems developed to date. In section 6, some steps towards more accurate simulation are briefly suggested.

5.1 Different ways of simulating pointing gestures

Face-to-face interaction is performed by gestures and speech in parallel. In many domains (e.g. form deixis), objects are often and efficiently referred to by pointing gestures. Thus, dialog systems will become more natural if the user has the possibility of 'pointing' at the objects which are visible on the screen.

The goal 'reference by pointing' can be achieved by various strategies. One fundamental decision must be made first: whether one wants to simulate natural pointing (as is the aim of TACTILUS) or to offer functionM equivalents. In the former case, there is the presupposed but questionable demand that man-machinecommunication should be performed by the same means as interhuman communication.

If the main emphasis relies on simulation, then the pointing device and its use must correspond to natural pointing as accurately as possible. In this case, the most adequate simulation will be pointing at a touch-sensitive screen (see section 6). But other devices (e.g. input via mouse-clicks) can also partially simulate natural pointing (see sections 5.3).

Functional equivalents to natural pointing include the following devices: Framing the referent or zooming in on it, highlighting it in different colours etc. (see Ffthnrich et al. 1984). On the one hand, the system can 'point' by these means. On the other hand, the user gets immediate teedback as to whether the system has recognized the intended referent. This advantage is paid for by the loss of 'naturalness'.

5.2 Historical remarks

Multimodal input, especially the possibility of pointing at visible objects, offers certain crucial advantages. For example, the use of simple pointing actions was already possible in the following systems: SCHOLAR (Carbonell 1970) allows pointing gestures in order to specify regions of geographic maps. Pointing in Woods' (1979) system, combined with simple descriptions, refers to substructures of a parse tree displayed on the screen. In NLG (Brown et al. 1979), the user can draw simple geometric objects through descriptive NL-commands and simultaneous tactile touches on the screen. SDMS (Bolt 1980)enables the user to create and manipulate geometric objects on a screenarrangement called 'MEDIA ROOM'. In all those systems, there exist predefined relations between the pointing gesture and its demonstratum. Referent identification is not dependent on context etc.

Currently, several projects are investigating problems concerning the integration of pointing actions and NL input, e.g.: In NLMENU (Thompson 1986), the user can select parts of a street map by means of a mouse-controlled rubber-band technique. Hayes (1986) oudines the integration of a deictic component into the Language Craft System, which should allow the user to click on items on the screen, e.g. the machines on a blueprint of a factory floor. ACORD investigates pointing actions with respect to various twodimensional objects, e.g. a map of the planetary system (Hanne, Hoepelmann, and F~ihnrich 1986) and a form for university registration (Wetzel, Hanne, and Hoepelmann 1987).

5.3 Pointing actions in TACTILUS

One aim of XTRA is the integration of (typed) verbal descriptions and pointing gestures (currently realized by mouse-clicks) for referent identification (Kobsa et al. 1986). The user should be able to efficiently refer to objects on the screen, even when s/he uses underspecified descriptions and/or imprecise pointing gestures (Allgayer, Reddig 1986). Hence the process of specifying referents is speeded up and requires less knowledge of specialist terms.

The deictic component of XTRA (called TACTILUS) is completely implemented on a Symbolics Lisp Machine (Allgayer 1086). It offers four types of pointing gestures which differ in accuracy. They correspond to three modes of punctual pointing (with pencil, index finger, or hand) and to the possibility of encircling the demonstratum. Thus, pointing becomes a two-step process: First, one has to select the intended degree of preciseness and then to 'point'.

These pointing actions are natural because of their ambiguity: There is no predefined relation between the spot where the mouse is activated and the object which is thereby referred to. Therefore, the system has to take into account additional knowledge sources for referent identification, e.g. verbal descriptions and dialog memory. From the user's point of view, the essential indication of this naturalness is the lack of visual feedback. In analogy to natural pointing, the identified referent is not highlighted.

5.4 Problems in processing mixed input

One essential problem is to assign a mouse-click to its corresponding verbal constituent. This task is not trivial since there is no guarantee that the user 'points' within the range of the deictic expression. Possibly, the click occurs too late because of the user being inattentive, not familiar with the system etc. One example is:

Here, the pointing action occurs next to 'the last entry'. But this is an anaphor and doesn't need to be amplified. On the other hand, there is the deictic expression 'this sum' without its correlated obligatory pointing action.

Therefore, the system has to recognize that '/ ' belongs to 'this sum'. This problem is aggravated by the fact that the words 'here'/'there' and 'this'/'that' are not only the most frequent deictic expressions but have anaphoric and text- deictic readings as well.

Matching mouse-clicks and phrases becomes even more difficult if a single utterance requires more than one pointing action. This case is called 'multiple pointing'. Examples include:

Hayes (1986) assumes that pointing actions are performed in the same order as their corresponding phrases. But until this hypothesis is confirmed empirically, it can only serve as a heuristic rule.

As soon as reference by pointing is possible, the use of incomplete expressions will increase. In these cases, additional knowledge sources are needed for referent identification, like descriptor analysis and case frame analysis (Kobsa et al. 1986). For example, the expression 'this' in the sentence 'I want to add Sis/" ' surely refers to a number in the present domain, because 'add' is categorized as an action to be performed with numbers.

5.5 Problems in generating mixed output

If the pointing actions of the system are also conceived as a simulation of natural pointing, the user is confronted with the same problems that have already been identified in the last subsection (Reithinger 1987). But, whereas multiple pointing can be simulated during input, there seems to be no adequate mode for simulating it during output as well: In normal communication, the hearer doesn't need to watch the speaker in order to understand him/her unless the occurence ofa deictic expression (or the sound of touching during tactile pointing) demands his/her visual attentiveness. Also, during typed dialog, there is no need to observe the output sentences permanently. In the case of multiple pointing, the possibility cannot be ruled out that the user might fail to notice one of the pointing actions.

6. Prospects of more natural simulation

Up till now, only certain kinds of tactile pointing gestures can be simulated on a screen. Negroponte (1981) oudines some future plans, e.g. the consideration of non-tactile actions such as eye tracking and body movements.

Simulation of tactile pointing gestures by mouseclicks has some serious limitations with regard to its 'naturalness'. Empirical investigations are needed to determine the extent to which mouse-clicks can be regarded as an equivalent of natural pointing. These investigations are currently carried out in the XTRA project.

In the case of natural pointing, the choice of a more or less precise pointing gesture is made automatically rather than consciously. But in TACTILUS, the user has tc, select explicitly the intended degree of accuracy. Empirical investigations must examine whether the user regards this as a disadvantage.

Furthermore, pointing via mouse-clicks differs from natural tactile pointing, because there is no physical contact between finger and demonstratum. A better solution would be the use of a touch-sensitive screen on which 'real-world gestures' (see Minsky 1984) are possible. Touch-sensitive screens allow highly natural pointing gestures (see Picketing 1986), but have some shortcomings, e.g. a restricted degree of resolution.

A problem just as serious as the aforementioned is the temporal dissociation of a pointing gesture and its corresponding phrase. This problem would be soluble if the system would accept input via voice. But this alone wouldn't be sufficient: There is no guarantee that spoken phrases and correlated mouse-clicks occur simultaneously. Furthermore, current voice-input systems have too small a vocabulary and cannot process fluent speech.

Therefore, the most adequate simulation would be the combination of voice input~output and gestures on a touch-sensitive screen. However, the state of the art with respect to the required devices is not yet sufficient.

Acknowledgements

The research described in this paper has been funded by the German Science Foundation (DFG) in its Special Collaborative Program on AI and Knowledge- Based Systems (SFB 314). I am indepted to my colleagues of the XTRA project for their helpful comments on an earlier version of this paper.

References

1. Allgayer, J. (1986): Eine Graphikkomponente zur Integration yon Zeigehandlungen in natfirlichsprachliche KI-Systeme. Proceedings der 16. GIJahrestagung. Berlin etc.: Springer.
2. Allgayer, J. and C. Reddig (1986): Processing Descriptions containing Words and Gestures. A System Architecture. In: C.-R. RoUinger, Hrsg.: GWAI/OGAI 1986. Berlin etc.: Springer.
3. Bolt, R. A. (1980): 'Put-That-There': Voice and Gesture at the Graphics Interface. Computer Graphics 14,262-270.
4. Brown, D. C. et al. (1979): An Experimental Graphics System with Natural Language Input. Computer and Graphics 4, 13-22.
5. Bfihler, K. (1982): The Deictic Field of Language and Deictic Words. Abridged translation of K. Bfihler (1934): Sprachtheorie, part 2, chapters 7 and 8. In: R. J. JarveUa and W. Klein, eds.: Speech, Place, and Action. Chichester etc.: Wiley. Carbonell, J. R. (1970): Mixed-Initiative Man- Computer Dialogues. Cambridge, MA: Bolt, Beranek and Newman.
6. Clark, H. H., R. Schreuder and S. Buttrick (1983): Common Ground and the Understanding of Demonstrative Reference. Journal of Verbal Learning and Verbal Behavior 22, 245-258.
7. Ekman, P. and W. V. Friesen (1969): The Repertoire of Nonverbal Behavior: Categories, Origins, and Coding. Semiotica 1, 49-98. F~ihnrich, K. P. et al. (1984): The Role of Graphics. Technical Report E3/GR, FhG, IAO, Stuttgart.
8. Hanne, K. H., J. P. Hoepelmann und K. P. Ffihnrirh (1986): Combined Graphics/Natural Language Interfaces to Knowledge Based Systems. Proceedings of the Artificial Intelligence and Advanced Computer Technology Conference, Wiesbaden, West Germany.
9. Hayes, P. J. (1986): Steps towards Integrating Natural Language and Graphical Interaction for Knowledge-based Systems. Proceedings of the 7th European Corlf,-rence on Artificial Intelligence, Brighton, England.
10. Kirk, L. and M. Burton (1981): Physical versus Semantic Classification of Nonverbal Forms: A Cross-Cultural Experiment. In: A. Kendon, ed.: Nonverbal Communication, Interaction, and Gesture.
11. Klein, W. (1978): Wo ist hier? Pr~iliminarien zu einer Untersuchung der lokalen Deixis. Linguistische Berichte 58, 18-40.
12. Kobsa, A. et al. (1986): Combining Deictic Gestures and Natural Language for Referent Identification. Proceedings of the llth International Conference on Computational Linguistics, Bonn, West Germany.
13. Levelt, W. J. M., G. Richardson and W. La Heij (1985): Pointing and Voicing in Deietic Expressions. Journal of Memory and Language 24, 133- 164.
14. Lyons, J. (1977): Semantics, Vols 1 and 2. Cambridge: Cambridge University Press. Minsky, M. (1984): Manipulating Simulated Objects with Real-world Gestures using a Force and Position Sensitive Screen. Computer Graphics 18, 195-203.
15. Negroponte, N. (1981): Media Room. Proceedings of the Society for Information Display 22, 109- 113.
16. Pechmann, T. and W. Deutsch (1982): The Development of Verbal and Nonverbal Devices for Reference. Journal of Experimental Child Psychology 34, 330-341.
17. Pickering, J. A. (1986): Touch-sensitive screens: the technologies and their application. Int. J. Man-Machine Studies 25,249-269.
18. Reithinger, N. (1987): Generating Referring Expressions and Pointing Gestures. To appear in: G. Kempen, ed.: Natural Language Generation. Dordrecht: Kluwer
19. Scherer, K. R. (1979): Die Funktionen des Nonverbalen Verhahens im Gespr/ich. In: K. R.
20. Scherer und H. G. Wallbott, Hrsg.: Nonverbale Kommunikation. Weinheim/Basel: Beltz.
21. Schmauks, D. (1986a): Formulardeixis und ihre Simulation auf dem Bildschirm. Ein Oberblick aus linguistischer Sicht. Memo Nr. 4, Sonderforschungsbereich 314, Dept. of Computer Science, University of Saarbralicken, FR Germany.
22. Schmauks, D. (1986b): Form und Funktion von Zeigegesten. Ein interdisziplin/irer Uberblick. Bericht Nr. 10, Sonderforschungsbereich 314, Dept. of Computer Science, University of Saarbriicken, FR Germany.
23. Sennholz, K. (1985): Grundzfige der Deixis. Bochum: Brockmeyer.
24. Thompson, C. (1986): Building Menu-Based Natural Language Interfaces. Texas Engineering Journal 3, 140-150.
25. Wetzel, R. P., K. H. Hanne and J. P. Hoepelmann (1987): DIS-QUE: Deictic Interaction System-Query Environment. LOKI Report KRGR 5.3/KR-NL 5, FhG, IAO, Stuttgart.
26. Wittgenstein, L. (1958): Philosophical investigations. Oxford: Blackwell.
27. Woods, W. A. et al. (1979): Research in Natural Language Understanding: Annual Report. TR 4274, Bolt, Beranek and Newman, Cambridge, MA.