6.1 Semantic Space

Every linguistic textbook in existence has a discussion of word (and fixed-phrase) meaning, and usually no time is lost rejecting the notion that meanings (or senses) can be usefully thought of as mental images. The reasons for this rejection are those touched on in the introduction concerning the inadequacy of images to represent abstract, hypothetical, or habitual events and relations, as if one were to try to understand Hamlet by watching a video with the sound turned off, or to render Aspects of the Theory of Syntax in pantomime. In this great age of visualization, however, there has been a return to the representation of word meanings--not by using pictograms or some other visual metalanguage to represent meanings, to be sure, but to see meanings as relations between words and to represent these relations. The theoretical, textual basis for one such attempt has been developed as the WordNet project at Princeton University under the direction of George Miller and a stunning and now famous visual interface for this project has been developed by Mark Tinkler of Plumbdesign. To grasp what Plumbdesign's Visual Thesaurus is representing, we must take a brief tour through basic word semantics.

WordNet is a large lexical data bank in which most of the standard semantic relations of word senses (synonymy, antonymy, hyper- and hyponomy, and several others) are coded for over 140,000 words. It is easy to see these relations as connecting the words via their senses into some sort of graph. The hyper/hyponym (i.e. general terms and specific kinds) relations (see Figure 6.9), which are worked out only for nouns and verbs, look like natural candidates for display as a tree (an acyclic, though not directed, graph). Miller and his associates do not trace all nouns from one most general noun root but rather from 25 different "beginner" (or top) words (which can be consolidated somewhat into 11 separate trees). Similarly, they have a number of verb lineages in 15 different areas (e.g. communication, possession, motion, perception, contact, competition, etc.) They do allow certain word senses to have more than one parent in these trees, so that they become "tangled trees" (i.e., cyclic).

WordNet can be accessed on line, and can be used as a thesaurus to find synonyms of words. Used in this way, it is similar to on-line versions of Roget's II: The new thesaurus. Synonymy is only one of the links, however, weaving word together in WordNet.

some synonym relations for hard

Figure 6.8
A diagram of words linked by synsets

Matters are not quite so simple as this overview suggests, however, because semantic relations are relations of word senses, not words. For example, hard and difficult are not synonyms on all senses: hard has senses which it shares with solid and heavy ("hard drinker"). Each of these latter words, of course, have senses they do not share with hard, but rather with some other words. We can represent this networking as at the left, where the red color represents the words and the shared senses are in boxes, which WordNet calls "synsets" and encloses with curly brackets instead of boxes. Thus {hard, difficult} is one synset which you would find under each word. Note that the graph at left omits senses for several words to reduce visual clutter (parenthetical numbers indicate the total number of senses) and that it only indicates synonym relations. One of the key ideas for WordNet was to use these synsets very much as dictionaries do to represent the senses. This has come to be called relational semantics, instead of a semantics that represents senses in a made-up metalanguage ("components," "abstract predicates", "mentalese"). These synsets are equivalent to pairings like hard1=difficult1, hard3=solid9, and hard7=heavy15, though this notation is scarcely perspicuous, unless you have two windows of WordNet open. Intuitively, one might think of synsets as marking an area of overlap between two words (or more than two), or, as George Miller says, as lexicalized concepts.

WordNet does not rely solely on synsets to represent senses, however. It also includes glosses (sense paraphrases) after the fashion of dictionaries, particularly when a particular sense has no synonyms, and even gives examples of uses. As WordNet has been elaborated, more "glosses" have been added, until in WordNet 2.0 all senses of the words have glosses.

WordNet display for imagine and related verbs

Figure 6.9
Words linked to their hypernyms

WordNet is a very large and complex space, with over 350,000 links of words to synsets, synsets to other synsets, and these links are of different kinds. You might want to compare the larger diagram, a portion of which is at the left, for verbs with hypernym links as well as synonym links.

If we imagine traversing this network, we see how WordNet could function as a web of word associations, all of which are "in" the lexicon. The words themselves are the points of slippage or transfer to other chains of related meanings. Scholars who use WordNet for various projects tend to ignore the word side of WordNet in favor of the synsets and their relations (since strictly speaking only synsets have semantic relations).


Figure 6.10
Classic Visual Thesaurus 3D display

Visual Thesaurus

WordNet itself is simply a database, although a large and complex one, with a plain vanilla, text-oriented interface for entering terms and senses of terms, selecting options, and receiving answers. A number of more graphic interfaces have been written for it, usually specializing in only one of the semantic relations. The most famous graphic interface for it is Plumb Design's Visual Thesaurus, which has undergone a major redesign since its first appearance in 1998. In its original form, it displayed semantically related from the synsets of a search word submitted by the viewer. These were connected to the search word by lines, creating stars or sometimes polygons. The resulting objects could be viewed in either two-dimensional or three-dimensional maps (with the latter, you can rotate the 3D object around the x and y axes Virtual Reality/CAD style). Indeed, being children of the basic Java graph applet (Figure 6.1), these displays move of themselves, either jiggling in the 2D fashion of Figure 6.1 or rotating about the x and y axes, imparting to the 3D objects the sense of words coming into consciousness and then fading away as they are replaced by others. All of the applicable semantic relations would produce solid connecting lines between words, so that it was not possible to see which words were connected to others by virtue of sharing a synset from those that were hypernym/hyponym pairs, "similar to" pairs (for adjectives), or "Also see" connections. So in Figure 6.10, hard connected to difficult as members of a synset, but they are both connected to demanding by an Also-see connection and to awkward by a "similar to" connection. Thus the carefully teased out semantic relations of synsets get lumped together into what is visually a general sense of "relatedness."

This is not optimal as a display of information, but it proved quite delightful to many critics as an evocation of groping for the right word. It has been exhibited as a piece of net.art ("Beyond Interface" exhibit, now on line at the Walker Art Gallery Gallery 9 and reviewed there by Susan Hazan). It has been enthusiasically celebrated in Le Monde Interactif by Vincent Fagot and by Robbin Murphy in Intelligent Agent ( A Burglar in the Treasure House). All of these reviewers comment on the mixture of database technology with images of thinking and the tracing of subtle filaments of meaning (Hazan's title is "Are the Engineers Holding Hands with the Artists?"), of the practical, even pedestrian use of synonym lists on the one hand and immersion in a space of animated words on the other. "Visualization of information" here shows very strong affinities with art. It is poised between an on-line reference tool and interactive net.art, a positioning that M. Fagot describes quite nicely:

S'il est incontestablement pratique, avec son navigateur qui permet de choisir la nature des termes recherchés (verbes, noms, adjectifs, ou adverbes), le site Visual Thesaurus appelle moins à une recherche ponctuelle qu'à une exploration prolongée. Une exploration d'autant plus enthousiasmante qu'elle se pare d'aspects ludiques et d'une esthétique aussi moderne que raffinée.

Plumbdesign Visual Thesaurus with difficult at center-www.visualthesaurus.com

Figure 6.11
Map of word SAGE

The current version improves on the display of information in two ways. First, the shared sense that items of a synset have is represented not as a connection of those words but by balls colored according to different parts of speech. Whe moused over, a ball displays the sense-gloss associated with the synset. Semantic relations are then drawn between balls. So Figure 6.10 shows the word sage with two adjective senses and three noun senses (the smaller darker, in fact red, ones). One of these noun senses is linked via synset to salvia (that is, the set is {sage, salvia}; hence salvia and sage are linked by solid lines to the ball). Another noun sense has only the sense-gloss, "a mentor in spiritual and philosophical topics who is renowned for profound wisdom," and the third noun sense has as a sense-gloss a desciption of the dried herb for seasoning. The latter two noun senses do not lead to other similar senses of words, but the two adjective senses do: the sense that spells out the {sage, sage-green} synset links has a similar-to (dashed) link to one sense of chromatic and the other ("having wisdom that comes from age and experience") has a similar-to link to one sense of wise. The solid lines connect words to sense-glosses; the dashed lines connect sense-glosses and are used for semantic relations like "is a type of," "similar to," "see also."

Second, each of these dashed connecting edges identify their connection type when moused-over. When all of the connections are on, the resulting map is usually quite large and benefits from the three-dimensional display which allows the viewer to rotate and select portions of the resulting large object. In the new interface, the screens for common, polysemous words display a great deal of information. Here for example is a screen capture of a the 3D display of the word hard with all relations on:

Figure 6.12
Visual Thesaurus: Screen capture of 3D display of HARD

Figure 6.12 displays a stunning amount of information very precisely, and in fact contains more, as each ball, when moused over, pops up a balloon with with its sense-gloss. Two points of interest are the balls representing major senses of hard with many dash-connected senses that have no other member of that synset. These are glossed "metaphorically hard: 'a hard fate,' 'took a hard look,' 'a hard bargainer,' and 'a hard climb'" and "not yielding to pressure or easily penetrated: 'hard as rock'". This shows the limitation of using only synsets of two or more to identify senses, which is indeed exactly why sense-glosses were introduced into WordNet in the first place. If indeed it is the case that we access all of the senses of words for a very brief interval when processing natural language, what wonderful things must be swimming about in our minds!

Again, my terms of admiration run to the contemplative, to the sheer intellectual grasp of what is involved in a lexical field, but PlumbDesign presents this new interface as an educational tool and package it to run off line. So it was nominated for a Webby Award in 2003 in the category of Education, not Art.

Apartment.

Such animation of and interaction with words is central to quite a bit of net.art. Another project that declared itself art from the outset is Marek Walczak and Martin Wattenberg's The Apartment, exhibited at www.turbulence.org and a nominee for the 2001 Webby Award in the category Arts. It was one of the opening displays at the Whitney Gallery's artport site and it was on exhibit at Ars Electronica for one year. Apartment comes with negligible documentation or explanation, suggesting that it is to be learned by exploration. Those who desire a pristine encounter with Apartment should go there now, before reading the following paragraphs culled from my explorations.

Sonnet 95 1st quatrain

Figure 6.13
Apartment display: Sonnet 95. 1sts quatrain

Apartment takes words and sentences of input and plots them into floor plans that are built ad hoc to locate the words. The rooms (and window) of the floor plans are drawn from a list of twelve types (living room, dining room, kitchen, closet, hall, foyer, bedroom, bathroom, library,office, and window). Each corresponds to a semantic category (group, glamour, food, secrecy, motion, change, intimacy, body, truth, work, and vision, respectively). Given a string of words as an input, it displays room outlines for the categories triggered by the words, and the words that it recognizes float up and hover over (in) their rooms. Some words will float between categories, in which case the wall between the two categories is erased. The words are in constant jittering, sliding, or swirling motion and each time words are added, the floor plan reconfigures itself so that the categories with words added become larger and their walls thicker, and new categories/rooms are added to locate new words. Thus words with the same category cluster together in a room, and the effect is somewhat like a do it yourself thesaurus based on 12 categories instead of Mr. Roget's 1000 plus, or the mapping of the written text as a lexical field, as if one were listening for key words or word clusters of certain types and plotting their relative preponderance. It is in any case a dynamic thesaurus, or field; in addition to the jittering and so on, words do float together as if testing plausible phrasal combinations.

Wattenberg, Wolczyk, and Feinberg say that several writers about words, space, and formal disorderings influenced their thinking about the site, such as Bachelard's The Poetics of Space and the work of the Oulipo group (English: Workshop of Potential Literature), but cite as most directly relevant Frances Yates' work the classical ars memoriae -- The Art of Memory (1966) wherein speeches were memorized by mentally locating their parts in an imagined house.

frame from 3D projection Sonnet 95

Figure 6.14
Apartment: 3D Interface

A second, 3D, multimedia display is also available for your apartment which is without parallel and very hard to describe. The floor plan with its words is mapped into a 3D space where the walls become bulging segments and are covered with slices of images which were previously saved from internet searches with the words as search terms. A script takes you on a tour of this phantasmagoric fun house with an audio accompaniment of odd machine and other noises and some of your words intoned repeatedly in a deep voice with British accent. It is a word your words have created, as interpreted through an alien or disturbed mind. At left is a screen capture from the 3D view as it passes from "closet" into "dining room"— while the voice is intoning "grace."

If you like your apartment, or simply want to publish your doodle, you can save it, and it will be added to a "city" of apartments made by others, at least in theory. I have not yet succeeded in making a save from the internet, and it may be that to do so you may have to be at one of the places where it is on exhibit. When a city gets quite large, a new city is started, but you can still add to the old city or even to one of its apartments. The newest city is number 13. Cities also can be viewed according to main category: that is, the apartments in the map can be redistributed according to their most central category or categories. For example, if your apartment has no bathroom (i.e., no body/bodily function words, medicines, diseases, etc.) it will be plotted at the periphery of a city view centered on the Body, but the same apartment would appear nearer the center of the city viewed from the perspective of some category your apartment does contain.

Apartment display of Dylan Thomas

Figure 6.15
Apartment display of Dylan Thomas,
"The Force That Through The Green Fuse"

Since the archive contains a special city of "favorite apartments" and hundreds of apartments in 13 cities, it is easy to see the uses that various unidentified viewer-writers have made of this apparatus. The favorites are generally made of many repetitions of a few words ("ego, ego, sex" or "run, fast, run") which demonstrate special powers of the word animation engine. Sampling the newest apartment, one can find similar exercises, along with little lyric poem-like things, simple probe strings, a slew of adjectives ("sad, content, moody"), haiku-style brevities ("tokyo, morning, girl"), and in the dimmed limbo of unrecognized words, throughout, evidences of the users' execrable online typing. The brevity recommends itself to avoid clutter: the image at the left is a one -third reduced screen capture of the apartment built around Dylan Thomas' "The Force that Through the Green Fuse," a poem of 22 lines. It is more readable when in motion, of course. Though it is not easy to make out, the words activate 11 of the 12 possible rooms.

When making and studying such visual representations of the words of a text, I find myself looking at it in a slow, receptive way as if I were looking at a random dot stereo-gram waiting for a significant pattern to emerge. But I am considerably less confident here that such a pattern will emerge. It was not put there, after all, by a maker the way an image is in a stereo-gram, and I am not sure what I am looking for. I am looking at a poem (in this case) that has been partially disassembled and then recast in a simply categorized, pulsing lexical field which suggests a pool of significations before the ordering of syntax has been imposed, or that lies beneath the sentences of the poem. And such a seeing, literary critics hold, is a way into understanding how the language of a poem works.