Lab 4 (due 2/1)


These instructions might get edited a bit over the next couple of days. I'll try to flag changes.

As usual, check the write up instructions first.

In this and future labs, I'll use klingon.tdl to refer to the tdl file named after your language.

Download a customized grammar start

Be sure to try this by class on Wednesday. Bring questions about how to best answer the questionnaire for your language to class. GoPost/Email if you hit bugs in the cgi script itself or don't understand the error messages it gives.

Import your test suite into [incr tsdb()]

These instructions will show you how to make a test suite 'skeleton' which [incr tsdb()] can use to create a new instance of your test suite whenever you need one (handy for comparison across time).

  1. Create a directory called tsdb inside your grammar directory.
  2. Inside tsdb, create two subdirectories: home (for test suite instances) and skeletons (for skeletons).
  3. Save a copy of Index.lisp in tsdb/skeletons
  4. Save a copy of Relations in tsdb/skeletons. (If your browser doesn't like files without extensions, here's another copy of the same file with .txt appended. You should save it as just Relations.)
  5. Make a subdirectory called lab4 inside tsdb/skeletons for your testsuite. (If you choose a different name for this subdirectory, you must edit Index.lisp accordingly.)
  6. Download the perl script and run it on your test suite from lab3:

     perl lab3_suite.txt
  7. (If the perl script doesn't like the formatting of your test suite, edit the test suite appropriately and/or complain about the perl script on GoPost.)
  8. Copy the .item file which is output by to tsdb/skeletons/lab4/item.
  9. Copy tsdb/skeletons/Relations to tsdb/skeletons/lab4/relations (notice the change from R to r).

Create and run an initial test suite instance

  1. Start the lkb
  2. Load your starter grammar
  3. Start [incr tsdb()] (within emacs, that's M-x itsdb)
  4. In the [incr tsdb()] podium, select Options > Database Root and input the path to tsdb/home.
  5. In the [incr tsdb()] podium, select Options > Skeleton Root and input the path to tsdb/skeletons.
  6. Optional: For future use, you can set these variables ahead of time in a file called .tsdbrc in your home directory. It should contain these lines, with path names edited appropriately:
    (in-package :tsdb)
    (setf *tsdb-home* "path-to-tsdb/home")
    (setf *tsdb-skeleton-directory* "path-to-tsdb/skeletons")
  7. In the [incr tsdb()] podium, select File > Create. You should see your test suite in the menu there. Select it, and get a test suite instance. Post to GoPost if this doesn't work.
  8. Make sure your grammr is loaded into the LKB.
  9. Once you have a test suite instance, select it (by clicking on it), then do Process > All Items.
  10. Observe that with the minimal vocabulary in your grammar, very few of your sentences will parse.
  11. Be sure to save (i.e., not overwrite or delete) this test suite instance, as you'll be asked to turn it in.

Expand lexical coverage (perhaps just a little)

When you created your customized grammar start, you created lexical types for transitive and intransitive verbs as well as at least one type of noun. Depending on the options you selected, you may have also created lexical types for determiners, case marking adpositions, a negative adverb and/or a question particle.

These lexical types are somewhat underspecified (e.g., no constraints pertaining to agreement). We're not going to fix that this week.

  1. Use emacs to explore the file lexicon.tdl and the lexical type definitions in klingon.tdl.
  2. Use the LKB to explore the expanded types and lexical entries.
  3. Find all the words in your test suite, and determine which ones correspond to the lexical types you already have. (NB: In most cases, a transitive verb appearing without its object is still a transitive verb, as it should introduce a two-place relation.)
  4. Use [incr tsdb()] to analyze the vocabulary in your test suite: Process > Vocabulary. The results which appear in your emacs window will give you a sense of which word forms have the highest token frequency in your list.
  5. For isolating languages: Add entries to lexicon.tdl for all of the words that match the types you have so far.
  6. For highly inflecting languages: For each word that matches a type you have so far, pick the most frequent form and add it to lexicon.tdl it that form. (NB: Even though a verb might be in, say, the 3sg present tense form the constraints on the lexical entry won't reflect this.)

    NB: The predicate names for your words should all be English glosses of the words (for MT purposes, and because it helps in dealing with a language you don't know very well). Furthemore, they should take the form

    is the English gloss and
    is a coarse-grained pos tag (e.g.,
    ). This is the standard DELPHIN/MRS format for predicate names.

Create a new test suite instance and parse again

  1. Reload your grammar in the LKB.
  2. In [incr tsdb()], choose File > Create and create a new test suite instance.
  3. Highlight the instance and choose Process > All Items
  4. Have [incr tsdb()] calculate the coverage (Analyze > Coverage) and overgeneration (Analyze > Overgeneration) your current grammar has and document it for your write up (also check out Browse > Errors).
  5. Play with the [incr tsdb()] menus and see what you can do.

Write up

Submit your assignment

ebender at u dot washington dot edu
Last modified: Thu Jan 24 2008