Lab 2 (due 1/16)

Overview

The goal of this lab is to make a start on the test suite that will be your development target, on the one hand, and to customize a first version of your grammar start on the other. I've selected phenomena to cover in this lab with an eye to starting with those that are essential to creating a working grammar. You'll probably want to work on these two subtasks in parallel, though they are described separately in the instructions below.

Back to top

Test Suite

The first task is to create positive and negative example sentences illustrating the following phenomena, to the extent that they are relevant for your language:

• word order
• the rest of the NP
• case
• agreement

Before you start, read the general instructions for testsuites and the formatting instructions.

Back to top

Starter grammar

The second task is to create a starter grammar by filling out the required sections of the Grammar Matrix customization questionnaire. In particular, you'll need to address these sections:

• General information
• Word order
• Number
• Person
• Case
• Direct-inverse (if appropriate)
• Lexicon

(Updated 1/13/09) In the word order section, you can skip the auxiliaries by saying "no" on that question for now. When we get to auxiliaries, you may of course revise this answer.

In the lexicon section, you should define lexical types for transitive and intransitive verbs and nouns. If appropriate, you should define determiners and case-marking adpositions.

If you have case and/or agreement, you'll need to define morpheme slots and morphemes for verbs and nouns as appropriate. In many languages, the agreement morphemes on verbs also mark, say tense. We'll ignore this for now, but return to it soon. If you want to define other affixes without giving them morphosyntactic content, you can.

Back to top

[incr tsdb()] profile

The final step for this lab is to use the [incr tsdb()] grammar profiling system to test the performance of your starter grammar over your test suite, and then examine the results. (You may find in doing so that you want to refine certain aspects of your starter grammar. You can do this by uploading the file "choices" which comes with your grammar into the customization system and then tweaking from there.)

Create a test suite profile

1. Create a directory called tsdb inside your grammar directory.
2. Inside tsdb, create two subdirectories: home (for test suite instances) and skeletons (for skeletons).
3. Save a copy of Index.lisp in tsdb/skeletons
4. Save a copy of Relations in tsdb/skeletons. (If your browser doesn't like files without extensions, here's another copy of the same file with .txt appended. You should save it as just Relations.)
5. Make a subdirectory called lab2 inside tsdb/skeletons for your test suite. (If you choose a different name for this subdirectory, you must edit Index.lisp accordingly.)
6. Download the perl script make_item.pl and run it on your test suite:

    perl make_item.pl testsuite.txt

7. (If the perl script doesn't like the formatting of your test suite, edit the test suite appropriately and/or complain about the perl script on GoPost.)
8. Copy the .item file which is output by make_item.pl to tsdb/skeletons/lab2/item.
9. Copy tsdb/skeletons/Relations to tsdb/skeletons/lab2/relations (notice the change from R to r).

Create and run an initial test suite instance

1. Start the lkb
2. Load your starter grammar. (The script file is in matrix/lkb/script.)
3. Start [incr tsdb()] (within emacs, that's M-x itsdb)
4. In the [incr tsdb()] podium, select Options > Database Root and input the path to tsdb/home.
5. In the [incr tsdb()] podium, select Options > Skeleton Root and input the path to tsdb/skeletons.
6. Optional: For future use, you can set these variables ahead of time in a file called .tsdbrc in your home directory. It should contain these lines, with path names edited appropriately:

   (in-package :tsdb)
   (setf *tsdb-home* "path-to-tsdb/home")
   (setf *tsdb-skeleton-directory* "path-to-tsdb/skeletons")
   

7. In the [incr tsdb()] podium, select File > Create. You should see your test suite in the menu there. Select it, and get a test suite instance. Post to GoPost if this doesn't work.
8. Make sure your grammr is loaded into the LKB.
9. Once you have a test suite instance, select it (by clicking on it), then do Process > All Items.
10. Explore the results, with functions such as Browse > Results and Analyze > Competence.
11. Be sure to save (i.e., not overwrite or delete) this test suite instance, as you'll be asked to turn it in.

Back to top

Write up

Your write up should include the following:

1. Documentation the choices you made in the customization system, illustrated with examples from your test suite.
2. Descriptions of any properties of your language illustrated in your test suite but not covered by your starter grammar and/or the customization system.
3. Documentation the coverage of your grammar over the testsuite. If there are examples that thare parsed incorrectly (unanalyzed grammatical examples, analyzed ungrammatical examples, or grammatical examples assigned surprising parses), reflect on why that might be.
4. Finally, if there are any places where the customization system seems unable to cope with the properties of your language (within the phenomena addressed in this lab), describe them here.

Back to top

Submit your assignment

• Be sure your write up and the text-file version of your test suite are included in your grammar directory.
• Create a tarball:

        tar czf yourname.tgz yourgrammar
      

• Upload the tarball to CollectIt

Back to top

Back to course page