Lab7 (due 5/17 11:59 pm)

Overview

• Collect translations of our MMT sentences into your language and add them to your testsuite
• Choose two phenomena illustrated in those examples but not handled by your grammar to fix. Post to Canvas by class on Tuesday for pointers on how to fix them. (I'll likely direct you to lab instructions from previous years.)
• Add a few further examples to your testsuite for those phenomena (as appropriate).
• Run initial testsuite & testcorpus profile.
• Fix the two phenomena you chose.
• Make sure your grammar can generate. Edit as necessary until it can. Post many questions to Canvas!
• Run final testsuite & testcorpus profile.
• Write it up!

Back to top

Collect MMT sentences

Download the English MMT sentences. There are 29 sentences in this file. Try to develop translations of them into your language. However, if you don't have enough info for some of the phenomena, it is okay to skip those examples. If you have info about the grammatical structures but can't find appropriate lexical items, it is okay to substitute similar ones, e.g. lion for cat, or canoe for car.

If your language obligatorily marks MOOD, pick indicative unless some other mood is actually required in a given context (e.g. interrogative mood in questions).

If your language obligatorily marks ASPECT, pick imperfective if possible. (If your language doesn't require aspect, leave it off.)

Create testsuite format IGT for each of the examples you translate and add these to your testsuite. In addition, crease a plain text file (called iso.txt, with iso replaced by your iso language code) with your 29 MMT sentences (in your language only) in the format your grammar expects (morpheme segmented or not), one per line, with a blank line between each. NB: I'm not looking for IGT here, just the actual strings your grammar expects. For any that you are skipping, put "EXAMPLE SKIPPED" on the corresponding line.

Choose two phenomena to develop

By class on Tuesday: Choose two phenomena represented by the MMT sentences not already covered by your grammar to work on this week. Post to Canvas with the IGT for the relevant examples and the phenomena you intend to work on. I will reply with pointers to instructionsn for those phenomena/develop some if necessary.

Extend your testsuite for those phenomena

Create additional positive/negative testsuite examples that illustrate your chosen phenomena and add them to the testsuite in the usual fashion. This testsuite should now include the MMT sentences plus ~5-10 more examples and should be called lab7.

Initial testsuite run

1. Create and run initial testsuite instances for both the linguist-provided data and your testsuite, using the initial grammar.

   Note If your tsdb/ directory is inside a shared folder on VirtualBox, it will not work.

2. For each of these, explore the results, collect the following information to provide in your write up:
   • How many items parsed?
   • What is the average number of parses per parsed item?
   • How many parses did the most ambiguous item receive?

Back to top

Develop analyses and test your grammar

Based on the instructions I've pointed you to through Canvas + answers to the many questions I hope you will ask, develop analyses for your two additional phenomena.

For the MMT sentences specifically, you can test your MRSs by looking at the output of this English grammar for the corresponding examples. We don't expect an exact match, but if things are different you should have a clear idea of why. And, of course, you are always welcome to post lots of questions!

Test generation

Test generation with both lkb & ace. Can you generate from short sentences? What about longer ones? To receive full credit on this lab, you need a grammar that can generate from simple transitive sentences and you need to have tested what happens with longer ones (e.g. sentences with clausal modifiers or clausal complements). See Lab 6 for detailed instructions on generation. And, of course, post lots of questions to Canvas.

Run both the test corpus and the testsuite

Following the same procedure as usual, do test runs over both the testsuite and the test corpus.

Again, collect the following information to provide in your write up:

1. How many items parsed?
2. What is the average number of parses per parsed item?
3. How many parses did the most ambiguous item receive?
4. What sources of ambiguity can you identify?
5. For 4 newly parsing or otherwise fixed items (2 in the testsuite, 2 in the corpus), do any of the parses look reasonable in the semantics?

Back to top

Write up

NB: While the test suite and grammar development is joint work, the write up should be done by one partner (the other will get a turn next week). The writing partner should have the non-writing partner review the write up and make suggestions.

Your write up should be a plain text file (not .doc, .rtf or .pdf) which includes the following:

1. A statement of what you were able to find for translations of the MMT sentences, and why.
2. Documentation of your analyses of the additional phenomena:
   1. A descriptive statement of the facts of your language.
   2. Illustrative IGT examples from your testsuite.
   3. A statement of how you implemented the phenomenon (in terms of types you added/modified and particular tdl constraints). (Yes, I want to see actual tdl snippets.)
   4. If the analysis is not (fully) working, a description of the problems you are encountering.
3. Documentation of what happened when you tried generating with the LKB. Did it work right away? If it didn't, but you were able to get it working, what did you have to do?
4. Documentation of your coverage over testsuite & test corpus for both the initial & final runs, including the answers to the questions given above.

Back to top

Submit your assignment

• Be sure your write up and the text-file version of your test suite are included in your grammar directory.
• Likewise, make sure to include your most current tsdb profile in the grammar directory (ideally inside tsdb/home/).
• If you're using svn, export the grammar so I don't get all your .svn files:

  svn export yourgrammar iso-lab7
  
  For git, please do the equivalent.

• Create a tarball:

        tar czf iso-lab7.tgz iso-lab7
      

• Upload the tarball to Canvas under the name of the partner who did the write up.

Back to top

Back to course page