CSE 109    Program 5    Due   10:45 PM Thursday 29 March 2012
          Late submissions:  9 PM 31 March, 10:45 PM 1 April

     In this programming assignment I ask you to develop a class Lex,
     which will act as a tokenizer for the StomAsm Language.  Your file
     p5.cc should contain exactly the code which can be found in the file
     /proj/csc109/p5/testLex.cc. When I test p5 I will discard your
     copy of p5.cc and use the code in /proj/csc109/p5/testLex.cc. THUS, YOU
     MUST NOT CHANGE THE CODE IN testLex.cc (your p5.cc).

     Part of the program involves using the class Word, which you developed
     for assignment p1. The expected output of the program is in a comment
     at the bottom of the program stored in /proj/csc109/p5/testLex.cc.

     First I describe how your class Lex should behave. It has the following
     methods(I don't indicate the paremeters or their types).  Note, the
     main job of Lex is take in a string and then return the various "tokens"
     in the string. You may very well develop other methods.

         set()         store a string of characters to tokenize
         str()         return the token as a Word (the string read in to
                         tokenize)
         count()       return the index of the current location in the string
         next()        Read the next token in the string and return its type,
                       i.e., one of the constants (see below). Store the
                       actual string read in a Word.  If at the end
                       of a string, REFUSE TO GO ANY FURTHER and return EOLN.
                       (If at the end of a string, repeated calls to next()
                       should return EOLN.)  Calls to next() should ignore
                       comments, which are denoted by the ";", and return
                       EOLN.


     Note.  Your class should have public (static) constants, corresponding
        to various tokens (value of constant in parentheses, where, e.g.,
        TIMES stands for '*', LPAR stands for ')', HALT stands for 'halt',
        GT stands for '>', etc.):
          IF(0), HALT(1), READ(2), WRITE(3), END(4), EOLN(5), LT(6), LTE(7),
          GT(8), GTE(9), EQU(10), NEQ(11), JUMP(12), JUMPZ(13), JUMPL(14),
          SPLUS(15), SMINUS(16), STIMES(17), SDIVIDE(18), LPAREN(19),
          RPAREN(20), ADD(21), SUB(22), MUL(23), DIV(24), COLON(25),
          SEMICOLON(26), PUSH(27), POP(28), COPYTOP(29), SWAPTOP(30),
          IDENT(31), NUMBER(32), JUNK(33)

     To submit your assignment, create a subdirectory of /cse109.121 called
     p5 (i.e., from your root directory /cse109.121/p5) for doing your work.
     Finally, indicate you wish the assignment to be collected by executing
     the unix command "touch DONE", which will create a file of 0 bytes
     named DONE.  Then all the files in the subdirectory will be collected.
     I will test your program using your Makefile and my copy of p5.

     Hints:
        (1)  It will be very helpful to have a "lookup table" that stores
          each of the tokens, e.g., "read", "+", etc., along with their
          corresponding values.  You should use your class Tree< , > from
          programming assignment 4. But what if the classes you developed
          for programming assignment 1 and 4, Word and Tree< , >, don't
          function well? In that case, I provide the files tree.t,
          node.t, word.h, and word.o (the files from my p1 and p4).
          I have stored these files in /proj/csc109/p5. If you use the .h
          and .o files, it is VERY IMPORTANT that you use them as I direct.
          It is not a good idea to aggravate me just as I am about to grade
          your assignment.  Your makefile should have the target "clean:'
          It should have two lines (in the given order):
                     rm -f *~ *.o p5
                     cp /proj/csc109/p5/word.o
                     

        (2)  To understand the expected output of p5.cc read the program
             in testLex.cc and see the comment at the bottom of testLex.cc
             (your p5.cc).

     As usual I warn you about unfair collaboration.  Read the syllabus if
     you are unclear about what constitutes unfair collaboration.