Gelex: Some Simple Examples

Some Simple Examples

First some simple examples to get the flavor of how one uses gelex. The following gelex input specifies a scanner which whenever it encounters the string "foo" will replace it with the string "bar":

%{
class FOOBAR

inherit

    YY_COMPRESSED_SCANNER_SKELETON

create

    make
%}

%%

foo     output ("bar")

%%

end

This description file is made up of three sections separated by %% marks. The text enclosed within the two signs %{ and %} in the first section is Eiffel text which is copied verbatim to gelex output class file. It specifies that the name of the generated class is FOOBAR and that its creation procedure is make, a routine inherited from class YY_COMPRESSED_SCANNER_SKELETON. This class contains the pattern-matching engine - a Deterministic Finite Automaton (or DFA for short) - which is optimized in terms of memory space, hence the name of the class. It also provides numerous facilities such as routine scan for analyzing a given input text. The second section, enclosed within the two %% marks, contains the only rule of this description file. foo is the pattern of this rule and the outputinstruction is the action. By default, any text not matched by a gelex scanner is copied to the standard output, so the net effect of this scanner is to copy its input file to the standard output with each occurrence of "foo" replaced with "bar". The third section of the file, after the second %% mark, is Eiffel text which is copied verbatim at the end of the generated class. In this example, it just contains the end of the class keyword.

Here's another simple example:

%{
class WC

inherit

    YY_COMPRESSED_SCANNER_SKELETON

create

    make
%}

%%

\n      nb_lines := nb_lines + 1; nb_characters := nb_characters + 1
.       nb_characters := nb_characters + 1

%%

feature -- Access

    nb_lines: INTEGER
    nb_characters: INTEGER
            -- Number of lines and characters

feature -- Analysis

    analyze 
            -- Count the number of characters and lines
            -- in input file and display the results.
        do
            nb_lines := 1
            nb_characters := 0
            scan
            print ("Number of lines = ")
            print (nb_lines)
            print ('%N')
            print ("Number of characters = ")
            print (nb_characters)
            print ('%N')
        end

end

This scanner counts the number of characters and the number of lines in its input (it produces no output other than the final report on the counts). The first section of the scanner description is similar to the one described in the first example. The generated class is named after the Unix command wc which counts the number of characters, words and lines in given files. There are two rules, one which matches a newline (\n) and increments both the line count and the character count, and one which matches any character other than a newline (indicated by the . regular expression).

A somewhat more complicated example:

%{
note

    description: "Scanner for a toy Eiffel-like language"

class EIFFEL_SCANNER

inherit

    YY_COMPRESSED_SCANNER_SKELETON

    KL_SHARED_ARGUMENTS
        export {NONE} all end

    KL_SHARED_STANDARD_FILES
        export {NONE} all end

create

    make
%}

DIGIT    [0-9]
ID       [a-z][a-z0-9_]*

%%

{DIGIT}+    {
                 io.put_string ("An integer: ")
                 io.put_integer (text.to_integer)
                 io.put_new_line
            }
{DIGIT}+"."{DIGIT}*  {
                 io.put_string ("A real: ")
                 io.put_real (text.to_real)
                 io.put_new_line
            }
class|inherit|feature|do|if|then|else|end    {
                 io.put_string ("A keyword: ")
                 io.put_string (text)
                 io.put_new_line
            }
{ID}        {
                 io.put_string ("An identifier: ")
                 io.put_string (text)
                 io.put_new_line
            }
"+"|"-"|"*"|"/"|"^"    {
                 io.put_string ("An operator: ")
                 io.put_character (text_item (1))
                 io.put_new_line
            }
"--".*           -- Ignore comments
[ \t\n\r]+       -- Ignore separators
.           {
                 io.put_string ("Unrecognized character: ")
                 io.put_character (text_item (1))
                 io.put_new_line
            }

%%

feature -- Scanning

    scan_file 
            -- Scan file whose name has been given as argument.
            -- Scan standard input if no argument has been given.
        local
            l_file: KL_TEXT_INPUT_FILE
            l_input: KI_TEXT_INPUT_STREAM
        do
            if Arguments.argument_count >= 1 then
                create l_file.make (Arguments.argument (1))
                l_file.open_read
                l_input := l_file
            else
                l_input := std.input
            end
            set_input_buffer (new_file_buffer (l_input))
            scan
        end

end

This is the beginnings of a simple scanner for a language like Eiffel. It identifies different types of tokens and reports on what it has seen. The details of this example will be explained in the rest of the documentation.