To: Adoram Rogel <adoram@hybridge.com>
Subject: Re: Flex 2.5.2 performance questions
In-reply-to: Your message of Thu, 19 Sep 96 10:16:04 EDT.
Date: Thu, 19 Sep 96 09:58:00 PDT
From: Vern Paxson <vern>

> a lot about the backing up problem.
> I believe that there lies my biggest problem, and I'll try to improve
> it.

Since you have variable trailing context, this is a bigger performance
problem.  Fixing it is usually easier than fixing backing up, which in a
complicated scanner (yours seems to fit the bill) can be extremely
difficult to do correctly.

You also don't mention what flags you are using for your scanner.
-f makes a large speed difference, and -Cfe buys you nearly as much
speed but the resulting scanner is considerably smaller.

> I have an | operator in {and} and in {pats} so both of them are variable
> length.

-p should have reported this.

> Is changing one of them to fixed-length is enough ?

Yes.

> Is it possible to change the 32,000 states limit ?

Yes.  I've appended instructions on how.  Before you make this change,
though, you should think about whether there are ways to fundamentally
simplify your scanner - those are certainly preferable!

		Vern

To increase the 32K limit (on a machine with 32 bit integers), you increase
the magnitude of the following in flexdef.h:

#define JAMSTATE -32766 /* marks a reference to the state that always jams */
#define MAXIMUM_MNS 31999
#define BAD_SUBSCRIPT -32767
#define MAX_SHORT 32700

Adding a 0 or two after each should do the trick.