Issue EVAL-TOP-LEVEL Writeup

Forum:		Public Review

Issue:		EVAL-TOP-LEVEL

References:	Loosemore's public review comment #29 (X3J13/92-1329)

		Section 3.2.3.1, "Processing of Top Level Forms"

		EVAL, LOAD, COMPILE-FILE

		Issue EVAL-WHEN-NON-TOP-LEVEL

Category:	CLARIFICATION, CHANGE

Edit history:	12 Dec 90, Version 1 by Tim Moore

		14 Dec 90, Version 2 by Moore w/ comments from Loosemore

		08 Mar 91, Version 3 by Moore, discussion from rwk and JonL

		06 Feb 93, Version 4 by Loosemore

		  (new proposal, update writeup)

Status:		Proposal LIKE-COMPILER (version 3) failed 4-9 at

		  the March 1991 meeting.

		Proposal LOAD-LIKE-COMPILE-FILE passed 5-2 at the

		  October 1993 meeting.



Problem description:


  Section 3.2.3.1, "Processing of Top Level Forms" specifies the 

  processing of top-level forms in the file compiler in great detail.

  Unfortunately, there is no such specification of the evaluator or

  loader's top level behavior.  If the evaluator or loader does not 

  process top-level forms in the same way that the compiler does, there 

  can be a surprising semantic gap between the compiler and interpreter.

  It may be impossible to evaluate forms that macroexpand into more than 

  one top-level form.  For example:


    (progn

      (defsetf foo setf-foo)

      (defun bar (u v)

        (setf (foo u) v)))


  If the DEFSETF form is not evaluated before the function definition is

  processed, the SETF form will expand incorrectly into:


    (funcall (function (setf foo)) v u)


  instead of:


    (setf-foo u v)


  The net result is that code which has been carefully structured to

  compile correctly under the order-of-processing constraints given in

  section 3.2.3.1 may fail to work when loaded interpretively.


  At least two implementations -- UCL and WCL -- have unwittingly 

  stumbled into this trap by implementing evaluators that perform

  implicit compilation before executing any code.



Proposal (EVAL-TOP-LEVEL:LOAD-LIKE-COMPILE-FILE):


  Replace the fourth paragraph of the description section of the

  dictionary entry for LOAD with:


    \funref{load} sequentially executes each \term{form} it encounters

    in the file named by \param{filespec}.  If the file is a source

    file and the implementation chooses to perform 

    \term{implicit compilation}, \funref{load} must recognize

    \term{top level forms} as described in {\secref\TopLevelForms} and

    arrange for each \term{top level form} to be executed before

    beginning \term{implicit compilation} of the next.  (Note, however, 

    that processing of \specref{eval-when} forms by \funref{load} is 

    controlled by the \kwd{execute} situation.)



Rationale:


  Everybody seems to agree that it's a bug for implementations *not*

  to be able to load files containing top-level forms such as that

  given in the problem description section.  Having the standard say

  something explicit about it will hopefully prevent implementors

  from overlooking this source of bugs in the future.



Current practice:


  UCL and WCL have had problems with this in the past.  It's not known

  whether or how they have been fixed.



Cost to implementors:


  In those implementations that do implicit compilation, minor hacks

  to make LOAD or EVAL to do stuff similar to the top-level COMPILE-FILE

  processing.  None in other implementations.



Cost to users:


  None.



Aesthetics:


  Neutral.



Editorial impact:


  Inserting the text included above.


  (It would probably be better to reorganize this material into a separate

  section describing program structure rather than talking about the

  behavior of LOAD and COMPILE-FILE, but that's too much work!)



Discussion:


  Sandra Loosemore says:


    I submitted this as a public review comment after I discovered that

    WCL was "broken" while trying to port some code last fall.  I had 

    completely forgotten that the exact same problem had previously 

    been presented as a cleanup issue to the committee until Kim Barrett 

    mentioned it while we were sorting through the comments after the 

    end of the public review period.


  Version 3 of this issue contained a different proposal, LIKE-COMPILER,

  which would have constrained EVAL rather than LOAD.  That proposal was

  defeated at the March 1991 meeting, with people who voted against the

  proposal arguing that


    (eval x) 


  should have completely equivalent semantics to


    (funcall (compile nil `(lambda () ,x)))


  The LOAD-LIKE-COMPILE-FILE proposal in version 4 avoids this problem

  by constraining LOAD only.  However, it may be implemented either by

  tweaking LOAD specifically or EVAL in general to process forms in 

  the required order.