Issue ARGUMENT-MISMATCH-ERROR Writeup

Issue:         ARGUMENT-MISMATCH-ERROR

Forum:         Cleanup

References:    ANSI CL spec (Aug 29, 1989 draft) pp.4-11,4-12,4-13,4-15,1-10

               88-002R p.1-26 (appears also on p.4-20 of ANSI CL draft) 

Category:      CHANGE

Edit history:  1-Feb-90, Version 1 by Moon

               9-May-90, Version 2 by Moon 

                (address another inconsistency, found by Kim Barrett)

               30-Oct-90, Version 3 by Pitman 

                (first attempt to formalize X3J13 amendments to v2)

               01-Nov-90, Version 4 by Pitman (comments by Moon, Barrett)

	       01-Nov-90, Version 5 by Pitman (more comments by Moon, Barrett)

               24-Feb-91, Version 6 by Loosemore (propose more amendments)

               14-Mar-91, Version 7 by Loosemore (change item 11)

Status:        v2+amendments (reflected in v5) accepted by X3J13, June 1990


Problem description:

  

  The draft ANSI Common Lisp specification is inconsistent about what kind 

  of error handling occurs when a function is called with arguments that 

  do not match its definition.

  

   1. For too few arguments, p.4-11 says "there must be at least n passed

   arguments," which probably (page 1-10) means "the consequences are

   undefined" if there are too few arguments.

   

   2. For too many arguments, p.4-12 says "an error of type error is 

   signalled."

   

   3. For unrecognized keyword arguments, p.4-13 says "the consequences 

   are undefined."  On the other hand, p.4-15 says "and error would be

   signalled."

   

   4. For keyword argument names that are not symbols, p.4-13 says "the

   consequences are undefined."

 

   5. For unrecognized keyword arguments supplied to generic functions, 

   p.4-20 and 88-002R p.1-26 say "an error is signalled."

 

  This is Symbolics issue #5.


Proposal (ARGUMENT-MISMATCH-ERROR:MUST-SIGNAL-WHEN-SAFE-OR-SYSTEM):

  

  Define that an error must be signalled in cases 1, 2, 3, and 5 of the

  argument mismatch situations in the problem description if the caller,

  the callee, and the point of functional evaluation all appear in a

  context where a `safe' optimization setting is in effect (i.e., 

  SAFETY=3).  In all other scenarios for these situations, the 

  consequences are undefined.


  Case 4 is treated the same, except that when :allow-other-keys t or

  &allow-other-keys is involved, error detection is optional (i.e., the

  consequences are undefined if the keyword argument names are not symbols).


  If any of the caller, the callee, or the point of functional evaluation

  was not user code, and was instead supplied by the implementation as a

  pre-defined definition, or as automatically generated code (e.g., as

  in method combination), then it must be treated as safe unless some 

  user code involved in the scenario is not safe.


  Clarify that a reference to the symbolic name of the function or to the 

  contents of the symbol-function of a symbol does not count as a functional

  evaluation.  For the purposes of this definition, functional evaluation

  occurs either explicitly due to a use of the FUNCTION special form, or 

  implicitly due to the use of a function name in the car of a normal

  functional form.


  The naive model which is intended is that the user can rely on error 

  checking in these situations if he has taken all reasonable steps to

  ensure that the situations will be safe.


  The exact time that the error will be signalled is implementation-dependent,

  but will always be prior to the execution of the body of the function being

  called.

  

Clarifications:


  The error might be signalled at compile time or at run time.  If the

  error is signalled at run time, it might be signalled by either the

  caller or the callee.


  The reason that this terminology is used, and not the normal, "should

  signal" terminology is because system code may be involved, and the

  user may not know in general whether system code was compiled `safe'

  or `unsafe'.  An implication of this definition is that all code

  compiled by the system will behave as if compiled safe unless some

  user code involved in the scenario is not. So, for example, if a user

  calls MAPCAR from safe code and passes a function which was compiled

  safe, the system is required to ensure that MAPCAR will make a safe

  call as well.



Proposal (ARGUMENT-MISMATCH-ERROR:MORE-CLARIFICATIONS):


  Amend proposal MUST-SIGNAL-WHEN-SAFE-OR-SYSTEM as follows:


  (1) Clarify that if the callee is a generic function, the generic function

      definition (if it was defined explicitly), the method definitions for 

      all applicable methods, and the method-combination definition must all

      be safe for the callee to be considered safe.

  

  (2) Clarify that for a form (COERCE <lambda-expression> 'FUNCTION),

      the value of the OPTIMIZE SAFETY quality in the global environment

      at the time the COERCE is executed applies to the resulting function.

  

  (3) Clarify that for a form (ENCLOSE <lambda-expression> <environment>),

      the value of the OPTIMIZE SAFETY quality in the <environment> 

      argument applies to the resulting function.

  

  (4) Clarify that for GENERIC-FUNCTION, GENERIC-FLET, and GENERIC-LABELS

      forms, the value of the OPTIMIZE SAFETY quality in the environment 

      in which the form appears applies to the resulting generic functions 

      and method definitions.

  

  (5) Clarify that for a form (ENSURE-GENERIC-FUNCTION . arguments), the

      value of the OPTIMIZE SAFETY quality in the environment passed as

      the :environment keyword applies to the resulting generic function.

  

  (6) Clarify that for a form (COMPILE <name> <lambda-expression>), the

      value of the OPTIMIZE SAFETY quality in the global environment at

      the time the COMPILE is executed applies to the resulting 

      compiled function.

  

  (7) Clarify that for a form (COMPILE <name>), if the original definition

      of the function was at a `safe' optimization setting, then the

      resulting compiled function must also be `safe'.

  

      Rationale: although many implementations do not save the information

      about the global environment for interpreted functions, this is

      consistent with the idea that safety is a lexical property that is

      captured at the time code promotion occurs.  Implementations that do

      not save the necessary information must treat interpreted functions

      as being always "safe".

  

  (8) Clarify that "automatically generated code" does not apply to the

      expansions of macros defined in the standard, which inherit safety

      from the environment in which the macro call appears.

  

      Rationale:  this term isn't very well defined.  

  

  (9) Replace the words "supplied by the implementation as a pre-defined

      definition" with "supplied by the implementation and part of the

      Common Lisp standard".

  

      Rationale:  the original wording prohibits implementations from

      providing *any* functions that do no argument checking, even

      internal functions that users might accidentally get their hands on

      as well as officially supported extensions.  There's no good 

      reason to prohibit this, since presumably users have to ask for 

      these unsafe functions explicitly.


 (10) Clarify that a call to a method via CALL-NEXT-METHOD must be 

      considered `safe' if the generic function, applicable methods,

      method-combination, and the definition and efunctuation of the

      CALL-NEXT-METHOD itself are all `safe'.


 (11) Specify that if CALL-NEXT-METHOD is called with arguments, the ordered

      set of applicable methods for the changed set of arguments for

      CALL-NEXT-METHOD must be the same as the ordered set of applicable

      methods for the original arguments to the generic function, or an error

      should be signaled.  [This is a change -- 88-002R p2-13 says this

      situation must always be detected.]


      Clarify that the comparison between the set of methods applicable to the

      new arguments and the set applicable to the original arguments is

      insensitive to order differences among methods with the same

      specializers.


 (12) Clarify that if CALL-NEXT-METHOD is called with arguments that specify

      a different ordered set of applicable methods and there is no next

      method available, the test for different methods and the associated

      error signalling (when present) takes precedence over calling

      NO-NEXT-METHOD.

     


Examples:


 A. Given ...


    (declaim (optimize (safety 3)))


    (defun     foo (x)                  (print 'test-failed) x)

    (defun     bar (&key x)             (print 'test-failed) x)

    (defmethod baz ((a integer) &key x) (print 'test-failed) x)


    (defun funcall-it (x) (funcall x))

   

    Then every implementation must arrange for an error to be signalled

    no later than the body (i.e., before (print 'test-failed) is executed)

    if the following forms also occur in safe code:


     A.1:  (foo)

     A.2:  (foo 1 2)

     A.3:  (bar :y 1)

     A.4:  (bar 1 2)

     A.5:  (baz 1 :y 7)

     A.6:  (funcall #'foo)

     A.7:  (funcall 'foo)

     A.8:  (mapcar #'foo '(1 2) '(1 2))

     A.9:  (mapcar 'foo '(1 2) '(1 2))

     A.8:  (mapcar #'1+ '(1 2) '(1 2))

     A.9:  (mapcar '1+ '(1 2) '(1 2))

     A.10: (funcall-it #'foo)

     A.10: (funcall-it 'foo)

     A.11: (funcall-it #'1+)

     A.12: (funcall-it '1+)

     A.13: (let ((x (locally (declare (optimize (safety 0))) 'foo)))

             (funcall-it x))

     A.14: (let ((x (locally (declare (optimize (safety 0))) '1+)))

             (funcall-it x))

     A.15: (let ((x (locally (declare (optimize (safety 0)))

		      (symbol-function 'foo))))

             (funcall-it x))

     A.16: (let ((x (locally (declare (optimize (safety 0)))

		      (symbol-function '1+))))

             (funcall-it x))


 B. Here are some examples of situations that might signal an error, but 

    are not required to signal an error.  In effect, the consequences are

    undefined in these cases, even if the surrounding code is declared safe:


          ;; Functional evaluation is not safe:

    B.1:  (let ((x (locally (declare (optimize (safety 2))) #'foo)))

            (funcall-it x))

    B.2:  (let ((x (locally (declare (optimize (safety 2))) #'1+)))

            (funcall-it x))


          ;; Point of call is not safe:

    B.3:  (let ((x #'foo))

            (locally (declare (optimize (safety 2)))

              (funcall x)))

    B.4:  (let ((x #'1+))

            (locally (declare (optimize (safety 2)))

              (funcall x)))


          ;; Callee is not safe:

          (locally (declare (optimize (safety 2)))

            (defun foo1 (x) x))

    B.5:  (foo1)

    B.6:  (mapcar #'foo1 '(1 2) '(1 2))

    B.6:  (funcall-it 'foo1)


Rationale:

  

  It's important for the document to be consistent and always say the

  same thing about each individual error situation.  It also seems

  important for the five error situations to be treated equally.

  

  Further, it's important that programmers be able to debug their code

  conveniently in a safe environment.  Once they start tampering with

  safety, they may run immediately into situations that expose

  variations in how implementations deal with function calling, but

  where safety is uniformly requested by the code, they should be able

  to insulate themselves from such differences.


  An exception is made in the name of efficiency for case 4 when

  :ALLOW-OTHER-KEYS T or &ALLOW-OTHER-KEYS is used.


Current practice:


  CLtL did not require this level of error checking, so it's entirely 

  likely that there are implementations which do not conform.


  Symbolics Genera conforms to this proposal.


Cost to Implementors:


  Some implementations already implement most or all of this, so their

  cost will be minimal.


  A few implementations may not implement this.  The cost will vary

  depending on the implementation. In some cases, it could take a fairly

  substantial amount of work to make these changes.


  An implementation not willing to make these changes might prefer to

  identify itself as implementing only a low-safety subset of the

  language, and simply refuse to compile code which was declared high

  safety.  This might be appropriate for certain delivery situations.


Cost to Users:


  More robust code.


Cost of non-adoption:


  The specification document will not even be self-consistent.


Performance impact:


  None.


Benefits:


  Language consistency.


Aesthetics:


  This is an improvement over the inconsistent situation which preceded it.


Discussion:


  The idea of making this merely "undefined" was discussed and rejected

  at the June 1990 meeting.  There was consensus that signalling an

  error was fine, but the main sticking points were saying under what

  situations the user could depend on such signalling, and at what time

  the signalling might be permitted to occur.  The following `amendment'

  to the previous proposal was proposed and adopted, with the intent

  that it be clarified later:


    "should signal"

           +

    [ efunctuation ]

    [ caller       ]  [ system or safe ]

    [ callee       ]

           +

    "and no later than the body of the callee"


  Versions 3-5 were an attempt to clarify what was voted upon.


  Version 6 is an attempt to fix some lingering problems in the

  writeup from version 5, mostly related to other situations in which

  function promotion can occur that were not dealt with explicitly

  in version 5.