Issue QUOTE-SEMANTICS Writeup

Forum:		Compiler

Issue:		QUOTE-SEMANTICS

Subsumes:	Issue QUOTE-MAY-COPY

References:	CLtL p. 55, 78, 86, 143

		Issue CONSTANT-COLLAPSING

		Issue CONSTANT-COMPILABLE-TYPES

		Issue CONSTANT-CIRCULAR-COMPILATION

Category:	CLARIFICATION

Edit History:   V1, 22 Jan 1989, Sandra Loosemore

		V2, 13 Mar 1989, Sandra Loosemore (discussion)

		V3, 22 Mar 1989, Sandra Loosemore (suggestions from Moon)

Status:		Ready for release



Problem Description:


Is it permissible for COMPILE and EVAL to coalesce or copy constants?

Are there constraints upon what kinds of objects may appear as

constants in code processed by COMPILE or EVAL, similar to those for

COMPILE-FILE?


CLtL p86 states that (QUOTE <x>) simply returns <x>.  On p55 it is

mentioned that the only self-evaluating forms that may be copied are

numbers or characters.  It is also stated that an implementation is

permitted to collapse (or coalesce) EQUAL constants "appearing in code

to be compiled" (p78), which is defined to mean self-evaluating forms

or objects contained in a QUOTE form (without reference to whether the

form is processed by EVAL, COMPILE, or COMPILE-FILE).


Because of its nature as a file processor, COMPILE-FILE generally must

cause copies of constants to be constructed when the compiled code is

loaded.  In a number of existing Lisp implementations, COMPILE also

causes constant objects to be copied and/or coalesced.  There is also

at least one implementation where constants are copied by EVAL in some

circumstances.


In the proposals that follow, "copying" is used to mean the process of

constructing an object that is "similar as a constant" (as defined in

proposal CONSTANT-COMPILABLE-TYPES:SPECIFY), but not necessarily EQL,

to the original.


The term "coalescing" is defined in the writeup for issue 

CONSTANT-COLLAPSING.



Proposal QUOTE-SEMANTICS:NO-COPYING:


State that copying or coalescing of constants appearing in code

processed by EVAL and COMPILE is not permitted; the resulting program

must reference objects that are EQL to the corresponding objects in

the source code.  The constraints on what kinds of objects may appear

as constants (described in issues CONSTANT-COMPILABLE-TYPES and

CONSTANT-CIRCULAR-COMPILATION) apply only to COMPILE-FILE.


  Rationale:


  This proposal is consistent with what many people think of as the

  "traditional" semantics for QUOTE.  It gives users maximum flexibility

  about what kinds of objects may appear as constants.



Proposal QUOTE-SEMANTICS:COPYING-ALLOWED-BUT-NO-CONSTRAINTS:


State that copying or coalescing of constants appearing in code

processed by EVAL and COMPILE is permitted.  Copying or coalescing may

only take place when the source code is "promoted" to being a program

by EVAL or COMPILE, not at runtime.  Function definitions are promoted

to being a program when the form enclosing the definition (e.g., a

FUNCTION or DEFUN form) is promoted.


Any object may validly appear as a constant in code processed by EVAL

or COMPILE.  The constraints on what kinds of objects may appear as

constants (described in issues CONSTANT-COMPILABLE-TYPES and

CONSTANT-CIRCULAR-COMPILATION) apply only to COMPILE-FILE.  For data

types where proposal CONSTANT-COMPILABLE-TYPES:SPECIFY does not define

the notion of "similar as a constant", an implementation is permitted

to copy objects of that type only if it has extended "similar as a

constant" to include that type.


  Rationale:


  This proposal is the most consistent with the semantics stated in CLtL.

  It gives users maximum flexibility about what kinds of objects may

  appear as constants.


  Allowing constants to be coalesced or copied has advantages for

  memory management; for example, constants can be copied to read-only

  memory that does not need to be garbage-collected.



Proposal QUOTE-SEMANTICS:SAME-AS-COMPILE-FILE:


State that copying or coalescing of constants appearing in code

processed by EVAL and COMPILE is permitted.  Copying or coalescing may

only take place when the source code is "promoted" to being a program

by EVAL or COMPILE, not at runtime.  Function definitions are promoted

to being a program when the form enclosing the definition (e.g., a

FUNCTION or DEFUN form) is promoted.


The constraints on what kinds of objects may appear as constants

(described in issues CONSTANT-COMPILABLE-TYPES and

CONSTANT-CIRCULAR-COMPILATION) apply to EVAL and COMPILE as well as to

COMPILE-FILE.


  Rationale:


  This makes the rules for handling of constants consistent between

  EVAL, COMPILE, and COMPILE-FILE.  It gives implementors maximum 

  flexibility in handling constants in EVAL and COMPILE.


  Allowing constants to be coalesced or copied has advantages for

  memory management; for example, constants can be copied to read-only

  memory that does not need to be garbage-collected.



Current Practice:


Implementations in which COMPILE attempts to copy all constants

include PSL/PCLS, Utah Common Lisp, and Kyoto Common Lisp.  UCL and

KCL both implement COMPILE effectively as a combination of

COMPILE-FILE and LOAD.


In Lucid Common Lisp, constants are not normally copied by COMPILE,

but since COMPILE does coalesce constants, it may cause QUOTE to

return an object which is not EQL to the object which appeared in the

source code.


Symbolics Genera has COMPILE copy list, string, non-displaced array,

and (I-Machine only) closure constants, but Moon says he thinks this

is wrong.


There is known to be at least one implementation where expanding the

DEFUN macro causes all constants in the body of the function to be

copied.



Cost to implementors:


Proposal NO-COPYING would involve a significant cost in those

implementations where constants are now copied or coalesced by EVAL

and COMPILE.  The aspect that is likely to cause the most problems is

that, in some implementations, the garbage collector assumes that

constants referenced in compiled code have been copied to read-only

storage and do not need to be scanned or relocated.  Changing this

would require major changes to the internal representation of

functions, memory management strategy, and/or the implementation of

COMPILE.


Some implementations would also require substantial changes to support

proposal COPYING-ALLOWED-BUT-NO-CONSTRAINTS.  Implementations that

would have garbage collection problems under proposal NO-COPYING would

have the same problems under COPYING-ALLOWED-BUT-NO-CONSTRAINTS,

unless they can define a copying behavior that will correctly handle

objects of all possible datatypes.


Proposal SAME-AS-COMPILE-FILE has no adoption cost above what is

required to support issues CONSTANT-COMPILABLE-TYPES and

CONSTANT-CIRCULAR-COMPILATION.



Cost to users:


Proposals COPYING-ALLOWED-BUT-NO-CONSTRAINTS and SAME-AS-COMPILE-FILE

may break some existing programs that assume constants in code

processed by EVAL or COMPILE are always EQL to the corresponding

objects in the source code.  Proposal SAME-AS-COMPILE-FILE may also

break existing programs that depend on referencing "undumpable"

constants in code processed by EVAL or COMPILE.  In both cases,

however, the behavior is already nonportable.  Both proposals would

permit implementations in which these programs now work to continue to

provide their existing behavior.



Benefits:


The semantics of QUOTE are clarified.



Discussion:


This issue subsumes issue QUOTE-MAY-COPY, which caused a very lengthy

debate on the cl-compiler mailing list.


This issue relates to conformance requirements.  Accepting either of

proposals NO-COPYING or COPYING-ALLOWED-BUT-NO-CONSTRAINTS would mean

that not all conforming programs could be compiled with COMPILE-FILE.

Some people may find this disturbing, particularly since one of the

goals of Common Lisp has been to try to eliminate differences in

semantics between compiled and interpreted code.


Loosemore supports proposal QUOTE-SEMANTICS:SAME-AS-COMPILE-FILE,

since it requires essentially no conversion cost for implementors and

does not break any user programs that are not already nonportable.


JonL White says:

  Since we have already passed the proposal that permits constants to 

  be "read-only" -- it is an error to modify them -- and have already 

  passed  the proposal that allows access to updateable structures -- 

  LOAD-TIME-EVAL -- then there is no excuse for being overly concerned

  with the storage address of quoted data.  People who have mistakenly 

  used structured constants as updatable data should convert over to 

  either LOAD-TIME-EVAL or DEFPARAMETER.


Kent Pitman says:

  The problem is that a lot of copying advocates have been going around

  trying to use "the need for copying" as leverage for restricting

  the set of things which I may quote. My view is that it is my write [sic]

  to quote whatever I want, and it's up to the person who thinks they

  can do something fun with copying to not get themselves in deeper than

  they can handle.


Jeff Dalton says: 

  I would agree [with Pitman's remarks] too.  My only quibble is that

  it's not just "the need for copying" that's used a lever.

  "Consistency with file compilation" is also being used as a lever.


UCL implements COMPILE by dumping and loading a temporary file using

the same mechanisms as COMPILE-FILE and LOAD.  Leigh Stoller (one of

the UCL compiler implementors) says that, even if this implementation

technique is disallowed by the outcome of this issue, they would

rather be nonconforming than change the implementation of COMPILE.  In

addition to the change being a lot of work, he says he thinks that

making COMPILE-FILE and COMPILE different would be "really dumb", and

that having different conformance requirements for compiled and

interpreted code would just encourage people to write programs that

can't be compiled correctly.