![[LISPWORKS]](../Graphics/LWSmall.gif)
![[Common Lisp HyperSpec (TM)]](../Graphics/CLHS_Sm.gif)
![[Previous]](../Graphics/Prev.gif){kind=small}
![[Up]](../Graphics/Up.gif){kind=small}
![[Next]](../Graphics/Next.gif){kind=small}
Issue: DOTIMES-IGNOREForum: Cleanup
References: DOLIST (p126), DOTIMES (p126-128), IGNORE (p160),
LOOP (X3J13/89-004), DO-SYMBOLS (p187), DO-ALL-SYMBOLS (p188),
DO-EXTERNAL-SYMBOLS (p187), DO (pp122-126), DO* (pp122-126)
Category: CLARIFICATION
Edit history: 20-Jul-90, Version 1 by Pitman
06-Mar-91, Version 2 by Pitman
15-Mar-91, Version 3 by Pitman
Status: For Internal Discussion
Problem Description:
The IGNORE declaration is described in terms of whether a variable is
`used'. But what constitutes a use? Several iteration primitives are
described as macros, and the nature of the expansion is not well
enough specified for the answer to this question to be easily
determined.
Consider:
Is A used? Would it be appropriate to write:
(DOTIMES (A 3) (DECLARE (IGNORE A)) (PRINT 'FOO))
In some implementations, this would seem to be desirable, while in
other implementations it would seem undesirable.
In some implementations, such as an older Genera release, when the
compiler sees an IGNORE declaration, it `commits' the fact that the
variable will not be used and sometimes cannot later back out and
produce a successful compilation when this assumption is later realized
to be wrong in the process of code-walking. Because such a program
is technically in error, this is a conforming situation. But it
means that the choice of whether to use IGNORE or not has potential
semantic impact.
This issue started out as being just about DOTIMES (hence the name)
but has been generalized to include all CL iteration forms.
Terminology:
For the purposes of this issue, the following terminology applies:
An "explicit use of a variable in a form" is a use that would be
apparent in the normal semantics of the form. That is, if all
subforms of that form were macroexpanded, but the form itself were
not. [This isn't how an implementation has to detect an explicit
form; it's just a definition of what explicit use means. The intent
is that an explicit use be one which does not expose any undocumented
details of the form's macro expansion (if the form is in fact a macro).]
An "iteration form" means a DOLIST, DOTIMES, LOOP, DO-SYMBOLS,
DO-ALL-SYMBOLS, DO-EXTERNAL-SYMBOLS, DO, or DO* form.
An "iteration variable" is a variable which is "explicitly used"
as a variable to be bound by by an "iteration form".
Voting Instructions:
Proposals which have a "+" in their name are compatible with any
other proposals, and may be voted in addition to the other proposals.
Proposal (DOTIMES-IGNORE:NEVER):
Clarify that iteration variables are, by definition, always `used'.
That is, clarify that using
for such an iteration variable has undefined consequences.
Rationale:
This is analagous to the situation in DEFMETHOD where a specialized
required argument is considered a use of the argument.
Example:
;; The following would be correct and should not be warned about.
(DOTIMES (I 5)
(PRINT 'FOO))
;; The following would not be correct and might be warned about.
(DOTIMES (I 5)
(PRINT 'FOO))
Proposal (DOTIMES-IGNORE:UNLESS-EXPLICIT):
Clarify that unless an variable in an iteration form is explicitly
used in the form which binds it, it might be considered "unused" by
the implementation, and that it is always permissible to use
for such a unused variable.
Rationale:
This doesn't force users to think about the macro expansion and tends
to treat iteration variables more like variables bound by LET, LET*,
and LAMBDA.
Example:
;; The following would be correct and should not be warned about.
(DOTIMES (I 5)
(PRINT 'FOO))
;; The following would not be correct and might be warned about.
(DOTIMES (I 5)
(PRINT 'FOO))
Proposal (DOTIMES-IGNORE:INVISIBLE-REFERENCES):
Introduce a macro
WITH-INVISIBLE-REFERENCES ({vars}*) {forms}* [Macro]
Within the context of this macro, any uses of the <vars> are not
regarded as `uses' of the <vars> for the purposes of compiler warnings.
Clarify that macros like DOTIMES which expand into code which do
invisible computations on user-supplied variables must do the equivalent
of wrapping WITH-INVISIBLE-REFERNCES around those invisible uses, so that
those uses will not be counted as uses.
Rationale:
This makes the mechanism for invisible uses available to users so that they
can write forms like DOTIMES which manipulate user-supplied variables
and yet warn about lack of explicit use of those variables.
Example:
;; The following would be correct and should not be warned about.
(DOTIMES (I 5)
(PRINT 'FOO))
;; The following would not be correct and might be warned about.
(DOTIMES (I 5)
(PRINT 'FOO))
;; The following would be correct. User might be warned that X
;; is not used. (This is a pathological case which wouldn't be a
;; normal use. For a `real' use see the expansion of DOTIMES in
;; the Symbolics Genera current practice.)
(LET ((X 3))
(WITH-INVISIBLE-REFERENCES (X) (INCF X) (PRINT X))
0)
4
=> 0
Proposal (DOTIMES-IGNORE:+IGNORE-FUNCTION):
Introduce a function
IGNORE &REST IGNORE [Function]
Accepts any number of arguments and returns NIL.
Function calls to IGNORE can effectively be optimized away by a compiler,
except where side-effects might occur. That is, (IGNORE X #'Y (Z)) is
equivalent to (PROGN X #'Y (Z) NIL), but is more perspicuous about the
user's intent to identify ignored items.
Rationale:
This leaves the original problem in place but provides a compromise
that people can learn to use. It also has the side benefit of providing
an obvious syntax for `ignorable' variables introduced by some macros.
It also provides an obvious syntax for ignoring FLET'd items.
Example:
;; The following might or might not be correct and might be warned about.
(DOTIMES (I 5)
(PRINT 'FOO))
(DOTIMES (I 5)
(PRINT 'FOO))
;; The following would not be warned about.
(DOTIMES (I 5)
(IGNORE I)
(PRINT 'FOO))
Proposal (DOTIMES-IGNORE:+STYLE-WARNING):
Clarify that any `unused variable' warning must be a STYLE-WARNING, and
may not affect program semantics.
Rationale:
This is similar to STATUS-QUO, but at least insures that the issue has
no semantic impact on code.
Example:
;; The following might or might not be correct and might be warned about
;; but could be suppressed in a context where style warnings were
;; suppressable. In either case, correct code would be generated.
(DOTIMES (I 5)
(PRINT 'FOO))
(DOTIMES (I 5)
(PRINT 'FOO))
Proposal (DOTIMES-IGNORE:+NEW-DECLARATION):
Introduce a new declaration, IGNORABLE, similar to IGNORE, except that
it means that the variable might or might not be used, but that in
neither case should a warning be issued.
Rationale:
A lot of people have asked for this. e.g., it would be useful for
places where variables pop out of nowhere due to the presence of some
macro. e.g., in New Flavors (think of it as a user program from the
point of view of this proposal) the variable SELF magically appears in a
DEFMETHOD. The definition of FLAVOR:DEFMETHOD might, therefore, want
to declare SELF to be IGNORABLE since not all Flavors' DEFMETHOD forms
actually use this.
Example:
Given...
(DEFMACRO WITH-FOO (X &BODY STUFF)
`(LET ((FOO (FROB ,X)))
,@STUFF))
Neither of the following two uses would be warned about:
(WITH-FOO (F) (G))
or (WITH-FOO (F) (H FOO))
Proposal (DOTIMES-IGNORE:+FUNCTION-DECLARATIONS):
Permit the IGNORE (and IGNORABLE, if it passes--see proposal
+NEW-DECLARATION) declarations to contain references to #'name
in order to refer to a function name instead of a variable name.
Rationale:
This would be useful, for example, in the expansion of DEFMETHOD
in order to declare that CALL-NEXT-METHOD and NEXT-METHOD-P might
or might not be used, but that no warning should be produced in
either case. Some user programs presumably have similar needs.
Example:
`(... (FLET ((CALL-NEXT-METHOD ...)
(NEXT-METHOD-P ...))
(DECLARE (IGNORABLE #'CALL-NEXT-METHOD
#'NEXT-METHOD-P))
...)))
Proposal (DOTIMES-IGNORE:STATUS-QUO):
Accept the fact that the use of (DECLARE (IGNORE ...)) for an iteration
variable will always cause a warning in some implementations and that
the failure to use it will always cause a warning in some other
implementations. The net effect of this will be that no code which uses
any iteration primitive, DO, LOOP, or otherwise without using all iteration
variables explicitly will be free from gratuitous whining of at least
some compilers. Since the presence of
is sometimes fatal to some compilers, people will learn to live without it,
or will introduce creative other ways to `use' the variable to muffle the
warnings, sometimes at a performance penalty.
Rationale:
Implements the status quo.
Example:
;; The following might or might not be correct and might be warned about.
(DOTIMES (I 5)
(PRINT 'FOO))
(DOTIMES (I 5)
(PRINT 'FOO))
Test Case:
(DEFUN COMPILER-WARNINGS-P (BODIES &OPTIONAL (FLAG NIL))
(FLET ((TRY (BODY)
(WITH-OUTPUT-TO-STRING (*ERROR-OUTPUT*)
(COMPILE NIL `(LAMBDA ,@BODY)))))
(IF FLAG WARNING (> (LENGTH WARNING) 0)))))
(LIST (TRY BODY)
#'(LAMBDA (X)
(CADR BODY)))))))
BODIES))
(DEFUN TEST-CASE ()
(COMPILER-WARNINGS-P
'(((X) (DOTIMES (A X) (DECLARE (IGNORE A)) (PRINT 'FOO)))
((X) (DOLIST (A X) (DECLARE (IGNORE A)) (PRINT 'FOO)))
((X) (DO-SYMBOLS (S X) (DECLARE (IGNORE S)) (PRINT 'FOO)))
((X) (DO-EXTERNAL-SYMBOLS (S X) (DECLARE (IGNORE S)) (PRINT 'FOO)))
(() (DO-ALL-SYMBOLS (S) (DECLARE (IGNORE S)) (PRINT 'FOO)))
((X) (LOOP FOR I IN X DO (DECLARE (IGNORE I)) (PRINT 'FOO)))
((X) (LOOP FOR L ON X DO (DECLARE (IGNORE L)) (PRINT 'FOO)))
((X) (DO ((L X X)) (NIL) (DECLARE (IGNORE L)) (PRINT 'FOO)))
((X) (DO* ((L X X)) (NIL) (DECLARE (IGNORE L)) (PRINT 'FOO))))))
Current Practice:
Symbolics Genera implements INVISIBLE-REFERENCES (except that the
macro is called COMPILER:INVISIBLE-REFERENCES) and by implication
it implements UNLESS-EXPLICIT and STATUS-QUO. It also implements
+IGNORE-FUNCTION, and probably implements +STYLE-WARNING.
Test case results...
Symbolics Genera 8.0.1: ((NIL T) (T T ) (T NIL) (T NIL) (T NIL)
(T T) (T T ) (T T ) (T T ))
Symbolics Genera 8.1: ((NIL T) (NIL T) (T NIL) (T NIL) (T NIL)
(T T) (T T) (T T ) (T T ))
Symbolics Cloe: ((NIL T) (T T ) (NIL NIL) (T NIL) (T T )
(T T) (T NIL) (T T ) (T T ))
In Symbolics Genera, the macroexpansion of (DOTIMES (A 3) (PRINT 'FOO)) is:
(PROG ((A 0))
#:G1689 (PRINT 'FOO)
(COMPILER:INVISIBLE-REFERENCES (A)
(< A 3))
Cost to Implementors:
NEVER: Relatively small. It's straightforward for macros to add a few
spurious additional references in order to assure that no unused warnings
occur.
UNLESS-EXPLICIT: Medium. This is similar to INVISIBLE-REFERENCES, but
it allows for alternate implementations with equivalent effect because
it doesn't publish the interface.
+IGNORE-FUNCTION: Relatively small.
+NEW-DECLARATION: None to medium. This involves some compiler work,
the amount of which may vary, for implementations that do enough
bookkeeping for it to matter. Implementations which don't ever warn
can just ignore this, of course.
+FUNCTION-DECLARATIONS: Probably relatively small.
INVISIBLE-REFERENCES: None to Medium. This probably involves some
compiler hacking for implementations who care to do this kind of
warning. Strictly, an implementation could simply never warn, and then
this would be free, but some implementations have already established
a precedent of warning, so their customers would probably insist that
they did the work to make warnings continue to work in places where
they were called for.
STATUS-QUO: None.
Cost to Users:
NEVER: Relatively small.
UNLESS-EXPLICIT: Relatively small.
+IGNORE-FUNCTION: Relatively small.
+NEW-DECLARATION: None.
+FUNCTION-DECLARATIONS: None.
INVISIBLE-REFERENCES: Relatively small.
STATUS-QUO: Potentially large nuisance value and medium amount of work
for some users.
When porting to a new platform, you get a lot of warnings. Sometimes
the warnings are useful, sometimes they are not. The worst part is
if they are not useful and you just want to ignore them but you can't
because there are other warnings you want to see and you don't have the
ability to turn off just these warnings. So over and over you have to
wade through these warnings just to get to the others.
Cost of Non-Adoption:
See cost of option STATUS-QUO.
Benefits:
Users don't have to guess about important aspects of language semantics.
Aesthetics:
Proposals NEVER, UNLESS-EXPLICIT, +IGNORE-FUNCTION, +NEW-DECLARATION,
+FUNCTION-DECLARATIONS, and INVISIBLE-REFERENCES definitely improve the
aesthetics in the sense that they give the user the power to make the
code mean a particular thing, rather than leaving it to the discretion
of the implementation what the user meant.
Proposal STATUS-QUO is not aesthetic.
Discussion:
Pitman thinks that anything but STATUS-QUO is livable, but has a
preference for either INVISIBLE-REFERENCES or NEVER. He thinks
any of +IGNORE-FUNCTION, +NEW-DECLARATION, and +FUNCTION-DECLARATIONS
would be nice additions, too.
Dalton is on record as opposing a WITH-INVISIBLE-REFERENCES macro
and an IGNORABLE declaration (see below), but he hasn't seen this
specific proposal (which admittedly isn't likely to change his mind).
Several Symbolics customers have complained about portability
problems due to this lack of specificity.
Gregor Kiczales says: ``in writing PCL, this IGNORE issue was a
major pain in the ass. I never really was able to get it right,
and it ended up showing through to the users.''
JonL White said of mail from Dalton in a much longer discussion:
``he claims that DECLARE-IGNORE is *not* just friendly,
optional advice to the compiler, but assertions about
the program. I may disagree with him, but I can certainly
sympathesize with his viewpoint. This issue in general
-- what *must* declarations do -- is a serious one.''
Margolin said ``We're past the deadline for making significant changes
in the language. We're trying to get the standard edited now. At this
time we shouldn't be making changes unless they fix real language
bugs. I personally don't feel that a style warning is sufficient
justification for a change to the semantics of DOTIMES.'' RWK
responded ``I feel that the style warnings *ARE* an important enough
issue, because these style warnings are by default dictating the
variable semantics, but in a manner inconsistent from implementation
to implementation.''
Some people reviewing version 1 made suggestions which are not
pursued in this version, but which are worthy of note:
- GLS suggested we might need a REPEAT macro, such that
(REPEAT n . body) meant the same as (DOTIMES (I n) . body).
This would paper over the DOTIMES problem, but would still
leave other similar problems unattended to. This angle was
therefore not pursued.
- Barrett and others raised the issue of what counts as a use.
Both reading and writing it, or just reading? Technically,
that's an orthogonal issue, although it has obvious interplay
with anything decided here.
- There was discussion by Dalton, Margolin, and others of changing
the semantics of IGNORE to mean `ignorable'. Some felt this
would be visually confusing. Some felt we should rename IGNORE
to IGNORABLE. Some felt we should introduce IGNORABLE in
addition to IGNORE. [An IGNORABLE declaration was added as an
option in v3 of this proposal. -kmp]
- Jeff Barnett (at Northrop) suggested that we should reconsider
the idea of dignifying a certain variable, IGNORE, as implicitly
ignored and then permit (DOTIMES (IGNORE 5) ...). He observed
that this also works for (MULTIPLE-VALUE-BIND (X IGNORE Z) ...).
This was already brought up as issue IGNORE-VARIABLE (tabled by
Masinter at Mar-89 meeting due to deadline constraints).
Fred White (at BBN) suggested that it should be possible to do
(PROCLAIM '(IGNORE IGNORE)). Pitman pointed out that
(PROCLAIM '(SPECIAL IGNORE)) almost works.
Neil Goldman (at ISI) raised the issue of
which again calls for the IGNORE variable.
- Eliot@CS.UMASS.EDU pointed out that he just tries to avoid
(DECLARE (IGNORE X)) in favor of just X at toplevel of a body.
Don Cohen (at ISI) disagreed that this was adequate, saying that
he felt that an implementor would be justified in finding such
a `use' to be gratuitous and still warning. (Pitman disagrees
with this disagreement, because sometimes macros expand into
such things and getting a use that is `more significant' might be
a data abstraction violation.) In any case, the use Eliot was
talking about really implements IGNORABLE, not IGNORE.
- There was some exploration into the area of whether DOTIMES should
create a new binding every time. RPG cited that the current
definition of DOTIMES is unintuitive to users of parallel lisps.
Dalton pointed out that if a new binding were made every time, then
the internal variable which DOTIMES stepped could be a different
one, and hence the user's variable would be unused as a matter of
course (i.e., without WITH-INVISIBLE-REFERENCES) since no hidden
operations on it would be needed. Bob Kerns (RWK at ILA) said that
if people were intending to close over an iteration variable, their
intent would still be clearer if they wrote
(DOTIMES (I N) (LET ((I I)) (SPAWN #'(LAMBDA () ...I...))))
rather than just
(DOTIMES (I N) (SPAWN #'(LAMBDA () ...I...)))
- Dalton raised the question of whether all this trouble was really
worth it and whether we shouldn't just specify the expansion of
DOTIMES precisely so users could just know what it would have going
on inside it. RPG said ``Using possible macro expansions to reason
about language design is poor methodology. DOTIMES should be
specified and its implementation choices outlined.'' Barmar
defended (as an example) the Symbolics implementation which
expands differently depending on the situation and worried that
optimizations would be harder if the expansion was dictated. The
idea of showing a space of possible expansions was also discussed
by RWK and others. Barmar said ``By specifying each
operator as independently as possible we have less trouble with
strange interactions.'' Dalton responded ``Except where we don't,
as with DOTIMES and IGNORE. Moreover, it is at least arguable that
when too much is separate the language is more complex and harder
to learn.''
- Neil Goldman cited three examples of problem situations--
- 1) in lambda lists, because a function is used in a role
where it may be passed arguments it does not need.
2) multiple-value-setq, where the values that need to be
consumed are not just an initial sequence of those returned.
3) macros (like DOTIMES), that require the provision of a
variable name that will be lexically bound in the expansion
and permit IGNORE declarations about it.
He suggested that the form of destructuring used by LOOP would
solve these problems. e.g.,
(defun arg1-is-integer (arg1 nil) ...) ;;second arg is not consumed
(defun CAR-is-integer ((car . nil)) ...)
;; first arg must be a CONS. Its CDR is not consumed
(multiple-value-setq ((onea oneb) nil three) ...)
Peter Norvig (at Berkeley) worried about ambiguities in this notation.
(defun optional-CAR-is-integer (&optional (car nil)) ...)
(defmethod CAR-is-integer ((car cons)) ...)
RWK wrote one particular message which Moon identified in mail as the
most important message in the long conversation and which Moon said he
hoped wouldn't get lost. GLS sent mail saying he agreed. The
message was long, but it was a good message, so KMP has picked some choice
paragraphs from it and reproduced them here. These are from the
message of 26 Jul 90 21:26 EDT from RWK@FUJI.ILA.Dialnet.Symbolics.COM:
``Variable semantics is a very fundamental aspect of the language,
and we have failed to specify an important part of it. Traditionally,
we have dismissed "style warnings" as being an "environment" issue.
However, in this case, I do not think it should be so dismissed.
Currently, we have implementations of CL which vocally complain about
opposite usages. Having bogus style warnings go off in this way is
a serious problem for people porting code, because it obscures
legitimate problems. Bogus warnings are a serious waste of time for
porters of code, and they also make venders of portable code look as
if they are careless programmers.
``I strongly feel that a well-written portable Common Lisp program
should compile with no warnings in any well-done Common Lisp
implementation. Otherwise, how is the user of a portable program
supposed to know if the warnings warn of actual problems, or are
just noise.
``I strongly feel that X3J13 should firmly nail down under what
circumstances "unused variable" and "unused local function"
warnings may be issued. It is our failure to do so which is
at the heart of the current issue, and as a porter of code, it
has been a recurrent problem.''
``There are also two capabilities which are missing, which have
been refered to as an IGNORABLE declaration and a
COMPILER:INVISIBLE-REFERENCES form, in BARMAR's message. I
actually think that both are needed. IGNORABLE states that it
doesn't matter if the user doesn't reference a variable.
COMPILER:INVISIBLE-REFERENCES (WITHOUT-REFERENCES ?) states that
the user is *expected* to reference a particular variable, and
that style warnings are appropriate if he does not.
``I think both situations are common in macros, and that because
these issues are so close to the core of the language semantics,
this should be regarded as something more than mere "style
warnings". I wouldn't require compilers and interpreters to
issue the warnings in any particular situation, but I do think
we can and should define the semantics of our variables sufficiently
to state when these warnings are and are not appropriate.
``I'd also like to see all of these extended to local functions
in the same manner as the DYNAMIC-EXTENT declaration.''
Pitman notes the following bug report, received by Symbolics. One very
important effect of solving this issue is that vendors will know what
to implement and users will know what to expect. Right now, vendors
are forced to implement their conscience, and they have no defense if
a user thinks they did the wrong thing.
Date: Fri, 14 Sep 90 17:45 EDT
From: J.P. Massar <massar@Think.COM>
Subject: Stupid warnings
To: bug-lispm@Think.COM
Look at all this crap. I don't care what anyone says about
how iteration variables REALLY aren't been used...
I JUST DON'T WANT TO SEE THESE STUPID MESSAGES AND I DON'T
PARTICULARLY FEEL LIKE GOING AND PUTTING STUPID #+ WHATEVER's
OVER THIS CODE.
SYMBOLICS SHOULD BE SHOT FOR ISSUING THIS WARNING.
...
The only little problem is--all Symbolics did was implement what it
felt CLtL said to implement. What the user is really complaining
about is that the implementation didn't do the same thing as he
expected, and the only way it ever will is if we resolve this issue
once and for all.
![[Starting Points]](../Graphics/StartPts.gif){kind=small}
![[Contents]](../Graphics/Contents.gif){kind=small}
![[Index]](../Graphics/Index.gif){kind=small}
![[Symbols]](../Graphics/Symbols.gif){kind=small}
![[Glossary]](../Graphics/Glossary.gif){kind=small}
![[Issues]](../Graphics/Issues.gif){kind=small}