function with proclaimed return type NIL causes a warning when uses as a defstruct slot initializer

[nikodemus]

Compiling a file with code such as following causes a full warning and a tertiary value of T from COMPILE-FILE.

(declaim (ftype (function () nil) arg-missing)) (defun arg-missing ()

(error "missing arg!"))

(defstruct foo

(bar (arg-missing)))

[nikodemus]

Let's try that again:

(declaim (ftype (function () nil) arg-missing))
(defun arg-missing ()
  (error "missing arg!"))

(defstruct foo
  (bar (arg-missing)))

[gb]

With that FTYPE declaration in effect, a full warning is also generated on:

(defun foo2 ()
  ;; The constructor generated by DEFSTRUCT contains a call to ARG-MISSING
  ;; and that's where the warning comes from in your example.
  (arg-missing))

for essentially the same reasons that warnings are generated for:

(defun foo3 (x)
  (the nil x))

and:

(defun foo4 ()
  (the nil (some-function)))

and:

(defun foo5 ()
  ;;; All STRINGs are BASE-STRINGs in CCL.
  (declare (ftype (function (unsigned-byte &key (element-type t))
                            (and string (not base-string))) make-string))
  (make-string 10 :element-type 'character))

and, to take a respite from the empty type:

(defun foo6 ()
  (the fixnum 1.0))

The last case involves an easily detected violation of type declarations, "undefined behavior", and possible runtime errors: it's pretty obvious that 1.0 isn't a FIXNUM, and these are situations in which a compiler is allowed to generate a full warning.

It's not clear to me that there's a significant difference between FOO6 and the other cases. Most (all ?) implementations will signal runtime errors for constructs like (THE NIL X) under appropriate SAFETY settings, and I think that all implementations would be justified in signaling (non-STYLE-) warnings at compile time in those cases.

Deciding that a compound FUNCTION type whose return value type is NIL denotes a function that doesn't terminate normally is (at least) an interesting and useful extension (since CL doesn't seem to provide another way of saying that and the return type of NIL is otherwise meaningless), but I'm not sure that it's something that nominally portable code should depend on. I can't find anything in the spec that suggests that that interpretation is anything but a possibly useful non-standard extension, and I wouldn't want empty (in some implementations, including CCL) types like (AND STRING (NOT BASE-STRING)) in FOO5 above to be treated as an indication that functions with that result type don't return.

That concern apparently isn't universal; given:

(defun double-it (x) (+ x x))

(defun test (f arg)
  (declare (type (function (t) (and integer float)) f))
  (funcall f arg))

a fairly recent version of SBCL will signal a runtime error on

(test #'double-it 5)

, complaining that the function in question returned unexpectedly. That's at least consistent with how other ways of specifying an empty function return type behave, but if this interpretation of empty return types is an extension, it makes me less than enthusiastic about adopting it.

If there's something in the spec that says that an empty function return type can be used in conforming code to indicate non-termination of functions to which a FUNCTION declaration applies, then clearly calls to such functions shouldn't warn about that. I can't find anything in the spec that supports that interpretation, and without that interpretation declaring that a function returns a value of type NIL is always a violation of type declarations (nothing can be of type NIL, of course) and a strong indicator of potential runtime errors or anomalous behavior, and I think that full warnings are both allowed and desirable.

If I'm mistaken about that and there is language in the spec that supports that interpretation, please reopen this ticket and point us to that language.

If it's simply a matter of extending a case that's otherwise not too useful (NIL as a result type) to express something that's otherwise not expressible (non-termination of functions) ... I'm somewhat sympathetic to at least the second part of that but have some concerns about the first part, and regardless of what anyone thinks of that extension (if that's indeed what it is) you should probably avoid having nominally portable code depend on it.

[nikodemus]

While I'm reopening the ticket, that is just to make sure you get to see this -- this isn't an issue of note for me.

I don't think the spec ever comes out and spells it out, but (FUNCTION * NIL) meaning "never returns" is the only consistent interpretation for that type -- and a necessary consequence of NIL being subtype of every type.

Taking that stance certainly isn't a recent invention for SBCL. It's been that way since CMUCL days, and from our perspective at least it is not an extension, and I'm pretty sure the CMU folks didn't consider it that either.

...but if CCL doesn't like it, we should be able to make the cross-compiler not depend utilize those declarations, so it's not problem.

For future reference, the chain of logic that leads to the interpretation of NIL functions never returning is:

A ftype declaration (FUNCTION * NIL) says that call sites are equivalent to

(THE (VALUES NIL &REST T) ...)

and since there are no objects TYPEP NIL, nothing can ever be returned, ergo the function can never return. The function returning (VALUES) doesn't qualify, since the return value would still default to NIL in this case.

Perhaps more importantly, having the compiler complain about inconsistent types for

(THE FIXNUM (THE NIL (SOMETHING)))

doesn't make any sense to me, since (SUBTYPEP 'NIL 'FIXNUM) must return T, T -- so obviously the types are consistent. Of course there aren't any objects that fit the description, but the types are still perfectly consistent. ...which leads us back to the only logical conclusion: SOMETHING can never return. :)

Cheers,

-- Nikodemus

[gb]

Replying to nikodemus:

While I'm reopening the ticket, that is just to make sure you get to see this -- this isn't an issue of note for me.

It's not particularly hard to understand the chain of logic that would lead one to believe that a function declared to return NIL is being declared to never return, especially if one believes that the design of the CL type system concerned itself with non-termination and wouldn't have just left the use of NIL return types largely undefined.

I don't believe the latter: the CL type system as it was first defined (in CLtL1) was too underspecified (and too non-deterministic) to allow implementations or applications to do much type-reasoning (or even type-checking in some cases.) I remember suggestions to make it more useful, but not many of those were seriously considered. X3J13 was generally concerned with issues like "are empty ranges like (INTEGER (0) (0)) definitely subtypes of NIL ?" and "can we say ANYTHING useful about the cases in which SUBTYPEP must return a true second value ?" and "are LISTs whose car is the symbol LAMBDA a legal first arg to FUNCALL, and is such a list of type FUNCTION or not ?" and with big-picture questions like "how many implementations and how much user code will break if is changed/clarified ?" It may seem hard to believe 20-25 years later, but there were people who were passionately opposed to the idea that LAMBDA lists couldn't be FUNCALLED and that they weren't FUNCTIONs; IIRC, one of the most vocal and passionate opponents of that idea was KMP. What I remember of that argument - and I hope that I'm not misrepresenting it - was that it was something like "sure, saying that the types CONS and FUNCTION were disjoint would make the type system saner, but saying that you can't just cons up a list and funcall it destroys something that's an essential and traditional part of Lisp."

There was also concern from some people that - in addition to its other flaws - the CL type system just wasn't strict enough. Rob MacLachlan? was one of those people; he wrote and maintained an informal proposal ("clprop.txt", which used to be on a server somewhere at cmu.edu and which may or may not still exist somewhere) which (IIRC) tried to address type-system issues as well as general issues of compiler semantic. I don't think that any of that (or much of it) ever made it as far as an X3J13 cleanup issue. I think that changes to the spec that would have made CL's type system stricter would have been controversial and met with significant opposition; intelligent, experienced Lisp programmers weren't really sure what FTYPE declarations meant or whether their purpose was primarily to serve as a form of documentation.

Larry Masinter used to maintain an archive of X3J13 cleanup committee email and issue status; a copy of this is available at < ftp://ftp.clozure.com/pub/x3j13-cleanup.tar.gz>. Some of the files in that archive have .Z extensions (are compressed with the old Unix compress utility) and the mail archive files are CR-terminated and may contain other control characters; mail transfer in those days often involved UUCP, so the messages are sometimes slightly out-of-order.

None of that material is a binding part of any language definition, but I propose that anyone who starts an argument with a phrase like "well, the spec doesn't say that explicitly, but clearly the intent was " should offer proof of having read those discussions before expecting that argument to be accepted.

(There was also a separate cl-compiler mailing list; I don't have archives of that list, but I think that I've seen some of it on the net in the relatively recent past.)

The point of this trip down memory lane is to point out that the CMUCL-derived extension under discussion (a) is an extension and (b) isn't supported by "the only logical conclusion" as to what a function return type of NIL means. If you believe that the CL spec somehow intended to describe a stricter type system than any language actually in the spec describes (and that that supporting language was just tragically omitted due to oversight), then you might believe that an empty declared return type "must mean something other than an undefined/anomalous case", and your conclusion that the extension's behavior is the only consistent interpretation is believable.

As I've already said, I actually believe that that interpretation is somewhay useful and somewhat sound and not contradicted by anything that the spec actually does say, but believe that that interpretation has a pretty serious flaw: it assumes that empty types (things equivalent to the type NIL) always appear in declarations intentionally, and that quite simply isn't the case.

Again, the type specified by:

(and string (not base-string))

is a non-empty subtype of STRING in some implementations and the empty type in others, so using that type-specifier as the return value in an FTYPE declaration in effect on a call to MAKE-STRING in an implementation where all STRINGs are BASE-STRINGs is either:

• an assertion that this call to MAKE-STRING won't return
• a programming error for which a warning is justified and desirable.

I think that the first interpretation in this case is - to use the technical term - "Just Too Damned Ugly Too Live." I can't think of any way of having both behaviors coexist, other than saying that saying NIL exactly means "doesn't return" and other, otherwise equivalent empty types are "likely indicators of programming errors". That's not quite ugly enough to kill on sight, but it's pretty close. Something somehow like that idea - where there was something very much like the type NIL that could only be used to indicate that a function didn't return - would seem to be preferable to the status quo, where an implementation has to choose between these two interpretations of the empty type in these contexts. (One consequence of the fact that "clprop.txt" wasn't considered too strongly - if my memory of that is accurate - is that some of the concerns it expressed were never addressed; another consequence may be that some of the extensions that CMUCL chose to implement weren't widely reviewed.)

I wouldn't try to argue that interpreting an empty return type as a "likely indicator indicator of a programming error" is the only logical conclusion; I think that kind of assertion in this case requires too much tunnel vision.

FWIW, all of the non-CMUCL-derived implementations that I looked at (LispWorks?, Allegro, CLISP, and CCL) signal a type error at runtime and high enough SAFETY on calls to FOO defined as:

(defun foo () (the nil (random 10)))

where that type error complains that RANDOM's result wasn't of type NIL. CCL was the only implementation that warned about that potential error. It seems reasonable to assume that that might change in the future in those other implementations. (Those implementations interpretations of the construct might also change, but the fact that they're currently silent at compile-time doesn't seem to mean that they're interpreting things as you may expect them to and doesn't mean that they'll remain silent.)

If anyone actually wants to continue this discussion here, it's not necessary to reopen the ticket to do so; we're trying to get a release out, and trying to keep ticket status accurate is helpful in that process.