Ticket #452 (closed defect: fixed)

Opened 5 years ago

Last modified 4 years ago

pathname namestring escape breaks externalization for unix shell commands

Reported by: james.anderson Owned by: rme
Priority: normal Milestone: Clozure CL 1.4
Component: other Version:
Keywords: pathname asdf Cc:

Description

given version "Version 1.3-RC1-r11719M (DarwinPPC32)", given a local asdf package in a unix file with pathname such as

/Development/Downloads?/bordeaux-threads_latest.tar.gz

asdf-install fails to install the package. the step which attempts to extract the tar table-of-contents fails, as the filename escape character, #
, appears in the command string. a patch to installer.lisp#tar-arguments ( http://trac.clozure.com/openmcl/browser/trunk/source/tools/asdf-install/installer.lisp?rev=11304#L291) to delete the escape character, which allows the install to succeed, demonstrates the problem.


patch diff

Index: installer.lisp
===================================================================
--- installer.lisp      (revision 11787)
+++ installer.lisp      (working copy)
@@ -291,7 +291,7 @@
 (defun tar-arguments (source packagename)
   #-(or :win32 :mswindows :scl)
   (list "-C" (namestring (truename source))
-       "-xzvf" (namestring (truename packagename)))
+       "-xzvf" (remove #\\ (namestring (truename packagename))))
   #+(or :win32 :mswindows)
   (list "-l"
        "-c"

listener transcript

? (asdf-install:install "/Development/Downloads/bordeaux-threads_latest.tar.gz")
0> Calling (ASDF-INSTALL::INSTALL-LOCATION) 
Install where?
1) System-wide install: 
   System in /usr/local/asdf-install/site-systems/
   Files in /usr/local/asdf-install/site/ 
2) Personal installation: 
   System in /Users/janson/.asdf-install-dir/systems/
   Files in /Users/janson/.asdf-install-dir/site/ 
0) Abort installation.
 --> 1
<0 ASDF-INSTALL::INSTALL-LOCATION returned (#P"/usr/local/asdf-install/site/" #P"/usr/local/asdf-install/site-systems/" "System-wide install")
0> Calling (ASDF-INSTALL::LOCAL-ARCHIVE-P "/Development/Downloads/bordeaux-threads_latest.tar.gz") 
<0 ASDF-INSTALL::LOCAL-ARCHIVE-P returned #P"/Development/Downloads/bordeaux-threads_latest\\.tar.gz"
0> Calling (ASDF-INSTALL::INSTALL-PACKAGE #P"/usr/local/asdf-install/site/" #P"/usr/local/asdf-install/site-systems/" "/Development/Downloads/bordeaux-threads_latest.tar.gz") 
 1> Calling (ASDF-INSTALL::EXTRACT #P"/usr/local/asdf-install/site/" "/Development/Downloads/bordeaux-threads_latest.tar.gz") 
 <1 ASDF-INSTALL::EXTRACT returned "tar (child): /Development/Downloads/bordeaux-threads_latest\\\\.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
/usr/bin/tar: Child returned status 2
/usr/bin/tar: Error exit delayed from previous errors
"
<0 ASDF-INSTALL::INSTALL-PACKAGE returned NIL
NIL
? (pathname-name #P"/Development/Downloads/bordeaux-threads_latest\\.tar.gz")
"bordeaux-threads_latest.tar"
? (namestring #P"/Development/Downloads/bordeaux-threads_latest\\.tar.gz")
"/Development/Downloads/bordeaux-threads_latest\\.tar.gz"
? (truename #P"/Development/Downloads/bordeaux-threads_latest\\.tar.gz")
#P"/Development/Downloads/bordeaux-threads_latest\\.tar.gz"
? (truename #P"/Development/Downloads/bordeaux-threads_latest.tar.gz")
#P"/Development/Downloads/bordeaux-threads_latest\\.tar.gz"
? (truename #P"/Development/Downloads/bordeaux-\\threads_latest.tar.gz")
#P"/Development/Downloads/bordeaux-threads_latest\\.tar.gz"
? (namestring  (truename #P"/Development/Downloads/bordeaux-threads_latest.tar.gz"))
"/Development/Downloads/bordeaux-threads_latest\\.tar.gz"
? (find #\\ (namestring  (truename #P"/Development/Downloads/bordeaux-threads_latest.tar.gz")))
#\\

;;; load patched installer.lisp

? (load (compile-file "/Development/Applications/LISP/ccl-1.3/tools/asdf-install/installer.lisp"))
;Compiling "/Development/Applications/LISP/ccl-1.3/tools/asdf-install/installer.lisp"...
;Loading #P"/Development/Applications/LISP/ccl-1.3/tools/asdf-install/installer.dfsl"...
#P"/Development/Applications/LISP/ccl-1.3/tools/asdf-install/installer.dfsl"
? (asdf-install:install "/Development/Downloads/bordeaux-threads_latest.tar.gz")
0> Calling (ASDF-INSTALL::INSTALL-LOCATION) 
Install where?
1) System-wide install: 
   System in /usr/local/asdf-install/site-systems/
   Files in /usr/local/asdf-install/site/ 
2) Personal installation: 
   System in /Users/janson/.asdf-install-dir/systems/
   Files in /Users/janson/.asdf-install-dir/site/ 
0) Abort installation.
 --> 1
<0 ASDF-INSTALL::INSTALL-LOCATION returned (#P"/usr/local/asdf-install/site/" #P"/usr/local/asdf-install/site-systems/" "System-wide install")
0> Calling (ASDF-INSTALL::LOCAL-ARCHIVE-P "/Development/Downloads/bordeaux-threads_latest.tar.gz") 
<0 ASDF-INSTALL::LOCAL-ARCHIVE-P returned #P"/Development/Downloads/bordeaux-threads_latest\\.tar.gz"
0> Calling (ASDF-INSTALL::INSTALL-PACKAGE #P"/usr/local/asdf-install/site/" #P"/usr/local/asdf-install/site-systems/" "/Development/Downloads/bordeaux-threads_latest.tar.gz") 
 1> Calling (ASDF-INSTALL::EXTRACT #P"/usr/local/asdf-install/site/" "/Development/Downloads/bordeaux-threads_latest.tar.gz") 
 <1 ASDF-INSTALL::EXTRACT returned "bordeaux-threads_0.5.1/
bordeaux-threads_0.5.1/bordeaux-threads.asd
bordeaux-threads_0.5.1/LICENSE
bordeaux-threads_0.5.1/site/
bordeaux-threads_0.5.1/site/index.html
bordeaux-threads_0.5.1/site/style.css
bordeaux-threads_0.5.1/src/
bordeaux-threads_0.5.1/src/allegro.lisp
bordeaux-threads_0.5.1/src/armedbear.lisp
bordeaux-threads_0.5.1/src/bordeaux-threads.lisp
bordeaux-threads_0.5.1/src/cmu.lisp
bordeaux-threads_0.5.1/src/condition-variables.lisp
bordeaux-threads_0.5.1/src/corman.lisp
bordeaux-threads_0.5.1/src/default-implementations.lisp
bordeaux-threads_0.5.1/src/ecl.lisp
bordeaux-threads_0.5.1/src/lispworks.lisp
bordeaux-threads_0.5.1/src/mcl.lisp
bordeaux-threads_0.5.1/src/openmcl.lisp
bordeaux-threads_0.5.1/src/sbcl.lisp
bordeaux-threads_0.5.1/src/unsupported.lisp
bordeaux-threads_0.5.1/test/
bordeaux-threads_0.5.1/test/bordeaux-threads-test.lisp
"
"ln -s \"/usr/local/asdf-install/site/bordeaux-threads_0.5.1/bordeaux-threads.asd\" \"/usr/local/asdf-install/site-systems/bordeaux-threads.asd\""
;;; ASDF-INSTALL: Found system definition: /usr/local/asdf-install/site/bordeaux-threads_0.5.1/bordeaux-threads.asd
<0 ASDF-INSTALL::INSTALL-PACKAGE returned (#P"/usr/local/asdf-install/site/bordeaux-threads_0.5.1/bordeaux-threads.asd")
;;; ASDF-INSTALL: Loading system ASDF-INSTALL::|bordeaux-threads| via ASDF.
; loading system definition from /usr/local/asdf-install/site/bordeaux-threads_0.5.1/bordeaux-threads.asd into #<Package "ASDF0">
;Loading #P"/usr/local/asdf-install/site/bordeaux-threads_0.5.1/bordeaux-threads.asd"...
; registering #<SYSTEM BORDEAUX-THREADS #xA3A2A3E> as BORDEAUX-THREADS
; registering #<SYSTEM BORDEAUX-THREADS-TEST #xA3C305E> as BORDEAUX-THREADS-TEST
;Compiling "/usr/local/asdf-install/site/bordeaux-threads_0.5.1/src/bordeaux-threads.lisp"...
;Loading #P"/usr/local/asdf-install/site/bordeaux-threads_0.5.1/src/bordeaux-threads.dfsl"...
;Compiling "/usr/local/asdf-install/site/bordeaux-threads_0.5.1/src/openmcl.lisp"...
;Loading #P"/usr/local/asdf-install/site/bordeaux-threads_0.5.1/src/openmcl.dfsl"...
;Compiling "/usr/local/asdf-install/site/bordeaux-threads_0.5.1/src/default-implementations.lisp"...
;Loading #P"/usr/local/asdf-install/site/bordeaux-threads_0.5.1/src/default-implementations.dfsl"...
(ASDF-INSTALL::|bordeaux-threads|)
? 

Change History

comment:1 Changed 5 years ago by rme

  • Owner set to rme

ccl::native-translated-namestring (instead of plain vanilla namestring) is probably the right thing to use here.

Perhaps we should update our bundled asdf-install; the current version has moved tar-arguments into port.lisp, so that would probably be a cleaner place for us to conditionalize.

comment:2 Changed 5 years ago by james.anderson

agreed, that tar-arguments &co are naive to expect a standard operator to do the right thing, when used is to supply arguments to implementation-specific ones.

independent of this, shouldn't the escape-logic for namestrings still satisfy the (naive?) expectation, that mappings are stable - even if not 1-1? as now implemented, this is not the case. [yes, i realize namestrings are of limited use. i tend to use logical pathnames for everything i can get away with. still, i'd not yet realized, that physical pathname namestrings are effectively a black hole.]

? (pathname "asdf.qwer.lisp")
#P"asdf\\.qwer.lisp"
? (pathname "asdf\\.qwer.lisp")
#P"asdf\\.qwer.lisp"
? (pathname "asdf.qwer\\.lisp")
#P"asdf.qwer.lisp"

? (namestring (pathname "asdf.qwer.lisp"))
"asdf\\.qwer.lisp"
? (namestring (pathname "asdf\\.qwer.lisp"))
"asdf\\.qwer.lisp"
? (namestring (pathname "asdf.qwer\\.lisp"))
"asdf.qwer.lisp"

? (equalp (pathname (namestring (pathname "asdf.qwer.lisp"))) (pathname "asdf.qwer.lisp"))
T
? (equalp (pathname (namestring (pathname "asdf\\.qwer.lisp"))) (pathname "asdf\\.qwer.lisp"))
T
? (equalp (pathname (namestring (pathname "asdf.qwer\\.lisp")))(pathname "asdf.qwer\\.lisp"))
NIL

comment:3 Changed 5 years ago by rme

  • Milestone set to Clozure CL 1.4

comment:4 Changed 5 years ago by rme

I am not sure what to do about asdf-install generally. I personally think it's fragile and a bad idea generally, and would like to unbundle it and have users get the upstream asdf-install directly.

Nevertheless, I'm going to commit a patch to our bundled asdf-install that attempts to deal with the namestring issue raised above.

In some light testing, I ran into trouble with Snow Leopard's tar command, which on extraction produces output like

x bordeaux-threads_0.6.0/bordeaux-threads.asd
x bordeaux-threads_0.6.0/LICENSE
x bordeaux-threads_0.6.0/site/
...

Note the (traditional) leading "x", which GNU tar doesn't print. That confuses asdf-install, so it's currently busted on Snow Leopard (and probably on other systems that don't use GNU tar).

comment:5 Changed 5 years ago by rme

(In [12898]) Conditionalize SYSTEM-NAMESTRING for Clozure CL. Use SYSTEM-NAMESTRING instead of NAMESTRING in several pertinent places. (see ticket:452)

comment:6 Changed 4 years ago by rme

  • Status changed from new to closed
  • Resolution set to fixed

Snow Leopard should now work with asdf-install as of r13115

Note: See TracTickets for help on using tickets.