public inbox for goredo-devel@lists.cypherpunks.su
Atom feed
* goredo gets confused with relative paths and symlinks
@ 2025-10-23 13:54 Rafael Fourquet
2025-10-23 14:10 ` goredo
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Rafael Fourquet @ 2025-10-23 13:54 UTC (permalink / raw)
To: goredo-devel
Hi, thanks for this clean and sane implementation of redo!
`redo` fails to see an executable do file when called from a directory
accessed through a symlink, and falls back to
calling `/bin/sh` on the do script, which is unfortunate when the
script depends on `bash` or other languages.
For example:
$ cd /tmp/goredo-test
$ tree -a
.
├── a
│ └── b
├── b -> a/b
├── .redo
│ ├── test1.dep
│ └── test1.lock
├── test1
└── test1.do
$ cat test1.do
#!/bin/bash
# a-b-c is not valid syntax for `/bin/sh`
function a-b-c () {
echo 1
}
a-b-c
$ cd b; redo /tmp/goredo-test/test1
redo ../test1 ...
test1.do: line 5: `a-b-c': not a valid identifier
err ../test1 (0.003s): exit status 2
So the problem is that from within `/tmp/goredo-test/b`, `../test1.do`
doesn't exist, and therefore makes goredo believe
it's not executable (in "Prepare command line" from run.go).
This simple change seems enough to fix the issue, but I don't know
whether if leads to other problems:
--- a/run.go
+++ b/run.go
@@ -375,7 +375,7 @@ func runScript(tgt *Tgt, errs chan error, forced,
traced bool) error {
// Prepare command line
var cmdName string
var args []string
- if err = unix.Access(doFile.rel, unix.X_OK); err == nil {
+ if err = unix.Access(doFile.a, unix.X_OK); err == nil {
cmdName = doFile.a
args = make([]string, 0, 3)
} else {
^ permalink raw reply [flat|nested] 8+ messages in thread
* goredo gets confused with relative paths and symlinks
2025-10-23 13:54 goredo gets confused with relative paths and symlinks Rafael Fourquet
@ 2025-10-23 14:10 ` goredo
2025-10-23 14:58 ` Rafael Fourquet
2025-10-24 12:03 ` Sergey Matveev
2025-10-26 14:34 ` Sergey Matveev
2 siblings, 1 reply; 8+ messages in thread
From: goredo @ 2025-10-23 14:10 UTC (permalink / raw)
To: goredo-devel
Hi,
welcome and nice to see a fellow redo user!
Let me be blunt with you:
Do not use symlinks (or hardlinks) in redo projects. Just make a copy of the tree at a proper location. Links violate redo's dependency model. I think, there is some more rationale in the manual, but this is the gist of it.
Due to this expectation, redo or .do script may at any point resolve their location or that of other files around it to their physical path. That creates further complications with symlinked directories.
This is not a missing feature but part of the dependency model.
–Michael
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: goredo gets confused with relative paths and symlinks
2025-10-23 14:10 ` goredo
@ 2025-10-23 14:58 ` Rafael Fourquet
2025-10-24 12:12 ` Sergey Matveev
0 siblings, 1 reply; 8+ messages in thread
From: Rafael Fourquet @ 2025-10-23 14:58 UTC (permalink / raw)
To: goredo-devel
Yes, I read more today about this topic in the list and in the doc.
I'd like to make two points however:
1) the "bug" I reported happens even when there is no symlink within
the "redo project" itself.
In my example, "a", "a/b" and "b" (symlink to a/b) could be located
anywhere else to trigger the bug.
My use case is that sometimes I will want to rebuild a target when I'm
in another unrelated project, which I might have accessed through a
symlink (e.g. I did `cd ~/projects; redo /tmp/goredo-test/test1`,
where `~/projects` is a symlink to `/mnt/whatever/projects`.
2) if 1) if not a sufficient motivation to recognize the behavior as a
bug, at the very least, I don't think goredo should then try to call
`/bin/sh test1.do`.
IOW, I think it's definitely a bug for goredo to interpret the failure
of ` unix.Access(doFile.rel, unix.X_OK)` to mean that the file isn't
executable, when in fact it just means that the file is non-existent
(and then to proceed blindingly with `/bin/sh` on another file
(`doFile.a`) which is actually executable and relies on another
interpreter). This leads to this impenetrable error I got initially
(`a-b-c': not a valid identifier).
Cheers!
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: goredo gets confused with relative paths and symlinks
2025-10-23 13:54 goredo gets confused with relative paths and symlinks Rafael Fourquet
2025-10-23 14:10 ` goredo
@ 2025-10-24 12:03 ` Sergey Matveev
2025-10-26 14:34 ` Sergey Matveev
2 siblings, 0 replies; 8+ messages in thread
From: Sergey Matveev @ 2025-10-24 12:03 UTC (permalink / raw)
To: goredo-devel
[-- Attachment #1: Type: text/plain, Size: 304 bytes --]
Greetings!
*** Rafael Fourquet [2025-10-23 15:54]:
>For example:
>$ cd /tmp/goredo-test
>$ tree -a
Here your message seems to be cut for some reason.
There are no exact examples.
--
Sergey Matveev (http://www.stargrave.org/)
LibrePGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 265 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: goredo gets confused with relative paths and symlinks
2025-10-23 14:58 ` Rafael Fourquet
@ 2025-10-24 12:12 ` Sergey Matveev
2025-10-24 13:45 ` Rafael Fourquet
0 siblings, 1 reply; 8+ messages in thread
From: Sergey Matveev @ 2025-10-24 12:12 UTC (permalink / raw)
To: goredo-devel
[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]
*** Rafael Fourquet [2025-10-23 16:58]:
>1) the "bug" I reported happens even when there is no symlink within
>the "redo project" itself.
Questions and problems with symlinks were raised a while ago in that
maillist and I remember that basically they are not solvable in general
case. spacefrogg's advice not to use symlinks is best we can do.
>IOW, I think it's definitely a bug for goredo to interpret the failure
>of ` unix.Access(doFile.rel, unix.X_OK)` to mean that the file isn't
>executable, when in fact it just means that the file is non-existent
As I can see, Access() will be called only if do-file is found. It
should return earlier if it was not. Or maybe symlinks completely breaks
absolute/relative paths resolution and indeed it deals with different
views to the same file?
http://harmful.cat-v.org/software/symlinks
https://unix.stackexchange.com/questions/79571/symbolic-link-recursion-what-makes-it-reset/79621#79621
--
Sergey Matveev (http://www.stargrave.org/)
LibrePGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 265 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: goredo gets confused with relative paths and symlinks
2025-10-24 12:12 ` Sergey Matveev
@ 2025-10-24 13:45 ` Rafael Fourquet
2025-10-24 13:59 ` Sergey Matveev
0 siblings, 1 reply; 8+ messages in thread
From: Rafael Fourquet @ 2025-10-24 13:45 UTC (permalink / raw)
To: goredo-devel
Thanks for answering!
> Here your message seems to be cut for some reason.
> There are no exact examples.
The archive seems to have it:
http://lists.cypherpunks.su/archive/goredo-devel/CAJoaZ9Lv3VL0B_hqzZWSd4FUGr3bOZVyDhWE2F2ie1OsqGRZ-g@mail.gmail.com/
> Questions and problems with symlinks were raised a while ago in that
> maillist and I remember that basically they are not solvable in general
> case. spacefrogg's advice not to use symlinks is best we can do.
I understand, and am fine with avoiding symlinks within my "redo projects".
But I will always have symlinks somewhere else, and when I call redo from
one of these paths containing symlinks, it would be a small QOL improvement
if that would work out of the box.
> As I can see, Access() will be called only if do-file is found. It
> should return earlier if it was not. Or maybe symlinks completely breaks
> absolute/relative paths resolution and indeed it deals with different
> views to the same file?
It seems that's exactly what happens. The do file was determined to exist
(and `doFile.a` does indeed exist, and is executable), but
the `Access()` call is done on `doFile.rel`, which does not exist in the context
of my example.
So `Access()` was meant to determine whether the file is
executable, but it returns an error because the file was inexistant.
Adding debug statements in the code reveals:
` unix.Access returned error: no such file or directory`
with this line added in the else branch:
`fmt.Printf("unix.Access returned error: %v\n", err)`.
I don't know the codebase of goredo, so I will accept if calling
`redo /some/dir/file` from within an arbitrary path on my fs
(which happens to contain a symlink) can fail.
But I don't think that calling `/bin/sh` on that executable file is acceptable.
It can lead to very confusing errors, and could possibly have disastrous
side-effects if the intended interpreter has commands which are
also valid in `sh` with different semantics.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: goredo gets confused with relative paths and symlinks
2025-10-24 13:45 ` Rafael Fourquet
@ 2025-10-24 13:59 ` Sergey Matveev
0 siblings, 0 replies; 8+ messages in thread
From: Sergey Matveev @ 2025-10-24 13:59 UTC (permalink / raw)
To: goredo-devel
[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]
*** Rafael Fourquet [2025-10-24 15:45]:
>The archive seems to have it:
>http://lists.cypherpunks.su/archive/goredo-devel/CAJoaZ9Lv3VL0B_hqzZWSd4FUGr3bOZVyDhWE2F2ie1OsqGRZ-g@mail.gmail.com/
Hm, that is strange. My mail server delivered it to the maillist
properly, but cut it while delivering to my mailbox. Very strange.
Will look why that could happen and investigate it.
>It seems that's exactly what happens. The do file was determined to exist
>(and `doFile.a` does indeed exist, and is executable), but
>the `Access()` call is done on `doFile.rel`, which does not exist in the context
>of my example.
That sucks. Symlinks can complicate things so much indeed.
>But I don't think that calling `/bin/sh` on that executable file is acceptable.
Agreed, calling /bin/sh on executable is an unacceptable thing. Code
just does not expect that there could be an error about missing file it all.
I will think about that later, at least not earlier than Sunday (pretty
busy right now). I am not sure that you fix is a good thing: I am sure
that there are many other strange unexpected things could happen because
of those symlinks. So maybe I should check if there is some
"desynchronisation" between relative and absolite paths occured and
preventing running further at all, as I think noone can give enough
guarantees of any workability in presence of symlinks. Currently can not
give right answer. Will see soon. At least I will make a new release
with your patch.
--
Sergey Matveev (http://www.stargrave.org/)
LibrePGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 265 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: goredo gets confused with relative paths and symlinks
2025-10-23 13:54 goredo gets confused with relative paths and symlinks Rafael Fourquet
2025-10-23 14:10 ` goredo
2025-10-24 12:03 ` Sergey Matveev
@ 2025-10-26 14:34 ` Sergey Matveev
2 siblings, 0 replies; 8+ messages in thread
From: Sergey Matveev @ 2025-10-26 14:34 UTC (permalink / raw)
To: goredo-devel
[-- Attachment #1: Type: text/plain, Size: 558 bytes --]
Greetings!
*** Rafael Fourquet [2025-10-23 15:54]:
>This simple change seems enough to fix the issue, but I don't know
>whether if leads to other problems: [...]
I see no problems with changing doFile.rel to doFile.a, so making
a new release. I am sure that won't guarantee any expected workability
in case of symlinks presence, but Access() indeed should be checked
against an absolute path anyway. Thanks for noticing all of that!
--
Sergey Matveev (http://www.stargrave.org/)
LibrePGP: 12AD 3268 9C66 0D42 6967 FD75 CB82 0563 2107 AD8A
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 265 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-10-26 14:34 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-10-23 13:54 goredo gets confused with relative paths and symlinks Rafael Fourquet
2025-10-23 14:10 ` goredo
2025-10-23 14:58 ` Rafael Fourquet
2025-10-24 12:12 ` Sergey Matveev
2025-10-24 13:45 ` Rafael Fourquet
2025-10-24 13:59 ` Sergey Matveev
2025-10-24 12:03 ` Sergey Matveev
2025-10-26 14:34 ` Sergey Matveev