Adam, I think you're correct in your assumption about the problem being with the pipe. IIRC, the shell spawns a child to manage each pipeline, so that it can properly handle redirects, signals, etc., as it sets up the pipeline. In the case of the original Bourne shell, the child shell process then does an exec on the last binary in the pipeline, so its exit status is used as the exit status of the entire pipeline. So, the explicit exec you are doing has no real effect, as it's happening in a child process, and not the parent shell. Likely also, YMMV depending on which specific shell you use. Not sure if this would work better?... exec sh -c "curl -s $URL/go.sh | sh" You still have parent & child shell processes, but I'm not sure if they'll exit differently. Maybe you'd need to background the pipeline to allow the parent shell to exit, and the background pipeline to take over? Or just do a loop, as Rob suggested. :) Gilbert On 2020-06-30 10:16 a.m., Adam Thompson wrote:

I've written a shell script that redownloads itself and re-executes itself, but I'm not quite seeing the behaviour I'd expect out of "exec sh".  Instead of one process, I get an infinite progression of shells that finally blows up when I hit ulimits.

The script:

===BOF=== curl -s $URL/targets \ | while read TGTNAME TGTIP ; do     (         echo -n "."

        mtr \             --interval 0.1 \             --gracetime 1 \             --timeout 1 \             --report-wide \             --report-cycles 600 \             --show-ips \             --aslookup \             --no-dns \             ${TGTIP} \         | awk -f parse.awk -v TGT=${TGTNAME}  \         | psql -q -b -h $PGHOST -U $PGUSER $PGDB     ) &    wait & done

# start all over again sleep 60 & wait curl -s $URL/go.sh | exec sh ===EOF===

The key items are the backgrounding of mtr/awk/psql, which appears to work properly - there's 12 lines in "targets" and I only see 12 mtr/awk/psql process groups at a time - the sleeping + waiting (which theoretically reaps all children and grandchildren, I think) and the final "exec".

Except instead of exec'ing, I run an infinite sequence of shells, each one a child of the previous.  E.g. from pstree(1):

# pstree -lp 1628 screen(1628)─┬─bash(1629)───sh(12707)───sh(12854)───sh(12986)───sh(13154)───sh(13309)───sh(13444)───sh(13579)───sh(13709)───sh(13842)───sh(13979)───sh(14101)───sh(14242)───sh(14396)───sh(14530)───sh(14655)───sh(14795)───sh(14935)───sh(15076)───sh(15203)───sh(15353)───sh(15490)───sh(15624)───sh(15753)───sh(15891)───sh(16023)───sh(16154)───sh(16282)───sh(16418)───sh(16551)───sh(16696)───sh(16842)───sh(16975)───sh(17112)───sh(17241)───sh(17388)───sh(17522)───sh(17664)───sh(17824)───sh(17956)───sh(18096)───sh(18233)───sleep(18279)

I think the problem has something to do with the fact the final exec is on the RHS of a pipe, and thus is executing inside a subshell, but how do I fix this?

Thanks, -Adam

-- Gilbert E. Detillieux E-mail: <gedetil@cs.umanitoba.ca> Dept. of Computer Science Web: http://www.cs.umanitoba.ca/~gedetil/ University of Manitoba Phone: (204)474-8161 Winnipeg MB CANADA R3T 2N2 Fax: (204)474-7609

I think you are correct on the RHS of the pipe. If you are not adverse to bash, this appears to work as expected: #!/bin/bash echo "i am doing something useful" sleep 3 & wait exec bash <(curl -s $URL/test.sh) if bash is a no-go, (simply needed for the <() operation), you may just need to download a temporary file and e.g. 'exec sh /tmp/script.sh' I think the crux of it is the exec replaces the current process with whatever you provide it. But on the RHS of the pipe, that current process is a subshell On Tue, Jun 30, 2020 at 11:06 AM Adam Thompson <athompso@athompso.net> wrote:

Even that doesn't quite work:

root@bgpmirror:~# pstree -g 3 -w -s screen ─┬= 00001 root /sbin/init └─┬= 04264 root sshd: /usr/sbin/sshd [listener] 0 of 10-100 startups (sshd) └─┬= 20865 root sshd: root@ttyp0,ttyp2,ttyp1 (sshd) └─┬= 73703 root -ksh (ksh) └─┬= 59765 root screen -U -q -i -fa └─┬= 01077 root SCREEN -U -q -i -fa (screen) └─┬= 03339 root /bin/ksh └─┬─ 93663 root sh -c curl -s https://lg.merlin.ca/ping/go.sh | sh └─┬─ 87697 root sh -c curl -s https://lg.merlin.ca/ping/go.sh | sh └─┬─ 25186 root sh -c curl -s https://lg.merlin.ca/ping/go.sh | sh └─┬─ 85474 root sh └─── 11715 root sleep 60

Argh.

FYI, I also tried "exec ( curl ... | sh)" which did not work, it produced an error instead.

A loop isn't suitable, because I want to be able to pick up any changes to the script on the fly - that's why I'm doing it this way in the first place.

-Adam

On 2020-06-30 10:36, Gilbert E. Detillieux wrote:

...

Adam,

I think you're correct in your assumption about the problem being with the pipe. IIRC, the shell spawns a child to manage each pipeline, so that it can properly handle redirects, signals, etc., as it sets up the pipeline. In the case of the original Bourne shell, the child shell process then does an exec on the last binary in the pipeline, so its exit status is used as the exit status of the entire pipeline.

So, the explicit exec you are doing has no real effect, as it's happening in a child process, and not the parent shell.

Likely also, YMMV depending on which specific shell you use.

Not sure if this would work better?...

exec sh -c "curl -s $URL/go.sh | sh"

You still have parent & child shell processes, but I'm not sure if they'll exit differently. Maybe you'd need to background the pipeline to allow the parent shell to exit, and the background pipeline to take over?

Or just do a loop, as Rob suggested. :)

Gilbert

On 2020-06-30 10:16 a.m., Adam Thompson wrote:

...

I've written a shell script that redownloads itself and re-executes itself, but I'm not quite seeing the behaviour I'd expect out of "exec sh". Instead of one process, I get an infinite progression of shells that finally blows up when I hit ulimits.

The script:

===BOF=== curl -s $URL/targets \ | while read TGTNAME TGTIP ; do ( echo -n "."

mtr \ --interval 0.1 \ --gracetime 1 \ --timeout 1 \ --report-wide \ --report-cycles 600 \ --show-ips \ --aslookup \ --no-dns \ ${TGTIP} \ | awk -f parse.awk -v TGT=${TGTNAME} \ | psql -q -b -h $PGHOST -U $PGUSER $PGDB ) & wait & done

# start all over again sleep 60 & wait curl -s $URL/go.sh | exec sh ===EOF===

The key items are the backgrounding of mtr/awk/psql, which appears to work properly - there's 12 lines in "targets" and I only see 12 mtr/awk/psql process groups at a time - the sleeping + waiting (which theoretically reaps all children and grandchildren, I think) and the final "exec".

Except instead of exec'ing, I run an infinite sequence of shells, each one a child of the previous. E.g. from pstree(1):

# pstree -lp 1628

screen(1628)─┬─bash(1629)───sh(12707)───sh(12854)───sh(12986)───sh(13154)───sh(13309)───sh(13444)───sh(13579)───sh(13709)───sh(13842)───sh(13979)───sh(14101)───sh(14242)───sh(14396)───sh(14530)───sh(14655)───sh(14795)───sh(14935)───sh(15076)───sh(15203)───sh(15353)───sh(15490)───sh(15624)───sh(15753)───sh(15891)───sh(16023)───sh(16154)───sh(16282)───sh(16418)───sh(16551)───sh(16696)───sh(16842)───sh(16975)───sh(17112)───sh(17241)───sh(17388)───sh(17522)───sh(17664)───sh(17824)───sh(17956)───sh(18096)───sh(18233)───sleep(18279)

...

...

I think the problem has something to do with the fact the final exec is on the RHS of a pipe, and thus is executing inside a subshell, but how do I fix this?

Thanks, -Adam

---

Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable

On 2020-07-04 00:35, Trevor Cordes wrote:

(Late to the party.)

You could use bash's eval in a while. Download your new code into a shell var (or a file you then open/read or slurp), like (psuedocode):

while (true) { newcode=`curl mynewcode` eval $newcode sleep }

That has no forks and allows you to update the code inside the while.

Other than that, I'm not sure why the shell is doing such weirdness. The infinite recursion is a bit of a surprise. Gilbert's supposition is probably on the right track.

If you have perl, it gives you finer-grained (C-ish level) control over exec-ish calls and might allow you to work around this problem. I find the Run3 module particularly useful. However, perl is probably overkill and it's not your favorite...

I don't mind Perl that much, but yeah it would be massive overkill. Ultimately, I gave in and had the script re-download itself, chmod itself, and then re-exec itself, and that's working more reliably than anything else I've tried so far, at the expense of leaving another file on disk in $CWD. If the extra file(s) ever become a huge problem, I could always add an EXIT trap, but it's not worth it yet. Thanks, everyone, for the various suggestions! -Adam