interesting unix pipe problem (stop whole pipe chain on first match)
For the punters out there... think you can find a better (shorter command line? no signals?) way to solve this interesting problem? I want to run an nmap on a network range, but have it quit (with output) the instant it finds the first "up" host. Nmap doesn't seem to have any way to achieve this. So I thought why not use unix pipes to do this? Oh ya, let's throw on a maximum run time too. So the goal is to run as quickly as we can to find one host on the network (the normal case), with the total time capped at something lower than nmap seems to want to take. And I need a way to still tell if the whole enchilada found something or not, and I want to wait for that result to come in. I need to take different action on whether we found a host or not. Note, nmap timeout and rtt options seem to apply per-host so even with sane-ish numbers, nmap will still often take 40-60s to run this command. Here's what I've come up with (view in monospace): echo 192.168.101 | \ timeout 30 bash -c \ " \ sint=\$(cat); \ (nmap -ddd -vvv --send-ip \ --host-timeout 5s --max-rtt-timeout 1s --initial-rtt-timeout 500ms \ --max-retries 1 -S \$sint.1 -sn -PE \$sint.2 \$sint.254-254 2>&1 & \ echo \$! >&3) \ 3>pidnmap \ | grep --line-buffered -P 'RCVD.{0,30}?ICMP.{0,60}?Echo reply' \ | (head -1 && kill \$(<pidnmap) 2>&1) \ " \ |grep RCVD || echo "no hosts found" I initially thought head would sigpipe the nmap once it got it's one line... and maybe it is, but nmap doesn't quit. So I have to explicitly kill the nmap later to make sure it truly dies rather than continue its 60s run. Hence the convoluted pid/kill stuff. I tried pkill -g0 at first, but it ate all the output and made it impossible to tell if we got output or not. And using return values (ideal) seems completely hopeless as soon as signals are involved. And no, setting /bin/timeout to the very minimum time I'm willing to tolerate (say 10s?) is not a good solution either, as I really want this command to return in the < 100ms it takes in the normal case. So, other than hacking nmap to immediately respond to sigpipe or adding a "quit after finding X up hosts" option, does anyone see a better UNIXy way to do this? The general problem isn't nmap specific... it's really a problem of shortcircuiting out of any long-running program when we see a certain line of its output. Heck, maybe there's already a coreutil or moreutil that does this generally somehow, but I couldn't find it faster than I could write the above! My gut says it's just got to be easier than this!! I couldn't believe how convoluted the solution quickly became.
Argh, sorry... for testing I was using a slightly bogus ip range... please change the 254-254 to 100-254. The corrected command line is below: echo 192.168.101 | \ timeout 30 bash -c \ " \ sint=\$(cat); \ (nmap -ddd -vvv --send-ip \ --host-timeout 5s --max-rtt-timeout 1s --initial-rtt-timeout 500ms \ --max-retries 1 -S \$sint.1 -sn -PE \$sint.2 \$sint.100-254 2>&1 & \ echo \$! >&3) \ 3>pidnmap \ | grep --line-buffered -P 'RCVD.{0,30}?ICMP.{0,60}?Echo reply' \ | (head -1 && kill \$(<pidnmap) 2>&1) \ " \ |grep RCVD || echo "no hosts found" And for some reason when I paste this from a my gui mail client it has syntax errors, but when I paste it from a tui mail client it works fine... YMMV. I guess I could post the one-line version: echo 192.168.101 | timeout 30 bash -c "sint=\$(cat); (nmap -ddd -vvv --send-ip --host-timeout 5s --max-rtt-timeout 1s --initial-rtt-timeout 500ms --max-retries 1 -S \$sint.1 -sn -PE \$sint.2 \$sint.100-254 2>&1 & echo \$! >&3) 3>pidnmap | grep --line-buffered -P 'RCVD.{0,30}?ICMP.{0,60}?Echo reply' | (head -1 && kill \$(<pidnmap) 2>&1)" |grep RCVD || echo "no hosts found"
AFAIK, the solution is to not use nmap. I suspect arping would be just as useful for your case, and it runs nearly instantaneously. Couple with the Linux version's "-f" flag, put it in the background, collect all the return codes somehow[*] and you're done in under 0.5sec. I prefer arping because it eliminates stupid firewall problems. Or fping, which can scan an entire subnet in a second or so using ICMP in a single process in a couple of seconds. -Adam On 2020-05-31 01:35, Trevor Cordes wrote:
Argh, sorry... for testing I was using a slightly bogus ip range... please change the 254-254 to 100-254. The corrected command line is below:
echo 192.168.101 | \ timeout 30 bash -c \ " \ sint=\$(cat); \ (nmap -ddd -vvv --send-ip \ --host-timeout 5s --max-rtt-timeout 1s --initial-rtt-timeout 500ms \ --max-retries 1 -S \$sint.1 -sn -PE \$sint.2 \$sint.100-254 2>&1 & \ echo \$! >&3) \ 3>pidnmap \ | grep --line-buffered -P 'RCVD.{0,30}?ICMP.{0,60}?Echo reply' \ | (head -1 && kill \$(<pidnmap) 2>&1) \ " \ |grep RCVD || echo "no hosts found"
And for some reason when I paste this from a my gui mail client it has syntax errors, but when I paste it from a tui mail client it works fine... YMMV. I guess I could post the one-line version:
echo 192.168.101 | timeout 30 bash -c "sint=\$(cat); (nmap -ddd -vvv --send-ip --host-timeout 5s --max-rtt-timeout 1s --initial-rtt-timeout 500ms --max-retries 1 -S \$sint.1 -sn -PE \$sint.2 \$sint.100-254 2>&1 & echo \$! >&3) 3>pidnmap | grep --line-buffered -P 'RCVD.{0,30}?ICMP.{0,60}?Echo reply' | (head -1 && kill \$(<pidnmap) 2>&1)" |grep RCVD || echo "no hosts found" _______________________________________________ Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
arping(8) solution: #!/bin/sh if ( ( for i in {1..254}; do ( arping -q -c 1 -w 1 192.168.0.$i; echo $? ) & done; wait ) | grep -q -m 1 0 ) ; then echo "subnet has hosts" else echo "subnet is empty" fi I'm sure it's possible to optimize that a bit further, but it seems adequate as-is. -Adam On 2020-05-31 08:53, Adam Thompson wrote:
AFAIK, the solution is to not use nmap. I suspect arping would be just as useful for your case, and it runs nearly instantaneously. Couple with the Linux version's "-f" flag, put it in the background, collect all the return codes somehow[*] and you're done in under 0.5sec. I prefer arping because it eliminates stupid firewall problems. Or fping, which can scan an entire subnet in a second or so using ICMP in a single process in a couple of seconds. -Adam
On 2020-05-31 01:35, Trevor Cordes wrote:
Argh, sorry... for testing I was using a slightly bogus ip range... please change the 254-254 to 100-254. The corrected command line is below:
echo 192.168.101 | \ timeout 30 bash -c \ " \ sint=\$(cat); \ (nmap -ddd -vvv --send-ip \ --host-timeout 5s --max-rtt-timeout 1s --initial-rtt-timeout 500ms \ --max-retries 1 -S \$sint.1 -sn -PE \$sint.2 \$sint.100-254 2>&1 & \ echo \$! >&3) \ 3>pidnmap \ | grep --line-buffered -P 'RCVD.{0,30}?ICMP.{0,60}?Echo reply' \ | (head -1 && kill \$(<pidnmap) 2>&1) \ " \ |grep RCVD || echo "no hosts found"
And for some reason when I paste this from a my gui mail client it has syntax errors, but when I paste it from a tui mail client it works fine... YMMV. I guess I could post the one-line version:
echo 192.168.101 | timeout 30 bash -c "sint=\$(cat); (nmap -ddd -vvv --send-ip --host-timeout 5s --max-rtt-timeout 1s --initial-rtt-timeout 500ms --max-retries 1 -S \$sint.1 -sn -PE \$sint.2 \$sint.100-254 2>&1 & echo \$! >&3) 3>pidnmap | grep --line-buffered -P 'RCVD.{0,30}?ICMP.{0,60}?Echo reply' | (head -1 && kill \$(<pidnmap) 2>&1)" |grep RCVD || echo "no hosts found" _______________________________________________ Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
Roundtable mailing list Roundtable@muug.ca https://muug.ca/mailman/listinfo/roundtable
On 2020-05-31 Adam Thompson wrote:
arping(8) solution:
Ya, I figured there would be other networky ways to do it. But I purposely was trying to use ICMP rather than arp, for the exact opposite reason you stated: I prefer braindead or firewalled/hidden hosts to not count in my scan. If you forget about the network aspect of the problem, though, surely there must be an easier way to stop a "producer" program earlier in the pipe once you get the output you're looking for? Would seem like a common-ish problem if your producer is long-running and not listening for the proper signals? Ignoring the network aspect, no one knows a better way? Maybe I need to write a little something for moreutils! Or maybe head could be tweaked to do the work? ... guess it wouldn't know who is the start of the pipe if it's >2 ps's...
participants (2)
-
Adam Thompson -
Trevor Cordes