I have CentOS 6 system - my tape backup server - that's connected to two different subnets ("100" and "158"). There is also a router (actually a firewall) that routes between subnets "100" and "158". The server's default gateway is the router IP on subnet "158". The server's primary management IP address is the address on subnet "158".
There are client systems on subnet "100", both management workstations and things to back up.
When I try to SSH to the management IP in 158, the default behaviour in Linux is to send the reply back out the 'closest' interface, which is the "100" interface. This breaks things, because the router is actually a stateful firewall and I suddenly have asymmetric routing.
If I "ifconfig down" the "100" interface, suddenly everything works again... except now a large volume of traffic has to run through the firewall.
I'm trying to follow the various guidelines I've found for doing "ip rule" rules, but none of them seem to work.
Has anyone here accomplished this sort of thing before?
[image: Avant logo] *Adam Thompson* Senior Systems Administrator *voice:* 204.789.9596 x24 | *email:* athompson@avant.ca | *web:* avant.ca
Sounds like you need a VRF for 158/MGMT, where it ONLY goes out the 158 network? I'm not sure if Linux does this, but that might be a place to start perhaps.
Theo
On Oct 28, 2015, at 1:57 PM, Adam Thompson athompson@avant.ca wrote:
I have CentOS 6 system - my tape backup server - that's connected to two different subnets ("100" and "158"). There is also a router (actually a firewall) that routes between subnets "100" and "158". The server's default gateway is the router IP on subnet "158". The server's primary management IP address is the address on subnet "158".
There are client systems on subnet "100", both management workstations and things to back up.
When I try to SSH to the management IP in 158, the default behaviour in Linux is to send the reply back out the 'closest' interface, which is the "100" interface. This breaks things, because the router is actually a stateful firewall and I suddenly have asymmetric routing.
If I "ifconfig down" the "100" interface, suddenly everything works again... except now a large volume of traffic has to run through the firewall.
I'm trying to follow the various guidelines I've found for doing "ip rule" rules, but none of them seem to work.
Has anyone here accomplished this sort of thing before?
Adam Thompson Senior Systems Administrator voice: 204.789.9596 x24 | email: athompson@avant.ca mailto:athompson@avant.ca | web: avant.ca http://avant.ca/ _______________________________________________ Roundtable mailing list Roundtable@muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable
Whoops, I should be a little more consistent with where I send from, since my work address isn't even subscribed to the list...
I'm following, more or less, this: https://blogs.oracle.com/networking/entry/advance_routing_for_multi_homed
But it doesn't work (as expected, anyway).
VRFs are done kind of stupidly IMHO in Linux, given that it's a host, not a router.
-Adam
On 15-10-28 02:11 PM, Theodore Baschak wrote:
Sounds like you need a VRF for 158/MGMT, where it ONLY goes out the 158 network? I'm not sure if Linux does this, but that might be a place to start perhaps.
Theo
The tooling used in this article is the same as what I used (and what I would have recommended if I'd documented my setup, or wanted to reach out to Adi to re-get his configs for the setup) when I multi-homed a system at home years over a private and public network. In that case though, there wasn't any issues with connected subnets being chosen over default gateway.
Theo
On Oct 28, 2015, at 3:46 PM, Adam Thompson athompso@athompso.net wrote:
Whoops, I should be a little more consistent with where I send from, since my work address isn't even subscribed to the list...
I'm following, more or less, this: https://blogs.oracle.com/networking/entry/advance_routing_for_multi_homedhttps://blogs.oracle.com/networking/entry/advance_routing_for_multi_homed https://blogs.oracle.com/networking/entry/advance_routing_for_multi_homed
But it doesn't work (as expected, anyway).
VRFs are done kind of stupidly IMHO in Linux, given that it's a host, not a router.
-Adam
On 15-10-28 02:11 PM, Theodore Baschak wrote:
Sounds like you need a VRF for 158/MGMT, where it ONLY goes out the 158 network? I'm not sure if Linux does this, but that might be a place to start perhaps.
Theo
Roundtable mailing list Roundtable@muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable
On 2015-10-28 Adam Thompson wrote:
I have CentOS 6 system - my tape backup server - that's connected to two different subnets ("100" and "158"). There is also a router (actually a firewall) that routes between subnets "100" and "158". The server's default gateway is the router IP on subnet "158". The server's primary management IP address is the address on subnet "158".
An ascii picture might help.
There are client systems on subnet "100", both management workstations and things to back up.
When I try to SSH to the management IP in 158, the default behaviour
SSH from what host to what host?
in Linux is to send the reply back out the 'closest' interface, which is the "100" interface. This breaks things, because the router is actually a stateful firewall and I suddenly have asymmetric routing.
Hmm, default is to use the closest IF and not the IF that the traffic came in on (for tcp)? I wouldn't have guessed that.
You could use the various tc, ip rules, iptables, routing, etc functions of linux to force something to a certain IF, but I'm pretty sure that something can't change. i.e. you'd have to say "all ssh from host w.x.y.z goes out on .158" -- you couldn't make it dynamic AFAIK: you can't say "send the packet back out on the IF it came in on" (AFAIK).
In your case you could always have 2 different sshd's running on diff ports if you really needed it on both subnets.
If this sounds like I guessed your scenario right, I can dig out the various rules for you. I've done it before.
-
Adam Thompson -
Adam Thompson -
Theodore Baschak -
Trevor Cordes