Subnetting, Proxy Arp and TFTP
Background
Elsewhere on this site are notes about setting up diskless X terminals from
old PCs. All these experiments were done with the BOOTP/TFTP/NFS server on
the same subnet as the diskless client. This all worked fine.
But I now have a subnetted home network and was eager to try to set up a
diskless client on a different subnet. I had heaps of trouble with this and
I made a lot of important discoveries along the way ... and hence I've put
them into this article.
Home network diagram
Internet
|
+-----------------+--------------------+
| ppp0 |
| |
| G |
| |
| eth0(10.0.0.5) eth1(10.0.1.5) |
+-------+---------------------+--------+
| |
+--+ +-------------+
| |
+---------+---------+ +----------+--------+
| eth0(10.0.0.1 | | eth0(?.?.?.?) |
| | | |
| S | | C |
| | | |
+-------------------+ +-------------------+
G is the Gateway machine, S is the Server and C is our
diskless Client. G is acting as a router and an IP Masquerading
gateway to the internet through the PPP connection. On the G gateway, the
two ethernet ports are configured as:
eth0 IP=10.0.0.5 BROADCAST=10.0.0.255 NETMASK=255.255.255.0
eth1 IP=10.0.1.5 BROADCAST=10.0.1.255 NETMASK=255.255.255.0
Its just a simple class C subnet design(??). The whole reason I have
two ethernet subnets is that I was too cheap to buy a hub. Each subnet is
just a crossover cable directly connected to 'S' and 'C'.
The 'S' server has its ethernet card configured as follows:
eth0 IP=10.0.0.1 BROADCAST=10.0.0.255 NETMASK=255.255.255.0
DEFAULT GATEWAY=10.0.0.5
Obviously, anything not meant for the 10.0.0.x network is meant to go through
the default gateway.
The 'C' client machine actually does have a hard disk and is often booted up in
Windows and works just fine with the following ethernet config:
eth0 IP=10.0.1.1 BROADCAST=10.0.1.255 NETMASK=255.255.255.0
DEFAULT GATEWAY=10.0.1.5
Under this setup, the masquerading on
the 'G' gateway seemed to work fine, with both 'S' and 'C' able to access the
internet correctly. The two machines could also connect to each other. I just
assumed this was an offshoot of turning on ip_forward'ing on the 'G' gateway.
The Problem
All I wanted to do was to boot up 'C' as a diskless client using 'S' as the
TFTP/NFS server. The BOOTP server was also on 'S' and I knew that BOOTP
broadcasts would not be passed through the 'G' machine at all ... unless I
helped it. At the time I was also mucking around the dhcp, so I started up the
DHCP relay on 'G' so that it would forward any DHCP/BOOTP requests onto 'S'.
So all I did on 'G' was:
dhcrelay 10.0.0.1
I could now reboot the 'C' machine from an Etherboot floppy. I could see it
sending the BOOTP request and getting a response, But it would not download
the kernel by TFTP.
Here is the relevant bootptab entry for our diskless client:
.client:\
hd=/tftpboot:vm=auto:ip=10.0.1.16:\
:sa=10.0.0.1:sm=255.255.255.0:ht=ethernet:ha=00a02406636a:\
:gw=10.0.1.5:bf=xterm1.boot3:rp="/tftpboot/10.0.0.5":
The sa= indicates the TFTP server to get the kernel from. The IP
address, default gateway should all be OK. I then tried booting off a Bootable
Linux CD on the 'C' machine to try a manual tftp. It also did not work.
I then started doing some packet tracing on the 'G' gateway. Using tcpdump,
I could see the tftp UDP packet come in from 10.0.1.16 (the 'C' client) on
the eth1 interface and then go out again on the eth0 interface to 'S'. 'S'
then responded, but the packet didn't seem to go out the eth1 interface.
The solution
In those packet traces I noticed something odd about the port numbers used
in some of the UDP packets. The 'G' gateway was actually changing the source
port of the UDP packet on the way through. What was happening was that the 'G'
gateway was performing masquerading on connections between the 10.0.0.x and
10.0.1.x subnets. This was not a problem for TCP based protocols like telnet
and ftp, but UDP ones don't quite work the same.
My Masquerading rule on the 'G' gateway was effectively this:
ipchains -A forward -j MASQ -s 10.0.0.0/16 -d any/0
i.e. Anything from 10.0.x.y goingto anywhere gets masqueraded. Thats
generally what you want for masquerading, but it also meant that packets
going from 10.0.1.x to 10.0.0.x would also get masqueraded. I changed the
masquerade rule to:
ipchains -A forward -j MASQ -s 10.0.0.0/16 -d ! 10.0.0.0/16
This allowed the two internal machines to connect to the internet OK,
but now they couldn't talk to each other, basically because the default
'forward' policy was to REJECT. I added in another forward rule:
ipchains -A forward -s 10.0.0.0/16 -d 10.0.0.0/16 -j ACCEPT
This allowed packets to be forwarded anywhere within the 10.0.x.y networks
with the important difference that the packets are NOT masqueraded.
Proxy Arp
My home network design is an example of being cheap and using
available components. Having seperate subnets does have some
advantages, but it would simplify things if the server and client
machines above actually thought they were on the same subnet. In
fact they can for most purposes.
Lets revisit my home network design, but this time, we'll assume that the 'C'
machine is a running machine with a proper IP address:
Internet
|
+-----------------+--------------------+
| ppp0 |
| |
| G |
| |
| eth0(10.0.0.5) eth1(10.0.1.5) |
+-------+---------------------+--------+
| |
+--+ +-------------+
| |
+---------+---------+ +----------+--------+
| eth0(10.0.0.1) | | eth0(10.0.1.1) |
| | | |
| S | | C |
| | | |
+-------------------+ +-------------------+
The S and C machines both have a subnet mask of 255.255.255.0 and use the
corresponding '10.0.x.5' address on the 'G' box as their default gateway.
Sounds sensible.
Proxyarp is a method of tricking IP to do things its not meant to.
Some background: When a machine on an IP network first attempts to contact
another host that it knows is on the same subnet as itself (ie it knows through
the netmask that its on the same subnet), then it sends out an ARP request.
This is basically a 'Who is IP address 10.0.0.3?'. The machine that has that IP address then responds including its own MAC address. From then on, packets to
'10.0.0.3' are sent with the MAC address of that machine that responded.
ProxyARP means to respond to ARP queries that aren't for the IP address in
question. In my setup, this means getting the 'G' system to respond to ARP
requests that arent for 10.0.0.5 or 10.0.1.5, but for 10.0.0.1 and 10.0.1.1.
To give you the basics:
- 'G' maintains the same config info for eth0 and eth1. ie to use a subnet
mask of 255.255.255.0, but we turn on proxyarp for both these interfaces
(more on this later).
- Both 'S' and 'C' have their netmasks changed to 255.255.0.0. They can keep
the same default router. But you can see that by changing to the netmask to
this value, both 'S' and 'C' would think they are on the same subnet.
So what happens when 'S' tries to send a packet to 'C'?:
- 'S' initially would send an ARP request saying 'Who is IP 10.0.1.1?'.
- 'G' has proxyarp configured on eth0 so it will say it is the machine
that has that IP address.
- 'S' now sends the packet to the eth0 interface on 'G'. Because of proxyarp,
'G' passes the packet through its eth1 interface out to 'C' and voila the
packet gets to its destination. The reverse happens for packets coming back.
Turning on proxyarp
For linux 2.2.x and beyond there is a configurable parameter in /proc.
eth0 /proc/sys/net/ipv4/conf/eth0/proxy_arp
eth1 /proc/sys/net/ipv4/conf/eth1/proxy_arp
or for all interfaces
all /proc/sys/net/ipv4/conf/all/proxy_arp
You just need to write a "1" to it to turn it on or a "0" to turn it off:
echo "1" >/proc/sys/net/ipv4/conf/eth0/proxy_arp
echo "1" >/proc/sys/net/ipv4/conf/eth1/proxy_arp
Jun 04,2001
|