Mar 03
Categories: linux, sysadmin Tags:

I run my NAS on an Intel SS4200-E [1] using Debian Stable. Since the SS4200 doesn’t have a graphics device, I use a serial console. Good set-up instructions can be found in the SS4200-E Wiki [2]. After the upgrade the kernel was not configured accordingly and I lost contact via the serial console. To complicate things, Squeeze upgrades the grub package to grub-pc (AKA Grub Version 2).

Here are some notes on how to fix this. The first step is to get the system to boot with a serial console by editing the kernel parameters. Use Grub to temporarily add these parameters to the: console=tty0 console=ttyS0,115200.

Once booted, make this permanent by adding the following to /boot/grub/grub.cfg.

GRUB_CMDLINE_LINUX_DEFAULT="quiet"
GRUB_CMDLINE_LINUX="all_generic_ide console=tty0 console=ttyS0,115200"
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200"

Run update-grub2 to apply the change. This will update the new Grub2 configuration in /boot/grub/grub.cfg. You may want to edit the old /boot/grub/menu.lst to add the same changes if you are still booting from the old menu.

[1] – http://www.intel.com/products/server/storage/ss4200/index.htm
[2] – http://ss4200.pbworks.com/w/page/5122751/FrontPage

No Comments
Nov 08
Categories: sysadmin Tags: , , ,

I recently noticed that the downstream bit-rate on my ADSL modem was fluctuating. This can happen because the noise on the copper pair between the house and the exchange can vary over time. There isn’t much you can do about that, but it is possible to somewhat influence the performance at the house end by reducing the length of the wire from the master socket to the modem, by using better quality wire and even by disconnecting the bell terminal at the master socket.

But before making any changes, I needed to be able to monitor the effects, so I thought this Muinin plug-in was in order. This plugin monitors the upstream and downstream bit-rates of my ZyXEL Prestige P-660HWP-D1 ADSL modem. I’m sure it’ll work with any Prestige modem. Here is an example plot.

Munin graph of modem bitrates

Munin graph of modem bitrates from zyxel_prestige_adsl_chandata plugin

You can get the script here: http://ccgi.ambrosia.plus.com/debian/zyxel_prestige_adsl_chandata. See the documentation therein for set-up instructions.

You can see the bitrate drop twice during Friday evening. On Saturday morning I moved the modem from an extension socket to the master socket. Initial indications were that I had improved the downstream bitrate significantly. However by the evening you can see that it subsequently dropped back to a rate slightly lower than at the same time 24 hours earlier.

No Comments
Oct 02
Categories: sysadmin Tags: , ,

How Shorewall Can Break OpenVPN

I’ve been administering OpenVPN for years now, and I’ve been quite happy with it. For the most part it’s given me very little grief. The only frequent problem comes from users with a personal firewall who unwittingly block all VPN traffic.

I’ve recently upgraded to version 2.0.9, without memorable incident. It has caused me, however, to revisit the problem I had with the old installation concerning MTUs. I had found — as have many others out there — that the only way to get a reliable tunnel over UDP was to fiddle with the various MTU and fragmentation parameters, settling on a configuration that apparently worked but without the satisfaction of understanding why.

This time round I left all that mssfix and fragment guff out of the configuration to see what would happen. Maybe it’s not necessary any more? Well apparently it won’t go away that easily. Once again I find that an OpenVPN client can connect reliably but cannot necessarily get any traffic through the tunnel. In fact it depends on the client: most users don’t have a problem, but one usually does. The distinguishing features of this client is that it’s across the pond in the US and that it’s connecting through a cable router (PPPoE). The evidence of the problem appears in the OpenVPN log like this:

Wed Oct  1 17:25:49 2008 69.19.60.64:33041 [fred] Peer Connection Initiated with 69.19.60.64:33041
Wed Oct  1 17:25:50 2008 fred/69.19.60.64:33041 PUSH: Received control message: 'PUSH_REQUEST'
Wed Oct  1 17:25:50 2008 fred/69.19.60.64:33041 SENT CONTROL [fred] 'PUSH_REPLY,route-gateway 10.0.10.254 255.255.255.0,route 192.168.0.0 255.255.255.0,route 192.168.10.0 255.255.255.0,route 192.168.150.1 255.255.255.255,route 172.16.0.0 255.255.255.0,route 172.17.0.0 255.255.255.0,ping 10,ping-restart 120' (status=1)
Wed Oct  1 17:31:21 2008 read UDPv4 [ECONNREFUSED]: Connection refused (code=61)
Wed Oct  1 17:31:31 2008 read UDPv4 [ECONNREFUSED]: Connection refused (code=61)
Wed Oct  1 17:31:41 2008 read UDPv4 [ECONNREFUSED]: Connection refused (code=61)
.
.
.
Wed Oct  1 17:34:15 2008 read UDPv4 [ECONNREFUSED]: Connection refused (code=61)
Wed Oct  1 17:34:26 2008 read UDPv4 [ECONNREFUSED]: Connection refused (code=61)
Wed Oct  1 17:34:35 2008 read UDPv4 [ECONNREFUSED]: Connection refused (code=61)
Wed Oct  1 17:34:45 2008 NOTE: --mute triggered...
Wed Oct  1 17:35:11 2008 fred/69.19.60.64:33041 3 variation(s) on previous 20 message(s) suppressed by --mute
Wed Oct  1 17:35:11 2008 fred/69.19.60.64:33041 [fred] Inactivity timeout (--ping-restart), restarting
Wed Oct  1 17:35:11 2008 fred/69.19.60.64:33041 SIGUSR1[soft,ping-restart] received, client-instance restarting

After some fresh research I now have a better grasp of what might be going on. Surprisingly, it would appear to be my Shorewall set-up that’s to blame.

Here’s the theory: the path between the client and the server has a reduced MTU (at the cable modem at the least) and that the path MTU discovery is not effective, causing the OpenVPN server to be unaware that its packets are not reaching the client. Why? Because the path MTU discovery is effected by sending an ICMP type 3 datagram from the node with the reduced MTU back to the sending server, and the example Shorewall configuration that I started from blocks all incoming ICMP except ping!

The example configuration I started from is in examples/three-interfaces/ in the distribution. The relevant section is as follows.

Ping/ACCEPT     loc             $FW
Ping/ACCEPT     dmz             $FW
Ping/ACCEPT     loc             dmz
Ping/ACCEPT     dmz             loc
Ping/ACCEPT     dmz             net

ACCEPT          $FW             net             icmp
ACCEPT          $FW             loc             icmp
ACCEPT          $FW             dmz             icmp

The default policy for net->$FW is DROP, so the configuration above does not permit any incoming ICMP packets except PING. Unless there’s some back-door exception for ICMP type 3 that I can’t see, the path MTU discovery will be broken by this configuration.

References

[1] – PMTU (Path MTU) Discovery – http://www.netheaven.com/pmtu.html

[2] – OpenVPN FAQ – http://openvpn.net/index.php/documentation/faq.html#mtu

No Comments
Sep 17
Categories: sysadmin Tags: , ,

I’ve been having intermittent problems with my internet access for a while (who hasn’t?) where everything would be really slow for a day or so and then it would mysteriously recover. I’d kick the modem, the squid cache, the local caching DNS server, etc but I was never sure where the problem lay.

Recently it got so bad that I investigated a bit harder. When it’s all gone bad (which is most of the time now, is seems) It looks like my DNS queries are transmitted as expected but the replies are never forthcoming. All the while I have solid connectivity. I can even flood ping the DNS servers that are not responding!

Before I go hassle my ISP, I thought I’d collect some evidence. Hopefully this will help me bypass the stupid “have you tried rebooting?” questions. So I’ve written a Munin plugin which I will configure to poll my ISP’s DNS servers. I expect that this will show that I frequently don’t get a response, in the evenings at least; I wonder if their traffic shaping is broken…

You can find the plugin at http://ccgi.ambrosia.plus.com/debian/dnsresponsetime. See the documentation therein.

Example daily plot from Munin plugin dnsresponsetime

Example daily plot from Munin plugin dnsresponsetime

No Comments