As discussed at the Apr meeting, here is my solution for having squid cache your Windblows (XP, mainly but should work for Vista/7) updates downloads.
I have just verified this is still working at all my customer sites by looking at the squid access logs and seeing HITs for the big CAB files from windowsupdate.com. This confirms what I have always been seeing while manually doing updates.
The magic line you require in your squid.conf is: refresh_pattern http://www.download.windowsupdate.com/ 0 80% 525600 reload-into-ims (no wrapping on the above line)
If someone wants my entire squid.conf, let me know. It's setup for a pretty common case where you want to squid all of your internal (192.168.*) computers. The only other major thing is make sure you have a pretty big squid cache and max object size. I recommend at least 3GB. If you have tons of disk, make it 50G or something. More never hurts.
I use: cache_swap_low 94 cache_swap_high 97 maximum_object_size 800000 KB
Which should nicely handle the Win Update case we're trying to achieve.
Now, to make it so you don't have to configure ANYTHING on a stock WinXP box to make it use squid:
Make a file on your web server's document root (ie: /var/www/html or similar) called wpad.dat. Make it world readable. Populate it thusly:
<? header("Content-type: application/x-ns-proxy-autoconfig"); ?>
function FindProxyForURL(url, host) { if ( isPlainHostName(host) || dnsDomainIs (host,"tecnopolis.ca") || dnsDomainIs (host,"nocachingdomain.whatever") || shExpMatch (host,"192.168.*") || shExpMatch (url ,"*windowsupdate.microsoft.com/*.js*") || shExpMatch(url ,"*windowsupdate.microsoft.com/*.asp*") ) return "DIRECT";
if (url.substring(0, 5) == "http:" || url.substring(0, 4) == "ftp:" || url.substring(0, 7) == "gopher:") return "PROXY 192.168.1.1:8080; DIRECT";
return "DIRECT"; }
Tweak the DIRECT domain/IP exclusions in the first if set. Change the 192.168.1.1 to your squid server's IP. Change the 8080 to whatever port you want to have squid listen to. There's a more standard default port, but I use 8080 as that's what Shaw used when I signed up ages ago and it's easy to remember :-)
You also require some funky apache config to make it work: <VirtualHost *:80> ServerName wpad.tecnopolis.ca ServerAlias wpad DocumentRoot /var/www/html Alias /wpad.dat /var/www/html/wpad.dat </VirtualHost>
(may be a bit redundant, but I am editing my slightly more complex setup to be usable by others)
Note, wpad.dat only applies to IE (Internet Exploder) to make Nutscrape-based browsers (Fireflax) you need to make another identical file (hardlink?) called proxy.pac and do some more funky apache stuff:
<VirtualHost *:80> ServerName proxy.tecnopolis.ca ServerAlias proxy DocumentRoot /var/www/html Alias /wpad.dat /var/www/html/wpad.dat Alias / /var/www/html/proxy.pac </VirtualHost>
More redundancy, but trust me, getting this stuff to work is mystical voodoo so cover all your bases.
AND, obviously add proxy and wpad as CNAMEs in your DNS server's config! They must resolve to your web server's IP.
Lastly, XP by default has the "autodetect proxy settings" enabled in IE, so you literally have to do nothing to have your entire network magically start using the proxy. If it doesn't seem to be working, double-check that IE indeed has that option enabled (Tools, connections, lan settings).
Like I said, it's been 10 years since I set this up and last looked it it, so hopefully I'm not overlooking some important point. It literally has worked all these years without tweaking on dozens of servers. 3 cheers for UN*X and it's stability of configuration and migration!
In case one has some older versions of IE, sym-linking wpad.dat to wpad.da solve some of those corner cases.
Some versions of IE and XP (especially the freshly unpatched kind) have a bug in proxy detection where it doesn't calculate the size of the string properly and null-terminates it one character short which is why the last letter is chopped off of the end (bad coder, no cookie).
May not still be the case anymore but I remember back when we rolled out wpad it gave me a really big headache. :)
On 2010-04-15, at 1:30 AM, Trevor Cordes wrote:
As discussed at the Apr meeting, here is my solution for having squid cache your Windblows (XP, mainly but should work for Vista/7) updates downloads.
I have just verified this is still working at all my customer sites by looking at the squid access logs and seeing HITs for the big CAB files from windowsupdate.com. This confirms what I have always been seeing while manually doing updates.
The magic line you require in your squid.conf is: refresh_pattern http://www.download.windowsupdate.com/ 0 80% 525600 reload-into-ims (no wrapping on the above line)
If someone wants my entire squid.conf, let me know. It's setup for a pretty common case where you want to squid all of your internal (192.168.*) computers. The only other major thing is make sure you have a pretty big squid cache and max object size. I recommend at least 3GB. If you have tons of disk, make it 50G or something. More never hurts.
I use: cache_swap_low 94 cache_swap_high 97 maximum_object_size 800000 KB
Which should nicely handle the Win Update case we're trying to achieve.
Now, to make it so you don't have to configure ANYTHING on a stock WinXP box to make it use squid:
Make a file on your web server's document root (ie: /var/www/html or similar) called wpad.dat. Make it world readable. Populate it thusly:
<? header("Content-type: application/x-ns-proxy-autoconfig"); ?>
function FindProxyForURL(url, host) { if ( isPlainHostName(host) || dnsDomainIs (host,"tecnopolis.ca") || dnsDomainIs (host,"nocachingdomain.whatever") || shExpMatch (host,"192.168.*") || shExpMatch (url ,"*windowsupdate.microsoft.com/*.js*") || shExpMatch(url ,"*windowsupdate.microsoft.com/*.asp*") ) return "DIRECT";
if (url.substring(0, 5) == "http:" || url.substring(0, 4) == "ftp:" || url.substring(0, 7) == "gopher:") return "PROXY 192.168.1.1:8080; DIRECT";
return "DIRECT"; }
Tweak the DIRECT domain/IP exclusions in the first if set. Change the 192.168.1.1 to your squid server's IP. Change the 8080 to whatever port you want to have squid listen to. There's a more standard default port, but I use 8080 as that's what Shaw used when I signed up ages ago and it's easy to remember :-)
You also require some funky apache config to make it work: <VirtualHost *:80> ServerName wpad.tecnopolis.ca ServerAlias wpad DocumentRoot /var/www/html Alias /wpad.dat /var/www/html/wpad.dat
</VirtualHost>
(may be a bit redundant, but I am editing my slightly more complex setup to be usable by others)
Note, wpad.dat only applies to IE (Internet Exploder) to make Nutscrape-based browsers (Fireflax) you need to make another identical file (hardlink?) called proxy.pac and do some more funky apache stuff:
<VirtualHost *:80> ServerName proxy.tecnopolis.ca ServerAlias proxy DocumentRoot /var/www/html Alias /wpad.dat /var/www/html/wpad.dat Alias / /var/www/html/proxy.pac
</VirtualHost>
More redundancy, but trust me, getting this stuff to work is mystical voodoo so cover all your bases.
AND, obviously add proxy and wpad as CNAMEs in your DNS server's config! They must resolve to your web server's IP.
Lastly, XP by default has the "autodetect proxy settings" enabled in IE, so you literally have to do nothing to have your entire network magically start using the proxy. If it doesn't seem to be working, double-check that IE indeed has that option enabled (Tools, connections, lan settings).
Like I said, it's been 10 years since I set this up and last looked it it, so hopefully I'm not overlooking some important point. It literally has worked all these years without tweaking on dozens of servers. 3 cheers for UN*X and it's stability of configuration and migration! _______________________________________________ Roundtable mailing list Roundtable@muug.mb.ca http://www.muug.mb.ca/mailman/listinfo/roundtable
The organization I work for is wants to set up a proxy server for Windows updates. We work with older computers, XP and 2000. We have a server available, and a couple of us want to use Ubuntu server. The individual who suggested a proxy server also suggested Squid. Great ideas. Do you have any recommendations to setup Squid for Windows 2000 and version of IE that will run on it?
Trevor, you offered to send your entire squid.conf file; could I get that please?
Also, my college suggested getting software to make the server a router, while I had considered getting a hardware router and just configuring the server as a proxy server. Any recommendations about topology or software?
Thanks, Rob Dyck
Ps. I'm the guy who was trying to set up a server using CRUX. The problem was an error message "Spurious ACK on isa0060/serio0. Some program might be trying access hardware directly" causing the kernel to panic trap. Never was able to get past that. So I gave up, instead installed Ubuntu 9.10 server. It was smooth; from start of download to complete up-and-running system was a single day.
On 2010-04-15 Robert Dyck wrote:
The organization I work for is wants to set up a proxy server for Windows updates. We work with older computers, XP and 2000. We have a
The setup I suggest *should* also support Win2k ok. I *think* I remember it being ok back in the day when I had customers with w2k. For sure it supports wpad.dat, so as long as it hits windowsupdate.com for updates, it should work. Even w98 may work.
server available, and a couple of us want to use Ubuntu server. The
Any distro should work fine. I prefer Fedora, but that's just me!
Trevor, you offered to send your entire squid.conf file; could I get that please?
I will email it directly. Obviously it will have to be tweaked for your site.
Also, my college suggested getting software to make the server a router, while I had considered getting a hardware router and just configuring the server as a proxy server. Any recommendations about topology or software?
All my squid caches are also routers/firewalls. Seems silly to dedicate multiple boxes to what 1 can do quite well, but that's my personal preference. All my routers/firewalls are 100% custom. If you're looking for an inexpensive turnkey and 100% managed solution, let me know as that's what my company does, we can even use your old hardware (tailored for micro and small business).
On 2010-04-15, at 9:29 AM, Trevor Cordes wrote:
On 2010-04-15 Robert Dyck wrote:
The organization I work for is wants to set up a proxy server for Windows updates. We work with older computers, XP and 2000. We have a
The setup I suggest *should* also support Win2k ok. I *think* I remember it being ok back in the day when I had customers with w2k. For sure it supports wpad.dat, so as long as it hits windowsupdate.com for updates, it should work. Even w98 may work.
We're part of the same group (it was yours truly who brought up the original topic).
The thing is we can't really control what machines we get. Sometime it's XP and other times it's 2K. Some times we are lucky and the XP machines have been updated to the previous update. It doesn't look like we'll be getting Vista or W-7 licenses in the near future. The machines couldn't run it anyway.
In addition to the Windows updates we also have firewall, anti-virus, and anti-spyware. We also install a ZIP package. It used to be Winzip but we found 7-zip does more things. 7-zip installs directly from the net. The idea was we could knock off 45min - 1 hour per machine installation. The connection to the net gets pretty slow sometimes. That's another issue beyond our control.
server available, and a couple of us want to use Ubuntu server. The
Any distro should work fine. I prefer Fedora, but that's just me!
The thing is we are "lightweights" when it comes to how to install this sort of thing. The reason I suggested Ubuntu server is because it's available and it is a very quick install. I have an hour at the end of the day to do extra things around the lab so I figured we could do it a little at a time.
We don't actually have a web server. I mention it because previous discussion suggested it. The machine we have at the moment is a single FreeNAS box. We have hardware ready to go. Just no idea how to proceed.
Trevor, you offered to send your entire squid.conf file; could I get that please?
I will email it directly. Obviously it will have to be tweaked for your site.
Again, we are "lightweights". The more directions the better.
Also, my college suggested getting software to make the server a router, while I had considered getting a hardware router and just configuring the server as a proxy server. Any recommendations about topology or software?
All my squid caches are also routers/firewalls. Seems silly to dedicate multiple boxes to what 1 can do quite well, but that's my personal preference.
Here is our current setup... AC Router <--> CLL Hub <--> Various machines
We don't control the AC Router. We can put in requests but they are having problems with Barracuda at the moment so it will be quite a while before they can help out.
My thoughts are if we can make the whole process transparent to the machines being installed/repaired it would save us a bunch of time. To do this I figured we'd need a router between the incoming connection and the hub. OTOH, if the machine were on the other side of the hub we'd end up having to configure the machines and restore them when we were done.
This is what *I* think we'll have to do... AC Router <--> CLL Router/Squid <--> CLL Hub <--> Various machines
I don't know enough about Squid to know if we can get rid of the router part of the machine then have it sit between the AC Router and our machines. Our machines are generally set up for DHCP so my concern would be where we get the IPs from.
All my routers/firewalls are 100% custom. If you're looking for an inexpensive turnkey and 100% managed solution, let me know as that's what my company does, we can even use your old hardware (tailored for micro and small business).
Bear in mind this may or may not last a while. Given the games M$ seems to be playing with licenses it may come down to Linux (and old Macs) or nothing. AFAIC, that would be fine. Unfortunately the short sighted behaviour of many employers means unless we give out M$ machines the clients are mostly ignored. It would be very hard to justify coming up with money to pay someone to install it for us (I don't get paid and I'm the senior volunteer - I don't think the supervisor gets paid either). I think I could persuade them to buy you a lunch at Subway though... :-)
Later Mike
On 2010-04-15 Mike Pfaiffer wrote:
In addition to the Windows updates we also have firewall, anti-virus, and anti-spyware. We also install a ZIP package. It used
To save yourself time, you should do as someone mentioned and slipstream at minimum SP3 into your XP install CD. This sounds daunting, but I will give you my cheatsheet and you can do it in 1 page of commands without any thinking:
(install packages first: wine, convmv, cabextract; and find the script on the net called geteltorito, or ask me for it)
tcsh set workdir=/tmp/Slip set spfile="/tmp/WinXP_SP3.exe" set spdir=$workdir/sp set indir=$workdir/new set cdrom=/dev/sr0 set cdmnt=/media/cdrecorder
mkdir $workdir cd $workdir
cabextract -d $spdir "$spfile"
mkdir $indir
mount $cdrom $cdmnt
cp -r $cdmnt/* $indir chmod -R 777 $indir
wine $spdir/i386/update/update.exe -s:$indir
convmv -r --upper --notest $indir/*
geteltorito $cdrom > $indir/boot.bin
find $indir | xargs touch -t 200804140000
umount $cdmnt eject $cdrom
cd $indir mkisofs -b boot.bin -hide boot.bin -hide boot.catalog -no-emul-boot \ -boot-load-size 4 -iso-level 4 -relaxed-filenames -D -V GRTMPVOL_EN -o \ $workdir/iso .
cdrecord dev=$cdrom $workdir/iso eject $cdrom rm $workdir/iso
============
Doing the above (on linux) never ceases to amaze me (I've never used wine before).
The thing is we are "lightweights" when it comes to how to install this sort of thing. The reason I suggested Ubuntu server is
Nothing wrong with Ubuntu at all, just use what you're fastest with configuring, that's what I say. You get used to a distro's way of doing things (mostly file system layout and /etc arrangement), and for me that's RedHat 6 (c 1999) or so :-)
We don't control the AC Router. We can put in requests but they are having problems with Barracuda at the moment so it will be
If you don't control the router then probably best to do the router-behind-router idea you outlined, just make sure to pick a different local subnet IP range!
My thoughts are if we can make the whole process transparent to the machines being installed/repaired it would save us a bunch of time. To do this I figured we'd need a router between the incoming connection and the hub. OTOH, if the machine were on the other side of the hub we'd end up having to configure the machines and restore them when we were done.
For easiest transparency (no settings to be made on client), you want the wpad to work, which means you must control your DNS, web and squid server. If you can live with a 60sec tweak on every client (setting proxy manually) then you can do away with DNS and web and just setup a squid server that can be in your existing subnet (no need for another router).
This is what *I* think we'll have to do... AC Router <--> CLL Router/Squid <--> CLL Hub <--> Various machines
As above, best/transparent solution but a fair bit harder to initially implement.
I don't know enough about Squid to know if we can get rid of the router part of the machine then have it sit between the AC Router and our machines. Our machines are generally set up for DHCP so my concern would be where we get the IPs from.
If you did squid but no DNS/web then the squid server would just be another peer on the LAN, just another box off the main switch. Ideally you'd want to assign it a static IP.
out M$ machines the clients are mostly ignored. It would be very hard to justify coming up with money to pay someone to install it for us (I don't get paid and I'm the senior volunteer - I don't think the supervisor gets paid either). I think I could persuade them to buy you a lunch at Subway though... :-)
I hear ya. I'm a bit over-subscribed for work, but I could possibly squeeze in some moments here and there if you can arrange remote ssh access from my IP. Going onsite would be a royal pain given my schedule though. The roundtable (incl myself) is also here to help out with questions.
If you've never done any BIND config before, that will probably be the biggest challenge for you. The apache stuff should be fairly easy and the squid stuff extremely easy with my conf file.
On 2010-04-15, at 12:20 PM, Trevor Cordes wrote:
On 2010-04-15 Mike Pfaiffer wrote:
In addition to the Windows updates we also have firewall, anti-virus, and anti-spyware. We also install a ZIP package. It used
To save yourself time, you should do as someone mentioned and slipstream at minimum SP3 into your XP install CD. This sounds daunting, but I will give you my cheatsheet and you can do it in 1 page of commands without any thinking:
I think we may have SP3 already slipstreamed. I seem to recall something on a CD. I'll check it out when I get in to the lab in an hour.
(install packages first: wine, convmv, cabextract; and find the script on the net called geteltorito, or ask me for it)
tcsh set workdir=/tmp/Slip set spfile="/tmp/WinXP_SP3.exe" set spdir=$workdir/sp set indir=$workdir/new set cdrom=/dev/sr0 set cdmnt=/media/cdrecorder
mkdir $workdir cd $workdir
cabextract -d $spdir "$spfile"
mkdir $indir
mount $cdrom $cdmnt
cp -r $cdmnt/* $indir chmod -R 777 $indir
wine $spdir/i386/update/update.exe -s:$indir
convmv -r --upper --notest $indir/*
geteltorito $cdrom > $indir/boot.bin
find $indir | xargs touch -t 200804140000
umount $cdmnt eject $cdrom
cd $indir mkisofs -b boot.bin -hide boot.bin -hide boot.catalog -no-emul-boot \ -boot-load-size 4 -iso-level 4 -relaxed-filenames -D -V GRTMPVOL_EN -o \ $workdir/iso .
cdrecord dev=$cdrom $workdir/iso eject $cdrom rm $workdir/iso
============
Doing the above (on linux) never ceases to amaze me (I've never used wine before).
I had wine configured properly ONCE. It never worked for me again. I'll follow your directions when things get organised. Who knows... Lightning may strike twice. ;-)
The thing is we are "lightweights" when it comes to how to install this sort of thing. The reason I suggested Ubuntu server is
Nothing wrong with Ubuntu at all, just use what you're fastest with configuring, that's what I say. You get used to a distro's way of doing things (mostly file system layout and /etc arrangement), and for me that's RedHat 6 (c 1999) or so :-)
I like the text interface of Ubuntu server. I'm not sure I'd go with a GUI at this point. No sense in wasting clock cycles unless we have to.
We don't control the AC Router. We can put in requests but they are having problems with Barracuda at the moment so it will be
If you don't control the router then probably best to do the router-behind-router idea you outlined, just make sure to pick a different local subnet IP range!
Makes sense. We don't want to compete with the AC.
My thoughts are if we can make the whole process transparent to the machines being installed/repaired it would save us a bunch of time. To do this I figured we'd need a router between the incoming connection and the hub. OTOH, if the machine were on the other side of the hub we'd end up having to configure the machines and restore them when we were done.
For easiest transparency (no settings to be made on client), you want the wpad to work, which means you must control your DNS, web and squid server. If you can live with a 60sec tweak on every client (setting proxy manually) then you can do away with DNS and web and just setup a squid server that can be in your existing subnet (no need for another router).
I understood about 80% of this paragraph. I think I should be able to understand the rest when we actually start the process.
This is what *I* think we'll have to do... AC Router <--> CLL Router/Squid <--> CLL Hub <--> Various machines
As above, best/transparent solution but a fair bit harder to initially implement.
The advantages also include not having to set it up every time.
I don't know enough about Squid to know if we can get rid of the router part of the machine then have it sit between the AC Router and our machines. Our machines are generally set up for DHCP so my concern would be where we get the IPs from.
If you did squid but no DNS/web then the squid server would just be another peer on the LAN, just another box off the main switch. Ideally you'd want to assign it a static IP.
I thought this may be the case.
out M$ machines the clients are mostly ignored. It would be very hard to justify coming up with money to pay someone to install it for us (I don't get paid and I'm the senior volunteer - I don't think the supervisor gets paid either). I think I could persuade them to buy you a lunch at Subway though... :-)
I hear ya. I'm a bit over-subscribed for work, but I could possibly squeeze in some moments here and there if you can arrange remote ssh access from my IP. Going onsite would be a royal pain given my schedule though. The roundtable (incl myself) is also here to help out with questions.
I'll have to go through channels to ask. Given the AC has installed Barracuda my guess is they probably will say no.
We could arrange to have the machine dropped off to you somewhere if that would be any better...
Worst case would be we work on it an hour per week and report the progress back here.
Sounds like this has captured the imagination of the group. I wonder if it might be worth a future presentation... ;-)
If you've never done any BIND config before, that will probably be the biggest challenge for you. The apache stuff should be fairly easy and the squid stuff extremely easy with my conf file.
As I mentioned I am a lightweight with this. The whole computer area is very broad and this isn't an area I had experience with until very recently.
Later Mike
On 04/15/2010 01:30 AM, Trevor Cordes wrote:
AND, obviously add proxy and wpad as CNAMEs in your DNS server's config! They must resolve to your web server's IP.
oooh, I've never seen this before, I had thought you were talking about DHCP option 252 (DHCP server can tell the client the location of the wpad file to use). Doing it in DNS is interesting!
Then again, I know almost nothing about Windows :)
Peter
On 2010-04-15, at 3:26 PM, Peter O'Gorman wrote:
oooh, I've never seen this before, I had thought you were talking about DHCP option 252 (DHCP server can tell the client the location of the wpad file to use). Doing it in DNS is interesting!
Then again, I know almost nothing about Windows :)
I have never seen option 252 work. I think I have a scar from the wall I banged my head against due to it. I wish it did, work especially on non-windows hosts as forgetting to set http_proxy is something I ALWAYS forget to do.
On 04/15/2010 03:31 PM, Sean Cody wrote:
On 2010-04-15, at 3:26 PM, Peter O'Gorman wrote:
oooh, I've never seen this before, I had thought you were talking about DHCP option 252 (DHCP server can tell the client the location of the wpad file to use). Doing it in DNS is interesting!
Then again, I know almost nothing about Windows :)
I have never seen option 252 work. I think I have a scar from the wall I banged my head against due to it. I wish it did, work especially on non-windows hosts as forgetting to set http_proxy is something I ALWAYS forget to do.
Didn't work when I tried it either - it's just that that's what I thought Trevor was talking about.
Ah, and the DNS wpad thing works on Mac OS X 10.6 too (as long as the user has opened System Preferences, clicked on "Network" clicked on "Advanced..." then on "Proxies" and then checked the box "Auto Proxy Discovery"... sigh).
Peter
-
Mike Pfaiffer -
Peter O'Gorman -
Robert Dyck -
Sean Cody -
Trevor Cordes