Home Graffiti About Sitemap Visual DevWork Email Brian GPG Key

GRAFFITI -- May 03, 2004 thru May 09, 2004 >> Link to the Current Week << Last Week << Mon   Tues   Wed   Thu   Fri   Sat   Sun >> Next Week Search this site : Welcome to Orb Graffiti, a place for me to write daily about life and computers. Contrary to popular belief, the two are not interchangeable.     About eMail - I publish email sometimes. If you send me an email and you want privacy or anonymity, please say so clearly at the beginning of your message.

MONDAY    Tues    Wed    Thu    Fri    Sat    Sun   
May 3, 2004

1859 - Good evening. A busy day, but fruitful. At work, some problem systems were brought into the fold, and mostly working right, it seems. Then, on the home front, you can see part of my Saturday's work pictured at left. That is the last 10 bags of boxwood hedges, which I finally got around to cutting up into small enough bits to bag for the green pickup tomorrow morning. I thought originally that I'd gotten about 2/3 done, when I did 10 bags worth on the first go-round. But now I know, it was just half. Anyway, what I was reminded of as I snapped that picture from an upstairs window was the scene in The Hobbit, with all the dwarves bagged up and waiting to be Troll supper. I know, it's not a perfect match - it was a wooded night, not a manicured rainy day, but the image persists in my mind. So I've got those out by the curb, and cleaned out a gutter that got clogged up with crud during yesterday's rains. Whew, another long day done, and it's time for supper.

After dinner, I'll do a bit more work on the Mailbox build, and see what progress I can make. If there's useful output from that effort, then you'll see it tomorrow morning. Now to find some food - have a great evening.

Top  /  Email Brian

Mon    TUESDAY    Wed    Thu    Fri    Sat    Sun   
May 4, 2004

0709 - Good morning. Wow! I mean, I knew it already, but a third of this year's gone already, winter is done, and I hardly even noticed most of the individual days. April was a light month around this site, with 981 average page reads per day (the annualized average is 1091 pages/day) However, I continue to be awed and honored by each and every one of you that comes by - there were an average 531 visits per day, from a total 10334 sites. And that doesn't count places like AOL that proxy web pages. Thank you, thank you, thank you for your patronage, your kind words and your patience. I am especially grateful for the outpouring of sympathy that both Marcia and I received when we had to send our little Sally on ahead, back in March.

In April, I had visitors on this site from 91 "countries" including the dotCOM, the dotNET, the unresolved, and even a few from far-off Arpanet, along with a great number of more terrestrial places, from Canada near the top of the pops, to Bulgaria, one of three countries with just one hit. Thanks to each and every one. Popular search terms start with the ever-popular (although certainly disappointing to most searchers for) "grafitti". There's "exotic fish" this time, which I didn't remember writing about (and in fact I didn't. Fish feature here from time to time, either in the Fish Heads lyrics, or some other way, and I mentioned the Exotic Erotic Ball at least once, but neither on the same page. Hmmm.) That old Chevy Cavalier continues to draw high, as does any mention of my third cousin, that talented quarterback for the Patriots. Overall, it's a pretty eclectic list of search terms that land people on my site.

Again, Thank YOU! I am honored to be your host.

Building a Mailbox with Debian, using Postfix with AV and UCE controls, Part the Second. Remember the baseline system: I'm running on VMware 4.5.1 as a "machine", with the following goals: Goals: Mailserver (SMTP, POP3, IMAP, AV, AS) Postfix Cyrus suite Amavisd Spamassassin ClamAV Web Server (WWW, PHP, SQL) Apache 2.x PHP4 MySQL Nameserver (DNS) Bind 9.3 At the end of the last article, I'd gotten Debian Sarge installed via the Beta 3 installer (now the 4th is out, can't keep up with reality). Postfix is also installed, and when I boot the system, there are currently only two ports open, 22 (for SSH) and 25 (for the MTA, Postfix in this case)... Sidetrip to Email Terminology - I just mentioned the acronym MTA. That stands for Mail Transport Agent. An MTA is an email server for transporting message from one machine to another over a LAN, WAN or the INTERNET. The protocol spoken between MTAs is SMTP, which stands for Simple Mail Transfer Protocol. The original definition of SMTP was written by the late Jon Postel in RFC 821 (http://www.ietf.org/rfc/rfc821.txt). Popular and well-known MTA servers include Sendmail, Qmail, Exim and Postfix. Of course, there's always Exchange... <me ducks and hides> Once received on a destination box, mail is (sometimes filtered, then) handed off to an MDA, or Mail Delivery Agent. The MDA receives the message from the MTA, and either places it into the user's mailbox on the local machine, or keeps in in some other type of store waiting for the user to pick it up. The single most common local machine MDA is procmail. For machines that store mail waiting for remote connections from individual users to pick up the mail, there are many program/suites that handle MDA duties and a lot more (we'll get there in a minute). These are programs like Courier-IMAP, Cyrus-IMAP and others. When I log onto a machine where local mail is stored for me (as with a few of the servers that I monitor), or if I'm checking my mail remotely, then I use an MUA, or Mail User Agent. These range from Mutt and Pine on the Linux command line, to the recent Thunderbird, Evolution, Kmail, Balsa, Netscape Messenger, Outlook, LookOut Express and many others. (I can't in good conscience recommend either of the latter two from that list, as there are new vulnerabilities seemingly every other week, some of which compromise a machine irretreivably.) An MUA may simply connect to a local mailstore in /var/spool/mail/USERNAME, or connect to a remote machine via the POP3 or IMAP protocols. POP3 (Post Office Protocol, Version 3, RFC 1939) is the client connection protocol most users are familiar with. POP3 clients generally download their mail from a remote server onto the local machine once (although there are provisions for saving messages on the server for later retrieval from another location). IMAP (Internet Message Access Protocol - Version 4 rev1, RFC 2060) is another method of connecting to a remote message store: one in which the messages are stored on the server, folders can be maintained and created on the server, and messages can be downloaded to the local machine by moving them from the remote store to the local folders. Our choices for MTA and MDA in this project, respectively, are Postfix and Cyrus-IMAP. Don't be alarmed, Cyrus does POP3 too, and SSL-enabled versions of both POP3 and IMAP to boot. Now, on with the show... Getting the (some of) rest of the software... Now, let's take a minute to learn a little bit about package management under Debian. The primary commandline tool is called apt-get. As you saw when I installed Postfix in the previous article, I simply typed (in the generalized form) apt-get install [package-name ...] and Debian uses the package repository information to determine any dependencies, asks permission to install them all (if the list is greater than the requested item(s)), fetches the pre-compiled binary packages from the specified Debian repositories, pre-configures, unpacks, installs, and post-configures all the packages. Finally, the software names are added to the list of installed binary packages on the system. Questions that might arise from the previous paragraph include things like: What specified Debian repositories? Okay, those were selected once, during the installation. If you find a faster one, or you choose to do some interesting things with your debian (like use special non-official Debian packages), then you'll need to edit the file /etc/apt/sources. Another question might be how to list the installed packages: Well, to get to that, we've got to step back in time a bit and use the reptilian hindbrain of the Debian package system, dpkg. Apt, though still a CLI (Command Line Interface) tool, is frosting on the dpkg cake. Apt makes debian packages simple for the neophyte and for those just wanting to get the job done. To get into the nitty and the gritty (some might say grotty) of packages, dpgk is your ... um, tool. Detail in intensity, but ... I digress. I use dpkg to list installed packages. In this case, I'll see if I've got Postfix installed. (I do, but...) mailbox:~# dpkg -l postfix Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err:uppercase=bad) ||/ Name Version Description +++-==============-==============-============================================ ii postfix 2.0.16-4 A high-performance mail transport agent Yes, that's nice, but didn't I install something else related to Postfix at the same time? Using the * to wildcard my query yields more results: mailbox:~# dpkg -l postfix* Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Installed/Config-files/Unpacked/Failed-config/Half-installed |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad) ||/ Name Version Description +++-==============-==============-============================================ ii postfix 2.0.16-4 A high-performance mail transport agent un postfix-dev (no description available) un postfix-doc (no description available) un postfix-ldap (no description available) ii postfix-mysql 2.0.16-4 MYSQL map support for Postfix ii postfix-pcre 2.0.16-4 PCRE map support for Postfix un postfix-pgsql (no description available) un postfix-snap (no description available) un postfix-snap-t (no description available) un postfix-tls (no description available) Ah, yes, postfix-mysql and postfix-pcre. There are other packages available for Postfix, I'll just get the -doc package for now. I'll explain why in a minute. Now this is what a clean single-package install looks like: mailbox:~# apt-get install postfix-doc Reading Package Lists... Done Building Dependency Tree... Done The following NEW packages will be installed: postfix-doc 0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded. Need to get 379kB of archives. After unpacking 1241kB of additional disk space will be used. Get:1 http://mirrors.rcn.net testing/main postfix-doc 2.0.16-4 [379kB] Fetched 379kB in 0s (402kB/s) Selecting previously deselected package postfix-doc. (Reading database ... 20184 files and directories currently installed.) Unpacking postfix-doc (from .../postfix-doc_2.0.16-4_all.deb) ... Setting up postfix-doc (2.0.16-4) ... That was a breeze. Now, you're going to be configuring and running an Internet-connected mail server, eh? Go dive into that documentation and read it. Not everything will make sense right now, but the only way to build the framework necessary to hang knowledge in the right places is to dive in and start reading. Want to know more about Apt? mailbox:~# man apt-get How about dpkg? (or more specifically, using the dpkq query capabilities): mailbox:~# man dpkg-query And now, go read those Postfix docs, if you can find them. Hmmm, don't know where to look, eh? I could guess ... probably in /usr/share/doc/postfix*/. But let's be sure. Did you just read that dpkg-query manpage? Then you already know how to query which files were installed by a specific package: mailbox:~# dpkg-query -L postfix-doc /. /usr /usr/share /usr/share/doc /usr/share/doc/postfix /usr/share/doc/postfix/html /usr/share/doc/postfix/html/access.5.html /usr/share/doc/postfix/html/aliases.5.html /usr/share/doc/postfix/html/architecture.html . . . /usr/share/doc/postfix-tls/ACKNOWLEDGEMENTS /usr/share/doc/postfix-tls/changelog.gz /usr/share/doc-base /usr/share/doc-base/postfix On through all 157 lines of output for that particular package (some are MUCH larger). So I was right. Now go read the Postfix docs. Then Part 3 of this series might start to make sense. I know, I said I'd install more packages today, and you only got one. But this is important, so the plan changed. See you back here soon...

Top  /  Email Brian

Mon    Tues    WEDNESDAY    Thu    Fri    Sat    Sun   
May 5, 2004

0626 - Good morning. Sasser claims networks, while the MS patch claims systems. Sheesh. None of my clients are directly affected, because the patches went in without a hitch, but still... Anyway, I've got to roll, so I'll post more later. I've a new encryption key, click the GPG Key link above, get that key and put it in your key ring in place of the old one. More on that story later, too. Byeeee.

Top  /  Email Brian

Mon    Tues    Wed    THURSDAY    Fri    Sat    Sun   
May 6, 2004

0727 - Good morning. Another day got past me. Of significance? She said it would be boring and she'd be home early. But the phone rang at 7 and Marcia was having a great time out with the girls (and a couple of the guys) from work - someone was up from another office and it was time for some quality networking time - she didn't get home until 11:40 and had a good time, after all. Wooo! And tonight they're coming here... um... okay. <grin> I guess that's just a larger audience to inflict my peculiar sense of humor upon.

While she played, I downloaded Knoppix 3.4 and started learning how to remaster it for both general knowledge purposes and to meet the needs of a specific client. I configured a dedicated splash page for new users, and set it up to ignore the DHCP stuff and use a pre-assigned IP address. That latter part didn't work in testing - I still had to type ifup eth0 after the boot was complete to the the address set properly, so I've got to find yet another config file and edit it properly. Soon.

Today, Larry and I are doing the initial assessment and documentation routine at a new client site that I'll be minding for several hours each month hereafter. This is always a fun time - lots to learn, new faces to meet, and a network, servers and systems to bring into line. I'd best be about it. I'll try to dance faster...

Top  /  Email Brian

Mon    Tues    Wed    Thu    FRIDAY    Sat    Sun   
May 7, 2004

0643 - Good morning. Up late, as Marcia's friend and cow orker from the Atlanta office was here and visiting. Up early as we have a busy day of backups to be done to a DG (that's Data General) Eclipse. Nope, there aren't any tubes in that model, but it's close, close. Now I must fly - see ya!

2237 - Hullo. The disk backups of the DG went well, but the day was downhill, other than that. Rocket, the box this site and several others is hosted on is dying. We're not sure how, and Rackshack tech support had several ideas, none of which seemed appropriate to the situation. Instead, Greg picked up a new box there, today. He was a busy beaver on it, and I joined in when I got out of the commute traffic, at about 5. We're configuring email, webs, mysql, lots of stuff. Notices out to customers, and reboots to custom kernels that fail to boot. It's not been a fun day, and it has been a long week. So we've stopped for the night, and you will probably see this. Rocket comes back to life each time after rebooting. I suspect power problems, but it'll be good to get away from RH7.1. It's totally unsupported now. The new box is RHEL3, and seems to be pretty stable, although I wish I knew why our custom kernel won't boot. We can't get enough of the console data to discern the problem. More when we know more, and if the place goes dark, hang tough, we'll be back on the air soonest. But I hope for a seamless transition. Now g'night.

Top  /  Email Brian

Mon    Tues    Wed    Thu    Fri    SATURDAY    Sun   
May 8, 2004

0822 - Good morning. We had a nice lightning and thunder show going on outside, while I worked here and Greg worked there, configuring and testing as we bring up a new box to replace a failing Rocket. Yep, if you missed last night's mini-post, Rocket is an unhappy camper. I just logged onto him for the first time this morning, a few minutes ago, and found his uptime to be something less than an hour. The system is clean, to the best of our ability to detect - that is, no interlopers have been found. We suspect hardware issues, but the range of things that the Rackshack techs will do for us is kind of limited, so we're limping along while we bring a new box up, transfer and update configs, copy data over, and generally have a totally unplanned weekend.

I was up at the crack of 0715, made some coffee, then gave Lucy a bath. She's pretty good about being calm during a bath, too, which is pleasant. In just a bit, when I've finished my coffee, I'll give the back yard it's weekly haircut, then it's back to the keyboard with me - Greg and I are starting up officially at around 10. Now to go test a thing or two... Have a great day!

Top  /  Email Brian

Mon    Tues    Wed    Thu    Fri    Sat    SUNDAY  
May 9, 2004

0904 - Good morning. We rolled DNS over to the new machine, named Zidane, at about 1610 yesterday afternoon. Since we'd set the $TTL down to 5 minutes the previous night, the transition started happening almost immediately... There was a lot of work that lead up to that point. We configured and tested and broke email several times in the process of bringing up the new system. You'd have thought it would be easy, a straight-across transfer of data and configs, then just update the IP addresses, right? Well, for webs and DNS, that was mostly true, though we needed to create a new SSL certificate for https access (that's working now too, I just made and tested that a few minutes ago.

But email was different. Greg and I had long talked about how we'd change our email setup to be much easier to administer, if only we had the time and opportunity to effect such a drastic change. Previously we were handling the virtual users in a mysql table, which allowed for easy modification of accepted users. But authentication for sending and receiving was handled by the native Cyrus SASL tools ... from the command line. So creating a new forwarding alias was easy - we only had to edit that virtuals table. However, adding a new real user, with mailbox and all, required an entry in that users table as well as several commands as the cyrus user, followed by copying the resulting authentication database from it's normal home in /etc into the chroot'd environment of /var/postfix/etc. That's a lot of manual work, for first time setup, and making changes was just as fun. Increasing someone's mail quota required a return to the command line.

Now Greg and I can do THAT all day long. But it was a process/interface that we could never delegate by domain to customers - too many fiddley bits to go wrong. Now we're using Web-cyradm - all user data including authentication is stored in MySQL tables, and the interface allows for per-domain delegation of authority to domain master admins. That rocks. But it meant major changes in configs and mail-handling that we were fighting with almost from the beginning of bringing the box up. We'd make a little change, test, figure out what was broken, fix that, and start again. Yes, there are Howto documents, and they are pretty good - they get an admin well within the ballpark, to the point where debugging is possible. But there are so many interlocking config files that it just takes time and tweaking. Greg did the bulk of that scutwork, and did a great job at it.

So the server came online at about 1615 for the world, and then we worked with Bob Thompson to shake the last kinks out of the email system - it took a couple of hours before everything seemed right. Shortly thereafter I called Jerry Pournelle (yeah, we pre-warned him about outages and config changes to come) and walked him through re-configuring his mail client, and most importantly, warning him that FrontPage would probably want to republish his whole web: all 265M of it. Yeah, we rsynced the whole shebang over from Rocket. But to FrontPage, it won't matter, this is a new machine, I'd better copy everything again. Bah! We'll see what comes, but getting updates online using a standard FTP client worked for Jerry last night, sending and receiving email worked fine. Marcia's email is reconfigured and she can publish...

Yes, Marcia's posting again. After a busy, busy week, Marcia finally got her Musings page caught up to date. Yay!

Back to the box... Last night Greg created a tool for users to modify their own passwords (for email - these are not shell accounts). Then we called it a night. I went downstairs for about 5 minutes, then came back up, installed Webalizer and tested it against my logs. Ooops. When we did the final rsync of the webs during DNS rollover, we forgot to stop and restart Apache. About 6 hours of web traffic wasn't logged - a small price to pay for transition to a new box, and on Saturday (traditionally the lowest traffic day, as the whole world is in weekend that day). Then I installed Mailman, and tried to figure out how I wanted to change things for that, going forward. The installation previously was nasty and fraught with peril - multiple installations into user subdirectories to cope with the conflicting security models of Mailman and Apache's Suexec. So I've side-stepped that altogether this time. More on that later. I nudged things here and poked them there until it all came up and the first list was getting mail out. Then I called it a night, shortly after midnight.

So far this morning, I've got the new SSL certificate created and in play, modified how I was talking to Mailman via the web, and organized a couple of small things. Now I've got to grab a shower and then we'll do the sunday shopping. Then back here to help Greg finish configuring the box, for logging, monitoring, etc, etc. So it's time for you to have a lovely day and me to get to work. Byeeeeee?

Top  /  Email Brian

Last Week << Mon   Tues   Wed   Thu   Fri   Sat   Sun >> Next Week

Visit the rest of the DAYNOTES GANG, a collection of bright minds and sharp wits. Really, I don't know why they tolerate me <grin>. My personal inspiration for these pages is Dr. Jerry Pournelle. I am also indebted to Bob Thompson and Tom Syroid for their patience, guidance and feedback. Of course, I am sustained by and beholden to my lovely wife, Marcia. You can find her online too, at http://www.dutchgirl.net/. Thanks for dropping by.

All Content Copyright © 1999-2011 Brian P. Bilbrey.


Except where otherwise noted, this site is licensed under the
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.