Hardware

(MCE 2005) Media Center PC Experience, Feb 2005

Internet

Why RBLs are out of control, and challenge/response systems

Yahoo will not let me have my own name back

NetGear Routers and DNS

I have a WGR614 Netgear WiFi NAT/router. Some DNS queries against my domain were failing:

dig @kelvinist.com www.kelvinist.com-- this worked
dig @kelvinist.com kelvinist.com -- this failed, with "server unreachable"

Other domains that I hosted had similar problems. At first the behavior was erratic, but that was mostly a matter of not using the "@" option and seeing irregular caching effects from different machines. That is, most external email kept working, but some started failing.

Running namedin foreground debug mode showed that both queries were reaching the server and it was responding in similar ways. This suggested the that packets being sent by named were not reaching the requesting machine.

I'm not clear on why it works this way, but apparently there is more outgoing UDP traffic on the second-level lookup. For some reason the NetGear blocks this outgoing traffic, but passes that for the third-level domain request. I could not succeed in getting a "port trigger" for port 53 to solve the problem. Instead that blocked original outgoing DNS requests (e.g. dig google.com from my linux server then fails on the forwarded DNS request). Instead, the fix was to "Disable SPI Firewall" under "WAN setup".

Unix use

PATH compression

Multiple shell scripts for setting up environment variables under different situations often leaves the PATH variable with duplicate entries, possibly overflowing the limit, and certainly becoming distracting. The following Perl script wrapped up as an alias reduces the PATH variable to unique instances, e.g. \a:\b:\a becomes \a:\b (preserving the order):

alias compress 'perl -e '"'"'$_=shift; split(/:/); foreach(@_) { $dir{$_} || ($out.="$_:"); $dir{$_}=1; } chop $out; print $out;'"'"

It can be invoked as:

setenv PATH `compress $PATH`

NT

Process priority

To keep GUI tasks (e.g. email and browser) responsive while running compute bound tasks, launch a cmd.exe shell in low priority mode and then all apps launched from it will be low priority (idle), and not distract noticeably from foreground tasks. This can be accomplished by creating a cmd.exe shortcut with the following start-up command:

%SystemRoot%\system32\cmd.exe /c start /low /b

Filename Completion

To get windows to complete filenames with the tab key, add the following to the registry

Windows Registry Editor Version 5.00
[HKEY_USERS\.DEFAULT\Software\Microsoft\Command Processor]
"CompletionChar"=dword:00000009

[HKEY_CURRENT_USER\Software\Microsoft\Command Processor]
"CompletionChar"=dword:00000009

TiVo upgrade experiences

How to upgrade your TiVo is covered many places (example). Here are my experiences with what can go wrong.

First TiVo upgrade: A Series 1 Philips DirecTiVo (model # no longer available), built circa 2001. This unit had space for two drives. I applied BlessTivo to a new drive and installed it. All went well. But, 18 months later, the unit failed completely. There was no video output at all, nor any sign that it was booting. I suspected the second drive had finally ruined the power supply. But, I could hear the drive spinning up (I think). It never started stepping though, so it was not even trying to boot.

Second TiVo upgrade: A Series 1 RCA Direct TiVo. I was expecting to add another drive, but upon opening it there was no slot to hold a second drive. It was a standard sized case (like the Philips). It might have been possible to modify the bracket in the RCA and mount both drives on top of each other, but I didn't do that. Instead, I used ''dd" to backup the 40G Fireball to a 250G drive. The Linux boot only recognized 127G as expected. "dd" copied 32G without apparent error. After that, I ran "mfsadd -x". It reported increasing the size to 127G (nearly).

However, on boot, there were two problems, about half of the shows were missing. They were present in the list of recorded shows, but attempting to view them produced an error saying they were not actually recorded. I had viewed those shows before the backup, so they were indeed there before the backup. Second, the menu background was gone. The menus all worked, but they simply appeared over the TV show, without the blue background.

Third TiVo upgrade: Another Series 1 RCA Direct TiVo, model DVR-40 (replacing the dead Philips). The internal arrangement was the same as the previous one. This time, I used
# mfsbackup -aqo - /dev/hdc | mfsrestore -xpi - /dev/hdd
to do the backup. That worked with no glitches.

Linux (Red Hat) server

Installing sqlgrey 1.6.7 on postfix

SQLgrey is a greylisting engine or the postfix MTA. The installation instructions on the web page were inadequate and the ones in the download were scattered about. Here is my summary to help other people running their own linux servers:

perl -MCPAN -e shell; install Bundle::CPAN // to get perl-DBI
also need DBD:MySQL, Net::Server, & IO::Multiplex modules
i) install IO::Multiplex from CPAN prompt failed. Instead,
ii) from CPAN prompt> make IO::Multiplex
iii) from IO::Multiplex download directory>make; make install
iv) Do the same for Net::Server::Multiplex
service mysqld; and use GUI applet service config to start mysqld every time
mysql ; > CREATE DATABASE sqlgrey; > GRANT ALL ON sqlgrey.* TO sqlgrey@localhost;
don't bother: rpmbuild -ta sqlgrey-1.6.7.tar.bz2 seems to build into /var/tmp/sqlgrey-1.6.7-build
but not to install anything
groupadd sqlgrey; adduser -g sqlgrey sqlgrey
make; make install

vi /etc/sqlgrey/sqlgrey.conf, first pass, the following need to be set:

conf_dir = /etc/sqlgrey
loglevel =3 # for early testing
user = sqlgrey
group = sqlgrey
db_type = mysql
pidfile = /var/run/sqlgrey.pid
prepend = 1
whitelists_host = sqlgrey.bouton.name
admin_mail = some real email address

add
*.groups.yahoo.com
*.yahoogroups.com
(rumored not to resend emails) to
/etc/sqlgrey/clients_fqdn_whitelist.local

Start by adding check_policy_service after reject_unauth_destination in /etc/postfix/main.cf :

smtpd_recipient_restrictions =
...
reject_unauth_destination
check_policy_service inet:127.0.0.1:2501

be careful not to put it after permit_auth_destination, or other early permit options.

Add to boot:

cd /etc/rc5.d # make install already created /etc/rc.d/init.d/sqlgrey
ln -s ../init.d/sqlgrey S65sqlgrey

download and install: http://www.vanheusden.com/sgwi/ (the web UI)

tar -xzf sqlgreywebinterface-0.6.tgz
vi config.inc.php; change db_pass to ""
mv sqlgreywebinterface-0.6 /var/www/cgi-bin/sqlgrey -- manual install in web hierarchy

in conf.d: 
<Directory "/var/www/cgi-bin/sqlgrey">
AllowOverride All
</Directory>
in /var/www/cgi-bin/sqlgrey/.htaccess:
AuthUserFile /etc/httpd/passwords
AuthGroupFile /dev/null
AuthType Basic
AuthName "Sqlgrey"
<Limit GET>
require user ervan
</Limit>

Important usage notes:
- /etc/sqlgrey/* -- all config info
- /var/log/maillog -- sqlgrey log info, if run in daemon mode
- /usr/sbin/update_sqlgrey_config -- updates global whitelist?
- service sqlgrey restart -- like other servicess
- /usr/bin/sqlgrey-logstats.pl < /var/log/maillog -- summary of what has happened but does not count user whitelist matches anywhere in output
- loglevel seems permanently set at 2, regardless of config
- optin/optout -- apparently only for multiple users on one system, i.e. disable greylisting for some recipients, not useful for single server
Serious weakness: when someone relays email (either an MX backup or a forwarding service) to you, it keeps trying, thus any spam that goes first to the relay will automatically pass the greylist test due to the relay's MTA retrying. MX backup relays just also use greylisting. Greylisting provides no protection from other forwarding services.

Spamassassin AWL failure on spammers spoofing From: address

Spammers spoof the from domain in the SMTP conversation (thus making the Received: line wrong) and the From: address in the header so as to trick Spamasassin's AWL test into thinking the email came from yourself. This either lets spam through or (with a high AWL) causes self email to not get through.

Every negative failure (spam getting thru) of spamassassin I have seen recently is of this form. Oddly, spamassassin seems to use this textual information rather than the hard IP (anybody know better?). Here is an elided example of such a spam:

  Received: from kelvinist.com (24-119-134-60.cpe.cableone.net [24.119.134.60])
  X-Mail-Format-Warning: Bad RFC2822 header formatting in by kelvinist.com
  (Postfix) with ESMTP id D991441C7
  From: "kevin" <_myname_@kelvinist.com>
  To: <_myname_@kelvinist.com>
  Subject: Come as much as a pr0n star! upto 500% more!
  X-Spam-Status: No, hits=-0.4 required=5.1 tests=AWL,BANG_MORE,BAYES_70,
  HTML_MIME_NO_HTML_TAG,MANY_EXCLAMATIONS,MIME_HTML_ONLY,
  NORMAL_HTTP_TO_IP,UPPERCASE_2_50 autolearn=no
  version=2.60-rc5
  X-Spam-Report:
  * -6.5 AWL AWL: Auto-whitelist adjustment

Using the "E" flag in procmail offers a way around this by selectively disabling AWL for self mail:

:0fw
* ^From:.*_myname_@kelvinist.com
| spamassassin

:0 fwE
* < 256000
| spamassassin -a

Ideas that did not work.

Attempting to solve the problem with a spamassassin rule that looks for the comment (quoted named) part of the From: address:

  header SPOOFING_FROM From =~ /^\s*"?[a-df-zA-DF-Z].*_myname_\@kelvinist.com/
  describe SPOOFING_FROM Attempt to spoof the from address to be me.  
  score SPOOFING_FROM 6.0

did not work. This interferes with the subsequent AWL calculation by boosting the positive AWL for my own mail for every spam that is received (or vice-versa, letting spam thru because self mail lowers the AWL). The "whitelist" command doesn't help, because that just subtracts 100 for real spam and my own loopback, leaving them the same relatively. Removing one's own email from the AWL only works temporarily. Permanently removing it via:

:0fw
* < 256000 
| spamassassin --remove-addr-from-whitelist-_myname_@kelvinist.com -a

does not work either because this prints a status message in the output stream, which ends up in /var/spool/mail/_name_. That prevents POP from working.

Home