wildpointer*referencing whatever comes to mind, by chet nichols

For those of you unfamiliar, NXDOMAIN is a DNS query response for non-existent domain. An example would be if you start typing google.com, but sneeze halfway through, and end up going to gegooele.com. That domain doesn’t exist, and you will ultimately receive an NXDOMAIN response from your upstream DNS server instead of an A record with the IP of the domain.

These days, a lot of ISPs have been doing NXDOMAIN redirection. Instead of handing back the NXDOMAIN response, they will intercept it and send you back an A record with an IP of one of their web servers (lets say 1.2.3.4 as an example). Your machine will ultimately think that gegooele.com has the IP 1.2.3.4, connect to that IP, and do a GET request with the HTTP host header of gegooele.com. The ISP’s web server, which specifically handles NXDOMAIN “redirected” HTTP requests, will then take that host header, pass it through some type of dynamic script, and display a nice custom error page with possible corrections, advertisements, etc.

Now, here’s where PowerDNS comes in: there are companies out there whose specialty is to sell DNS appliances and software solutions, solely for the purpose of doing NXDOMAIN redirection. However, if you are in the market for doing something like this (which is a whole separate discussion altogether, since there are strong opinions about this subject), you can save yourself a chunk of change by instead using PowerDNS (pdns-recursor, to be exact) with built-in Lua scripting. PowerDNS is open source, free, awesome.

We’re going to skip the build/compile details for now, and assume you have the latest PowerDNS recursor installed, and that Lua scripting has been compiled in. Chances are, it has. If not, check out http://www.powerdns.com and download the latest source. Once it’s installed, you just need to write a Lua script to modify DNS results, configure your recursor to use the script, and start up pdns-recursor. Let’s get to it.

First, you need to define a Lua script to handle DNS responses. Currently, Lua will only handle two types of responses: nxdomain, and preresolve. Preresolve is a response to give before any resolution has taken place. This is to basically hijack requests with a static response.

To define, we’ll add this line to our recursor.conf:

lua-dns-script=/opt/pdns/bin/nxdomain.lua

Now, let’s write our script. At it’s most basic level, it’s really easy:

function nxdomain (ip,domain,query_type)
ips={}
if query_type ~= pdns.A then return -1, ret end
ips[1]={ query_type=pdns.A, content="1.2.3.4" }
ips[2]={ query_type=pdns.A, content="5.6.7.8" }
return 0, ips
end

This script will first make sure our NXDOMAIN response is the result of a request for an A record. As long as it is, we will build an A record response to return instead of an NXDOMAIN. Per this code, the client will ultimately receive an A rotor response with two records.

Now that our script has been written and added to our recursor.conf, let’s start up the recursor and try it out!

$ /opt/pdns/sbin/pdns_recursor –config-dir=/opt/pdns/etc

Assuming the script is loaded successfully, you will see the following from the startup output:

Nov 25 03:09:36 Loaded ‘lua’ script from ‘/opt/pdns/bin/nxdomain.lua’

Looks good. Now, let’s see what happens when we try to hit our non-existent gegooele.com:

$ host gegooele.com
gegooele.com has address 1.2.3.4
gegooele.com has address 5.6.7.8

And there we go! NXDOMAIN redirection without the need for a commercial solution. Of course, there are more things you can do [with the Lua script]. WIth the proper log level, you can have it log every time the handler is called. For example, you can add a simple print line:

print ("nxdomain handler received: ", ip, domain, query_type)

You can also have it make decisions based on incoming IP, hostname/domain, and query type. However, the point of this was just to show the basics with what you can do. For more Lua syntax, and PowerDNS capabilities with Lua, check out the Lua website and PowerDNS wiki. Enjoy!

A while back, I had put a Perl script together to read in some large files, do some data mining, and dump out the results I wanted to see (well, maybe they weren’t the results I was hoping for, but results nonetheless). Now, when the script ran, it would go through it’s first few routines nice and quick-like, but when it went to the following routine, it would become unbearably slow. The first couple of times, I didn’t think much of it; I liked feeling that the script was hard at work (of course).

However, after a few more runs, it was getting kind of ridiculous: the routine in question was the one that did reads from the file handle. Now, I can understand on a multi-gigabyte file it might take a while (read in each line, parse it through a regex, and populate a data structure). However, it was causing the entire system to hang, and that made no sense at all given it’s relatively easy instructions. Here’s an example of what I was doing:

1
2
3
4
5

my $fd = IO::File->new("data.out",O_RDONLY);
 
foreach my $line(<$fd>) {
   @data = $line =~ /regex_pattern/;
}

Simple enough. So, on the next run, I started tracking the PID. Now, I saw two things: 1) the CPU was pinned, and 2) memory utilization was going through the roof. Bingo. Basically, my multi-gigabyte file was being read into memory, eating up all the remaining free physical memory, which was forcing the box to start swapping (which was eating up the CPU). Now it made sense why things were going ridiculously slow.

Now, what didn’t make sense was the fact this was happening at all. For each line of data, I was only pulling about 10% of it into my data structure (bytes parsed vs bytes per line), so I shouldn’t see memory utilization match that of the file. Additionally, I had worked with large files like this before, on systems with less memory, and never had an issue. What was different?

I took lunch, thought about it, and somewhere in there thought to myself, “hrmm, I wonder if it has to do with the foreach loop.” In the past, I had always used a while-loop, but didn’t really consider there might be an extreme difference between how each was interpreted when reading a file handle. I went back to my desk and modified the code to use a while loop instead:

1
2
3
4
5

my $fd = IO::File->new("data.out",O_RDONLY);
 
while(my $line = <$fd>) {
   @data = $line =~ /regex_pattern/;
}

Sure enough, it tore right through the file, did it’s parsing, and was done very shortly after. Interesting! So, what happened?

Basically, it’s a major difference in how Perl handles while loops vs. foreach loops.

In a while-loop, Perl will blindly shift lines one-by-one off of the array/fd you pass until it reaches an EOF. Once it does, it breaks out and you’re done. This is how it should work (and does exactly what we expect it to do), so that makes sense.

Now, in a foreach-loop, the Perl interpreter needs to know the end of the array before it begins iterating over it (whereas with a while loop we just go until EOF). If the data is already an array, that’s not a problem. However, if the data is a file handle, then the only way it can know the end is by reading the entire file into memory and ultimately create an array of the data to return back to the foreach iterator.

Most of the time, people aren’t running through multi-gigabyte files, so doing a foreach or a while on a file handle won’t really be noticeable, even though there are two completely different logic paths being followed behind the scenes.

Ultimately, use a while loop when reading off of a file handle. It’s cleaner, uses less overhead, and has obviously proven itself to be more fit for this task.

There are a lot of Perl hackers out there (many of whom I have worked with) who may end up commenting or shed some deeper technical insight. By all means, I would love to hear it! If you think I’ve done a good job summarizing and detailing the issue, that works just as well! Enjoy!