Sam Pointer

Time and again I hear the following: “I’ve updated DNS with the new [A record, CNAME], it’ll take a couple of hours to propagate.”

No it won’t.

If you’ve previously tried to look up the non-existent name it’ll take as long as the minimum TTL value in the last field of your SOA record (see section #5 of RFC2308) for any caches to expire the NXDOMAIN response. If you’ve not then it will most likely be instant, depending on a few factors.

 

Image credit: xesc
 

I think this myth most likely comes from three places.

Firstly, most people don’t run their own DNS, they rely on a 3rd party provider. Most cheap providers do not use an authoritative nameserver that is database driven, such as PowerDNS or BIND with the DLZ (“dynamically loadable zones”) patches compiled in (I think this ships with modern 9.x versions). Instead, when you submit a form through their web UI you queue up some process behind the scenes that will produce a plain flat zonefile which will be reloaded on a schedule: usually every 15 or 30 minutes.

Secondly, back in the bad old days AXFR - a full zone transfer at the expiration of a timeout - was the only option for updating slave servers with the changes to a given zone. Depending on the value set in the SOA this could be quite some time. These days we have IXFR (“incremental zone transfer”), where the master server takes the initative to update all of its slaves when it detects a change, rather than waiting for them to time out and pull the changes. This delay, and the NXDOMAIN caching it would produce then slaves were queried before an AXFR would introduce a compound delay that certainly looks like propagation, but isn’t.

Thirdly propagation does need to take place in some instances. When you create a new domain you need to wait for the glue records to be circulated between the TLD authoritative servers, for instance. For most people for whom DNS is a once-a-year deal I suspect this idea of “a bunch of servers on the Internet need to sync up” gets a bit ingrained, doubly so if you’re mainly registering new domains more than you’re moving or mucking with existing ones beyond repointing the odd www. A record.

The O’Reilly “Cricket” DNS book, whilst nominally about BIND, is well worth the read for anyone who has more than a passing interest in DNS, or administers any operating system on a network. The first few chapters on DNS theory are worth the price of the book alone.
 

Edit: I’ve either proven my point or have not been clear by seeing most of reddit.com/r/sysadmin miss the point completely by discussing the expiration of positive caches, non-adherence to TTLs, etc. This is not what I am trying to discuss here.

What most people misunderstand is that when they add a new record to a zone (‘admin.nowhere.com’, for example - which has not existed before) they think that there is some propagation time. There is not, unless you’ve tried to look up that non-existent record before or your slaves are doing AXFRs with huge timeouts, or are distant in latency terms and doing IXFRs. When you register a new domain there is propagation time as the various TLD or cTLD root servers swap glue and NS records about. The mistake is thinking that the latter process has to occur for new records in an existing domain.

As for “propagation” meaning the gradual expiration of positive caches and the disemation of the information (an IP or MX change, for example), this isn’t propagation in the common DNS parlance. Propagation, in this context, is a “transmission” and is actively performed by the authoritative DNS server, not your local DNS resolver cache expiring.

I perhaps should have added a fourth reason: people confuse the cache TTL time-out of existing records with the lookup of a new record. There is nothing to expire with a new record, save the SOA value for NXDOMAIN as discussed above, and nothing to propagate.

For some reason I keep finding myself writing Perl code. In 2011. Over the last couple of months I’ve written enough Ruby to make my head spin, and yet, in a fix, I find myself back in the arms of Perl.

Recently I needed to parse some Apache virtual host configurations into some Nagios configuration file stanzas, a BIND zonefile and a hosts file for a project for my employer. Perl. On some private boxes not yet big enough to warrant a full Nagios installation I needed a script to run the CLI from the Clickatell SMS Gem in response to various /proc values. Perl.


Picture credit: reidrac

I like to think that my Perl is well written. I avoid implicits: $_, and the like, `use strict`, pass “-w” and declare all of my variables up-front. Sometimes I even entertain taint checking. It’s not perfect by any means, and not everybody’s style, but if you’re familiar with a C-like language you can probably read it. If you’ve written pseudo-code, you can read it. If you’re a patterns infected developer professional you’ll wince a bit. No FactoryFactories here. Objection orientation has its place, but it isn’t knocking up a quick 5 minute script to remove all of the commas from a YAML file driven by SOAP calls via wget in a panic.

I don’t think that anybody really pretends that Perl is the best thing to begin a new large-scale development project with today. But that doesn’t mean that Perl is down and out. Whilst Ruby gained frameworks and conferences and spawned religions, Perl sat there in it’s varied and glorious assorted version 5 point releases on nearly every UNIX-like box on the planet, just waiting for someone to ignore the framework bling and me-too bits and get stuck into good old text manipulation and nicking bits off of CPAN.

I’m by no means a language fanatic; I like all sorts. And I’ll tackle ActiveState or Strawberry if you force me, too. Perhaps it’s familiarity. Perhaps I simply don’t know Ruby or Python or anything else as well as I think I do. Or perhaps for your common or garden UNIX-style, “text as a univeral interface” basic string hackage, Perl is the crusty mig-welding old nutter who might just get you home?

1983 was a pretty amazing year. Lotus 1-2-3 was released, the IBM PC XT was released, Pioneer 10 becomes the first man-made object to leave the solar system. We get a woman and a black man in space, there is nuclear panic, the first NES is released, Brinks Mat, the IRA and Maggie Thatcher. Also, DNS was invented. If you want the history, here is a link to the Wikipedia article. With it, DNS brought us the great naming conundrum.

As a Systems Administrator (or a variant thereof, whatever the title), I have strong views on DNS naming conventions in your average corporate network. I’ve seen some obtuse and downright sadistic host and DNS naming convention abuses; I want to avoid more of the same. So, below, is my take on the definitive guide to your average internal namespace. Feel free to comment.

Use full words: don’t omit vowels or use cryptic two letter abreviations

‘dc’, ‘prn01’, ‘ps1-LON’. Wrong, wrong, wrong. Just learn to type and use full words. And what is this obsession with omitting only vowels? I mean: ‘exchngsvr’. Is it really worth it? Does it roll off of the fingers that much more easily? Really? It makes things much more obvious to you and non-technical people alike if you just use full words in logical domains. What would you both as an admin and a user prefer?

    app14.svr.internal.corp.com

or

    sales.servers.internal.corp.com

A logically configured name that can be read almost as a sentence can’t be a bad thing. To the business it makes the whole thing seem less like a Heath Robinson cranky geek outfit and a modern proper infrastructure.

Properly Configure and Use the Domain Search List

Make sure your domain search list is properly ordered and contains everything sensible for your outfit. Not only does it simplify configuration to single, obvious (full!) words, it comes into it’s own if you ship Virtual Machines between locations, copy configuration information to backup sites, or otherwise sychronize configuration between differently named domains.

Let’s say you ship a VM to a DR and production location. Each is handed out DNS information via DHCP and they have their domain search lists set to dr.corp.com and production.corp.com respectively. All of your scattered, site-specific configuration goes away. Want to talk to an SMTP server? Call it ‘mailhost’ in your configuration and at the DR site the DNS search list will cause a lookup for mailhost.dr.corp.com - likewise for the production location.

Not forgetting the users, having a domain search list that enables them to refer to hosts as ‘sales’, ‘fileserver’, ‘sage’ without full qualification makes everybody’s life easier.

The Cricket Book

Read it. It is getting slightly out-of-date, but the fundamentals still apply. Not optional.

Be Careful of Split Horizon Namespaces

… or revealing a different view of the world to your internal machines. Better that www.corp.com resolves to the same address no matter where you’re coming from and you handle the traffic from one point of ingress only. When the box moves in a year, or something else changes you’ll have an outage. A subdomain of your real (or /a/ real) domain is preferable to an unqualified ‘.corpnet’ internal domain, or a dummy domain in another TLD unresolvable externally. You have a proper chain of DNS delegation making any future delegation and rearrangement of your DNS configuration exponentially more trouble-free.

Get ready for DNSSEC, Test Your Resolver

DNSSEC is coming. When is open to debate, but it pays to keep on top of things. The DNS-ORAC have kindly created a test to determine if your resolver chain can or cannot receive large responses, a problem with additional traffic required for a DNSSEC lookup. Run it and do something about any problems.

Don’t neglect it

DNS is critical to any modern network, be it the Internet as a whole, or your little part of it. A bit of thought and a bit of discipline will make for a better infrastructure for you and your clients.

I recently had to solve a problem for a client at quite short notice. It is a twist on the normal domain canonicalization steps that we might use.

The client had three major requirements:

• The client had 20 publically accessible domains, of which 3 were “canonical” and represented their presence in three different terriorities. The rest were legacy, commerical variants, or “catch-alls”. They had 4 internally-resolvable domain to access various management aspects of the application (all ACL protected too, of course!).
• The client required that if a user visit the site with a plain domain name, it should be canonicalized to the www. version with a 301 redirect
• The site must be SSL protected.

SSL certificates only existed for the three main domains.

Usually, within this particular part of infrastructure we would move the SSL burden to the load balancer, and have that issue a 302 redirect to the https protected site automatically, preserving any of the path portion of the URL that had been supplied too.

As SSL certificates existed only for the three canonical domains this wasn’t an option: we can’t issue a redirect to https until we canonicalized the domain (the certificates were not wildcards, even for the three that existed). Canonicalization would take place after the connection would hit the load balancer. It just would not work.

I took the decision to use the Apache instance fronting-up the application server to handle all of these tasks, as it was the natural place for them to sit. The load balancers were relegated to fail-over, session persistence and NATing only.

A new Apache was recompiled that included:

• mod_alias (part of the default build)
• mod_rewrite
• mod_proxy

Now, to tackle domain name canonicalization.

When using name-based virtual hosts Apache serves content from the first default vhost where the requested domain does not match the ServerName or ServerAlias of any of the other named vhosts.

I took advantage of this and had the default vhost configured with an internally resolvable hostname, thereby creating a catch-all for any requests not to canonical domains.

The following configuration directives were used:

RewriteEngine On

# Alternative hostname, with or without a www. prefix
# Repeat for all non-canonical domains
RewriteCond %{HTTP_HOST}    ^.*notcanonical\.test$ [NC,OR]
RewriteCond %{HTTP_HOST}    ^.*alsonotcanonical\.test$ [NC,OR]

# Canonical hostname without www. prefix
# Repeat for all canonical domains
RewriteCond %{HTTP_HOST}    ^canonical\.test$ [NC,OR]
RewriteRule ^/(.*)          https://www.canonical.test/$1 [L,R=301]

RewriteCond %{HTTP_HOST}    ^anothercanonical\.test$ [NC,OR]
RewriteRule ^/(.*)          https://www.anothercanonical.test/$1 [L,R=301]

There are some important details in here.

• By redirecting straight to the desired canonical domain with https:// prefix the request cycle is shortened to only one possible 301 redirect. It would be possible to simply let the URL iterate over the conditions multiple times, perhaps for ease of maintenance and DRYness, but you risk increasing the response cycle length.
• The ‘NC’ flag ensures that case is ignored, and we add ‘OR’ to override the explicit AND for RewriteConds.
• When we issue the Rewrite we the ‘L’ flag is included to stop multiple iterations through the request cycle, and we override the default 302 issued (temporary) with a 301.
• The brackets within the RewriteRule regex capture any matches into a named variable ($1). We use this in the 301 target URL to preserve any of the path portion of the URL. The hostname and port are not parsed with this type of rule.

The final part of the puzzle is configuring two named virtual hosts for each canonical domain. The first listens on port 80 for plain http and is there to handle the situation whereby the user visits the canonical domain, but does so via plain http. Within the vhost container I used the RedirectMatch directive of mod_alias to issue a 301 redirect to the https:// version of the URL.

RedirectMatch 301 (.*)$ https://www.canonical.test$1

The second named host listens on port 443 and is configured to handle SSL in the usual manner. Within the vhost container we proxy the back-end application with something like:

ProxyRequests Off
< Proxy * >
    Order deny,allow
    Allow from all
</Proxy>
ProxyPass / http://appserver.internal/
ProxyPassReverse / http://appserver.internal/