#Linux

Internal Auto-Renewing LetsEncrypt Certificates

I have a well-documented obsession with pretty URLs, and this extends even to my internal home network. I have way too much stuff bouncing around in my head to have to remember IP addresses when a domain name is much easier to remember. LetsEncrypt launched to offer free SSL certificates to anyone, but the most crucial feature of their infrastructure, and one someone should have figured out before then, was scriptable automatically renewing certificates. Basically they validate you do in fact own the domain using automated methods, then issue you the new certificate. Thus, your certificates can be renewed on a schedule with no interaction from you. Traditionally, they have done this by placing a file in the webroot and looking for that file before issuing the certificate (see my earlier blog post about Zero Downtime nginx Letsencrypt Certificate Renewals Without the nginx Plugin for more detail about this.) But what happens when you want to issue an internal certificate? One for a service that is not accessible to the outside world, and thus, not visible using the webroot method? Well, it turns out there is a solution for that too!

Incrementally Migrating from Apache to nginx

I am currently in the process of migrating a bunch of sites on this machine from Apache to nginx. Rather than take everything down and migrate it all at once, I wanted to do this incrementally. But that raises a question: how do you incrementally migrate site configs from one to the other on the same machine, since both servers will need to be running and listening on ports 80 and 443? The solution I came up with was to move Apache to different ports (8080 and 4443) and to set the default nginx config to be a reverse proxy!

Backing Up and Rotating MySQL Databases the Easy Way

Here’s a little quickie for you. Say you have a small MySQL server floating around your house that you want to have regular backups of. You do want regular backups right? In my case, the biggest motivation was wanting a regular way to grab a recent MySQL dump of an internal tool I use at home to develop against. After poking around the Internet a bit, I was surprised that, other than mysqldump itself, there doesn’t seem to be a simple tool out there that you can slam into a cronjob and let it do it’s thing. So, like any good hacker, I decided to brew my own. After all, when you have 256,428 different solutions, why not make solution 256,429? :)

The Brilliance of Linux

I’ve been a Linux user for many, many years. Going all the way back to Red Hat 5.2, which I picked up to install on an ancient Packard Bell 486 in the late 90s. Since then there’s always been at least one Linux machine in my dorm, apartment or house somewhere. At various times I’ve even run it for my desktop OS, although these days I use macOS for that. For much of that time, Linux was the choice of hackers, but was definitely not a choice for everyday users and required a significant amount of technical knowledge to run. That’s not true so much anymore, but growing in that environment I learned a lot about how computers and operating systems work.

Options Have Meanings, or, How I Made an rsync Seven Times Faster

Warning: Doing this is making a clear tradeoff between security and speed. Do not do this on the public Internet or across a network you do not trust. rsync is one of those tools that is in every computer user’s toolkit. It’s fantastic for moving large amounts of data around and for migrating data from one system to another. rsync also has a ton of options and, after awhile, you get to where muscle memory means you just type the same few options over and over again. With me, that was -avz, archive, verbose, compression. Recently, I was migrating several terabytes of data from a NAS to a computer. As is often the case, I fired up an rsync job and watched it. It maxed out at about 35 megabit. Across a gigabit switched internal network.

gitcreate

I’ve created a new repository on my GitHub account where I can commit some of the little scripts I’ve written for use on my server. The first one I’ve committed is gitcreate, a small script that automates the creation and bootstrapping of git repositories. I realized that, when I was creating a new repo on my server, I do the same things over and over. Create the repo, then add in some frameworks for whatever little thing I’m playing with at the time. Well, gitcreate can do all that for you. Create the repo and bootstrap in things like the most recent versions of CodeIgniter, jQuery, and Bootstrap. That way, when you clone the repo to start working, you’re already ready to start coding. Like most of my stuff, it’s licensed under the New BSD License.

The Stupid Simple Guide to Setting Up Your Own DNS Server

I’m a developer, first and foremost. I like writing code. To me, maintaining servers, configuring things, troubleshooting network issues and the like -  these are things I do to support my primary interest and job as a developer. I’m not ignorant of these things, but all things considered they’re not my favorite things to do. One thing I will admit I’ve been ignorant over the years is DNS. Oh sure, I know at a high level how it works. I even know a bit about the different record types. I knew enough to have my own domain name, configured using Godaddy’s DNS servers to point to my server. But actually running my own name server? Something I’ve never done and, for some reason, had this unnatural fear of. Well, no more. I’m now running my very own shiny new name server and, actually, it wasn’t really as difficult as I thought. And because this was a learning experience for me, I figured I’d walk you through what I did as well. Picking  a Server There are two big players in the “DNS Server software” space: BIND and djbdns. BIND is the 900 pound gorilla that has been around forever and ever, and is insanely difficult to configure. djbdns is from the same guy who wrote qmail - I’ll let you be the judge of that. But after researching and actually attempting to install both of these, I eventually gave up. Both just came across as being too complex for a simple name server handling a couple of domains, and the documentation for both was equally complex. That’s when someone on Twitter pointed me to MaraDNS. I looked it over and was surprised to find good, readable and simple documentation that made it look easy to install. So I decided to give it a whirl. Here’s what I did. Note that this install is for a Gentoo system. Yours will be different if you’re using something else. Installing and Configuring MaraDNS First step is to install it. emerge maradns And let Portage do its thing. Once it’s installed, you really only have to worry about a few files. In /etc/mararc, you need to check to be sure you’re binding to the right interfaces. In my config, I bound it to the loopback and to the main interface: ipv4_bind_addresses = "x.x.x.x, 127.0.0." After that, you tell it to be authoritative, and what domains you are wanting to serve records for. csv2 = {} csv2["robpeck.com."] = "zones/robpeck.com" Note the period at the end of the domain name - it’s important. Each entry in the csv2 array should map to a zone file. I put mine in the “zones” subdirectory (which, in Gentoo, lives under /etc/maradns). mkdir -p /etc/maradns/zones Then, with your favorite editor (which should be vi :P), you create your zone file. The one for robpeck.com (partially) looks like this: robpeck.com. NS ns1.epsilonthree.com. robpeck.com. NS ns2.epsilonthree.com. robpeck.com. +3600 A x.x.x.x robpeck.com. +3600 MX 0 robpeck.com. www.robpeck.com. +3600 CNAME robpeck.com. So what are we doing here? Well, here it helps to know something about the different types of DNS records. I’m not going to cover all the different types of records - this is a good list of common ones and Wikipedia has a full list. The important ones you need to know are NS (Name Server), A (the main record), MX (mail server records), and CNAME (alias). The “+3600” is setting a timeout on the records to one hour (3,600 seconds). By default, the server will send one day (86,400 seconds). Here, I’m telling the server what the name servers are (strictly speaking, this isn’t required, but I added it all the same) and that the main address for people requesting “robpeck.com” is this IP address. I’m also saying that people who request “www.robpeck.com” should get the IP address for “robpeck.com.” I also add an MX record that points to robpeck.com with 0 as the priority (the first (and only) server). That’s it! Restart MaraDNS: /etc/init.d/maradns restart And you can test it out. dig @localhost robpeck.com A You should get a big long printout, but what you want to see is these two lines: QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0 robpeck.com. 3600 IN A x.x.x.x Assuming the above is the correct address, congratulations, your DNS server is now resolving properly locally. Delegating your Domain The next step is delegating your domain to your own server. I’m not going to cover this in too much detail because how it happens depends on the registrar. In general, this is a two step process: Register your name server’s IP address to a name. At NameCheap, when you’re in the domain screen this is done under Advanced Options > Nameserver Registration. Under GoDaddy, this is under the “Hosts” section of the domain information screen. You need to add at least two “nsX.domain.com” entries, but they can both point to the same IP. Delegate your domain to the names you just created. At NameCheap, you would go General > Domain Name Server setup, and Specify Custom DNS Servers. Then, enter the two (or more) names you just created “nsX.domain.com”. I can’t remember how I did this in GoDaddy, but I remember it was pretty apparent. That’s it! They say it takes 24-48 hours, but I started seeing requests hit the new name server within about an hour. Of course, since I wasn’t actually changing IP addresses, there was no real downtime. As of now, all my domains are being served off my own nameserver. It’s kind of a neat feeling of accomplishment, knowing you’re not relying on someone else’s DNS setup - they’re just providing you a name. This makes domain transferring much easier and adding new records much easier. And seeing as how I’m currently in the process of transferring all my domains away from GoDaddy, this will ease the transition.

Do Version Numbers Matter?

The recent announcement by Linus Torvalds that the next release of Linux will be 3.0 has provoked rather furious discussion around the Internets about whether or not the incrementing of the version number is warranted. Linus himself has said that “absolutely nothing” has changed. “It will get released close enough to the 20-year mark, which is excuse enough for me, although honestly, the real reason is just that I can no longer comfortably count as high as 40.” This got me to thinking about the nature of version numbers. Once upon a time (when versions were driven more by engineers and convention, and less by marketing), a version number meant something. Major, minor, revision. A major new release that modified significant portions of the code from the previous release incremented a major version number. Version numbers less than 0 were beta releases. Linux has been at 2.x since 1996, and at 2.6.x since 2003. Mac OS has been at 10.x since 2001 (even though the current version of OS X is significantly different from the original release in 2001). Meanwhile, Google Chrome has blasted through major 11 “versions” in three years. Mozilla is planning to release versions 5, 6, and 7 of Firefox this year. You can’t tell me that they are going to change major parts of Firefox three times this year. In this case, version numbers are purely being driven by marketing. They need to “catch up” to Chrome and Internet Explorer. But we live in a different world now. One where, arguably, version numbers are becoming less and less important. The growth of “app stores,” I think, is desensitizing your average user to a version number. While apps in the app store still have versions, I couldn’t tell you what “version” any of the apps on my iPhone are (other than the OS), and I bet you can’t either. Any of the apps I’ve installed from the Mac App Store I could not tell you the version of them. I just know that, when I see the number on the icon, I know I need to do updates. The updates happen, and I get a new version with whatever new features are there (or, in the case of the Twitter app, whatever features have been removed). Then there are web apps which are versionless. What version of Gmail do you use? You don’t. You use Gmail. Sure, there’s probably a revision number or something in the background, but the user has no clue what version they’re using. And they don’t need to, because there’s no action they need to take. So version are numbered in a wide variety of ways depending on the product and overall seem to be becoming less important as the growth of broadband, “app stores,” web apps, and automatic updates make thinking about version numbers less important. So why does it matter if Linus ups Linux to 3.0? Ultimately, it’s just a number.

BASH Quickie: Backing Up MySQL Databases

In some ways, after years of doing programming and scripting, I’m now sort of rediscovering the power of the shell. Tonight, I was working on my server and remembered that I needed to start backing up my MySQL databases (which you do also … right?). So instead of writing a script to do that, with a little research, I was able to come up with a way to: Dump each database to a separate SQL file, with a timestamp. bzip the file. Keep 5 days worth of backups for each database, rotating the oldest backup off. Here’s what I came up with: cd /backup/mysql; for i in $(mysql -BNe 'show databases' -u root -p<password>); do mysqldump -u root -p<password> $i | bzip2 > $i-`date +"%Y%m%d"`.sql.bz2; rm -rf $i-`date -d "-5 day" +"%Y%m%d"`.sql.bz2; done > /dev/null 2>&1 Shoved that in my crontab. Works great. Linux rocks.