Options Have Meanings, or, How I Made an rsync Seven Times Faster

By · Published · linux, networking

Warning: Doing this is making a clear tradeoff between security and speed. Do not do this on the public Internet or across a network you do not trust.

rsync is one of those tools that is in every computer user's toolkit. It's fantastic for moving large amounts of data around and for migrating data from one system to another.

rsync also has a ton of options and, after awhile, you get to where muscle memory means you just type the same few options over and over again. With me, that was -avz, archive, verbose, compression.

Recently, I was migrating several terabytes of data from a NAS to a computer. As is often the case, I fired up an rsync job and watched it.

It maxed out at about 35 megabit. Across a gigabit switched internal network.

Read More...

Harvesting Nest Thermostat Data For Fun And Profit

By · Published · nest, influxdb, php, collectd

Okay, no profit in this, but it certainly is fun!

I have two Nest thermostats in my house and, after some teething pains (yay the life of an early adopter) they have been pretty solid. But they're also black boxes that I know little about. I know they're collecting mountains of data and sending it back to the Google mothership. Wouldn't it be nice to get at some of that data and build my own reports?

Read More...



Extending ngResource To Access Metadata

By · Published · javascript, angularjs, howto

AngularJS's built-in ngResource is a great tool for natively supporting REST APIs in your Angular application. But what happens when you need to support something besides a simple call that retrieves a list of JSON objects? You quickly run into the limits of ngResource.

Here's a great case where you might need to do something more complex: paging. Say you want to get a list of objects, and there's 10,000 or so of them. You don't want to send 10,000 objects to your frontend app. You want to send a portion of them, but you still need to indicate to the app that there are more.

Surprisingly, considering how widespread this pattern is in web development, there does not seem to be a native way to accomplish this. But you can extend ngResource. Here's how I did it.

Read More...

What I use: 2016

By · Published · apple, mac, osx

Since it's been awhile since I wrote a post about what I use in regards to software, hardware, etc. Perhaps it's time that I did that again. So here's a list of what I'm using in 2016:

Read More...

Collecting Data From pfSense Using collectd

By · Published · pfsense, networking, php, collectd

So I've recently been on a graphing thing, wanting to collect all kind of data from my home network. And collectd seems to be a good candidate for doing that. With a huge number of plugins, it can collect and send just about anything you can think of to a time series database (I'm using InfluxDB for this).

But, there's a significant hole in my data collection: my pfSense firewall. Well, not anymore!

Read More...

Finding Multi-byte Characters in MySQL Fields

By · Published · mysql

So I was recently helping a client with an issue in MySQL where a migration failed to transfer the full contents of some fields. This amounted to a little over 1% of the total messages transferred. In doing some research, we discovered that the one thing every message had in common was the presence of multi-byte (high unicode) characters. In many cases, this was due to a user pasting some text from Microsoft Word.

Read More...

Why I'm (Almost) Quitting Facebook

By · Published · facebook, social media

So this is something I've been meaning to write for awhile now.

It's time we had a talk about Facebook.

I was an early adopter of Facebook. Not as early as some, but before it was open to everyone. Back when it was a social network for college students and you needed a .edu address to join. Back when you still had to choose a "network" (I think I'm still technically in the Auburn network somewhere buried down in my settings somewhere). I've been on Facebook probably more than 10 years at this point.

But now, I've finally decided to call it (mostly) quits on Facebook.

Read More...