So I came across an interesting quirk in MySQL the other day. Let’s say you have a table schema and some values that look like this:
And then let’s say you pass the following SQL statement to MySQL:
If you come from a loosely-typed language such as PHP, you would probably expect clicks for page_id 1 to now be 1. But that’s not the case in MySQL. After the query is run, the table will still look like this:
Not only does the query fail, but it fails with no warnings given. It appears that mathematical operations on null values silently fail.
There are a couple of ways around this. The first and most obvious is to set NOT NULL and a default value on the column. In the example above, this would work. The NULL value in that field becomes a 0 and you can to normal mathematical operations on it. But what happens if, for whatever reason, you can’t do that? We actually have this situation in a few places at dealnews, where NULL represents a distinct value of that field that is different from 0. In this case, you can use COALESCE() to fill in the appropriate value for the field.
Edit: Brian Moon informs me that this is actually part of the SQL specification. So hooray for specifications. Still, it’s kind of arcane; in working with MySQL (and PHP) for a decade now, this is the first time I’ve ever actually encountered this. Hopefully this helps someone who was as confused as I was.
Over the years, I’ve seen a number of blog posts relating to common questions that should be asked of programmers. Obviously, this is going to depend on exactly what position you are hiring for, but there are some good “gateway” questions that can be used to determine whether or not an applicant you are interviewing can … well … even program at all. If they even have the mindset that makes a good developer.
A common one I’ve seen tossed around is Fizz Buzz. The challenge goes something like this:
Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.
Now, to anyone who even has a basic understanding of programming, this is super simple to solve using a modulus operator. But apparently many people applying for even entry-level development jobs cannot solve this problem. According to the article linked above, even one “senior developer” took 15 minutes to solve this problem.
Earlier today, a friend posted something on Facebook that inspired what I think it another good, intermediate to difficult level programming question that also looks for pattern recognition. The relevant part of the post began by stating: “This year July has 5 Fridays 5 Saturdays and 5 Sundays.” There is the question! It would go something like this:
The month of July 2011 has 5 Fridays, 5 Saturdays and 5 Sundays. Calculate the next 50 times there will be a month that has 5 Fridays, 5 Saturdays and 5 Sundays.
Woah, so how to go about solving this problem? Well, look at a picture of July 2011. Notice something interesting about this month in relation to the question? This month has 31 days (the most any month can have), begins on a Friday and ends on a Sunday. And that’s the solution! It’s any month with 31 days that begins on a Friday!
With this in mind, it’s pretty easy to come up with a PHP solution:
Note that I make use of PHP’s strtotime function, because it is the Swiss Army Knife of date manipulation in PHP. This would need to be adapted for use in another language.
So now tell me: what are some other questions you’ve been asked or asked in an interview?
So today, out of nowhere, Xcode 4 finally landed as an official release. After seemingly forever in beta, and me quipping more than once about it’s similarity to Duke Nukem Forever, Apple finally pulled the trigger and released it. But something changed.
Xcode now has a price. And that has left me, as both a Mac user and a Mac developer, with a lot of questions.
It’s either $4.99 if you’re not a registered, paid Apple developer, or free if you are a registered, paid Apple developer (with all its $99 per year price tag glory). Supposedly there’s some crazy accounting reason that they have to charge for it. This, of course, leaves open the possibility that Xcode will soon be free again once OS X 10.7 arrives. But, it also leaves open the possibility that Xcode will no longer be distributed with OS X and will always have a price tag. It may not even stay $4.99. It may be $49.99 or $499.99.
There are additional questions, too. Does this mean that Apple is still distributing Xcode as a bundle with GNU GCC? Because there are things (such as MacPorts) that rely on the underlying foundation provided by the developer bundle that don’t actually use Xcode. Before, those were completely free. Now, they cost $4.99 unless they have split the underlying compiler from the IDE. And if they are still distributing it with GCC, that leads to all kinds of crazy interesting licensing questions.
But I think the worst part is that there is now a barrier to entry, however low, to being a developer on a platform that is already a minority in market share. I can’t understand how Apple potentially believes that it is good and right to trade short term profits for long term growth in the number of potential developers. For the future of the Mac platform, I sure hope this isn’t their line of reasoning.
So, let me tell you a little story.
My first dabbling in programming came courtesy of QuickBASIC back in the MS-DOS and Windows 3.1 days. This was the late 80s or early 90s, so I would have been 10 or 11 at the time. I stumbled across the Qbasic environment included with MS-DOS by accident and found Nibbles. And, after playing it, I discovered that I could change things by making changes to the strange text presented on the screen. I could change colors and speeds. But it would be a couple of years before I really understood what I was doing.
When Windows 95 came out (and along with it, Visual Basic 4), I talked my parents into getting me a copy. I don’t remember how much it cost but it was probably a lot because it was one of the few Christmas presents I got that year. But boy did I run with it. I’ve periodically felt guilty over that expense because I didn’t actually make anything really useful with it, but it was instrumental in furthering my education. Now I could do things on my computer far beyond what poor ol’ Qbasic was capable of. So I wrote lots of silly little programs. I put together a “family newsletter” one year that was installed and ran as a piece of software. I was pretty proud of that. I even wrote some software for my high school as part of a software development and AP Computer Science courses.
Eventually, I would move on to other things. Other versions of Visual Basic, Java, C, a brief foray into LISP and Forth-based languages for programming MUDs, and eventually web programming. First in Perl, then in PHP. I even landed my first paying programming job while still in high school, writing applications for a local transit contractor. At first, these were Visual Basic applications. But by the time I left (August of 2000) everything was going to the web and so were we.
But I can trace everything - my entire career, and my consuming passion for software engineering - back to Qbasic and Nibbles. A silly little game about a block snake, and a free development environment included with the operating system. Had I not stumbled on Qbasic and Nibbles, there’s a chance I would never have been a developer.
This is not about $4.99. I spend more on coffee in a week than that. My worry is about that 11 year old kid out there somewhere who may never get the opportunity to stumble across Xcode or the sample applications in /Developer and realize the raw power they possess. This is an area where Apple, a company with billions in cash on hand, should be happy to show a loss. It would be to the benefit of their platform, both now and in the future.
One of the great benefits of the Mac platform has been it’s low barriers of entry to developers. Sure, one could argue that the hardware is more expensive (and I could counter-argue that, for the quality of the equipment you are getting a bargain), but the development tools have always been freely available online and included with the machine. You could dabble in programming to your heart’s content. Sure, if you want to put something in the app store(s), you had to pay for admission, but there was nothing stopping you from getting all the way to that point, or even distributing your creations on your own.
But this new trend of charging for the development tools - even if it is a paltry sum - sends, to me, a worrying signal about the course Apple intends to tread. They’ve now moved the gate from the last step to the first step. It’s a course that Microsoft, as above, once tread.
In some ways, after years of doing programming and scripting, I’m now sort of rediscovering the power of the shell.
Tonight, I was working on my server and remembered that I needed to start backing up my MySQL databases (which you do also … right?). So instead of writing a script to do that, with a little research, I was able to come up with a way to:
Dump each database to a separate SQL file, with a timestamp.
bzip the file.
Keep 5 days worth of backups for each database, rotating the oldest backup off.
Here’s what I came up with:
Shoved that in my crontab. Works great. Linux rocks.
I have more than 20 various IM accounts set up in Adium on my Macintosh. But during the day, the only one I really want to be active is the one I use for work. The remainder, I want to leave logged in, but showing as away with a warning not to bother me unless it is important. But half the time I forget to set all those accounts as away, or I forget to set the work one as available, or some other issue that would arise out of a manual process interferes and too often it doesn’t get done.
Enter AppleScript. I whipped up a surprisingly easy AppleScript to do just this:
Because the work account is the first one, this makes it easy. It just sets all accounts as away and then sets the work one available. I shove this in my MarcoPolo ruleset to fire when I arrive at the office.
The script to reverse the change when I leave is even easier. This is fired when I leave the office:
To completely eliminate the need for me to touch the phone to provision it. I want to be able to create a profile for it in the database, then simply plug the phone in and let it do the rest. And…
To eliminate per-phone physical configuration files stored on the server. The configuration files should be generated on the fly when the phone requests them.
So the flow of what happens is this:
I create a profile for the phone in the database, then plug the phone in.
Phone boots initially, receives server from DHCP option 66. Script on the server hands out the correct provisioning path for that model of phone. Reboots with new provisioning information.
Phone boots with new provisioning information, begins downloading update SIP application and BootROM. Reboots.
Phone boots again, connects to Asterisk.
At this rate, provisioning a phone for a new employee is simply me entering the new extension and MAC address into an admin screen, and giving them the phone. It’s pretty neat.
**Note: **there are some areas where this is intentionally vague, as I’ve tried to avoid revealing too much about our private corporate administrative structure. If something here doesn’t make sense or you’re curious, post a comment. I’ll answer as best I can.
Creating the initial configs
I used the standard download of firmware and configs from Polycom to seed a base directory. This directory, on my server, is /www/asterisk/prov/polycom_ipXXX, where XXX in the phone model. Right now we deploy the IP-330, IP-331 and IP-4000. While right now the IP-330 and IP-331 can use the same firmware and configs, since the IP-330 has been discontinued they will probably diverge sometime in the not too near future.
With the base configs in place, this is where mod_rewrite comes into play. I added the following rewrite rules to the Apache configs:
To understand what these do, you will need to take apart the anatomy of a Polycom boot request. It requests the following files in this order: whichever bootrom.ld image it’s using, [mac-address].cfg if it exists or 000000000000.cfg otherwise, the sip.ld image, [mac-address]-phone.cfg, [mac-address]-web.cfg, and [mac-address]-directory.xml. So, we’re going to rewrite some of these requests to our scripts instead.
Generating configs on the fly
We’re going to skip the first rewrite rule (we’ll talk about that one in a little bit since it has to do with plug-in auto provisioning). The one we’re concerned with is the next one, which rewrites [mac-address]-phone.cfg requests to our provisioning script. So each request to that file is actually rewritten to provision.php?mac=[mac-address].
Now, in the database, we’re keeping track of what kind of phone it is (an IP-330, IP-331 or IP-4000), so when a request hits the script, we look up in the database what kind of phone we’re dealing with based on the MAC address, and use the variables from the database to fill in a template file containing exactly what that phone needs to configure itself.
For example, the base template file for the IP-330 looks something like this:
The script outputs this when the phone requests it. Voila. Magic configuration from the database.
There’s a little bit more to it than this. A lot of the settings custom to the company and shared among the various phones are in a master dealnews.cfg file, and included with each phone (it was added to the 000000000000.cfg file).
Now, on to the next rule.
Generating the company directory
Polycom phones support directories. There’s a way to get this to work with LDAP, but I haven’t tackled that yet. So, for now, we generate those dynamically as well when the phone requests any of its *-directory.xml files.
This one’s pretty easy since 1) we don’t allow the endpoints to customize their directories (yet), and 2) because every phone has the same directory. So all of those requests go to a script that outputs the XML structure for the directory:
We do this for both the 000000000000-directory.xml and the [mac-address]-directory.xml file because one is requested at initial boot (the 000000000000-directory.xml file is intended to be a “seed” directory), whereas subsequent requests are for the MAC address specific file.
Getting the log files
Polycoms log, and occasionally the logs are useful for debug purposes. The phones, by default, will try to upload these logs (using PUT requests if you’re provisioning via HTTP like we are). But having the phone fill up a directory full of logs is ungainly. Wouldn’t it be better to parse that into the database, where it can be easily queried? And because the log files have standardized names ([mac-address]-boot/app/flash.log), we know what phone they came from.Well, that’s what the last two rewrite lines do. We rewrite those PUT requests to a PHP script and parse the data off stdin, adding it to the database.
A little warning about this. Even at low settings Polycom phones are chatty with their logs. You may want to have some kind of cleaning script to remove log entries over X days old.
Passing the initial config via DHCP
At this point, we have a working magic configuration. Phones, once configured, fetch dynamically-generated configuration files that are guaranteed to be as up-to-date as possible. Their directories are generated out of the same database, and log files are added back to the same database. It all works well!
… except that it still requires me to touch the phone. I’m still required to punch into the keypad the provisioning directory to get it going. That sucks. But there’s a way around that too!
By default, Polycom phones out of the box look for a provisioning server on DHCP option 66. If they don’t find this, they will proceed to boot the default profile thats ships with the phone. It’s worth noting that, if you don’t pass it in the form of a fully-qualified URL, it will default to TFTP. But you can pass any format you can add to the phone.
In this case, what we’ve done is look for a MAC address in Polycom’s space (00:04:f2) and pass it option 66 with our boot server. But, we’re passing the same thing no matter what kind of phone it is! How can we tell them apart, especially since, at this point, we don’t know the MAC address.
The first rewrite rule handles part of this for us. When the phone receives the server from option 66 and requests 000000000000.cfg from the root directory, we instead forward it on to our index.php file, which handles the initial configuration. Our script looks at the HTTP_USER_AGENT, which tells us what kind of phone we’re dealing with (they’ll contain strings such as “SPIP_330”, “SPIP_331” or “SSIP_4000”). Using that, we selectively give it an initial configuration that tells it the RIGHT place to look.
Now, all you do is plug the phone in, and everything else just happens. A phone admin’s dream.
Keeping things up to date
By default, the phones won’t check to see if there’s new config or updated firmware until you tell them to. But his also means that some things, especially directory changes, won’t get picked up with any regularity. A quick change to the configs makes it possible to schedule the phones to look for changes at a certain time:
This causes the phones to look for new configs at 1AM each morning and do whatever they have to with them.
The reason all this is possible is because Polycom’s files are 1) easily manipulatable XML, as opposed to the binary configurations used by other manufacturers, and 2) distributed, so that you only need to actually send what you need set, and the phone can get the rest from the defaults. In practice this all works very well, and cut the time it used to take me to configure a phone from 5-10 minutes to about 30 seconds. Basically, as long as it takes me to get the phone off the shelf and punch the MAC address into the admin GUI I wrote.
I’ve been using Google Chrome as my primary browser for the last few months. Sorry, Firefox, but with all the stuff I need to work installed, you’re so slow as to be unusable. Up to and including having to force-quit at the end of the day. Chrome starts and stops quickly But that’s not the purpose of this entry. The purpose is how to live with self-signed SSL certificates and Google Chrome.
Let’s say you have a server with a self-signed HTTP SSL certificate. Every time you hit a page, you get a nasty error message. You ignore it once and it’s fine for that browsing session. But when you restart, it’s back. Unlike Firefox, there’s no easy way to say “yes, I know what I’m doing, ignore this.” This is an oversight I wish Chromium would correct, but until they do, we have to hack our way around it.
Caveat: these instructions are written for Mac OS X. PC instructions will be slightly different at PCs don’t have a keychain, and Google Chrome (unlike Firefox) uses the system keychain.
So here’s how to get Google Chrome to play nicely with your self-signed SSL certificate:
On your web server, copy the crt file (in my case, server.crt) over to your Macintosh. I scp'd it to my Desktop for ease of work.
** These directions has been updated. Thanks to Josh below for pointing out a slightly easier way.**
In the address bar, click the little lock with the X. This will bring up a small information screen. Click the button that says “Certificate Information.”
Click and drag the image to your desktop. It looks like a little certificate.
Double-click it. This will bring up the Keychain Access utility. Enter your password to unlock it.
Be sure you add the certificate to the System keychain, not the login keychain. Click “Always Trust,” even though this doesn’t seem to do anything.
After it has been added, double-click it. You may have to authenticate again.
Expand the “Trust” section.
“When using this certificate,” set to “Always Trust”
That’s it! Close Keychain Access and restart Chrome, and your self-signed certificate should be recognized now by the browser.
This is one thing I hope Google/Chromium fixes soon as it should not be this difficult. Self-signed SSL certificates are used **a lot **in the business world, and there should be an easier way for someone who knows what they are doing to be able to ignore this error than copying certificates around and manually adding them to the system keychain.
At dealnews, as I’ve written before, we run Asterisk as our telephone system. I find it to be a pretty good solution to our telecom needs: we have multiple offices and several home-based users.
And, for the most part, for hard telephones, we use Polycoms. We run mostly IP-330s, with a couple of IP-4000s and a few new IP-331s. We also have softphones, a couple of PAP2s and a couple of old Grandstreams from our original Asterisk deployment in 2007 that I’m desperately trying to get out of circulation. But it’s mostly Polycoms.
Recently, I changed how we were doing provisioning. I’ll write a more in-depth post about this later, but the short of it is that since Polycom phones use XML for their configuration information, we now generate them dynamically instead of creating a configuration file. It’s what I should have done back in 2007 when we bought our first round of Polycoms.
But this presented me with a problem: how do I re-provision the older phones - some of which I don’t have easy physical access to (at least that doesn’t involve an airplane ride) - to use the new configuration system?
In doing some research, I discovered that Polycom allows you to set, via certain commands, the provisioning server from within a config. With this information, I crafted a custom re-provisioning config that looks like this:
And included it at the top of the 000000000000.cfg file (one of the default files downloaded by each Polycom phone):
Then, using Asterisk, I issue the check-config command:
The phone should reboot, pick up its new config, then reboot again with with proper new provisioning information from the new provisioning provider.
Next post, I’ll show you how to use PHP and mod_rewrite to eliminate the need for per-phone config files.
My home entertainment center is probably second only to my computer(s) inn “things I interact with every day.” Barely a day goes by when I don’t spend a little relaxing time watching TV or movies.
I have a Hitachi 1080p 42-inch television, an Onkyo receiver attached to a 5.1 surround sound system (Polk Audio subwoofer and Energy speakers), a DVD player (that rarely gets any use anymore), a VCR (that gets even less use) and a PlayStation 3. But the star, and my single favorite piece of equipment in my living room is my AppleTV.
Yup. My AppleTV.
You might be asking why I profess love for a device that many people consider to be a failure. After all, the way some people, including some of my coworkers, talk about this device, you’d think it was Battlefield Earth bad. The kind of bad that you ask for your money back after using. The kind of bad that makes you regret waking up that day, and makes you want to drown your sorrows with a pitcher of Natural Ice.
And yet I, as an AppleTV owner, am trilled with it.
I love it simply because of its typical Apple simplicity: it’s all the best parts of a HTPC without all the bull** that comes with having a HTPC. Powerful enough to be usable, and yet simple enough that my wife - whom I love, but is most definitely not a computer person - can figure it out. It was simple enough to set up that all I had to do was plug it into my TV and get it on the network. And, it integrates incredibly well with the rest of the Apple products in the house.
And now, Apple has come out with a new AppleTV, and I could not possibly be more thrilled, because it addresses almost all the issues I had my current AppleTV, and with an upgrade price of $99, it’s a no-brainer. I might buy one for every TV in the house.
Let’s go through some of the differences:
No onboard storage. I have two AppleTVs. One in the living room - a 160gb model, and one in the bedroom, a 40gb model. You know how much storage space I’m using on them? Zero. Nothing. I stream everything off my iMac upstairs. Sync’ing is slow, and I have way more content than could even fit on the 160gb model. Moreover, streaming from iTunes shares works seamlessly, so there’s really no reason to use the local storage at all. Apple did away with it.
No composite. Non-issue. I use all HDMI. The new AppleTV has only three plugs on the back: power, HDMI, and ethernet. Perfect.
Movies from the iTunes store are rental-only. I don’t quite agree with this, but it’s not very strong. I never purchased a movie from the iTunes store. But I did rent on more than one occasion, so I don’t foresee this being an issue, especially because of …
Netflix support. That’s right. You can stream all the free content on Netflix straight to my AppleTV. This in and of itself is enough reason for me to want to upgrade.
In other words, it’s as if Apple fixed the device to exactly reflect how I use my current one. Since Steve Jobs never called me, I can only conclude that there were a lot more people out there using AppleTVs in the way I use mine. Frankly, at this point, the only things that it’s missing that I really wanted were 1080p and Hulu.
Two days ago Apple announced Ping: a social network geared towards music sharing. And a bunch of iPods too. Personally, I was more excited by the new AppleTV (I have two of them and absolutely love them) but more on that later. This is about Ping.
My thoughts on **Ping:
Apple’s first real attempt at social networking reminds me of Google’s countless attempts to get into the social networking space: they’re like that guy that shows up to the party really late - I mean beyond fashionably late - when the party is already over and everyone else is already drunk and thinking about stumbling across the street to IHOP or Taco Bell. They say they were at the library studying and now they want to go out and drink, but the keg has floated, the bars and liquor stores are already closed and all you want to do is eat a burrito supreme and find some sofa to pass out on.
Ping is a good first start, but it has some problems:
What is the target here? Am I supposed to follow people or artists or both or what? And what are they supposed to do? All this feels like is Twitter or Facebook + iTunes. The people I’m following can share messages and pictures? Yep. Twitter in iTunes. I can like and share and post comments? Yep. Facebook in iTunes.
Why not allow independent artists into the fold? Some of my favorite artists (such as Matthew Ebel - check him out if you love piano rock) are independent. Right now there are like 10 artists you can follow, and that Lady Gaga is one of them makes me want to break something. The only ones on there I’m remotely interested in following is Dave Matthews Band and maybe U2.
I can’t access it in any way other than in iTunes. No web access. While this means I can fire it up myself on my computer and laptop, and (currently) on my iPhone via the iTunes application, I can’t check Ping at a friend’s house. I can’t go to the Apple store and check Ping. Everything has to go through iTunes, and this absolutely cripples it. Think that’s overkill? Go to the Apple store and watch for 15 minutes how many people walk in and use one of the computers to check Facebook.
I can only “like” and “share” content I purchased from iTunes. I have purchased 58 songs from iTunes over the years, out of 3,621 songs in my library. About 1% of my library is available.
If Apple fixes these (and other, more minor) problems, Ping could be really cool. The problem is that these aren’t code fixes. They’re not something they can test and roll out a change for. These are conceptual problems relating to what their idea of Ping is versus the what the rest of the world is going to use it as. The question is, will they be Google and throw this out here, not maintain it and mercifully kill it a year later (a la Google Wave and the impending death of Google Buzz), or will they adapt and change it to better suit the needs of the public? Because that’s the thing about social networking: you have to embrace the users thoughts, opinions, and ideas. It’s a lesson digg just learned the hard way and a lesson that frankly, given Apple’s reputation as wanting to control everything, I don’t see them embracing.
As a side note, I will however salute Apple for not giving into Facebook if the rumor is true. Facebook plays fast and loose with people’s information, and I really don’t like how it seems to have become the de facto standard for social network usage (and thus the reason you can comment with your Facebook login). That, and Zuckerberg. I hate that guy.
Still, Ping is yet another player in this social networking space. A space that is becoming increasingly full …
I’m already Facebooked, Myspaced and Twittered. I’m LiveJournaled, Wordpressed, and Youtube’d. I’m Flickr’d, LinkedIn’d, Vimeo’d, Last.fm’d and Gowalla’d. I’m on any number of dozens of message boards and mailing lists that predate “Web 2.0” and the social networking “revolution,” and I follow nearly 100 various blogs and other feeds via RSS. They’re on my desktop, on my laptop, on my tablet and in my phone.
And now, apparently, I’m Ping’d as well. Le sigh.
Now, to be fair, I don’t check all these sites. I last logged into Myspace about 9 months ago. I last used Gowalla about a year ago. I usually only look at Youtube, Flickr or Vimeo when I need something, and haven’t updated a LiveJournal in about 3 years. But at what point does all this interaction - this social networking - become social overload?
Are any of these services adding value to my life? And at what point does a social network - Ping, in this case - simply become yet another thing I have to think about and check? Or will it become yet another service I sign up for, try for awhile and ignore?