Ionrock Dot Org

My Weblog

Noticing Small Features

We recently returned from a tour. While on the road, I'm still working. I have a slick bean bag chair, curtains and MiFi router, which makes it possible to get my work done while cruising down the freeway. This is something I've discussed before.

My two recent annoyances with my setup are not having an automatic way of completing an email address and slow updates to the server. So far, the slow updates will probably involve setting up a local IMAP server to sync to and from. It is something that I've started looking into, but haven't spent that much time on it. It ends up being easy enough to just ignore email and save the time. What is rather annoying is having to enter email addresses.

It is not that bad, but it is relatively time consuming. My typing could be much better and it can be easy to make mistakes. Also, there are many times I'm copying the email address from some other resource such as a web page or IRC, in which case, there is a lot of buffer management to deal with. Again, not a huge deal but enough to warrant looking into solutions.

Fortunately, like most things in the Emacs landscape, there was already a good solution in place. The BBDB or the Insidious Big Brother Database. I initially tried using the tips mentioned on emacs-fu but found it didn't work. Fortunately, I found the solution was incredibly convenient and directly in the Wanderlust docs. The result, is now I'm collecting email addresses for completion much like I would get in gmail in addition to having an actual addressbook I might consider using. Pretty nice ROI for googling a bit.

This is realization you need a feature is a pretty common occurance in Linux. The conceptual basis is usually there to create a solution, but often times it takes a bit of work to really get things configured. Emacs is very similar, with the exception being it is almost expected that the configuration might very well be writing a new mode or piece of functionality yourself. It serves as a good reminder that I shouldn't go recommending Emacs to my parents anytime soon. Although it would be pretty cool if I did get a call from my dad asking about setting up a keybinding for some lisp function he wrote.

Experiments with Diesel: Repeater

It feels as though web development has begun to focus on other parts of the stack. Up until recently, the framework decisions seem to be the biggest focus, with MVC based patterns reigning in the masses. At this point though, there is a wealth of documentation and options that make trying out the latest and greatest MVC framework slightly passe. To combat this stagnation in hip web technology, the focus has changed slightly to server technology.

Here are few recent articles to make point regarding the state of web development:

• Unicorn is Unix
• Python is Unix
• Announcing Tornado
• Announcing Diesel

While most of these ideas and concepts are not relatively new in the realm of computer science, they are new to me. I represent a rather large audience of web developers who did not necessarily see web develop from socket libraries to Rails. Instead, my experience began with PHP and didn't include any understanding of what actually happens on the web. Fortunately though, my own forays into web development exposed some consistent patterns that helped me understand what was really happening. So, with these new-ish ideas coming around, it seemed like a great learning opportunity to become better acquainted with more the lower level aspects of web development.

While I have a few projects laying around my home directory for each of the ideas mentioned above, this one is specific to Diesel. I'm somewhat partial to for social reasons. When thinking of an idea for something that might very well benefit from being asynchronous (that is not a chat server), I came up with the idea for a database hub.

The idea for Repeater is that you set up a proxy that will "repeat" the requests to a set of services. For example, if you were using CouchDB, you could set up Repeater, which would be a round robin proxy for the different instances and allow updating all the instances with one request. This doesn't work yet! But that is the idea.

What does work is a basic proxy that will balance between some set of services. It works pretty well from the stand point of basic tests not failing, but I have no idea if it would be extremely slow in practice. The thing that got me stuck is how to handle more requests without blocking when requesting the other site. From the code, you can see where I started messing around with threads to make sure I don't block. The gotcha in all this was that I couldn't figure out a way to effectively yield a sleep or join where the join is when the thread has finished making the request. I'm sure there is a way to do it, but my experiments were not very fruitful.

My overall impressions are pretty good. I'm still a bit hazy on the applications where an async system is an order of magnitude better than a threaded or forking (or both) system. The chat example seems obvious because you have a direct relationship between a client connection and a single process that needs to make responses in order. This is similar to a database connection in that it may need to handle a large amount connections, but at the same time, as soon as you start doing things like requests over the web, it seems inevitable that blocking will be an issue. Although, it seems very solvable.

One Problem with Music Piracy

I read this article by Lily Allen on music piracy. While I'd like to think that music should be free, after looking at a few offers we've received from labels it is difficult to deny that music piracy has been considered part of the equation.

The big issue is not that record labels are struggling. This is a good thing. Labels have been loan sharks for entirely too long so good riddance. But, as crappy as big labels can be, there are a wealth of smaller labels that do aim to release great music to people because they appreciate what artists are doing. These range from friends that have friends in bands they want to help out, all the way up to labels who have established themselves as viable businesses. For bands, often times a label is the sole party that really can break a band.

There are are plenty of other ways a band can break, but the most common is through a label. The label puts out a record, pays for press, helps improve the bands stature and spreads the word with the result being the band finds a fanbase. The most important thing in this whole scenario for a band is finding the fans. While the critics can be nice to sway and Carling massive attention on blogs has its perks, the real deal is to get your music to fans. As a band, the closer you get to your fans and the more direct relation you build, the better the payout.

In the past bands could make money selling records. It was a big lottery, but it was still possible. Bands also were given the responsibility to do with their money as they chose. A band gets a big advance from the label and uses the funds to build their business (or buy drugs). Now, labels don't make nearly as much money selling music, so they are looking for other revenue streams. This means "partnering" up with an artist. The negative term has become the 360 deal where a label effectively can act as a manager, taking a percentage of the gross income. The more common occurrence from what I've seen is that labels aim to find targeted income areas to cover costs through the artist.

This in and of itself isn't too bad. It can be a huge help keeping the band funds liquid. If a band lands a huge tour or gets massive success, every t-shirt size they don't have is money lost. If a label can make sure you have enough shirts, hoodies and vinyl, that is a definite benefit. Labels can also help find placements and synchs for songs, which can be very lucrative. Again, very nice.

The problem is that while these benefits can be helpful, they can also preclude you from building your own business. If your label is your merchandising outlet, you probably can't make t-shirts yourself. If you have a three record deal, that is probably 3-5 years lost building part of your business. You won't be able to build up your own relationships with another manufacturer or put together your own fulfillment facilities. Likewise, if your label owns some of your publishing, that can end up being a lot of money. Say you have one song that gets 5 placements a year. If you average $2k per placement that is $10k. Over 20 years, that is $200K, which seems pretty good. Take 25% and give it to the label, pay another 20% in taxes and split it up between band members and it quickly becomes rather depressing. Couple that with the fact some labels are asking for actual percentages of tour revenue as well. Also realize that the band is most likely having to pay a lawyer, manager and booking agent (and in some cases a tour manager). Many of these support staffs used to take flat rates, but now all focus on percentages.

Before, if a band got 12% of a record sale they were doing pretty well. They could go on the road and make good money. Now, you don't get any money from record sales even though your percentage might be 50%. In addition to that, you are effectively taxed by your "team" on everything else you do. 5% for your lawyer, 10-15% for your manager, 10-15% for your booking agent, 10-20% for a tour manager, and 10-15% from your label. So, between 45% and 70% of the money you make disappears before you see it. Even if you do "break" and get a decent following, getting $5k a show doesn't mean much over the course of a year. In the famous words of Bill Cosby, "You have eaten yet!" Even if you take home half of the $5k for 30 dates ($2500 * 30 = $75k) you've yet to pay for gas, food, lodging, equipment and taxes. When you consider this money has to cover all the band members living expenses both on the road and off, things start to get rather tight. This doesn't even consider off road maintenance such as wardrobes and props (fog machines and can lighting is not cheap).

I honestly have to agree with Lily that music piracy doesn't really help the artists. It does help destroy the old system, which I do think is a good thing. But, unless there are some provable models for being an artist that don't involve taxes by everyone that helps you break, it doesn't seem like it is any better for artists. In a sense it is potentially worse because at least before, labels sold a product. The initial investment or R&D were directly tied to selling music. Now, everyone wants to get shares in the artists career.

The music business has been compared to the tech industry and venture capital, but I don't think that comparison is quite right. Music doesn't scale the same way technology does. Craigslist is a great example where a really small group of people have been able scale a small business from one city to a whole country. A band simply can't all of a sudden put out an album a day while touring non-stop. They need others to help keep the message out there and no one works for free. The result is that in order to break and make enough money to be reasonably stable, the actual revenue needs to be pretty huge.

With all this said, the fact still remains that I love playing music. It is an honor that people want to invest money in the music that I help create. While we plan on trying to be as shrewd as possible in terms of music as a business, the performers and artists inside just want to get our music out to as many people as possible. It is really all about fans and making that connection, so if that means giving up some ownership in order to help get our name out there, I'm kind of OK with that. The only requirement is that if you want a share of our music, you need to really earn it and be a fan.

Introducing Focusr

One aspect of time management that is critical to success is finding a way to focus on tasks. For many people, myself included, it is a pretty serious battle that takes tons of practice and creative techniques for fooling yourself to stick to the task at hand. One such technique is the Pomodoro Technique. I haven't read the book or would consider myself an expert by any stretch, but the basic idea seems simple enough to run with it despite formal training.

In a nutshell, you give yourself 25 minutes to complete a task and then take a short 5 minute break before moving onto the next task. From what I understand, the book emphasizes using an egg timer that is visible to make the whole process convenient. Seeing as I'm a programmer and there are multitude of ways built into my desktop to get my attention, it seemed like a good opportunity to create a simple tool.

The result is Focusr. This is really simple timer that helps to complete Pomodoro like cycles. You say you want to start a task, it starts the 25 minute timer, lets you know when the times up and does the same for the break. Rinse and repeat. It is super simple and surprisingly effective.

You can grab it from the web or install it with easy_install or pip. It uses libnotify's

notify-send

command to do the actual notification. Also, I created a simple Emacs function so I could start it easily.
(defun pomo ( ) "Start a pomodoro task 25 minutes working and 5 off" (interactive) (setq msg (read-string "What do you want to work on? ")) (setq cmd (concat "focusr " msg)) (comint-simple-send (make-comint "pomodoro-task" "bash") cmd))

While I'm sure buying the book could be helpful, it seems more helpful to understand what Pomodoro is actually doing. For myself, it presents a attainable period of time focus on a task. I've read over and over again that one key to better productivity is breaking large tasks into smaller tasks. This is easier said than done though. By taking on the day in 25 minute chunks you're forced to consider how you can break up tasks such that you finish a task with in the time limit. In addition to getting better practice breaking up tasks, you also are exercising your estimation skills and getting a better understanding of how much work you can really do.Like I said before, the concepts are really simple with or without formal training.

For myself, I also appreciate the obvious openness of the system. Becoming more productive is partly effectively utilizing systems while always evolving your techniques. As a person you have an innate ability to hack around your own efforts. I think this technique is simple enough that it can be used many different ways to help keep your mind guessing, which in turn helps to truly learn how to get more focus.

Issues with TDD

I'm making a concerted effort to do a lot more testing at work. Up until this point, tests have always been an afterthought. Something to provide a sanity check that helps to be sure things don't randomly break. There was obviously a little voice in my head nagging me that I should do a better job testing. The goal is to create a more stable environment where we can better evaluate a release. This is an important goal because it establishes the real point of testing, providing better software.

One observation I've found is that unless you start doing TDD or have some sense of what the tests should look like, it can be very difficult to get tests in order. Some might argue that if the tests are difficult to write, it is indicative of the code needing some refactoring. I honestly couldn't tell you if that is true, but my gut suggests that might very well be the case. On the other side of the spectrum, great tests do not make users happy through more usable applications or libraries. There has got to be a balance.

One thing I'm noticing in our own application is a set of global state that seems to always muck up the works. It can be a pain in the neck to always pass around variables instead of relying on a global. You have to consider if that mutable variable you just gave to one object will be modified and potentially affect another object's use of the variable. One solution is to try to pass variables where ever possible, but at that point we've somewhat missed some of the advantages of objects. With that in mind the globals we do have seem relatively benign as they traditionally are wrapping some storage or persistence piece that doesn't logically make sense to always pass around. This doesn't even begin to answer the questions of handling thread safety.

When I start thinking about all these things it really makes me frustrated. It is definitely a morale killer because improving the code slows momentum to a crawl. You want to refactor things to get something more testable and reliable, yet in doing so, you can easily create regressions by changing the previously working tests. Along the same lines refactoring tests seems like a recipe for disaster as you are changing the one bit of code that stands as a benchmark for functionality and quality. Today was particularly frustrating because it felt like every time I started working on cleaning up some piece of code, it became intertwined in the code I was trying to isolate. At this point it seems clear that there is real need, which is a good thing. Hopefully tomorrow things might be a bit clearer on how to start untangling things in a way actually helps improve the reliability of the code and makes for a better user experience.