Ionrock Dot Org

My Weblog

Front Loaded Mercurial

I'm going to have to go back and see how I can avoid laying a big fat patch bomb on a repo and I'm not happy about it. There is no one to blame but myself. That doesn't make it any nicer. My big issue is that for all the cool features of Mercurial there is a consistent front loading requirement. You cannot simply work and then later construct your commits that you'll be pushing. MQ does help with this sort of thing and I'm going to have to find out just how much tomorrow, but it would have been really nice if I could have started coding and whan I finished have a convenient way to go through all the files and commit them in reasonable chunks.

The astute reader will recognize that this issue really just a sign of bad DVCS habits and I'm not about to argue otherwise. Still, I'm very much a part of the "not a great coder" club, and as such, seem like a good candidate for how to help out the normal developers using these powerful tools. One might also suggest that I open a ticket, or even better, contribute a patch. Again, my "not a great coder" club membership explicitly states that any gripes need to stay far away from those folks getting a lot of work done (a la the mercurial devs), hence I'm totally fine leaving my whining here on my blog. My bet is bringing it up here will do more to improve my own habits than suggesting to others they are real problems.

Next time I'm really going to do a better job manaing my patches. Feel free to hold my feet to the fire in the future seeing how I've done.

The Real Current State of the Music Industry

I just read this blog from Tim Bray. Let me start by saying Tim is a smart guy that deserves to be listened to. With that in mind, I think TIm's view of the music industry is a little stuck in the mainstream, which I believe has an impact on his perspective. The thing that I've been seeing from the standpoint of a band that is trying to make things happen, is the focus on licensing. People have essentially given up on selling music for profit and instead have turned to those select few who need music for advertising. The money is much better and the amount of work required is an order of magnitude smaller in comparison to what is needed to be successful selling a CD.

When you think of most larger bands, they have a label, management, booking agent and quite possibly press person. This is important because this reflects the amount of work it takes for a record to be anywhere close to profitable. Breaking a band is such a great concept because it suggest that they are selling their music to the masses. The reality is that they are selling their music at an extreme discount in hopes that they will be noticed and in turn considered for other opportunities that pay exceptionally well. When Apple says they want to use a band for their commercials, the respective band realizes that they have just hit the jackpot. The band represents something "cool" and Apple is responsible for paying for that credential. Apple will have to pay something serious not only because the precieved publishing value is high, but b/c Apple does a good job marketing and will garner a large audience.

The way these sorts of licensing deals usually work is that there is an amount of money up front for publishing. That is usually is money givent to the song writer and the artist performing the track. If you google a bit, you'll see the 200% concept where 100 goes to the writer and the other 100 going to the artist that performed the track. With this publishing side of thing wrapped up, the next thing to consider is the mechanical royalties. This measures the audience and pays according the number of people who heard the track. That is why a band that has a song in a super bowl commercial would get paid incredibly well. The audience is huge.

The whole point being is that the real money is in licensing. Sure, bands make money on the road, but that is negligible compared to a great placement with the new hot HBO series or commercial. A band can get 40k right out of the gate for something like the title track of a show. This easily dwarfs the profits and cost of selling a record. This means that the focus for fans may be noble and respectable, but doesn't translate into making a living. To make a real living in writing and performing rock music, you have consider how your music could be used to advertise a product or represent something.

This is not to say that bands should simply become jingle writers. Honestly, there is a huge amount of content, so making an effort to rack up on licensing doesn't make financial sense when there are legitimate (albeit less profitable) means of selling music.The problem is that people working with artists are realizing that there is less money using traditional means and they hope licensing can make up the difference. This leads to labels and bands alike considering how they can write music that no only reflects themselves artistically, but also might be a good track for a movie.

Personally, I hope some concept of selling music returns in one way or another. Pop music has become truly commercial in the jingle sense and indie music is not far behind. There are some great bands making great music of course and many of them are commercially viable. My concern is the bands that wouldn't sound good selling something getting overlooked where in the past they had an excellent chance of being successful. There is also many cases of artists that have tapped into the web in a way that maximizes thier profit and cuts out the middleman. Currently, that feels like a niche group, but hopefully the patterns they use can become more standardized and people more knowledgible. For the moment, the style of those artists making money from audiences on the internet is not clashing or at ends with each other. For many artists, it just isn't the case where heavy direct communication and song writing contests work within their artistic vision.

Serious About Time Management

This morning I watched this video on time management. It was striking partly because the speaker was diagnosed with terminal cancer and truly had a very limited schedule. He had a family and while he felt OK for the moment, it was all but decided that his time would be cut short. Most the practical tips on becoming a better steward of time were similar to other methods. Things like making a todo list, avoiding interruptions, keeping responses short and keeping an empty inbox were all methods that I had heard before. The only thing that really differed was that the speaker was a guy who was not only reputable for his time management skills prior to his disease, but excelled at managing the limited resource he had left.

It seems like this details wouldn't have a huge impact, but it provides an excellent context for the topic of time management. Prior to this, time management always focused on getting more done. I think internally this always seemed like a good thing because like anyone else, I've had dreams of being someone that other look up to for the things I've done. The problem is that simply getting more things done doesn't improve your quality of life. What it does offer is an opportunity to get back that extremely scarce resource of time for more worthwhile activities.

What you find worthwhile is something that is up to you. For me, like the speaker, it is spending time with friends and family. Part of that in my case is playing music. As this is where I'm placing my importance, it is easy to push non-important tasks to the background. For example, our upstairs is not the cleanest area of the house. Neither my wife nor I are that excited about cleaning things up. In one sense I could feel bad that I'm not a better house keeper, but today I realized that cleaning up the upstairs doesn't really rank very high on what is important in my life. As silly as it sounds, it was one of the first times I had something specific I intended to work on and decided against it without feeling guilty. I'm not planning on avoiding anything mundane, but it does make it easier to prioritize.

None of this is really new, but for whatever reason, the practical message of the speaker seemed to resonate with me. I think in the past time management always seemed to be a chore, but now I'm seeing how it is a critical part of life. Keeping a todo list used to be a lot of pressure, but the act of keeping one and creating my own system will be helpful, even if it is imperfect. For once, I feel as though seriously managing my time is actually something helpful and practical. I think none of the ideas or perspectives were terribly different, but the context definitely was. Hopefully this time my todo list will stick and I can really get a grasp on what I need to get done. I've tried many times before, but for whatever reason it seems like now I finally get it.

Finally, just to put a bit of a programming spin on things, I'm using Emacs and Org Mode for keeping at todo list. It has a ton of great features and I found a nice cheatsheet to help with the different commands. Wish me luck!

App Server Performance Thoughts

It is always interesting to see that CherryPy is included in Python web server benchmarks. I think it is a testament to the code base being considered a standard option as well as signifying that it is a reasonably fast base to consider other options. Often times it is not the fastest option, but at the same time, rarely is the most time spent simply responding to the request. Databases and application logic traditionally takes much longer than serving the response.

More Time Management

The past day or so I've really been trying to be improving my use of a todo list. Org-mode has been very helpful and I'm trying to figure out how I can better use my iphone. I'd imagine a web app that writes to an org-mode file or something similar might do the trick, but that seems to be a long ways off.

Some Org-Mode Workflow

I've got to say that org-mode has been treating me rather well. To contrast this, I also started playing with Evernote on my iphone. In theory, Evernote seems really helpful for working with todo lists. It has been reasonably nice for things like going to the store to buy a list of things. One feature that seemed really helpful was the ability to take a picture of something as a note. I took a picture of a pedal board I was making for Lauren that had some measuremeants on it. Unfortunately, for some odd reason, it wasn't saved. I think this happened when I tried to write a note that went along with the photo. It wasn't a huge deal and I'm sure user error was part of the equation, but I really hope that the photo feature is able to let jot down some notes in addition to simply having a picture.

tutorial. I had read it plenty of times before but it didn't really click what was happening. I was always frustrated that the agenda side of org-mode required configuration. That was probably more a function of my own lack of understanding tweaking my .emacs file than a real negative. Now that I'm more familiar with Emacs Lisp, doing small configuration details in my .emacs doesn't seem like nearly as big a deal. After all, that is what it is there for! Paste's paster command. We used IronPython for it and it ended up being somewhat helpful. If I were to do something like that I think I would use Paver. In fact, this gives me an idea!

First Day Working on the Road

Yesterday was the first day I had a chance to really work in our van and even though we drove through almost all of the nothingness that is west Texas, it went pretty well. My Verizon MiFi router was a little champ and the signal was pretty good considering we were out in no mans land for much of the time.

fabric, so that might be a helpful tool doing something interesting with my VM.

My Org-Mode Server

I went ahead and started working out how to get my todo list online. I started off pretty simple and ended up with a relatively nice system. The basic idea is that I can push my org files to my webserver and edit them. Likewise, I can pull from the server. It started with some simple paver scripts that uploaded the files and quickly became an actual application.

import os from mercurial import commands, ui, hg from paver.easy import * import subprocess IONROCK_HG = 'ssh://eric@ionrock.org/path/to/todos/' REMOTE_TODO = IONROCK_HG # '/local/dev/path/to/todos' @task def server():     import cherrypy     cherrypy.tree.graft(TodoServer(base_url='/'), '/')     cherrypy.quickstart() @task def create_repo():     cmd = subprocess.call("fab create_repo:hosts='ionrock.org'", shell=True) @task def commit():     conf = ui.ui()        user = conf.username()     repo = hg.repository(conf, '.')     files = [f for f in os.listdir('.') if f.endswith('.org')]     commands.add(conf, repo, *files)     commands.commit(conf, repo, addremove=True, message='Syncing org files')     commands.push(conf, repo, REMOTE_TODO) @task @needs('commit') def pull():     conf = ui.ui()        user = conf.username()     repo = hg.repository(conf, '.')     commands.pull(conf, repo, REMOTE_TODO)     commands.update(conf, repo) @task @needs('commit') def update():     cmd = subprocess.call("fab update_todos:hosts='ionrock.org'", shell=True)     from fabric import run def update_todos():     run('cd /home/eric/htdocs/todo && hg up') def create_repo():     run('cd /home/eric/htdocs/todo && hg init') import os import re import posixpath as path import difflib from selector import Selector from webob import Response, Request from webob.exc import * from mercurial import commands, ui, hg import datetime class TodoFile(object):     def __init__(self, fn):         self.fn = fn         self.html_diff = difflib.HtmlDiff()         self.diff = difflib.Differ()         self.matcher = difflib.SequenceMatcher()         lines = [l for l in open(fn, 'r')]         self.matcher.set_seq2(lines)     def _hg(self):         conf = ui.ui()         user = conf.username()         repo = hg.repository(os.path.dirname(self.fn))         return conf, repo, user     def __str__(self):         return ''.join(self.read())         def read(self):         return [l for l in open(self.fn, 'r')]     def write(self, new):         f = open(self.fn, 'w')         clean = re.sub('\r', '', new)         f.write(new)         f.close()         conf, repo, user = self._hg()         date = datetime.datetime.now().strftime('%m-%d-%y %H:%M')         commands.commit(conf, repo, message='Web write on %s' % date)             def is_different(self, new):         self.matcher.set_seqs(new.split('\n'), self.read())     def diff_txt(self, new):         return list(difflib.context_diff(new.split('\n'), self.read()))     def diff_html(self, new):         return self.html_diff.make_file(self.read(), new.split('\n'))            class TodoStore(object):     def __init__(self, directory):         self.dir = os.path.abspath(directory)     def get_todo(self, name):         for fn in os.listdir(self.dir):             if fn.endswith('.org') and (fn[:-4] == name):                 return TodoFile(os.path.join(self.dir, fn))         return false     def all(self):         return [fn[:-4] for fn in os.listdir(self.dir) if fn.endswith('.org')] class Auth(object):     def __init__(self, creds, login_url, success_url=None):         self.login_url = login_url         self.success_url = success_url         self.creds = creds     def __call__(self, f):         def func(env, sr):             sess = env['beaker.session']             if sess.get('auth.user'):                 return f(env, sr)             req = Request(env)             sess['auth.after_login_url'] = req.url             sess.save()             return HTTPSeeOther(location=self.login_url)(env, sr)         return func     def login(self, env, sr):         res = Response()         sess = env['beaker.session']         flash = sess.get('flash', '')         if flash:             sess['flash'] = ''             sess.save()         res.write('''<div>%s</div>         <form action="%s" method="post">           <label for="username">Username</label>           <input type="text" name="username" value=""><br />           <label for="password">Password</label>           <input type="password" name="password" value=""><br />           <input type="submit" value="login" />         </form>''' % (flash, self.login_url))         return res(env, sr)     def handle_login(self, env, sr):         req = Request(env)         post = req.POST         sess = env['beaker.session']                if post.get('username') and post.get('password'):             if self.creds.get(post['username']):                 if self.creds[post['username']] == post['password']:                     sess['auth.user'] = post['username']                     url = sess.get('auth.after_login_url', self.success_url)                     sess.save()                     return HTTPSeeOther(location=url)(env, sr)         sess['flash'] = 'Error logging in.'         sess.save()         return HTTPSeeOther(location=self.login_url)(env, sr)             class TodoServer(object):     def __init__(self, **config):         self.conf = {             'todo_dir': os.path.dirname(os.path.abspath(__file__)),         }         self.conf.update(config or {})         self.auth = Auth(self.conf.get('creds', {}),                          self.url('login'),                          self.url())                 self.store = TodoStore(self.conf['todo_dir'])         self.router = Selector([             ('[/]', {'GET': self.listing}),             ('/login[/]', {                 'GET': self.auth.login,                 'POST': self.auth.handle_login             }),             ('/{name}/edit[/]', {                 'GET': self.edit,                 'POST':  self.auth(self.update)             }),             ('/{name}[/]', {'GET': self.read}),         ])     def url(self, extras=None):         extras = extras or ''         if isinstance(extras, list):             extras = '/'.join(extras)         return path.join(self.conf['base_url'], extras)     def _header(self):         return '''<html><head>         <title>org todo server</title>         <style type="text/css">         body {             font-size: 2em; font-family: sans-serif;         }         </style>         '''     def _footer(self):         return '''</body></html>'''     def edit(self, env, sr):         res = Response()         req = Request(env)         name = req.urlvars['name']         td = self.store.get_todo(name)         res.write(self._header())         res.write('''         <form action="%s" method="post">         <input type="submit" name="submit" value="save" /><br />                <textarea rows="50" cols="80" name="new_body">%s</textarea>         </form>         ''' % (self.url('%s/edit' % name), str(td)))         res.write(self._footer())                 return res(env, sr)     def update(self, env, sr):         req = Request(env)         name = req.urlvars['name']         new_body = req.POST['new_body']         todo = self.store.get_todo(name)         todo.write(new_body.strip())         location = self.url('%s' % name)         return HTTPSeeOther(location=location)(env, sr)     def read(self, env, sr):         res = Response()         req = Request(env)         name = req.urlvars['name']         res.write(self._header())         res.write('''         Home | Edit         <hr />         <pre>''' % (self.url(), self.url('%s/edit' % name)))         td = self.store.get_todo(name)         res.write(str(td))         res.write('</pre>')         res.write(self._footer())                 return res(env, sr)     def listing(self, env, sr):         res = Response()         res.write(self._header())         res.write('<ul>\n')         for f in self.store.all():             res.write('<li>%s</li>\n' % (self.url(f), f))         res.write('</ul>\n')         res.write(self._footer())                 return res(env, sr)     def __call__(self, env, sr):         return self.router(env, sr)

Google of Food

Google is often considered a benchmark in the tech industry for some epitome of a field. It is pretty clear that Google is at the top of their game when it comes to search. One of the technical reasons for this status is due to their infrastructure. Much has been said about how they are able to use standard PC hardware and Linux to create highly distributed systems. The other day it occurred to me that someone should really do the same with food.

When we were driving up to San Francisco, we went by a slaughter house. It occurred to me that it must take quite a bit of fuel to move around so many animals. It was also a pretty disgusting place. It used up a ton of space and generally seemed like an unhealthy place to be producing massive amounts of food. When we saw Food Inc., there was an organic natural farmer that had a wealth of crops and livestock for sale it seemed really efficient. Normally, people look for ways to build bigger and bigger assembly lines. After all, that is what the food industry really is, a huge assembly line for growing, processing and packaging food.

The problem is that you can really only scale one large system so far. At some point you'll have to consider the implications of using a single system that has many single points of failure. For example, when gas prices rise, it effects almost every factor of food production on a number of levels. If farmers could more easily raise their own livestock using local food sources and do the slaughtering and packaging on site, you end up cutting out massive amounts of fuel shipping thousands of pounds of food.

It seems like someone should do something similar using local smaller farms. A distributed network of self sustaining farms could very easily help reduce the costs of transportation and improve the quality of the food. There would be a different kind of problem organizing these small farms, but the issues, while complex, would be orthogonal to problems of creating food at a large scale. This is the same problem companies like Wal-Mart and Target have had to deal with, so applying similar techniques to food seems not only plausible, but very profitable.

The big unknown is how the government could facilitate this movement. It already is primarily responsible for the current state of the food industry because of its subsidies and attention to storable food. Smaller farmers can make the cycle of production simpler which means if one farmer fails or there is a small shortage, you are dealing with a smaller scope that is most likey limited and easy to contain. With this in mind perishable foods can be given attention and small farmers would have a means to participating in the market as a whole. The big necessities from government would be to try and improve the patent situation for seeds, fix the subsidy laws that fund the major food corporations flawed practices and make an effort to reward those farmers who are able to be profitable without relying on government subsidies.

I'm not an expert on this stuff, but it seems like the principles might very well apply. My guess is that as organic foods continue to become more important, companies like Wal-Mart will have to find a way to give customers healthy food on a large scale and faciliate the creation of orginizational tools necessary to feed a nation on the spoils of many small distrubuted farms.