Ionrock Dot Org

My Web Log

If a one liner is long...

Today I came across a rather large line of Ruby code and it made me think. The line was primarily a long string, but I have seen many lines that are really long doing a set of chained operations. So, the question is, if a one liner is exceptionally long, is it still considered a one liner? The term "one liner" refers to one line of code doing an extreme amount of work. Things like Golf are good examples of writing one liners.

Personally, I don't think it can still be considered a one liner. But I'm also pretty rigorous at keeping my lines in the 80 column category, so I'm rather biased. With that said, I would also argue that placing everything on one line doesn't really make it clever, which a one liner should generally be. The idea is that you've expressed an operation in an extremely concise way, that in most cases, uses the features of the language. So, my answer is no. What about you?

Posted: 2008-04-22T18:43:52Z

Ruby Priority Queue

Once again Uche brought an interesting bit of computer science to my attention in BC. He started using the Python heapq module to help speed up an index in BC. I had never used the module before and wasn't exactly clear at first glance how it worked. I took some time last night to implement something similar in Ruby to help get a better understanding. I ran some exceptionally unscientific tests to see if it was incredibly slow and as far as I could tell, it seemed quick enough to use. If anyone is interested in grabbing a copy you can download here . I don't plan on making any updates or maintaining something serious here, but if others have suggestions or notice any errors please send them along.

Posted: 2008-04-15T20:16:36Z

The Problem with Leaders

Like the rest of us, I've been hearing quite a bit about politics lately. I'm a staunch supporter of keeping my political beliefs to myself, so this is not about what or who I support. Instead, I'd like to point out a pattern that is difficult to overcome.

I think Lessig has essentially brought this up with the Change Congress movement. The issue being that people are expected to vote based on some semi-qualified basis. Yet, this means that they have the responsibility to educate themselves on the issues along with how the parties and candidates plan on dealing with them. In short this doesn't scale. We don't have time to do a lot of research so we depend on other sources to help us make a decision.

The two basic categories of sources are media and associations. The media provides coverage of the issues, candidates and generally everything people are interested in. They are commercial though and must cater to the desire, not the needs, of its audience in order to survive. Associations are groups such as churches, employers, unions, clubs, or anything else that functions as a group. The issue here is that as people participate in the group, they essentially offer their impact to the group leaders in the form of endorsements. A good example is when the Teacher Union, AFL/CIO, or the AARP endorse a candidate. The group has offered to support the candidate as whole, and while the members are free to choose as they wish, the group has been the target of persuasion by the candidates and political parties.

The problem is essentially a scalability problem. No one has the resources to effectively communicate because the resources are not available in one way or another. Candidates and parties cannot speak to the public as a whole, so they turn to groups in hopes the members follow the leadership. The media must present what will be watched by an uninterested audience in order to support itself through advertising. Again, the message becomes tainted and/or misdirected in order to meet orthogonal needs.

There really is no solution, but the recent social networking movement and technology has been somewhat helpful in changing the landscape. By empowering more people to be producers of content and news, there are more opportunities to communicate messages that are unencumbered by outside forces. This doesn't imply there is truth in the messages. It simply means they are not impacted by a need to survive. The problem now is that there is a wealth of information and no way to meter and decipher it all. Again, I don't really have a solution outside of continuing to improve our use of semantic technology to find meaning and allowing interfaces that meet the needs of users.

This is actually what makes technology so interesting to me. It is the center of information, which has proven to be powerful in almost all facets of life. While answers are not easy, it is exciting to think that, at the very least, I'm working towards changing the way society works by helping further the discourse.

Posted: 2008-04-11T04:59:25Z

XSLTemplates 0.6

The other day I released XSLTemplates 0.6. The big change is including a template constant middleware that accepts a dictionary of items to include in the template by default. It automatically includes the WSGI environ dict in the template params.

The big question is how to use it? Well, one tactic is to go ahead and add one instance around the whole app adding any configuration details. In Bright Content we list the URLs of some services that will be used in the paste config file. I then wrap my BC instance with a TemplateConstants middleware that places environment dict and application configuration details found in the .ini. You can also use it more granular by wrapping different classes with their own instances. Really the idea is to use it where you have the same parameters in many different calls so you don't have to always remember to add them to the set_params call.

The next big change is using the xsl:output element to set the content type of the output. Currently you set the content type using the calling middleware. This is not very WSGI-ish b/c there is an actually calling middleware in the sense an application controls the value. It would be more in the spirit of WSGI to pass along that control to the template renderer. With that in mind, you can set the media type attribute in the xsl:output element and that will be the content type used in the response.

This is probably the last release of XSLTemplates for a little while as we migrate things into Akara. After that, I hope we can create some built options to allow relatively granular installs of different Akara components for use in WSGI apps.

Posted: 2008-04-03T18:31:21Z

REST is not Made Up of Verbs

I have been doing a few interviews as of late for my day job and it really hasn't been going well. I'm going to go ahead and take responsibility for the fact that my interview skills need some serious work. Beyond this though, no one knows much of anything about REST and/or HTTP. While I wouldn't say I'm an expert, it is clear that my knowledge ends up faring well against most the senior developers that I've been interviewing.

The point here is not to place myself upon some kind of expert pedestal. In fact, the point is just the opposite. I'm not an expert. When I was learning PHP I heartily ignored HTTP any way I could because I felt it was a waste of time. When I was forced to edit a few Perl CGI scripts, I felt the explicit Content-Type headers were silly. Little did I know that the explicitness was not explicitness at all, but rather simply using HTTP in an application. I doubt the code I edited revealed this subtlety at the time, but, it still was an opportunity for me to dig deeper that was ignored.

In learning WSGI and Python web development, it became clear that I would need to deal with HTTP. The first part was using simple status codes. This seemed like a hassle at first. It became quickly apparent that I could use HTTP to communicate much of what I meant simply by providing a decent status code. My next big step was working with the content type. I saw very quickly why all the PHP scripts for forcing a download actually worked. In the past this was semi-understood, but now I saw the real value. I also learned about content negotiation and how powerful it could be in creating a RESTful service.

All of this was learned within the scope of the AtomPub spec writing process. I watched as folks discussed caching, URI templates and linking. It became clear how using and understanding HTTP makes creating RESTful services intuitive, simple and efficient. There is no silver bullet, but understanding RESTful Web Services does a great job of doing the most with what you have.

So, what are my suggestions for really understanding REST? The first step is realizing the world is not about CRUD. REST doesn't get you anything better than SOAP or XML-RPC in this case. One could argue that, since browsers don't support PUT and DELETE out of the box, RESTful services are more difficult. I'm not saying that RESTful CRUD is unimportant, but simply it is not the real hype.

The big deal about REST is Linked Data. When you consider the division between a resource and a representation along side using linking to define the flow of state, the stateless web becomes a more understandable application platform. You can think of resources simply as URIs and representations the file you get back from the request of the resource. This is somewhat over simplified, but more or less this is how you think about things in REST. The next thing to understand is a representation or resource is not an object. It sure can be, but it is not by default. This may seem odd from an OO perspective, but the web is a creation of documents, not objects. Sure, these documents are often compositions of object data, but from the standpoint of a RESTful service, they are all documents.

This distinction is huge because it sets expectations. You will never just assume you can point some function at a URL and magically get an object. I'm not saying you won't write a method or two to make a GET request and build an object. You will and that is fine. But the fact is, you understand that when the document changes formats, you might have to change your code. Might is the operative word here because being RESTful means making an effort to be robust.

When you think of the web in terms of documents, the work flow of how to deal with data changes. Instead of thinking in terms of an object, there is an actual document that you reference. Any sort of language specific tool you use to work with the document is separated from the document itself. When you consider manipulating something like XML, you have tools that let you maintain the document as a whole. Contrast this with ORM tools that take small pieces of data and place them in data structures. When you are talking about the web, working with documents is much simpler than thinking in terms of serialization between a view, data structure, ORM, and database.

Another area I think causes confusion and misunderstanding for REST comes from the fact that it is more functional (in the Lisp/Haskell/Erlang sense) than object oriented. Functional languages require that variables be immutable. This is a direct parallel to HTTP requests. A web application cannot pass along a reference to some object that is tethered to an instance on the server. This means programming in this fashion is only working against you.

What is interesting about REST as a movement is that it is not meant to be innovative. REST aims to analyze what is successful about the web and apply the principles to web application development. In a way, it is what folks came up with after a code review of the web. I am hoping that more people make an effort to understand REST for what it is. In many cases it really takes some effort to think in RESTful terms. This is especially true when you are accustomed to thinking in objects. Hopefully, more people will continue to catch on and see the benefits, so we can finally move on to even better misunderstood principles for the Web 3.0 code review!

Posted: 2008-03-21T19:44:50Z

Blogging From Emacs

Thanks to the magic of Pymacs I can now script Emacs with my favorite language. It's rather thrilling to think about in a way because I spend so much time in my editor. Part of me wonders if I am wasting my time with Emacs. Things like better auto-completion could be nice and there are times where a more robust viewport into the filesystem could really be helpful.

What makes it all worth while is knowing that I am learning how to solve problems. When you have an IDE that is configured for you to automagically put all the pieces together, there is always the chance you misunderstood what is actually happening under the hood. This is usually not that big of an issue. It is pretty harmless because you can usually figure out the problems when they arise. Yet, even though you have good documentation and some helpful hackers at your side, there is still a reasonable chance that you go down the totally wrong path because you made incorrect assumptions.

I think Emacs helps to avoid this kind of problem. I can't tell you how many times I've seen some slick screenshot of a cool mode I want to try. Most of the time they don't work, but everytime I give it a go, I learn a little more. While I really can't code elisp, I have been able to undestand a good deal more about it by working with Pymacs. It is these little battles that make me realize the struggles I have with Emacs force me to learn new things and think methodically about how to solve problems. It also makes it clear how important a user interface can be when trying to accomplish a specific task. None of this is rocket science, but as someone who loves to learn, it is a ton of fun.

Posted: 2008-03-20T01:09:17Z

Bright Content Deployed

So, this week the Semantic Technologies conference went live. This is the first commercial deployment of Bright Content as well as its first foray into more generalized content management. I did want to take a second to say thanks to Uche and Eric for having me on the team. A big thank you goes to Zepheira for using Bright Content and seeing its potential.

The big question then is where you can see Bright Content on the Semantic Technologies Conference site. The answer is everywhere! The site uses XSLT for rendering content via a core data file and XSLTemplates . Each page then is created by adding a new template that extends the base XSLT. The templates are stored in an AtomPub store and could potentially be administered using AtomPub, but for our purposes, we left that out. Beyond the pages there are a few programmatic helpers out there such as scheduler, which uses Bright Content to mess with iCal files.

The other big feature of the site is comments. More specifically, we are referring these as annotations since they are really more than simple comments. Again, we are using Bright Content's AtomPub store. The comments are simply posted to the comment store and can be queried via a key to get a feed. The comments are moderated via Atom's control element.

This was all very simple due to Bright Content's core libraries. The first main library is XSLTemplates that has become relatively simple and easy to use. The other primary core lib is the Bright Content Atom store. The store is based on Amplee and has helped drive its development. Specifically, we have used Amplee for its indexers. Amplee allows you to create indexes that can be driven by whatever sort of scheme as needed. This means indexing pieces such as Atom categories or in the case of commenting, the Atom Threading Extension in-reply-to elements. The model can definitely be applied to generic XML, but at the moment we are using Atom. With that said, the indexing can occur on both the Atom entry and/or the actual content. So, in theory, one could do some crazy image processing to see if the picture is a person or not and make a feed of people pics.

The indexing piece of Amplee is nice because you can specify how the index is stored as well as how the actual data is stored. We are using a file storage and memory for indexes, but Sylvain has hooked up S3, SVN, and a wealth of other basic storage facilities that are all very nice. This means if one index is rarely used and large, we can persist to disk or S3 while those we use all the time can be directly in memory for faster look ups.

At the moment we are still actively working on getting Bright Content to be something someone could easily install. For the moment though, it is becoming apparent the XML with indexes model is very robust. Our pattern for applying business logic seems to be pretty solid in that it was easy to layer the store with our requirements. Overall, I would say the project was a huge success!

Posted: 2008-02-26T21:52:58Z

Paying Attention

Lately I've been constantly making note of my opinions. It is a frustrating practice because it really means realizing how often opinion stray to the negative. I've been told I'm a "nice guy", so at the moment I feel safe in my internal dialogs. Yet, it is obvious that if left unchecked, a person could easily bring negative stigma.

One piece that is often portrayed in films covering successful individuals is the aspect of drive that consumes. This is usually measured by the person's relationships, which traditionally fail in one way or another. It is interesting because in monitoring my opinions, my quest to be a better developer causes me to neglect my relationships. In this case, I am very glad my internal opinions caught my attention. Although it is a huge challenge to become a truly great developer, it is even more of a challenge to do so while treating those you love like they deserve to be treated.

I should point out my "neglect" is never a factor of my actual opinions of anyone. It is really a physical issue where my lack of sleep simply puts me on edge and I get rather snippy and annoyed easily. The truth is I have great friends and family who I could not live without. More over, like most successful folks in the movies, I have an amazing wife who seems to pull me together as I consistently fall apart. She makes life easy, fun and enjoyably challenging all the time. So, baby, I know I get tired and snippy sometimes, but I love you and always find myself in love with you.

Posted: 2008-02-13T21:17:27Z

Java Feel

Over lunch today I watched the closure controversy . What struck me was the nature of the language design and how its creators think of its use. The speaker made it clear that Java was designed with users in mind. I personally have never found Java very usable, but that is simply my opinion. Java's success makes it clear that its usability is a viable feature.

The speaker also alluded to the Java designers acknowledging that most developers were not very smart. Of course he did not say this directly, but the essence of what he mentioned was that Java was made for the "common" programmer. Personally, I have always taken great pride in my work as a developer. I have been diligent about learning technologies, languages and patterns that help to develop my skills as a programmer. For this reason, I am attracted to tools that provide grace and elegance in their power. This is probably why I have never liked Java, its design focused on a tool for getting things done. It has no concern for the quality or elegance of the output.

I'm not saying programming is brain surgery but, if it were, would you want your doctor operating with a drill press and a table saw on your head? Likewise, would you want to see sky scrapers built with only wood and nails? This concept is faulty at its core because without progressing the field into the complex, there will never be any progress towards stability and eventual elegance. We as people can see the art in architecture, fashion and even medicine. None are considered common trades that anyone can do, yet we settle for mediocrity in programming because of the perceived lack of impact.

The problem with this is that there would most likely be a serious lack of available developers. I wonder if that would be an appropriate cost compared to the amount of money spent fixing bugs and poorly programmed applications. One solution to provide a reasonable hierarchy of developers would be something similar to the medical field. There are different classes of nurses with each being qualified to do different types of development. While this is somewhat true today in that most developers are thought to have started with testing and moved up towards and architect, it might actually be worthwhile to have some sort of a guild or union type group to help make the qualifications more official and performed by those who actually understand programming. This could easily cause more problems then it would solve, but again, the costs of bad programming is probably much higher than one would think. Even if a guild type system was created and caused something of a stir, the potential money saved from lost data, broken systems and better designed applications could potentially make the market a much better place for those creating and using software.

Posted: 2008-02-04T21:07:25Z

Documentation Matters!

The last few few weeks Uche and I have been working on a project. It has been a great experience in that I have learned a ton. It has also been extremely frustrating because it has made clear my skills documenting are terrible. The biggest issue is that I simply don't enjoy writing documentation. There are a very select few who really enjoy documenting the mundane and technical, many of whom I've had the privilege of helping. It is too bad that I didn't manage to pick up a desire to be a better documentarian.

The biggest issue I have is adding enough detail. I might outline the basics, but you always have to consider when something goes wrong. This doesn't mean that every possibility needs to be documented, but having a few obvious routes for debugging can be a real help. Again, what denotes a piece of information is always going to be subjective, but one man's trash is another man's treasure, so you never know when the small reference to a specific class that handles a critical piece can help someone save an hour.

I doubt I will ever be a pro when it comes to documenting. I would rather write code for the time being. It is clear though, that as my code becomes used by more people, it will need better documentation. While now is not the best time, hopefully I'll catch the documentation bug and get my head out Python/Ruby mode for a little more reStructuredText/Wiki Text.

Posted: 2008-01-29T22:37:01Z
next