Ionrock Dot Org

My Weblog

Losing Faith in HTML, Javascript and CSS

Lately, I've been doing a good deal of front end development and it has gotten to the point where Flash (Flex more specifically) is looking really appealing. The issue is that you can't develop and test an issue once. No matter how much you consider browser quirks, utilized tested javascript frameworks and known CSS techniques, you're still likely to run into browser bugs. Running into a browser bug is not a big deal in many cases, but as the code becomes more interactive, debugging an issue in one browser often introduces regressions in another.

For a huge majority of sites, this isn't an issue. When you think of news sites, for example, there is a limited number of interactions you need from the user. The primary use case is consuming content, which will most likely come in rather standard forms such as text, audio or video. When the primary goal of the site is to consume input from the user, things can get a little tricky. This is especially when you consider sites that make an effort to innovate the ways you interact with the application. Clearly, this is where the web is lacking and where the web is moving.

One proposed solution is HTML 5. This might very well help, but the problem is that HTML 5 will not necessarily provide tools for creating entirely new and innovative user interfaces. I suppose the canvas aspects might very well be where that can take place, but it seems like it doesn't really add anything new that is not offered currently through Flash.

In the past, I've tried to avoid Flash and its non-standard friends. Java applets seemed terrible when I first encountered them and while I did have an appreciation for some of the more interesting Flash designer sites, it wasn't text. At this point though, I'm questioning this allegiance to simple characters. There is a voice in the back of my head screaming about i18n and accessibility, but at the same time, there is a timer that constantly reminds I could be doing more interesting things with my time than tweaking pixels in Javascript and CSS.

I believed jQuery would save us all, and for the most part it does a pretty awesome job. The problem is not that the tools haven't improved, but that the model of rendered HTML has hit some pretty serious limitations. Manipulating the DOM has been pretty successful, but the rendering is never consistent, which means extremely defensive programming and eventually bland interfaces for users.

At this point Flash feels a little like Java. You feel a little dirty using it compared to something like Python, simply because there is a definite black box involved, yet, Flex is a great step and might be enough. There doesn't seem to be a huge difference in my mind between writing HTML for a proprietary browser and writing ActionScript for a Flash engine. Still, I would feel better if there were a Mozilla of Flash.

I'm going to try and work more with Flex if possible. There is still a voice in my head telling me that the accessibility is a problem and google can't read it, but I'm at the point where doing anything non-trivial in HTML and Javascript seems like I'm spinning my wheels. I'm sure working in something like Flex has plenty of issues. It did feel pretty similar to many of the GUI toolkits, non of which I had huge success with. At the very least, I know when it is working for me, it should work for everyone. That is a pretty nice feeling.

When It's Time To Start Over

I recently read about a campaign to fix outlook and it got me thinking. Why hasn't Microsoft considered starting fresh on any of their software? My personal choice would be a reset on Internet Explorer. The browser wars are over and Microsoft won. It's too bad the web lost. At work we've been considering trying out Flex for some more intensive UI elements and it occurred to me that the consistency alone would probably be worth losing the hacky tests we have to keep up in order to feel confident releasing javascript intensive code on multiple platforms.

Random web developer wish lists aside, it is tough to recognize when some piece of software should be rewritten from scratch. Weighing the pros and cons rarely provides a true measure of the whether you'll be successful or not. Yet, even without a good means of measuring needs, it is clear that a rewrite can be very helpful. I think Apple provides an excellent model for rewrites. They have successfully rewritten their operating system, a web browser, and an office suite, to great success (in my opinion at least).

One theme in all these rewrites has been the inclusion of other pieces of software. Gecko (khtml) has been critical to providing a new suite of tools simply by making it possible to pay attention to other aspects of the applications. Likewise, OS X utilizing FreeBSD was not necessarily innovative, but rather an effective means of raising the level of abstraction.

Like programming languages, abstracting away low level detail enables thinking at a level closer to how people think. In programming, this allows programmers to reduce the complexity. When you can safely say "x = 5" without having to think about memory management or the scope of the variable, you create space in your available mind for other details. In the same way, introducing a well established library or piece of code can help take an application to the next level by moving the innovations closer to the user.

Going back to Microsoft, I wonder why they have not really made efforts to capitalise on these kind of changes in the software landscape. I'm sure lawyers would be involved in the equation, but as a participant in the software landscape, it is clear it hurts users. It is always difficult to standardise on things as competition and differences are where innovation occurs, but just like programming languages, when there is a low level standard to build on, using something else just adds complexity.

I think the time to start over is never clear. One sign that it might be worth it though is a low level library or system that might free up your resources for features closer to the user. At the end of the day software is there for people to use, so anything you can do to improve the experience for the person using is worth more than keeping a code base out of convenience.

Seriously Mutable

I ran into this today and wanted to write it down somewhere both for myself and posterity (ok, mostly for myself).

As you may know, python dictionaries are mutable. This means you can do things like this:

x = {'foo':'bar'} def change_it(d, new_value):     d['foo'] = new_value change_it(x, 'baz') print x # should print {'foo': 'baz'}

As you can see, the variable x can be changed anywhere. With that in mind what I was doing was basically this:

x = {'foo': 'bar'} def change_it(new_value):     y = x['foo']     y = new_value change_it('baz') print x # we get {'foo': 'bar'} as expected

So, this might give you the impression that when you access a value in a dictionary that value is not a reference to the original. But, what about this?

x = {'foo': {1: 'one'}} def change_it(new_value):     y = x['foo']     y.update(new_value) change_it({2: 'two'}) print x     # do you expect {'foo': {1: 'one'}} ?

You will actually get:

{'foo': {1: 'one', 2: 'two'}}

The reason this happens is because the reference you get back is a mutable object, so in the above code, the variable "y" points to the same mutable dict "x['foo']" points to. We can test this with this bit of code:

x = {'foo': {1: 'one'}} y = x['foo'] print y is x['foo'] # True

When you consider python always passes by reference this makes sense. At the same time, it can be a little sneaky if you're not thinking about it since it is usually pretty simple to just use code like "y = x['foo']" and expect it to act like a copy.

Thanks to a few folks in #cherrypy on oftc.net for clearing up my explanation on this.

More Fun with Mercurial

The other day I had something of an issue at work. I was working on retooling our testing environment when there was a need to provide a fix for something in production. I couldn't reproduce the issue, so I decided to add some extra logging to help try and gather some data on the issue. With the code in place, it became clear that I didn't know how I was going to move those changes to the production repo while keeping my other work safe.

After looking into the issue further, I thought rebase might be helpful. Rebase is a great extension, but it wasn't going to provide a fix (that I know of). The rebase extension allows you to choose the order of two existing heads. The classic example is when you are working on a feature, you pull to get the most recent changes and you want to upgrade to the latest from the remote repo, while keeping your changes "in front" or after the pulled code in the history. My description might be a little off, but it was how I understood the process.

In my situation the scenario was as if I already rebased and did so incorrectly. Fortunately, the transplant extension came to the rescue. What I wanted to do was effectively recreate my local repo and correct the order of commits so my unfinished work was "in front of" my production fix. To put things plainly, I had a sequence of commits 'ACB' where 'C' was unfinished, so I wanted to move it to the front and have 'ABC'. What I gathered is it is not really possible to reorder the commits since the time is always attached to the changeset. But, I was able to push my production fixes without having to push my working changes, which was good enough for me.

I started by cloning the remote repo. Then I transplanted the production changesets I needed from my local repo. Then I pushed back to the remote repo. I then transplanted the rest of my changes to the new clone. Just for good measure I pull my new remote changes into my local repo and merged to see what would happen in terms of history. It actually made it clear that things had been transplanted at different points in time and reordered. Here is what it looked like:

> ls local > hg clone ssh://user@remote/hg/repo remote > ls local remote > cd remote > hg transplant -s ../local ... interactively choose changesets to apply ... > hg transplant -s --continue # if any merges failed > hg push

Being able to push my production changes without having to also push my working changes means the person doing the release can merge with default without having to exclude my working changesets. This doesn't seem like a huge win, but I think it is pretty helpful way to avoid someone working with changesets they didn't write themselves. It seems like it is a decent work flow as well. Keeping your own "production" or "pusher" repo as an intermediary for a remote production repo can be a helpful way of making sure you introduce atomicity while still keeping your changes in VC. I've found the more commit points you create, the easier it is to see where things might have gone wrong. The downside is that your changes might become interspersed with other changes. Rebase definitely helps this case and I believe using a local production repo for pushing also provides another means of keeping merges simple and obvious.

TwitrView: A Small Announcement

When I first started using twitter for Ume, I thought it might be kind of cool if people did interviews over twitter. The character limit is an interesting constraint, there is a real time aspect, and the person getting interviewed can review an answer. For myself, I enjoy handling email interviews because you have a chance to edit your thoughts, which means readers get a more interesting and informative response. With that in mind I set out to write a crawler of sorts to monitor a set of twitter accounts and compile an interview between some people.

The idea for a crawler type of application was also partly stemmed from my desire to get more comfortable with threads. In addition to my own perceived concurrency issues, I started playing with BerkleyDB, and later, Tokyo Cabinet. In the end though, simplicity won and I dropped the persistence and just do everything on demand.

The result is TwtrView! All it does is pull the last 50 tweets from the users you enter and see what messages include @replies to any of those users. This makes it possible to see an interview as well as find conversations folks might be having. It actually has been somewhat useful because when I see a conversation someone might be having, I enter the two usernames and I can see where things came from.

There are definitely some limits. I'm using on twitter account for the API calls so if something doesn't work, it might have had too many requests. If people use this, then I'm sure OAuth will become much higher on my list of things to do. Also, the requests are made from my server, so it is very much network bound in terms of processing.

I hope someone else might find it useful. If you have any issues feel free to send me an email or comment below.

Where is the PC for the Web

I just read this article by Seth Godin regarding the Next Google. While it is a lot of fun to talk about the "next" big thing, what struck me was the competitive perspective Microsoft has taken regarding Google. Microsoft became the definition of PC through partnering with IBM compatible computers. For whatever reason, it was appealing to IBM and the computer industry to effectively work together to create an ecosystem for personal computing. It is unclear why the web has ushered in an environment of extreme competition when the societal impact is easily as important.

In my mind, the collaborative nature of both PCs and the web seem very similar. Both considered the environment of available more important than exclusivity. This addresses needs of users to feel that what they create and consume is something they control. All and all, it makes sense that when you enable people to communicate under their own terms, the environment is extremely valuable.

So why then when Microsoft had such a huge portion of the market chose to focus on web presence instead of collaborating with folks like Yahoo! and Google? Why did they waste the time fighting a browser war? Why did win the war and leave the winning browser to rot? It just doesn't make sense to me. Nothing they have done promotes the users need to communicate on their own terms. Google meets users needs by providing a way to find things to consume. If Microsoft were acting as they did before, it seems like the focus should be on integrating their creative software with the web. This is not a web based word processor, but rather an obvious way for people to create web content that is Google compatible.

In reality, I'm OK with how things have gone. I could do without testing Internet Explorer, but I am glad that Apple has gained market share. I'm glad that Microsoft has been ignoring its duties as the singular provider of operatiing systems in order to create subpar web entities. The lack of success seems to only help new systems to become the "next" big thing. With that said, I do wonder what sort of web we'd have if Microsoft had taken a compatability approach to the web instead of the exclusivity path.

More on Using Linux

Recently, I started using my Linux partition on my macbook and so far, things have been running pretty smoothly. I'm enjoying stumpwm with my two monitors and I'm pretty comfortable with the setup. That said, there are always some small oddities and frustrations that seem to creep in.

My biggest gripe is sound. It is unclear what Apple does to make the internal speakers sound reasonable. What is clear is that Ubuntu is not doing the same thing. While I was pretty happy my Apple keys worked for turning up and down the volume, the actual sound out of speakers is pretty terrible. It nothing headphones doesn't fix, but still a hassle.

Another issue is the sound server. In linux there are a set of back ends that "talk" to your sound hardware. Gnome (or Ubuntu) seems to adopted Pulse Audio, which is a compatibility layer on top of the lower level back ends. From a programmers prospective, this abstracts some of the more complicated issues and allows for a simpler API. From the users perspective, things just work and all seems to be well. In actuality, what really happens is that Firefox seems to eventually stop playing sound (ie Flash) and both Firefox and the Pulse Audio session need to be restarted in order to listen to hip new indie music. I'm sure much of this process has to do with the Flash plugin, but from the standpoint of a user, it's a pain.

Generally, I've found everything to be rather stable, with the exception of Firefox. It seems to always crash at some point. Flash is most likely the culprite, although I really don't spend very much time on sites using Flash. It just occurred to me that it is possible ads might be the main source of bad Flash, so I'd wonder if an ad blocker plugin might help. I'm not really a fan of ad blockers since advertising is such a critical part of the web ecosystem, yet I'm downright sick of Firefox crashing, so it might just be the way to go.

There are other issues of course. I'd like to gnome terminal, but I can't get the fonts to look like a regular xterm. I'm using Gnome with stumpwm so Adobe Air apps don't complain. This adds the gnome tool bar which is rather unnecessary. Gripes aside, it has been really easy to get back into Linux on the desktop. It is a lot of fun to have a system you can tweak to your liking. It is also kind of fun to have things to work on. While my sound issues are definitely a pain in the neck, it also gives me something to try and fix, which generally means understanding more about my system. Most of the time it is not the most useful knowledge, but often times it is really helpful stuff. For example, setting up my VPN to only be used on selected traffic. I can't say I'd reccommend using linux to everyone, but if you are using OSX because it's a *nix, it is worth trying out linux on th desktop.

Checking out the Zoo

Over the weekend we went to the Austin Zoo. It was a really good time. One of the tigers was in some play area where they put meat in the trees, so he was really active "hunting" for the surprises.

The Austin Zoo isn't a huge zoo. There is not an enormous facility or any sort of major features. From what I gather, it is much more focused on helping animals that were forced to enter into captivity. This makes zoo feel a little closer to a rescue facility than a typical zoo. Unfortunately, this also makes it clear that the zoo needs better funding. You can see cages where they simply put sheet metal on top with some large rocks to secure the makeshift roof. There empty cages that serve as storage for tools. Sometimes the cages seem a little smaller than expected. While I'm sure the zoo keepers know what they are doing, it does make one wonder what kind of oversight is involved in running a zoo. I also wonder if the Austin Zoo is really associated to the city or it is just a name some group was able to use.

In any case, my impression is not meant to be condemning. Instead I'd hope someone reading this might consider how they can help. We made sure to spend some money in the gift shop as well as make a donation. It wasn't much, but if more people visit, it can only help improve the situation. And despite the humble surroundings, it really is a ton of fun.

I know for a fact that non-profits are hurting right now. The economic downturn really puts a damper on this critical part of our society since they are already being run through the extra resource people have. When it seems like there isn't any extra, that means the non-profits get cut out. This is really a shame if you ask me. If you work, you are likely to pay taxes and that means funding non-profits as the government is a pretty huge contributor. When you pay taxes, you effectively lose the choice of where that money goes and in effect, you might very well be funding something you don't believe in. When you contribute to a non-profit, it is your choice. You can remain local if you'd like or try helping a global issue. The important thing is that you get the chance to make a choice.

This kind of turned into a rant, so my apologies. I'm not asking for money or anything, just sharing an observation I had at the zoo.

Catching Up

Since SXSW, I've been in something of a catch up mode trying to finish up a major upgrade to the latest jQuery at work and it has been pretty time consuming. I work on a pretty Javascript heavy application at work that has gone somewhat stale. They say if it's not broke, don't fix it and we did just that. The impetus then to go ahead and take the plunge to upgrade jQuery was a bug in IE 8 that had to do with the scrolling plugin we were using. A few tests were put together and we found that the latest jQuery would do the trick, so that started us down the path. We found out later the bug was actually an IE 8 release candidate bug that had been fixed in the released version. It was somewhat disheartening to do so much work on a bug that wasn't actually a bug. Since we have a shiny new jQuery, I'm not all that upset.

In our case the 80/20 rule was very much true. The first step was to change the script tag to the latest version, run the tests and see what breaks. After changing some of the API calls, the vast majority of things worked. I'd say around 80% in fact! The things that didn't work were a bit more troubling. One of the major changes was that we moved from using the old interface library to using the new jQuery-UI. This seemed to be a good idea as there were some helpful updates to the widget APIs as well as an entirely new widget and theme pattern that could be helpful. Unfortunately, what caused the most trouble was upgrading our code that used the old widgets.

One interesting aspect the jQuery team added was the inclusion of themes for UI widgets. This required making a difficult decision regarding how a UI widget is implemented in the DOM. The basic options are to 1) ask the user to declare the different components on page or 2) as the user to declare a container where the widget will fill in the required DOM elements. The first option is a tedious one since you will need to code for many different cases and define not only the required elements, but rigidity of the markup. The second option is also difficult because you have the obligation to expose the HTML the widget will write in order to allow setting the state of the widget (ie when a user goes back). The jQuery team chose the second option and it seems to be the correct decision. While it definitely caused me quite a bit of grief understanding how the widgets changed, in the end, the pattern seems like the best trade off.

Another aspect of the upgrade that I thought was interesting was the disparate documentation out there for making a jQuery plugin. I have been writing most of my javascript code as jQuery plugins in order to keep everything within their framework. My reasoning is that I can push the cross browser issues to the library and hopefully save myself some time. The problem is that there are quite a few plugin tutorials that all offer slightly slightly different views of how to properly write a plugin. After reading some mailing list posts and seeing the new widget API in jQuery-UI, it is clear that this problem is being addressed, albeit with caution. The unanswered questions regarding how to write a plugin stems from the flexibility found in Javascript. You can write code in such a wide variety of styles that choosing one or even recommending one is difficult. While I was not able to utilise it, the jQuery-UI widget API is a huge step towards providing a plugin pattern that I hope will eventually be extended to general jQuery plugins.

This upgrade has been pretty tough and I'm sure part of that was my own fault. My tendency was to try not to change the way the widget worked. My logic was that the algorithm had worked previously, so changing things radically would only present the opportunity for new, unknown bugs. While I still think this is true, one side effect of reimplementing things is that you have the opportunity to see what the underlying library is doing with less distractions as well as understanding potential pitfalls the previous author might have already solved. It is also relatively painless to make a go at reimplementing things and drop it if it becomes obvious you are on the wrong path. As I move forward I think my goal will be to make creating one off revisions of things as quickly and easily as possible. I'm not sure how best to do that but I have feeling if I streamline my experimentation workflow, the result will not only be more ideas getting fleshed out, but a better understanding of bugs that I'm fixing.

Finally Learning Lisp

I've tried and failed to learn Lisp more times than I can count. At this point there are no theories as to why it has been such a challenge. Fortunately, I'm pleased to announce this last foray has proved to be much more fruitful.

It started with the simple task of wanting a better view of a csv file. At work we export data and one of the simple formats is csv. Since my computer life increasingly revolves around Emacs, it seemed like it would be nice to find a way to convert a buffer with a CSV file to a table in org-mode. Emacs makes it really easy to send a selection to shell command, so the pattern was already in place to implement this in Python. After my Python version was done though, it seemed clear it should be relatively easy to write it in Lisp.

The first steps involved doing a little string manipulation. Taking a line and turning it into an org-mode table line was pretty easy:

(defun addpipe (line)   (concat "| " (mapconcat 'identity (split-string line ",") " | " ) " |")))

That effectively splits the line by commas and immediately puts it back together, replacing the comma with a pipe. It then adds a pipe on either end.

Next up was to select the text in the current buffer, split it by new lines, and format each line placing it in another buffer for viewing.

((defun csv2org ( )   (interactive)   (setq buffer-text (buffer-substring (point-min) (point-max)))   (setq prebuffer (delete "" (split-string buffer-text "\n")))   (setq mapped-buffer (mapcar 'addpipe prebuffer))   (with-current-buffer (get-buffer-create "*csv2org*")     (dolist (line mapped-buffer)       (insert (format "%s\n" line))))   (set-buffer "*csv2org*"))

I'm sure smarter folks than me a more elegant way of doing this, but it worked for me and fit my understanding of the problem. For example, creating the "*csv2org*" buffer most likely has its usability issues. That said, I'm pretty happy that I was able to get what I wanted basically working. Obviously there is a great deal more to learn when it come to working with Emacs in Lisp, but this experiment is getting marked down as a success.

While there were not any major revelations, things started clicking more than before. One helpful bit was becoming more comfortable with executing Lisp in Emacs. Using the scratch buffer, it became easy to play with the function and setup a simple development pattern for trying things. Also, all the Emacs and Lisp bookmarks I've recorded over the past couple years ended up providing useful tips for getting further along than before.

For my next trick, I'm going to implement a simple way to paste code to http://rafb.net/paste. Fortunaely, I've already started making progress and despite not finding much information on the URL Package, it seems like it shouldn't take much to get it working good enough.