Ionrock Dot Org

My Weblog

The Social Web is Boring

Has anyone else noticed that most discussions on social media have gotten a bit stale? Everyone is a social networking/media guru. There are folks that actually do understand social media but it’s not because it is social media they are understanding. The folks who understand the impact of branding and quality content are the ones who are continuing to be successful utilizing the social web. This is not really surprising as it parallels the SEO concept as well. I remember when people felt it revolutionary that instead of focusing on search terms you simply try to be a good writer and gather an audience. It shouldn’t be surprising that then that social media has taken a similar turn.

With the marketing angle and business model of social media exposed, what else could be interesting about it? Well we’ve seen the fall of MySpace and everyone thinks Facebook is up next. It is not a matter of why, but simply when. Social networks have an Achilles heel call size. The reason is because social networks have to provide the interface to people and make it evolve along side relationships. That is really, really hard. People change all the time and so does technology. The big shift with Twitter and Facebook was that the expectations of the user switched from being browser based to mobile enabled. It moved the social network out of the browser and into real life by allowing updates from mobile devices and in doing so, made MySpace, which really requires a browser, irrelevant. The problem is as each social network gets larger and larger, the technology changes and as such, the expectations of the users.

With that in mind, it still seems like social networking and social media are at least technically interesting. The question is how do you keep the massive user base while continuing to evolve the service alongside technology and user expectations? It is really simple, you don’t. In technology there is always a point where a system becomes too large for a host of reasons. What you do is refactor and reorganize the code to reflect different abstractions. Eventually you start breaking the system up into more manageable parts. That is why we now have things like Foursquare, even though Twitter seems to already do geo locations. People end up wanting one tool to do one thing really well. Where have I heard that before… Unix!

The fact is we’ve learned to scale and realized the potential in relationships. It is reset time. Computing started simple and boomed when every day people got involved. The original Windows made it possible for you to slap some clip art on a layout and send your grandparents a homemade card. It was communication and relationships evolving through a better user interface. The time has come once again for this kind of user interface shift and it has already happened.

Yup, I’m talking about the iPad. I was a skeptic, but I now see the light. It is going to change the world and that is that. Do I have one? Nope, not yet, but I do want one. The thing that tipped me off that I was wrong about the iPad before I even got one was some example music software. We are recording our latest music with a guy that doesn’t want to use computers. The result is more time consuming and in all honesty rather frustrating. But, I realize that with the iPad, a person gets to have a customized interface without the cruft of the operating system. The iPad has taken the single user iPhone interface, made it big and provided the tools for others to build interesting interfaces with it.

The timing is impeccable because as I’ve been saying only a few paragraphs earlier, social media and social networking is boring. The iPad is the new form factor to change everything. No more browsing the Internet at home, finding a link and sending it to your friend. Now, you hang out with your friend, remember you saw something cool, pull out your iPad and show them. There are still all the same network enabled capabilities but they are services no one really cares about. The real deal excitement is getting out from behind the desktop and experiencing relationships by utilizing a screen. Social media is boring because you are forced to have an awkward relationship through a computer screen.

MongoDB Schema

Unfortunately, I can’t remember where I read it, but there was a blog post that mentioned the importance of schema in a schemaless database. It hit me as somewhat close minded at the time. After all, they don’t call them “schemaless” for nothing. Slowly there has been a change in how I view the issue.

Contracts have been a theme recently in my programming life. Tests essentially create contracts. The environment you run in can be rather limited and yet more robust thanks to contracts with the underlying system. Contracts are what let you sleep at night because without some guarantees you’d never get anything done. The key with contracts is that they happen at the right time and place. Abstraction (and much of OO) basically is a way to take some level of complexity and hide it behind a contract. Before when you had to clean up a file name, verify it wasn’t written to the data store and replicate it to three other systems. Now, you call the “save” method and forget about the details. The benefit most people site is flexibility, but the reality is that the contract behind the “save” method lets you abstract the more complicated process down to one thing, letting you free of some mental space for new problems.

The idea then behind a schema for something like MongoDB is that you get to stop thinking in terms of JSON or Python dictionaries and begin to use real objects. The reason is the same as the save method example above. We need a contract. A schemaless database is advantageous because it moves the verification and contracts for the data to the application level where it belongs. The RDBMS can help by saying something is a string or int, but past that you’re working with really basic types. You’re going to need some larger concepts to keep everything in place. ORMs are one example. In the case of MongoDB, you still need a schema because you need the contract. The only difference is that there is a slightly more automatic translation between a strings, ints, lists, and dicts along side the fact MongoDB doesn’t require setting everything up front.

It is always interesting to find how some buzzword or concept really ends up being a slightly different perspective on system design. Most people (myself included) hear “schemaless” and think, “I can ditch my ORM! Declare nothing! Everything is just a dict!”. The reality is that in the case of MongoDB you have a database that can understand JSON. That is pretty much it. There are interesting outcomes to that reality, but no where does it say you never have to verify your data. That is what a schema does.

MongoUI – Barely Better Than a Terminal

At work we use MongoDB for storing most of our critical data. We started with customized key value store we called Faststore. It was built in Python and used BsdDB. It was a pretty successful system in that it was reliable and performed very well. The thing that killed Faststore was that there wasn’t a good way to replicate the data. We tried considering a different key/value database in the core such as Tokyo Cabinet, but it quickly became clear the speed wasn’t there for our critical operations. After some research, we settled on MongoDB for our storage needs. So, far the decision has been a rather successful one. We did hit a rather nasty bug that required some significant downtime, but beyond that, MongoDB has been excellent. This is especially true when you consider the amount of data.

As we’ve used MongoDB one theme that I’ve hit is that is can be a little hard to view the data. Things like CSVs and tables in terminals can be a little easier to read than a pretty printed dictionary. In order to help making the perusing of data a little bit easier when developing with MongoDB, I made a little tool called MongoUI. MongoUI is really simple. It is only a little better than a terminal. That said, it makes basic browsing a little easier.

Pretty much all you have to do to use it is download it (hg clone http://bitbucket.org/elarson/mongoui), install it (cd mongoui && python setup install) and then run it (mongoui). Once it is running, point your browser to localhost, port 9000 and you’ll see a simple list of your databases. Clicking on a DB will show you the collections and clicking on the collections gives you a couple fields to query. The first field is for excluding/including fields. The second is for providing parameters ({‘foo’: ‘bar’}). You can include a count by clicking the count checkbox and change the include to an exclude by clicking the exclude checkbox.

The output is still pretty much just a pretty printed dictionary, but the browser is a slightly easier place to scroll up and down than a terminal. There is no writing or editing of documents, and I seriously doubt I’d add that anytime soon. It also is not meant for huge databases. For example, if you have 20k collections in a DB, it will show a really big list. Same goes for large collections. I’m not paging or anything like that. It really is meant make basic queries on local tests databases a little bit easier.

If it seems helpful I’ll try to develop the viewing of the data bit more as well as possibly including some editing bits. That said, it is no where on my todo list. The other thing I’d like to try and incorporate is a good RESTful API. My biggest complaint about MongoDB is that the protocol to the DB is not via HTTP. It was a basic design decision, but it would be nice at times to have a basic HTTP enabled caching proxy in front of a MongoDB for things like saving queries as views. Likewise things like replication and load balancing might also be interesting uses. That said, I don’t really have a need for it at work, so I don’t know that I would get very far.

Feedback is always welcome!

Mobile is the Future

I read a great article the other day by someone who used to work at Nokia on how the US is so radically different than the rest of the world when it comes to cellular technology. It was really interesting to see how Nokia as a company was driven by research in its own local market, which happened to be a great predictor of how the rest of the world uses mobile phones. What was also really interesting was the impact of SMS on the mobile industry, and more specifically, for carriers in terms of profits. These profits (along with a wildly different geography) also seemed to help with providing better service to carrier customers. For example, no one wanted antenna on their phones because they got in the way of putting it in your pocket. In the US though, an antenna meant better reception and fewer dropped calls. That never happened, so users just assumed the phone always worked.

Articles like this are really interesting as programmer because reinforces the importance gathering data. Nokia’s reel claim to fame in the story is its research. It payed closed attention to its users and acted accordingly. As a result, Nokia sell millions of phones at a really high price all over the world and have customer loyalty. While it is something I’m not very good at, it is a good reminder that being able to gather data can make the difference between tweaking a setting and rewriting an application. A wrong hypothesis can be expensive, and data helps to make better guesses as to what the problem is. It is something I personally want to get better at.

The other theme of the article is the importance of mobile in terms of technology. The author described impact of SMS on the industry, which was enormous. Any company that has a technology focus needs to see the importance of mobile phones as a platform. Also, the platform is not the iPhone. The author made it clear that the iPhone application business model doesn’t work, which is something I have felt for a while now. The numbers just aren’t there. You sell the app for a couple bucks to a million people and you might have made a couple million dollars. But that’s it. There is nothing else to really be done except releasing updates and hope people keep buying it. The problem is that if it gets extremely popular, there is a really good chance Apple will just release its own version and put you out of business. The other tact is to release system services for other companies to use, but again this seems like a bad idea. You’re selling some service to the guy that only make a few million dollars off some application. Some percentage of those sales is pretty weak in terms of revenue and you still have the same problem of Apple simply destroying your business by releasing their own version. Apps just don’t make sense in the long term and doesn’t reflect the rest of the world.

The reason for all this is that defaults rule. SMS changed everything because it was a built-in feature of every phone. There are tons of phones now with data plans, but the reality is that everyone texts. This is not because texts are better than Safari or Twitter, but because interoperability is taken care of. The web flourished because browsers felt the same. The mobile industry gets the same feeling from SMS. The article states that every day users won’t buy apps and after thinking about it a bit, I have to agree. The one thing that could change that would be the iPad. The iPad has the opportunity to change the way people think of computers and it is built completely on the idea of apps. Before you had a filesystem and there was a disconnect between files and applications. The iPad is destroying that assumption and people seem to appreciate the simplicity. If the new assumption is that applications are the baseline and it is a pattern that naturally works on a phone (which it arguably has), it seems possible that the idea of the app store could become widespread. A person would have their iPad as their computer and their iPhone as their laptop with both synced via the cloud and apps. I don’t know that it will work like that but it could.

The point is that in either case, mobile computing is only getting more important. One interesting side effect is that mobile platforms are only now considering how to handle multiprocessing. This is not even how to mimic threading, but rather how to do two things at once. It is getting solved quickly, but it is interesting because developers have been focused on how to handle the problem of writing apps for more than one processor. It appears it doesn’t matter because while desktops and laptops are getting more cores, people might just stop using them in favor of a different kind of machine. It still means the web and servers are still very important, but we can probably keep punting needing things like functional paradigms in order to handle the multiple processors all our users supposedly will be utilizing in the future.