A Little Siggnal Amidst the Noise

Back at Chicago BARcamp 2010 I participated in ‘BARcompany’, a startup/hackathon event. I was totally unprepared for the informal atmosphere however – I was expecting teams to form, but they expect us to bring them. So, for kicks I started hacking on Siggnal. I had something sort of working by the end (I worked on it while attending other talks, riding the train back and forth, etc) It was a Twitter based web app, and learning OAuth for the first time didn’t help, especially on top of (re)learning Rails. A few weeks ago I brought the project back to life and took the one extra step to make it personally useful.

Smoke Test (or should that be Smoque?)

After a brief ‘smoke test’, the first order of business was to get it off Rails 3 RC and on to the latest release. I might have tried to do this in a more controlled fashion, but I just did a bulk upgrade of everything. The only thing that had an issue was the Twitter gem itself, which had jettisoned built-in OAuth support in order to allow interaction with other libraries. The consequence of this was that I was immediately confronted with the the hardest part of the application. (Of course, given how much trouble it gave me initially, it was also about the only part of the application.) Of course, being a weekend project I didn’t want to try and learn both Rails and testing-Rails at the same time, so it had no tests. Actually, a propensity for previously working features to break while I was woking on the next encouraged me to think about testing more, perhaps with some preparation.

Step One: Personally Useful

Siggnal is an attempt to build on FollowCost, taking it from quantity to quality. The obvious (I won’t say best) way to do this is mark tweets as signal or noise and determine if a user is contributing or detracting from the overall feed quality. During BARcamp I’d gotten as far as collecting votes. The great missing feature was displaying a sorted list of users to actually make judgements. So greatly needed, in fact, that I added that one page without approaching the test backlog. Thankfully nothing blew up, and I had a marginally useful application while circled back to learn about testing.


Being by now even more out of practice with Rails, and clearly lost on rails testing, I turned to the famous Ruby on Rails Tutorial to try and get a consistent from-the-ground up presentation. The tutorial was based on TDD, so it should have me covered there. Some immediately welcome additions were Spork and Autotest although as advertised they (Spork especially) can get a little wonky at times. Another minor improvement was replacing the ‘welcome’ controller with ‘pages’, which just seemed more natural all around.

Pain, Suffering, and OAuth

One of the major challenges has been working with Twitter. The OAuth setup requires a pair of keys to get going, which is then used to acquire another set of keys for the actual traffic. Even during BARcamp I was toying with Heroku, figuring that a live application would be more impressive – of course I didn’t actually get enough done during the weekend to bother. In any case, the issue is that Heroku operates via a version control repository, and it’s generally unadvisable to check keys into version control. Heroku’s work around is to set environment variables. So I’ve got a shell script (not checked in) locally that must be used to run the server if it’s actually going to talk to Twitter.

Testing: more Pain and Suffering

Testing Twitter raises even more challenges. The normal OAuth flow requires going to a Twitter page, possibly logging in, and then clicking to allow the application. I’ve little doubt that I could automate this process with enough effort, but it turns out there is an easier way – you can get a single-user OAuth access pair, which is good enough for the test environment. Just set up another not-checked-in script to run tests in and it’s possible to simulate the logged-in state.

There is one more gotcha – the time you’d want to use ‘real’ Twitter connections is during integration tests. Integration tests are a little more hands-off, and there isn’t a good way to inject the access keys. So far I’ve had to turn to overriding controller methods, but somehow this doesn’t feel quite right, and I’m not sure how to keep it from affecting the other tests.

It’ll Take a Miracle

Ultimately the entire model is unsustainable – if every (or even many) request to my server has to go out to Twitter and wait for a response, it simply won’t scale. I’ve heard of some work on async Ruby servers, and of course I could change platforms. The real solution however, is to move the Twitter integration into the client and make the server, um, serve it’s primary role as vote collector and summarizer. My goal isn’t really to make a client (although I simply had to enable links in tweets) – it is to get clients to integrate the Siggnal service.


Meanwhile, it has occurred to me that if I’m going to write a Javascript application, I might step back to another idea using a third party service, where I would only be responsible for one end, and then circle back to Siggnal after getting a little client side experience.

Posted Thursday, March 31st, 2011 under Devlog.

Comments are closed.