upstream agile - software

Über Tests: fast, isolated integration tests for distributed apps

2012-09-15T00:00:00+00:00

TL;DR: when you split up your monolithic (web) app, you can either write integration tests spanning all the parts, which makes them slow and hard to set up, or you test each part in isolation and lose test coverage around the edges of your components. Über Tests try to solve the problem by testing those edges.

Suppose you have written a nice little Rails app (they all start out that innocent don’t they). Over time you add more and more features and suddenly you are looking at 15k LOC and a test suite (of course you have unit and integration tests for everything, don’t you) that takes 20 minutes to run.

You decide to go with the trend and split up your code base, say into one or more API apps and a frontend app. The backend(s) now expose a number of endpoints via HTTP and the frontend app uses those instead of directly talking to the database. This reduces coupling between the different modules and as your test suite gets split up with the app, you now only have to run the tests for the part of the app where you were making changes. Sweet.

There’s one problem though: what do you do with your integration tests? Keep them all in one place? Split them up?

After some headscratching you come up with three ideas:

Airbake - if it crashes you will get an email. Err, maybe not the best idea.
Write integration tests across the entire system. In addition to firing up your database, you simply run all your API services, too. The clear benefit here is that you don’t even have to change your existing tests. On the other hand those tests will run even slower now, with the added overhead of multiple processes and possibly databases. Didn’t we say we want fast tests? Also, setting those up will be a nightmare.
Test each piece of your app in isolation. When a feature you are testing requires a certain service/API you just stub it using for example WebMock. This approach allows you to split up your test suite, so you don’t have to run all the tests all the time. In addition they will run faster, too, as your stubbbed API will respond a lot quicker than a full stack. The downside here is that you are not testing the edges of your components anymore, e.g. how do you make sure your frontend app stays in sync with the backend API? If you change the API, will the frontend still work?

Here’s an example. Suppose you have a signup API endpoint and it returns the following JSON:

{"id": 1, "email": "joe@example.com", "access_token": "12345"}

You now want to change that API to return other data than just the user:

{
  "user": {"id": 1, "email": "joe@example.com", "access_token": "12345"},
  "links": [
    {"rel": "settings", "href": "https://api.example.com/account/settings"}
  ]
}

Which parts of your frontend do you have to change now? Are you not going to make any mistakes when stubbing out the new API? (yes you will).

Here’s where the idea of Über Tests™®© comes in: we take approach three, which gives us the advantage of fast and decoupled tests and add another suite of tests. These tests will make sure that the different components of our distributed system still work together. Here’s how it works:

First we write a small client for our API, something like this:

class SignupClient
  def initialize(http_client)
    @http_client = http_client # bear with me
  end

  def call(params)
    response = @http_client.post('/signups', {user: params})
    response.success?
  end
end

Step 2, we use that client for testing our backend (RSpec integration test example):

describe 'The signup API' do
  let(:signup_client) { SignupClient.new(http_client) }
  let(:http_client) {
    Faraday.new {|connection|
      connection.request :url_encoded
      connection.adapter :rack, app
    }
  }

  it 'signals success when sending valid user attributes' do
    expect(signup_client.call({email: 'joe@example.com'})).to be_success # let's not discuss RSpec syntax here
  end
end

As you can see we are using Faraday as our HTTP client. The cool thing about this is that it allows us to specify a Rack endpoint, and hence our integration test session (referenced by app) as the HTTP adapter. When calling the client in the test it will in turn call the post method of the integration test session. Nifty eh?

Okay, what do we have so far? We have a client for our API and since we are testing it along our backend we know those two are going to play along like BFFs. What’s left? We use that same client in the frontend:

class SignupsController
  def create
    if client.call(params[:user])
      redirect_to account_path, notice: 'Konichiwa!'
    else
      render 'new'
    end
  end

  def client
    SignupClient.new(Config.http_client)
  end
end

(By now we should probably put that client into its own gem so we can use the same code in the frontend app and for the backend tests.)

We set up the Config constant in environments/production.rb like so:

Config = OpenStruct.new http_client: Faraday.new {|f| f.adapter :net_http}

and for our test environment:

Config = OpenStruct.new

Lastly we add another test (using RSpec/Capybara) for our frontend:

describe 'Signing up' do
  let(:http_client) { stub(:http_client) }

  before(:each) do
    Config.stub(:http_client) { http_client }
    http_client.stub(:post) { {"user": {<...>}, "links": [...] }
  end

  it 'redirects to the account page on success' do
    visit signup_path
    fill_in 'Email', with: 'joe@example.com'
    click_button 'Sign up'

    expect(current_url).to eql('/account')
  end
end

In that test we use our API client but we stub the HTTP client, so that the test does not rely on the actual API endpoint to be running, but uses the code that we have already made sure works with it.

Now when we change our API there are two possible scenarios:

The change only affects the API Client. In this case we have to change the client’s code, which causes our frontend tests to break because the stubbed HTTP response don’t match the API client’s expectations anymore, so we have to fix only those.

The change causes the API client’s method signatures to change. In that case the same tests in the frontend app will fail where the frontend code calls the API client in an incorrect way.

In both cases our Über Tests catch the error and we can fix the code and tests. The rest of our integration tests reside in the different parts of our app: We have a bunch of backend tests that test the API endpoints and the stack behind it, plus a set of tests for the frontend app that only test that app using a stubbed API.

When you change something in the frontend you don’t need to run your backend tests again, as there’s no way you could have broken anything there. When you change your backend but nothing in the client you only have to run those tests. If you change the client run the client’s tests and the frontend tests, which are fast now because the run against a stubbed backend. Happy end.

12 Rules for Writing Web Apps in 2012

2012-05-12T00:00:00+00:00

The following is a list of ideas I pulled out of my head over the course of today. While it was slapped together rather spontaneously it is based on six years of writing my own web apps, looking at others’ code and basically a lot of headscratching and crying.

None of these ideas are new, others have written about them over and over. Still most of the world around me is stuck in uncomprehensible code, coffee breaks for running tests and banging their head against the wall.

The intention of this post is not to piss on anyone’s coding style but to lay a foundation for writing better apps in the future – for myself and hopefully for others, too. The code examples are in Ruby but most of it should apply to other OO languages as well.

Business Models

Your business models are the core of your application. They model your business domain and nothing else.

1. Models are plain old Ruby objects.

They don’t extend a framework base class like ActiveRecord::Base.

2. Your models do not persist themselves.

Instead they use delegation. This decouples them from your persistence framework and makes them unit testable.

Don’t do this:

class User::ActiveRecord::Base
  def update_profile(attributes)
    ...
    save
  end
end

Instead do this:

class User
  def update_profile(attributes)
    storage.load(user_id).update_attributes(attributes)
  end
end

3. All execution flow is explicit.

No Callbacks. The various lifecycle callbacks most persistence frameworks now offer make your life easier if you have four models with 10 lines of code each. In larger apps they lead to unpredictable chains of code execution and tight coupling.

Don’t do this:

class User
  after_save :create_welcome_message

  def create_welcome_message
    Message.create! text: 'Welcome User!'
  end
end

Instead do this:

class Signup
  def complete
    user.confirm
    Mailbox.create_welcome_message_for(user)
  end
end

4. Models do not expose their properties publicly.

Instead they offer an API for others to use.

Don’t do this:

Comment.create author: user.name

Instead do this:

user.sign comment

5. Each model encapsulates its persistent data.

If other objects want to access that data they have to ask the respective model.

Don’t do this:

class Comment
  def create
    ...
    storage.load_user(author_id).comment_count += 1
  end
end

Instead do this:

class Comment
  def create
    ...
    user.notify(:comment_created, self)
  end
end

Persistence

6. Use whatever database or service is best for the current job.

Don’t be afraid to use more than one. Don’t use a relational DB for full text search. Use a full text search engine. Don’t store logging data in your main database, use a logging service. And don’t kill -9 your MongoDB.

Some of you data may be write heavy, other read heavy and in need of caching. Research your options, don’t just rely on what’s the default. It will make your life easier and your app faster.

Front-end

7. Your front-end is a RESTful JSON API.

The internet is meant to be connected. Data must be free. Apps need to communicate. You want to build your service on top of others. Others will want to build on top of you.

A proper RESTful JSON API is easy to explore and understand. REST scales, it’s what the internet is made of.

Your human facing front-end is either a client side JavaScript app or another server side app.

8. Your app provides OAuth2 authentication.

OAuth2 is easy and open. It has security built in. It builds the basis for others to use your API.

Testing

9. Your API is fully tested with integration tests.

This is 2012.

10. Your business models are unit tested.

As in real unit tests, no database access in tests. There will be many, they must run fast.

11. Your tests run within two minutes.

If they are slower make them faster. If you can’t make them faster split your code base. There is not enough going on on Twitter and Hacker News to wait for slow tests. More importantly your customer shouldn’t pay you waiting.

Frameworks

12. Prefer small and simple

Rails is great, but do you really need all of it? Since 3.0 it has been split into smaller parts. Maybe Sinatra + ActiveRecord will do the job? A 50 line Gemfile may be impressive but it sure hosts a ton of bugs. Less code means faster, simpler, error free-er.

The bottom line

I can’t promise you anything but these are the effects I’m trying to achieve.

Testability

By decoupling objects they can be truly unit tested individually. Your tests will be easy to write and they will run fast.

Scalability:

Of your code base – decoupling means you can split your code base into smaller apps as it grows. Avoid the big spaghetti monster.
Of your team – instead of one large project you now have many small ones that more people can work on simultaneously without stepping on each others’ toes.
Of your storage layer – by using the right tool for the job your tools will get your further. A lot further than abusing MySQL into doing things it was never meant to do.

Flexibility

Your main app being an API means you can add mobile apps and other clients easily. Your components being independent means you can swap backends or frameworks and even programming languages.

Happiness

Fast tests, easy to understand relations, small code base – pain free coding, short development cycles, bring out the unicorns.

Conclusion

I have tried these ideas out to variying degrees. I will continue to do so over the next months. Maybe I’ll start crying again. Then it’ll be time for another post. Until then, let’s move forward, let’s write better software. For the unicorns.

Follow me on Twitter: @langalex, Github: https://github.com/langalex

Http Caching assets in Rails 3.1

2011-10-20T00:00:00+00:00

I’ve been wrestling with this for the past few hours and finally got it working so I’m going to share this, hoping it might save others a few hours of their lives.

I’m currently working on a Rails 3.1 app that is being deployed to Heroku. We use the asset pipeline to serve all our assets and we want those to load fast. By default Rails sets ETag headers, which means that for every (web page) request the browser checks on all the assets, usually resulting in a 304 Not Modifed response from the server. It’s faster than serving the assets each time but it still means one round trip per asset. We can avoid this by setting an Expires headers with a far future date.

To do this add the following lines to your production.rb:

config.serve_static_assets = true
config.static_cache_control = "public, max-age=#{1.year.to_i}"
config.assets.compile = true
config.assets.digest = true

This will precompile the assets when you push the app to Heroku and set an Expires header with an expiry date 1 year in the future.

Don’t forget that you must reference your assets through the Rails asset helpers like asset_tag and image_tag (in *.scss files you can use image-url() etc. instead of the url() call that comes with CSS).

Ruby TDD Workshop

2011-08-10T00:00:00+00:00

Nachdem wir unsere ersten JavaScript Workshops erfolgreich durchgeführt haben, machen wir jetzt mit dem nächsten Thema weiter: wir bieten einen Workshop Test Driven Development in Ruby an.

Du bist erfahrene(r) ProgrammiererIn und bringst ein Feature nach dem nächsten an den Start. Nur irgendwann schleichen sich Fehler ein. Refactoring geht nicht mehr, und neue Anforderungen bringen Probleme. Was dir fehlt sind Tests. Wir zeigen dir wie’s geht.

Wann: 30. September 2011, 10-18 Uhr
Wer: Trainer: Thilo, Alex, max. 10 Teilnehmer
Wo: bei co.up/Upstream in Berlin Kreuzberg

Mehr Infos und zur Anmeldung.

Sublime Text 2 with RVM on OSX

2011-07-30T00:00:00+00:00

While everyone is still waiting for Textmate 2 from time to time an interesting alternative pops up. One of those alternatives is version 2 of Sublime Text.

One of the key features of a text editor for programming is being able to run my tests/specs from within the editor. Sublime offers a build command that runs a ruby file and shows the output within the editor. Not as pretty as TextMate with its HTML output but it’s something to start with.

The problem is that Sublime simply runs ruby, which in most cases will invoke the ruby interpreter that ships with OSX. Instead I want it to use my project specific .rvmrc file and run the Ruby version and Gemset I’ve specified for RVM. It took me a while to figure this out and I think there must be an easier way but this works:

Create a file ~/bin/rvm_ruby that looks like this:

#!/usr/bin/env ruby

file = File.expand_path(ARGV[0] || (STDERR.puts('you must specify a ruby file'); exit(-1)))
dir = File.dirname file
while dir.size > 1
  if File.exist?(dir + '/.rvmrc')
    exec("source \"$HOME/.rvm/scripts/rvm\"; cd #{dir}; ruby #{file}")
  else
    dir = dir.sub(/\/[^\/]*$/, '')
  end
end

puts "Could not find any .rvmrc above #{file}"
exit -1

Make it executable by running chmod +x rvm_ruby. Now edit the file ~/Library/Application Support/Sublime Text 2/Packages/Ruby/Ruby.sublime-build and change it to look like this:

{
    "cmd": ["/Users/$YOUR_USERNAME$/bin/rvm_ruby", "$file"],
    "file_regex": "^(...*?):([0-9]*):?([0-9]*)",
    "selector": "source.ruby"
}

That’s it. Now when you edit a Ruby file and press Cmd-B it runs the file with the correct Ruby version.

By the way, you can do the same with the RSpec package.

JavaScript Workshop for Designers 2

2011-05-19T00:00:00+00:00

We are doing another iteration of our JavaScript for Designers Workshop (in German). Our designer Kristina and I (Alex) will be teaching the basics of programming in the browser for a full day. You don’t need to have any knowledge about programming. You don’t even have to be a designer :)

The workshop is scheduled for July 15, early bird tickets are available for €300 until June 10.

For more information visit jstraining.de.

Testing the AnyTime date picker with Cucumber/Selenium

2011-03-20T00:00:00+00:00

On Cobot we are making extensive use of the AnyTime date picker. When it came to integration testing with Cucumber/Capybara, until recently I got away with using the Rack::Test driver and doing

And I fill in "2010-01-01 12:45:00" for "Date"

Yesterday I finally needed to enter a date in a Scenarion that was using Javascript (with the Selenium driver. With JavaScript enabled the above step doesn’t work anymore because when Selenium tries to type into the text field Anytime pops up and resets the text field’s contents. After some fiddling around I decided that I would make selenium click the actual buttons on the date picker. Not only did this seem to be the least hackish way, it would also mean I would be testing the real thing, i.e. clicking on buttons like a user would. This becomes especially important once you start with internationalization and different time formats (e.g. 24h vs. 12h system), where you want to make sure AnyTime generates the proper date string.

Long story short, it took me almost a day to figure out how to do this. I played around with a lot of variations, but in the end this is what you have to do:

When /I select the time "([^"]+)" from "([^"]+)"/ do |time_string, label|
  time = Time.parse(time_string)
  And %Q{I fill in "" for "#{label}"}
  find_field(label).click

  click_on_selectors ".AnyTime-btn:visible:contains(#{time.year})",
    ".AnyTime-mon#{time.month}-btn:visible",
    ".AnyTime-dom-btn:contains(#{time.day}):visible:first",
    ".AnyTime-hr#{time.hour}-btn:visible",
    ".AnyTime-x-btn:visible"
end

def click_on_selectors(*selectors)

  def recurse(*selectors)
    if selectors.any?
      wait_for_css_selector_fn(selectors.first,
        "$('#{escape_javascript selectors.first}').click(); #{recurse(*selectors[1..-1])}")
    else
      'window.__capybara_wait = false;'
    end
  end

  page.evaluate_script "window.__capybara_wait = true"
  page.evaluate_script recurse(*selectors)
  wait_until 10 do
    page.evaluate_script "!window.__capybara_wait"
  end
end


include ActionView::Helpers::JavaScriptHelper
def wait_for_css_selector_fn(selector, after)
  <<-JS
    (function() {
      var time = new Date().getTime();
      var runDelayed = function() {
        if(!$('#{escape_javascript selector}').length) {
          if(time < new Date().getTime() - 5000) {
            throw('waited too long for #{escape_javascript selector}')
          } else {
            window.setTimeout(runDelayed, 100);
          }
        } else {
          #{after};
        };
      }
      window.setTimeout(runDelayed, 100);
    })();
  JS
end

In you Cucumber feature you can then call:

When I fill in the time "2010-01-01 12:00" for "Date"

What the above code essentially does is open the date picker and click on all the required buttons. Just as important (and that was the tricky part) it asynchronously waits for the necessary events (open/close date picker, date shows up in text field). Enjoy.

Euruko 2011 call for papers

2011-01-14T00:00:00+00:00

Its very exciting to organize a conference, especially if it is the well known EuRuKo. The preparations for this year EuRuKo in Berlin are in full effect. We just published the call for papers and I hope we get a good deal of interesting talk proposals from you until the deadline at the 22th February. So stretch you fingers and send in a proposal

…[if] you have researched something about Ruby, developed a gem, found a unique usage for Ruby or you had a life changing experience with Ruby.

For details see the official post. We will choose wisely from the submissions so that you’ll never fell the urge to leave the track on this single track conference :).

JavaScript under scrutiny at Plat_forms 2011

2010-12-22T00:00:00+00:00

… we consider their participation a glimpse of the future …

With these words our team (Alex, Frank and Me) got accepted with JavaScript into the web development platform comparison “Plat_forms 2011“. Big words :)

Plat_forms is a contest in which teams of three programmers compete to implement the same requirements for a web-based system within two days, using different technology platforms. It will be held in Nürnberg from 18th to 19th January.

The purpose of the Plat_forms is to provide new insights into the real pros, cons and emergent properties of each platform by analyzing various aspects like usability, structure, performance, scalability etc.

Sadly we were the only applicant for Javascript, so our results for JavaScript platform will be treated noncompetitively in the evaluation. The other participating platforms are our beloved Ruby, PHP, Perl and good old Java.

Although we use Ruby as our main language we are getting more and more comfortable with JavaScript. Just recently JavaScript got a huge boost with faster VMs, server side execution, powerful libraries and more possibilities on the client side commonly regarded as HTML5. So our usage of JavaScript increased over time and now ranges from rich client interfaces over special server side tasks to small JavaScript only apps.

With our participation with JavaScript at Plat_forms 2011 we want to push our skills and boundaries further. We are excited and anticipate the insights that our participation with JavaScript will reveal.

Diplom thesis: a distributed application with CouchDB and Javascript

2010-10-27T00:00:00+00:00

Today I finally released the thesis I wrote at Upstream during the best part of the last year. The title is "Implementation of a distributed application using the document-oriented database CouchDB” An outliner as a replicable distributed system". I investigated the capabilities of CouchDB in general and CouchApps in particular to see if they're a good choice for implementing a distributed system. Before that, several other approaches to implementing such a system were examined. The main part is a description of the concept and realisation of the outliner (see below). Where it made sense, snippets of the source code have been used to illustrate certain issues. Also included is a presentation of the used technologies, which include Sammy.js as routing framework, JQuery for the user interface, Cucumber/Culerity and JSpec for testing purposes, CouchDB Lounge for scaling and Amazon EC2 for deployment.

In the course of the research I implemented an outliner that allows collaborative editing of documents. An outliner is an editor you can use to organize text into a tree structure. The idea for this has been heavily inspired by Upstream's SaaS product doingText. The application I wrote is a CouchApp, which means it is Javascript code which is served by a local CouchDB instance. It therefore runs locally in the browser and is also usable when users have unreliable internet access or none at all. Using CouchDB's built-in replication and conflict detection features, the documents can be exchanged and synchronised whenever there is an internet connection available. Quite a part of the development effort went into the handling and resolution of conflicts that may occur during replication.

You can view or download English and German versions of the thesis on my blog. There you can also find links to the deployed application and to the source code on github.

Along the way I learned tons of things, mainly about CouchDB and concepts of modern web applications. It was also great to be able to use Javascript for more than just UI enhancement, this gave me the chance to properly grok Javascript as a first class language. Thanks a lot to Upstream for sponsoring my work and the professional support along the way!

Upstream Summer Party

2010-08-16T00:00:00+00:00

Sorry for the short notice but we have been busy with being on vacation :)

Tomorrow (August 17th 20:00) is the date for our annual summer party. As last year’s location ceased to exist we are moving to the Kiki Blofeld.

Be there and let us buy you a drink or two.

Transactions in CouchDB

2010-07-30T00:00:00+00:00

So we’ve been told that all these fancy new NoSQL stores don’t support transactions because that wouldn’t scale, and we’d just have to live with that. So yes, technically, CouchDB doesn’t support transactions, yet it still does. In a way.

What CouchDB doesn’t support is transactions that span multiple read/write operations, i.e. write document a, then write document b, if something goes wrong, roll back both writes. What it does support is single document “transactions”, i.e. a document is either written completely or not. So if our application requires a transaction, all we have to do is make sure that transaction happens in a single document.

Here’s our use case: at Cobot, our coworking space management service, we have a feature where a coworking space can charge a coworker for a one time service, e.g. usage of a meeting room. The way it works is that the manager of a space goes to cobot and enters an amount and description, e.g. “$10” and “meeting room”. At the end of the month, a cron job sends out invoices for all the one-time charges.

Our problem lies with this cron job. The job has to find all the charges for a coworker that haven’t been invoiced yet, create an invoice and mark the charges as invoiced. Without transactions, if something went wrong between creating the invoice and marking the charge as invoiced, the charge could end up being invoiced twice, because it hasn’t been marked as invoiced the first time.

Clearly we need to throw CouchDB away now and go back to a proper™ (a.k.a. relational) database, right? Well, no. As I said earlier we can try to move the transaction into a single document.

In our relational past, we would have had an invoices table and a charges table. The invoices table would carry all the invoices data (have fun with that), and the charges would have a field for amount and description, as well as a boolean field invoiced.

In the world of NoSQL (documents, no transactions), instead we have 2 documents: an invoice document that contains all the invoice data and a charge document with the amount and description again. Oh, and the documents also have _id_s.

Now when we create the invoice, instead of marking the charge as invoiced, we add the charge’s id to an array invoiced_charges in the invoice.

That was pretty easy. Now the tricky part is, next month, to determine which charges have been invoiced already. The first approach could be to make two requests:

Fetch all the charges by using a view that just emits every document that is a charge

function(doc) {
  if(doc.type == 'charge') {
    emit(doc.id, null);
  }
}

Then fetch all the charge ids from the invoices:

function(doc) {
  if(doc.invoiced_charges) {
    doc.invoiced_charges.forEach(function(id) {
      emit(id, null);
    });
  }
}

Then on the client side we can throw all the charges whose ids are in the list of invoiced charges.

But we can do better: We can use a list function to do the filtering within CouchDB. First we have to combine the above views into one:

function(doc) {
  if(doc.type == 'charge') {
    emit('charge', null);
  }
  if(doc.invoiced_charges) {
    doc.invoiced_ids.forEach(function(id) {
      emit('_invoiced', null)
    });
  }
}

Now we add the following list function:

function() {
  // some helpers
  Array.prototype.index = function(val) {
    for(var i = 0, l = this.length; i < l; i++) {
      if(this[i] == val) return i;
    }
    return null;
  }

  Array.prototype.include = function(val) {
    return this.index(val) !== null;
  }


  // interesting stuff
  var used_ids = [];

  send_json_results(function(row, sender) {
    if(row['key'] == '_invoiced') {
      used_ids.push(row['id']);
    } else {
      if(!used_ids.include(row['id'])) {
        sender.send_row(row);
      };
    };
  });

  // more helpers

  // this just makes sending json from a list function easier
  function send_json_results(callback) {
    send('{"rows": [');
    var first_row = true, sender = {};
    sender.send_row = function(json) {
      if(!first_row) {
        send(',');
      }
      first_row = false;
      send(JSON.stringify(json));
    };

    while(row = getRow()) {
      callback(row, sender);
    }
    send(']}');
  };
}

(Disclaimer: this is not exactly the same code as used in cobot and I haven’t tested it again.)

Apart from all the helpers, what this essentially does it put all the invoiced charges ids into a list and check every charge against that list, only sending it down to the client if it hasn’t been invoiced already.

As the view is sorted by the keys (‘_invoiced’, ‘charge’) we can be sure that all the invoiced charge ids have been collected before the list function checks the first charge.

There you have it, transactions in CouchDB.

P.S. If you are writing your stuff in Ruby (and possibly Rails), Couch Potato now has support for creating and querying list functions. See the readme.

Experimenting with node.js/mite

2010-06-03T00:00:00+00:00

Recently on one of my research fridays I decided to work on a problem that’s been bugging us for a long time. We are big fans of the timetracking service mite and use it for all our projects. The problem is that mite is designed around accounts. You can have many users per account, and then you can run reports across all the users that worked on specific projects. On most projects we’re working with independents who have their own mite accounts. Our problem is that we can’t run reports across multiple mite accounts.

Mite does have a nice JSON API though, so there must be a way to fix this. The result of my work is mite.enterprise, which allows you to enter multiple mite accounts and then report on those in one go. You can find the source on github and it’s running at mite-e.upstre.am so you can use it straight away.

While the app itself is relatively simple it posed some challenges I want to share here:

Making multple API requests and still be fast

The way the app works is that every time it needs data from mite it makes all the neccessary queries to the mite api and immediately delivers it to the browser. It doesn’t have its own persistence or cache layer for holding data. In order to keep things fast these requests to the mite API need to run in parallel. While I could have written a standard Rails app and spawn a new thread for every request I decided to try something new: node.js - node is the new cool kid on the block - a server side Javascript framework where everything is handled asynchronously, similar to Ruby’s eventmachine.

In order to run the API requests in parallel I wrote a Javascript function that fires off the requests and collects all the data in an asynchronous fashion:

function DataCollector(no_of_requests, callback) {
  var datas = [];
  return {
    collector_callback: function() {
      return collect_request_data(function(data) {
        datas.push(data);
        if(datas.length == no_of_requests) {
          callback(datas);
        };
      });
    }
  };
};

var mite_client = {
  time_entries: function(params, callback) {
    // go to the mite api and pass the data to the callback
  }
};

var project_ids = [1, 2, 3, 4, 5],
  data_collector = DataCollector(projects.length, function(datas) {
  // do something with the collected data, e.g. send it to the browser
});

project_ids.forEach(function(project_id) {
  mite_client.time_entries(project_id, data_collector.collector_callback());
});

I think it’s best to read this from bottom to top. At the bottom, for every project id that was given to the app (I simplified the code for readability) I ask the mite client for all the time entries for that project. I need to pass the mite client a callback, which in turn gets passed the actual data. Since I don’t want to act on the data immediately but want to wait until all the requests have returned, I’m asking the DataCollector instance for a callback. DataCollector uses a closure so it can collect all the returned data. The data is collected through functions that are created by its collector_callback function that I pass to the mite client where they are used as callbacks. Because the callback functions all have access to the datas array they can push their data on it until enough data is collected - only then the final callback to process the data is called.

While this might seem a little complicated and confusing at first I think it actually is pretty cool. By using all the callbacks the server app never waits for any I/O operations to complete, hence it is lightning fast and can probably handle thousands of concurrent clients.

Storing account data securely

In order to talk to the mite api we need the accounts’ API keys. I didn’t want to have to worry too much about security and how to store this data on the server so I chose a different approach: I’m not storing any data on the server at all. Instead, I’m using the browsers’ local storage. This is a small key value store that most modern browsers support. The data is stored on the hard disk of the client, hence there’s no danger that anyone will steal all the API keys from the server, because they are distributed all across the web.

Another advantage of that approach is that I don’t need a signup/login process for the app. When a user goes to the mite.enterprise site they can start using it immediately. No email address, no password. The app just looks into the local storage and loads the according data.

Nice and fast GUI

I wanted to keep things small and simple, so I decided to keep all the HTML/CSS out of the server. My node.js server only serves JSON and static files. For handling user interactions and rendering templates I’m using the excellent Sammy framework in combination with mustache.js

Sammy is like Sinatra but it’s implemented in JavaScript and runs in the browser. You can map URLs to actions, but instead of loading a new page for every request you do some AJAX requests (or not even that) and then replace whatever parts of the page you want using jQuery.

Conclusions

Node.js is an awesome library. First it lets me implement servers that are fast as hell, secondly I can implement them in the same language I use for the frontend. With local storage I don’t have to implement a signup/login process. Instead, users can use the app immediately and their data is stored on their computer. With Sammy I can quickly put a rich and responsive GUI on top of my servers which only have to deliver JSON. I guess it’s the future :)

Oh, and mite just became much more useful for us, too.

Capybara - Mix and match acceptance testing

2010-04-26T00:00:00+00:00

The more layers you add when testing your application the slower these tests become.

A unit test takes a fraction of a second to run. A simple integration test maybe up to a second. When JavaScript and a real browsers gets involved probably longer. But unnecessary long test runs hurt productivity. Up until a couple of weeks ago I had to make trade-offs either in speed, test-coverage or maintainability of my test suite when it came to real full stack integration/acceptance testing. Here is how I perceive these problems.

A standard Cucumber (a framework to write natural readable acceptance tests) setup uses Webrat which interacts with your app through a simple Rails integration session, so no JavaScript. The best what Webrat could do in this scenario, when it comes to JavaScript, is to recognize some common patterns in your html and send according requests - but still, most of the JavaScript code is not covered.

Webrat can also be used to drive Selenium, which interacts with your web app through a real browser. This will cover Javascript, CSS and possible browser issues. But compared to Webrat Selenium is snail mail slow.

With Culerity or Steam there are alternatives to Webrat which use HtmlUnit: HtmlUnit is a “GUI-Less browser for Java programs”. It models HTML documents and provides an API that allows you to load pages, fill out forms, click links, etc… just like you do in your “regular” browser. They are significantly faster than scripting a normal browser with Selenium at the cost that you don’t see how design works or if any browser issues cause trouble and it is still slower than Webrat without a special backend.

In order to run acceptance tests which require full JavaScript support separately from those who don’t to get shorter test runs I use two separate environments as they are generated out of the box either by Cucumber or Culerity. This leads to duplication of the setup code and step definitions, which means that there is more to maintain and more that could possible break.

Capybara addresses the shortcomings of a two environments solution and still allows you to be flexible in what backend you use for integration testing on a per scenario basis in Cucumber. It does that by using Cucumber’s tags feature.

Capybara “is inspired by and aims to replace Webrat as a DSL for interacting with a web application. It is agnostic about the driver running your tests and currently comes bundled with rack-test, Culerity and Selenium support built in”. In the future we certainly will see other backends. I saw that a backend for env-js (a pure JavaScript browser environment) is a already in the making.

Below is an example that runs one scenario with Selenium and the other with Culerity.

  @selenium
  Scenario: Searching a blog post using autocomplete
    Given a blog post titled &quot;Autocomplete made easy&quot; containing &quot;Some example&quot;
    When I go to the search page
      And I fill in &quot;Search&quot; with &quot;Auto&quot;
      And I follow &quot;Autocomplete made easy&quot;
    Then I should see &quot;Some example&quot;

  @culerity
  Scenario: delete found comments
    Given a comment by author &quot;Thilo&quot;
    Given a comment by author &quot;Tilo&quot;
  When I search for &quot;ilo&quot;
    And I check the element with xpath &quot;//p[contains(@class,'delete')]/input&quot;
    And I press &quot;Submit&quot;
  Then I should see &quot;Are you really sure?&quot;
  When I press &quot;Yes&quot;
  Then I should see &quot;Comments were delete&quot;
  And I should see &quot;0 Comments&quot;

You might have noticed the XPath expression in the second scenario. From time to time you will have to deal with XPath expressions when writing custom steps as XPath is used to locate elements within the DOM. The default steps can be scoped with CSS selectors or XPath, Capybara takes care of the conversion to XPath then.

Using Capybara with Cucumber is straightforward: First get the gem:

gem install capybara

On OSX you may have to install libffi, you can install it via MacPorts with:

sudo port install libffi

Then go to your rails project and generate the cucumber files as followed:

script/generate cucumber --capybara

And you are ready to go to mix and match your acceptance with tags. For further informations like using capybara without cucumber, asynchronous JavaScript etc. checkout Capybara’s README.

Capybara definitely eases maintaining acceptance test and will give impulses to improve other tools, e.g. env-js.

Are you stuck in a maintenance nightmare or simply want to bring your testing skills to the next level? Get in touch with us.

Product Launch: cobot - coworking space management service

2010-04-01T00:00:00+00:00

Back in december I attended the “Berlin Coworking Day”, a meetup of the German coworking scene. In the talks I saw a recurring problem was the lack of proper software to help with all the tasks that running a coworking space requires. Since we had been running our own coworking space for a while the problems were not new to me.

So, after a few hours I decided to leave the meetup and start to scratch my own itch. Since upstream was taking the month off anyway I had enough time to play around.

After roughly 4 weeks of development we released the first version of our coworking space management software Cobot - it obviously didn’t have a lot of features, but enough to help our own needs.

The basic principle is this: a coworking space owner signs up her space and gets her own cobot site under its own subdomain. There she adds payment plans, payment methods, invoicing details etc. Anyone who wants to start coworking at this space signs up and chooses a plan and payment method.

From then on cobot helps the owner of the space in various ways. It manages payment tracking and invoicing (can be fully automated with Paypal), it helps to keep track of day to day tasks and problems (“the printer ran out of paper”) and provides a platform for communication between coworking space and coworkers.

In addition it allows a space owner to track key metrics of their space. From turnover to occupancy this gives coworking spaces the numbers they need to optimize their plans and the number of coworkers they can fit in.

Even for our small coworking space (which is now full) cobot has made things a lot easier already (we actually know who hasn’t paid yet now). Our next step is to use cobot’s wifi login/presence tracking system so we know how full we really are, and maybe fit in a few more people.

The first responses were quite positive and we started talking to a number of coworking spaces, mostly in Germany and the United States. With the feedback we’re getting and constantly improving the site we’re confident to become the first choice for many of the coworking spaces that are being started all over the world these days.

If you are interested in how cobot works you can read more details and see screenshots in the guides section of the site.

If you do have a coworking space we hope you hop over to Cobot and sign up - you can try it out for free with up to 5 coworkers. After that it’s only 1 EUR/1.50 USD per coworker and month.

If you want to follow along the development, there’s a twitter account (@cobot_me) and the Cobot blog.

From Journeyman to Client

2010-02-13T00:00:00+00:00

Agile project management makes the client an integral part of the development process. This article explains what your responsibilities and powers as a client are, and helps you to understand project management the way we see it.

In the traditional (waterfall) model of software development someone writes a specification of the whole software and then the programmers fail to implement it within the given time/budget. To be honest I don’t know a lot about this way of doing software, but I’ve seen it fail.

But I do know a bit about how we write software. In contrast, agile development works in small steps. There is no upfront specification, as we acknowledge that this specification would be incomplete and quickly outdated by changing requirements. Instead we work in short iterations (at upstream typically ~2 weeks) where we only plan ahead for a short amount of time.

By aggressively prioritizing what is most important we are able to release software that fulfills the needs of the business with less clutter in a shorter amount of time.

For me this sentence is really the essence of agile project management as it shows its most important point: prioritizing what is important for your business.

What is software the fulfills the needs of the business? It’s a software that solves your specific problem. If you want to sell something online it allows your customers to click a buy button. If you need to send invoices it will put a bunch of numbers on a sheet.

What is less clutter? Less clutter means if you want to sell something online it neither has a tag cloud nor a facebook like wall for every product where users can post their video reviews. If you need to send invoices it will not let you upload pictures for it.

What is less time? Less time means we deliver your product faster because we spend less time on implementing features because there are fewer features. It does not mean that we can program faster because we are doing agile (seriously we can’t).

What is prioritizing? You do the things first that are most important.

So what is important? And here is where it gets difficult.

What is important is the client’s decision. The problem is that a lot of things can be (thought of as being) important. For a web site, the start page has to have a maximum conversion rate, SEO is important, scalability is important (isn’t it?), usability is important, choice of technology is important, the database schema is important… you get the idea.

But do you know what’s really important? That your business problem is solved. And do you want to know how many kinds of business problems there are? Well, there’s exactly one: make (more) money.

I think it actually is that simple, but from what I’ve seen it can become really hard to focus on that one goal, when you think you have to consider all that other stuff as well, but let me say this again: as a client it’s not your responsibility to think about the database and scalability (you won’t have to scale in the beginning). Your one and only problem is: how do I make money with this. Everything else comes after it.

A good way to focus on solving that problem might be to ask yourself this question: How do I make (more) money?

Case Study

Let’s stick to the example of an online shop and let’s assume you want sell e-books. Of course you look at the competition first, and there is amazon.com. They sell e-books and from what it seems they are making a whole lot of money from it, so that must to be a good starting point. From their site you collect a bunch of requirements you also want in your shop:

Categories: Books must be in categories so people can find them.
Tags: With tags people can click on tag clouds and then see all the books to that tag.
Related books: “People who have bought this might also like…” - I love that feature
User reviews/ratings: when users review the books we can make a top list and then everyone will buy the top books.
Licensing: You have talked to the publishers already and they have these complicated licenses where you may only sell a certain book in a certain country but only at full moon when it’s raining - so the shop needs to take all this into account.
Internationalization: We need at least English, Spanish and Mandarin to reach more markets
Statistics: You want to know at what time and weather conditions people buy books from where, on which browser and operating system and which IP address they used to do it. From that data you will be able to optimize the hell out of the start page.

So, that looks like a reasonable list of requirements doesn’t it? With this shop you could steal a nice market share from amazon right? Well, let’s take that list to the test: does it help me to make money?

categories: no
tags: no
related books: no
reviews/ratings: no
licensing: no
internationalization: no
statistics: no

Surprised? I hope at least a little bit. So what would make you some money?

In order to make money
as the owner of the shop
I want people to buy a book

With all the fancy features above you forgot (sorry, I forgot) the only important feature of the site: a button that says “Buy this e-book”. By the way the format of that feature request comes from the BDD (behavior driven development) people. This is the generic template:

In order to <business value>
as <role>
I want <feature>

This is a brilliant template that you can use to take all your feature ideas to the test. Remember that the only valid business value here is to make money. And that the only role that is important here is you as the stakeholder of the project.

Let’s look at your features:

Related books

While this is a popular feature at amazon, your yet to be launched shop wouldn’t even have enough data at the beginning to make any sensible claims about what books are related. Again, at the beginning this won’t matter at all. You can always add it later, when the money making assertion becomes true.

Reviews/Ratings

While reviews can be a good way to promote books and build trust for your customers, when you launch your site there won’t be any reviews because nobody has bought a single book yet. So unless you charge people for being able to write reviews this is not going to pay your rent.

Licensing

Sure, we can build you a system that grabs the weather from weather.com, can calculate the moon phases over the sahara and figure out people’s countries by their IP address, but it’s going to cost you. How about this: You start and market your shop in only one country. This can be your home country or the country with the simplest licensing model. After you have sold enough books there we can start working on the rest (and you can pay us from the money you have made already).

Internationalization (I18n)

Again, we can build you any system you want, but be warned, some people read and write from right to left or top to bottom. WIth I18n you are opening a whole new can of worms. How about we do I18n later, after you have proven that your business model works in one country (or language region, which might cover a lot of countries (thinking English/Spanish/Mandarin)).

Statistics

Sure it’s important to measure the success (or failure) of your business. But before you have a business there isn’t really much to measure is there? And how about this: for now every time someone buys a book you open up your spreadsheet app on your computer and put a date and a price in it. I know it may seem archaic but it works. If you want to go crazy you can even use a real book made of paper, but since you’re selling e-books you probably don’t want that.

All these examples boil down to this: test your features agains the BDD template: can I make more money now when we implement this. A lot of them are really not that important now and you can use something simpler instead.

Conclusion

Deciding what is important is the one big way that you can control the project. Programming takes time and we don’t usually have a lot of ways to make that faster within a project. But choosing wisely the features that you really really need to start selling your products can make the difference between a 4 week and a 6 months project. We do the best we can to help you with those decisions but ultimately they are yours.

If we spend 6 months working on your project and it’s still not done then that’s not a programming problem, it’s a management problem, because we have been working on too many features, of which many probably weren’t as important as you thought (no offense). On the other hand if we make it in a few weeks, we'll congratulate you and bring some drinks to your early launch party.

(On a side note we recently released cobot.me after only 4 person weeks of development it doesn't have a lot of features, but that’s a different post.)

Git meets Tracker, a possible Workflow

2010-01-17T00:00:00+00:00

When working with our clients we use pivotal tracker to manage requirements and priorities. Which works very well, as it keeps overhead low and offers a very focused communication channel for feature related questions.

The last step for a successful delivery of a feature requires our customer to accept that feature or hand it back to us if something isn’t satisfying. To enable our customers to do that we provide them with a staging environment, which is a non public version of our customers’ product with the latest code integrated so that he can try out new features before they go live. We deploy to the staging system directly from the master branch usually, from time to time we introduce feature branches if we think something takes longer or might need refinement after more customer feedback. By the time of an actual live deployment we normally have ironed out any remaining issues which shown up during the client’s review on the staging system.

This simple system works pretty well and effective if you are in control of the staging environment, have only one person as the customer to talk to and your developers’ communication works like in a hive mind (which it does, mostly :)). This system reached its limit when we worked with PaperC lately, one of our valued clients.

We coached their programmer in BDD and pair programming some time ago and still working with them very closely to get improvements and new features out of the door fast. The limits with our simple workflow became visible when I paired with Patrick together for PaperC and they also had a pair on site with us at co.up. At first delivered tickets began to stack up waiting for acceptance, also because the staging system became unstable as issues weren’t fixed fast enough.

Things got more complicated as we put features on staging which weren’t supposed to make it into the next live deployment as they where part of a bigger, not yet thought to an end, interface change. These features just should be shown in action to see how things can go on from what we did so far. This incident cost us some extra time to put these feature related commits out of the master branch into a feature branch. So we switched to create a feature branch for everything that was more than “add a new input field to that”. This left us with a lot of branches we had to keep track of when deploying to the staging system, but at least we could undo merge commits easily when a feature wasn’t suppose to be deployed live. Then we started a discussion on how we could do this in a better way. Before I describe the workflow which I think is best suited, as we already employed it successfully in the past, I’d like to mention two suggestions I’m not comfortable with an why.

One idea was the introduction of pair branches, where every pair commit their work to. And these branches would be merged into master for acceptance. Apart from not really resolving the issue, it just mitigates it onto a pair branch, and it contradicts the principle of collective code ownership.

Another proposal was to create a special production branch and selectively cherry-pick commits which are supposed to go live. This points into the right direction, as it gives you more control of what goes live, but it might leave you with a lot of commits to pick from, if you have a developer like me who commits rather often. And it doesn’t give you a single point to roll back from if a feature that has multiple commits get pulled off the release.

So now for my preferred and practically proven solution. First off, it requires an extra server which I refer to as experimental. It is a lightweight (not so close to production setup) staging system with a database schema that should be easily migrate-able up and down like on a CI server. It should be fed with some test data automatically. It doesn’t have to be as much data as on staging, just enough to try out most features.

Now to the git workflow part. Smaller changes and bug fixes go directly to master as always. Bigger features should be done in separate branches and merged to master once things are done (Don’t forget to delete the remotes if the feature gets accepted). The master branch should be deployed to the experimental server at least once a day, likely more often so that features can be quickly accepted. Features that aren’t necessarily ready for production yet can be deployed to experimental for acceptance testing simply by changing the branch from which the experimental environment should be deployed from the master branch to the feature branch and rebuild the database if required. Before an imminent release the master should be pulled to staging so that any remaining issues can be fixed. Such fixes should happen on the staging branch and can later be merged back to master. If everything is fine, merge staging to production and deploy to the live system if you are ready. This way you always have a clean production branch, it’s easier to pull off features from a release and get faster feedback for completed features without worrying to mess up the next production release.

What is your approach to managing commits and releases? Do you find this post helpful? Feel free to leave some feedback in the comments.

Upstream spreading to Scandinavia

2010-01-12T00:00:00+00:00

As of last week upstream is now present in 2 more countries: Finland and Sweden. We see a strong increase in the web development market in the scandinavian area so we want to be there before the rest of the world notices. Ok ok, so actually that’s not the reason. Both Frank and I moved for personal reasons.

Frank is now situated in Turku, Finland and I’m in Lund, Sweden. We will both be staying there for about half a year and continue to work for upstream remotely. As our company pair programs all the time so are we“ only remotely now, using iChat’s screen sharing/audio chat and good old Textmate. It actually works smoother than it did when we tested it in the office’s wireless network. Scandinavian communication infrastructure is pretty good.

While we are here in the north we will be attending and possibly speaking at a few conferences in the area like Nordic Ruby and the Scandinavian Web Developer Conference (SWDC).

If you are interested or know anyone who would want to hire us for a web project either in Sweden or Finland please drop us a note. While remotely working for the Berlin office is ok it would be nice to work on our own. We create high quality, fully tested webapps in a few week’s time, we help with code quality and scalability problems, we can do training/coaching on various programming/agile topics, remotely or on site.

Oh and if you don’t have a project for us but live in the area say hi so we can invite you for a beer or two. :)

Upstream Locations - please zoom out until you see Australia.

Time-out

2009-12-02T00:00:00+00:00

Upstream is taking the december off. Except from support for running projects we haven’t taken on any new projects for this month. The reason is that we wanted some time to think about the future of our company, but also to have time for other things.

We have already done this for a weekend in January when we went to Maine. It was a fun weekend but we spent a lot of time walking around at the beach and in the sauna, which left too little time for developing any significant ideas. So this time we are doing it for a whole month.

For the thinking part we will be doing a few meeting throughout the month to talk about different aspects, like the kinds of customers we want to attract in the future, the kind of work we want to be doing, how much and where we will want to work and also how much money we want to be making with all this. We are not going to do any fancy brainstorming-like things. Instead everyone spends their time doing whatever he or she thinks will produce the best ideas - be it sleeping in every day, long breakfasts or running around in the park. We will then gather occasionally to collect and exchange ideas.

For the “doing other things” part: first of all we want to give our <a href=”“https://co-up.de/about.html>new office and coworking space</a> some love. The walls are all white and empty yet and there are still a few ikea boxes stacking up. Another part is to attend some of the many events Berlin has to offer, but that we mostly have to skip because of work.

I already spent yesterday at the coworking day, where Germany’s coworking community gathered to discuss the issues of setting up and running coworking spaces. That was my first day off and I already got a few new ideas from it that could help upstream in the future. Next up is TedX Kreuzberg, where the smart people of Kreuzberg will come and tell us about their brilliant ideas. I’m very much looking forward to that.

I will report back on our progress here from time to time. So far I can only say: “Please try this at home”. Not having to worry about projects and day to day problems all the time is very refreshing and I am confident 2010 will be a great year for us.

Upstream on the Move

2009-11-15T00:00:00+00:00

We just moved into our new office at Adalbertstrasse 7, 10999 Berlin, the old one just got too small. Too small for Upstream and too small to offer a good co-working experience. During the last years, also with the help of our co-working space, we were able to grow our network of skilled freelancing professionals which allow us to do more and bigger projects and offer our customers the best people for the job.

We had a good time with our old office. For example we co-worked together with Jan Lehnardt (@janl) of CouchDB fame, taught Ruby and TDD to then still new developers like Lena Herrmann (@kilaulena), found skilled developers like Patrick Hüsler (@phuesler), found great designers like Kristina Schneider (@kriesse), won good PHP Developer like Robin Mehner (@rmehner) over to Ruby, ran DevHouses and Cockpit Nights to learn and share, coded with international recognized personalities like Pat Allan (@pat), had lots of well known guests in our office and some good parties too. But the old upstream office just got to its limits. I’m happy that all our co-workers moved with us.

The new office will allow us to keep a productive and inspiring work environment, which I think is essential to deliver great products, and evolve all the great things we did so far. For example Alex will expand our efforts in providing an affordable space for independent workers that combines a relaxed and social atmosphere with the infrastructure and productivity of a well equipped work place with co.up (@co_up). This way I think we will improve our network even further and get peoples and ideas together for the best outcome possible.

Another great thing about the new office is that its interior and its central location next to Kottbusser Tor underground station allow us to keep hosting the Ruby User Group Berlin meet-up as part of our involvement in the local developer scene.

So if you are in need of a desk (see co.up for details), want to get in touch with the tech scene or are looking for skilled people for your project, visit us in our new office.

I look forward to a great time with more great people and hope that our recently started community photo wall will grow fast with photos of well known and yet not so well known faces.

Awesome presentations with boom amazing

2009-11-13T00:00:00+00:00

Boom amazing in action at JSConf.EU

Last weekend at JSConf.EU I gave another talk on CouchDB and CouchApps. This gave me an excellent excuse to hack on boom amazing instead of preparing my talk. Boom amazing is my personal presentation tool. Instead of slides my presentation is laid out on a single large surface and I can pan around, zoom in and out and rotate my viewpoint to show specific contents.

In the end I managed to add a bunch of new features and still get my talk done. The following post explains how boom amazing works and how you can create your own fancy presentations with it.

What is behind it?

Boom amazing consists of a few relatively simple building blocks:

One or more SVG files that contain all the texts and graphics
a CouchApp that includes some HTML and CSS, and JavaScript
some client side JavaScript to handle manipulating SVG and moving around (using JQuery and the jQuery SVG plugin)
CouchDB: stores all the data: the presentation itself (as document attachments), the CouchApp and all the camera positions and data for replaying presentations
A browser that displays the SVG file, I use PlainView which has a fullscreen mode.

Generating SVGs on OSX…

… pretty much sucks. I’ve tried a few tools already but still haven’t found anything that is simple and stable. I first tried Adobe Illustrator which can export to SVG, but first of all it’s slow, it sometimes crashes and I’m not willing to pay that much money just for drawing a few circles and write some text.

My current setup consists of Vector Designer for drawing and Scribus for converting to SVG. While I do like working in Vector Designer very much it doesn’t have an SVG exporter, it limits the maximum zoom and it has problems with the kerning when I set the font size below a certain level. I might be able to work around the zoom and kerning problems by simply using larger page sizes (this time I used 2x2 meters). I then export my drawing to the EPS format which works okay, except that the day before the conference Vector Designer suddenly created file sizes of 115MB instead of a few hundred KB like before. Which brings me to Scribus. It’s an open source vector drawing app with a truly ugly user interface, but it can import EPS and export SVG. At least sometimes. For me it worked in ~40% of the cases. Everything else was crashes, unknown import errors and missing data.

There’s a command line tool called pdf2svg (which you can install using MacPorts) which I also tried, but the resulting SVGs are much larger than what Scribus creates and the browser refused to display them.

There is hope though, a one man company called Bohemian Coding is working on a vector tool called Sketch and their previous apps look really slick.

If you’re not on OSX or can tolerate working with X11 apps there’s also Inkscape which has built-in SVG support.

Using it

In order to use boom amazing all you have to do is download it and push it into CouchDB.

git clone git://github.com/langalex/boom_amazing.git
cd boom_amazing
curl -X PUT http://localhost:5984/boom_amazing
couchapp push boom_amazing

After that go to futon and upload your SVG file into any document in the boom_amazing database. Then you go to http://localhost:5984/boom_amazing/_design/boom_amazing/index.html. When you move your mouse to the top a toolbar appears where you can select your presentation from the dropdown. Once the file has loaded you can:

pan around by click & dragging the mouse
zoom by pressing & holding [alt] and moving the mouse up and down
rotate by holding down [ctrl] and moving the mouse left and right

Whenever you like a position you can save that by clicking the save button in the toolbar. After you have recorded all positions you can then jump back and forth between them with the Previous and Next buttons or by pressing [b] and [space] - which is what I do when presenting.

Hacking

If you are interested in extending the app here are the basics of how it works:

In _attachments/javascripts/app.js you find a small sammy application which controls all the actions like loading and saving data. Sammy maps the hash part of the URL to different actions, so when a link points to e.g. #/slides/23 sammy will automatically call the code to load slide (slide refers to a saved position) 23.

screen.js includes all the logic that handles mouse and keyboard events to move around. It also does all the computations for things like smooth transitions between positions and rotating around the center of the screen instead of just (0, 0).

The way this all works is that the jQuery SVG plugin loads the file and turns everything into DOM nodes. The application then takes all the nodes and puts them into a single group node. A group node can have an attribute transforms which expects a rotate, scale and translate parameter - the rest is up to the browser.

Conclusion

The feedback I have been getting for the talks where I have used boom amazing has been great so I am going to continue to use and extend it. Right now performance with larger SVGs is an issue but I don’t have any ideas wether I can do anything about it“ except for buying faster laptops. There are also still a bunch of basic features missing, like editing or removing saved positions, or reordering.

I would love to see other people to pick up boom amazing and use it for their presentations. It certainly is more work than putting together a bunch of slides but then again I think it is worth the effort, and the results easily make any fancy Keynote presentation look like 1990. :D

Unit Testing CouchDB Views with Couch Potato

2009-10-30T00:00:00+00:00

I just released Couch Potato 0.2.14 and amongst other things it has a new feature i think is pretty neat: you can unit test your (JavaScript) views using RSpec and Ruby.

You can declare a view in Couch Potato like this:

class Comment
  property :post_id
  view :by_post_id, :key =&gt; :post_id
end

This will generate a pair of map/reduce function and push them to Couch Potato. The map function looks something like this:

[javascript] function(doc) { if(doc.ruby_class == ‘Comment’) { emit(doc.post_id, 1); } } [/javascript]

And here’s a unit test for that function:

describe Comment, 'by_post_id' do
  it &quot;should map to the post_id&quot; do
    Comment.by_post_id.should map({:ruby_class =&gt; 'Comment', :post_id =&gt; 3}).to([3, null])
  end
end

As you can see all you have to do is pass a Ruby Hash that looks like your CouchDB document and the expected results. You can also pass in multiple results if you expect your map function to emit multiple key/value pairs.

Testing a reduce function works the same way:

describe Comment, 'by_post_id' do
  it &quot;should reduce to the number of comments&quot; do
    Comment.by_post_id.should reduce([], [1, 1, 1]).to(3)
  end
end

For testing re-reducing you simply call .should rereduce(...).to(...).

How it works

So how come you can test JavaScript functions in pure Ruby? Well, by stealing other people’s tricks. I recently contributed a few patches to mustache.js which is a new templating library ported to JavaScript by @janl. It has a test runner currently implemented in Ruby which generates JavaScript code on the fly, runs it using Spidermonkey and reads back the results. I have added a few steps to this process:

A custom RSpec matcher collects the map function, a Ruby Hash representing the input and the expected output
The input is converted to JavaScript using the JSON gem
JavaScript code is generated that runs the document through the map function and prints the resulting JSON.
The Ruby code runs spidermonkey, collects the output and parses it back to Ruby using again the JSON gem
The results are compared to the expected values.

You can see how it works by looking at the code for the RSpec matchers.

I think this addition lowers the barrier to test your views quite a bit. Although I’m usually not a big fan of “one language to rule them all” and love writing JavaScript, being able to write all the necessary tests in Ruby when working on a Ruby project makes things way easier.

Testing Couchapps with Cucumber and Culerity

2009-10-25T00:00:00+00:00

On last week’s RailsCamp UK I started hacking on a new CouchApp called HejHej. Its purpose is to help me learn Swedish, but what’s more important here: I wrote this app BDD style using Cucumber, the famous BDD tool and Culerity, my humble addition that allows me to test any webapp (including client side JavaScript) with it.

The whole thing is available on Github so you can check out the features and steps there. In the following post I will show the necessary steps to test your own CouchApps with Cucumber.

First of all you will need to install some software:

Java - most computers have it installed already (OSX does). If not go to java.sun.com
JRuby - Culerity is written in Ruby but wraps a Java library called HTMLUnit, hence we need both. You can get JRuby at jruby.org. Just download the tar and extract it to any directory.
Culerity - comes as a RubyGem that you will have to install into JRuby. To install run JRUBY_HOME/bin/jruby -S gem install culerity --source=http://gemcutter.org
Cucumber - jruby -S gem install cucumber
RestClient - helps to clean up the test database: jruby -S gem install rest-client jruby-openssl

Next change to your CouchApp’s directory and create the following directory structure:


  CouchApp root
  |- features
     |- support
        |- env.rb
        |- paths.rb
     |- step_definitions
        |- common_culerity_steps.rb

Copy and paste the following into your env.rb:


  require 'rubygems'
  require 'culerity'

  require 'cucumber/formatter/unicode'

  require 'restclient'

  Before do
    RestClient.delete "#{host}/#{database}" rescue nil
    RestClient.put "#{host}/#{database}", ""
    system "couchapp push"
  end

Warning: the before hook you see there will delete and recreate your CouchDB database before every run, so make sure you are using a separate database for testing than for your actual “production” database.

This is what your common_culerity_steps.rb should look like:


  require 'culerity'

  Symbol.class_eval do
    def to_proc
      Proc.new{|object| object.send(self)}
    end
  end unless :symbol.respond_to?(:to_proc)

  Before do
    $server ||= Culerity::run_server
    $browser = Culerity::RemoteBrowserProxy.new $server, {:browser => :firefox, :javascript_exceptions => true, :resynchronize => true, :status_code_exceptions => true}
    $browser.log_level = :warning
  end

  def host
    'http://localhost:5984'
  end

  def database
    'hejhej'
  end

  at_exit do
    $browser.exit if $browser
    $server.close if $server
  end

  When /I press "(.*)"/ do |button|
    button = [$browser.button(:text, button), $browser.button(:id, button)].find(&:exist?)
    button.click
    When 'I wait for the AJAX call to finish'
  end

  When /I click "(.*)"/ do |link|
    When "I follow \"#{link}\""
  end

  When /I follow "(.*)"/ do |link|
    _link = [[:text, /^#{Regexp.escape(link)}$/], [:id, link], [:title, link]].map{|args| $browser.link(*args)}.find{|__link| __link.exist?}
    raise "link \"#{link}\" not found" unless _link
    _link.click
    When 'I wait for the AJAX call to finish'
  end

  When /I follow \/(.*)\// do |link|
    $browser.link(:text, /#{link}/).click
    When 'I wait for the AJAX call to finish'
  end

  When /I fill in "(.*)" with "(.*)"/ do |field, value|
    find_by_label_or_id(:text_field, field).set value
  end

  When /I attach "(.*)" to "(.*)"/ do |value, field|
    $browser.file_field(:id, find_label(field).for).set(value)
  end

  When /I check "(.*)"/ do |field|
    find_by_label_or_id(:check_box, field).set true
  end

  def find_by_label_or_id(element, field)
    begin
      $browser.send(element, :id, find_label(/#{field}/).for)
    rescue #Celerity::Exception::UnknownObjectException
      $browser.send element, :id, field
    end
  end

  When /^I uncheck "(.*)"$/ do |field|
    $browser.check_box(:id, find_label(field).for).set(false)
  end

  When /^I select "([^"]+)" from "([^"]+)"$/ do |value, field|
    find_by_label_or_id(:select_list, field).select value
  end

  When /^I select "([^"]+)"$/ do |value|
    $browser.option(:text => value).select
  end

  When /I choose "(.*)"/ do |field|
    $browser.radio(:id, find_label(field).for).set(true)
  end

  When /I go to the (.+)/ do |path|
    $browser.goto host + path_to(path)
  end

  When /I wait for the AJAX call to finish/ do
    sleep 0.4
  end

  When /^I visit "([^"]+)"$/ do |url|
    $browser.goto host + url
  end

  Then /^I should see "(.*)"$/ do |text|
    Then "I should see /#{Regexp.escape(text)}/"
  end

  Then /^I should see \/(.*)\/$/ do |text|
    # if we simply check for the browser.html content
    # we don't find content that has been added dynamically,
    # e.g. after an ajax call
    div = $browser.div(:text, /#{text}/)
    begin
      div.html
    rescue
      #puts $browser.html
      raise("div with '#{text}' not found")
    end
  end


  Then /I should see the text "(.*)"/ do |text|
    $browser.html.should include(text)
  end

  Then /I should not see the text "(.*)"/ do |text|
    $browser.html.should_not include(text)
  end

  Then /I should not see "(.*)"/ do |text|
    div = $browser.div(:text, /#{text}/).html rescue nil
    div.should be_nil
  end

  Then /I should see no link "([^"]+)"/ do |text|
    $browser.link(:text => text).should_not exist
  end

  Then /I should not find the page "([^"]+)"/ do |url|
    no_exception = false
    begin
      $browser.goto host + url
      no_exception = true
    rescue => e
      e.message.should =~ /404/
    end
    no_exception.should be_false
  end

  Then /^"([^\"]*)" should be chosen$/ do |field|
    find_by_label_or_id(:radio, field).should be_checked
  end

  Then /^"([^\"]*)" should be checked$/ do |field|
    find_by_label_or_id(:check_box, field).should be_checked
  end

  Then /^"([^\"]*)" should not be checked$/ do |field|
    find_by_label_or_id(:check_box, field).should_not be_checked
  end

  Then /^"([^\"]*)" should be selected$/ do |selection|
    $browser.option(:text => selection).should be_selected
  end

  When 'show response' do
    p $browser.url
    open_response_in_browser
  end


  def find_label(text)
    $browser.label :text, text
  end

  def open_response_in_browser
    tmp_file = '/tmp/culerity_results.html'
    FileUtils.rm_f tmp_file
    File.open(tmp_file, 'w') do |f|
      f < < $browser.div(:id, 'content').html
    end
    `open #{tmp_file}`
  end
</code>

Make sure to change the database name (and host if necessary).

Finally the paths.rb


  module NavigationHelpers
    def path_to(page_name)
      case page_name
      when /start page/
        "/#{database}/_design/#{database}/index.html"
      else
        raise "Can't find mapping from \"#{page_name}\" to a path."
      end
    end
  end

  World(NavigationHelpers)

Now that you are done with the prerequisites you can write your first feature. Create a file start_page.feature in the features directory and paste the following:


  Feature: start page
    In order to feel welcome
    As a user
    I want to be welcomed on the start page

    Scenario: go to the start page
      When I go to the start page
      Then I should see "Welcome"

Now you can run your first feature:

jruby -S cucumber features/start_page.feature

Congratulations, you can now test drive your application. Explore the step definitions in the common_culerity_steps.rb in order to learn how to traverse links (When I follow “link”), fill out forms etc.

In order to make the feature pass you should add the text “Welcome” to the index.html of your CouchApp.

DevHouseBerlin^2

2009-10-06T00:00:00+00:00

Last weekend the second DevHouseBerlin took place. A public event and the perfect excuse for you to not be available for your non-geek-, girl- and boyfriends during the weekend.

DevHouseBerlin stands in the tradition of the SuperHappyDevHouse. For one Weekend we opened up the office space from finnlabs, rocket rentals and us again to host everybody who is curious about technology in general and programming in particular, offering the opportunity to work on, show off or discuss whatever you like together. We provided music, drinks - especially Club Mate -, food - super delicious deluxe sandwiches - and other basic infrastructure, although our supposedly capable new WiFi router failed miserably on serving 30+ devices.

The 25 available seats were gone weeks before the actual event so we didn’t blog beforehand so that you wouldn’t be disappointed. But as we will move to a bigger office soon we will scale the next DevHouseBerlin too. So here are some highlights and results you missed.

We had an international guest and hero with us, Pat. During the last weeks he got the full blown Berlin tech culture while pairing with us, giving his sphinx talk at rug-b (Ruby User Group Berlin) and joining the dev house, where he obviously had fun although he was constantly besieged by fans ;). He even managed to get some new features for thinking sphinx done (1, 2).

We also planned a realtime battle challenge, a programming game, in which robots controlled by programs are fighting each other. But despite collaborative effort we couldn’t get it running for OS X so alex, till, lukas and pat took instead the challenge to re-implement it with ruby and javascript. A slightly too big endeavor for these two days with so many other things going on. See their progress and fork it on github.

A major source of distraction was the Segway which @jodok parked in the office. Although you look totally dorky it is a lot of fun, even just riding down the corridor.

Dispite the calamity on the corridor Till and Christian proved that you can write web apps fast and well with PHP, they rewrote planet php, a php blog aggregator, in a couple of hours on the first evening.

Dmitry and Hans-Gunther brought a 19” server rack to play around with the Xen Hypervisor, well a 19” rack is certainly the right stage for Xen. Hans-Gunther also saved our evening by providing back-up Internet through his G1 while our router was recovering.

Something else which had to do with big scales was Mahout, a bunch of machine learning libraries built on Hadoop, an open-source implementation of frameworks for reliable, scalable, distributed computing and data storage. Some really mind bending stuff if you ask me. Too bad I missed the presentation by MaineC. If you are still studying you might want to attend her Mahout course at the TU-Berlin.

You and I also missed the discussion about the best editor – emacs or vim – and - even worse - CouchDB in 30 minutes. An Introduction to CouchDB, the schema-free document-oriented Database, given by Jan, the european mister CouchDB himself. Of course he was around all the time during the DevHouse and gave help to get on track with CouchDB and related stuff.

I could rave on and on how cool the second DevHouseBerlin was, but just take a look at the wiki page where you can find more projects, people and photos. So that even if you were not with us this time you might take something out of it.

See you next time at DevHouseBerlin.

Ruby Metaprogramming: Dynamically Inherited Methods™

2009-09-19T00:00:00+00:00

Yesterday I added a new feature to Couch Potato, my persistence layer for CouchDB that I call Dynamically Inherited Methods™ and found the implementation I came up with interesting enough to share. Here’s the problem:

class User

  def self.string_attr_reader(name)
    define_method name do
      instance_variable_get("@#{name}").to_s
    end
  end

  string_attr_reader :nickname
  attr_writer :nickname
end

user = User.new
user.nickname = 123
user.nickname # => '123'

This code adds a macro to the class user called string_attr_reader. When you call this macro on the class and pass it a name it generates a method that returns the instance of the same name converted to a string. This all works perfectly until you want to override the generated method to add some behavior:

class User
  def nickname
    super.gsub('6', '7')
  end
end

The enhanced method now replaces all occurrences of 6 with a 7 - except it doesn’t. When you run the above code you will get an exception where Ruby complains there is no super to call. That’s because we defined the nickname method in the User class so we didn’t inherit it, hence no super.

The one way out of this is to use alias_method, or alias_method_chain when you are using Rails:

class User
  def nickname_with_six_replaced
    nickname_without_six_replaced.gsub('6', '7')
  end
  alias_method_chain :nickname, :six_replaced
end

First of all this isn’t very beautiful anymore, and secondly using alias_method_chain when you could use standard object oriented metaphors (like inheritance) doesn’t make a lot of sense - except that you can call yourself a metaprogramming programmer, yay.

Anyway, there is another way - actually involving much funkier meta programming - to solve the problem:

class User
  def self.string_attr_reader(name)
    accessor_module.module_eval do
      define_method name do
        instance_variable_get("@#{name}").to_s
      end
    end
  end

  private

  def self.accessor_module
    unless const_defined?('AccessorMethods')
      accessor_module = const_set('AccessorMethods', Module.new)
      include accessor_module
    end
    const_get('AccessorMethods')
  end
end

The updated code now dynamically creates a Module called User::AccessorMethods and includes it into the User class. All methods generated by string_attr_reader are now added to that new module instead of the class. The result is that when overriding those methods you can now call super because they’re are inherited from the new module.

class User
  attr_writer :nickname
  string_attr_reader :nickname

  def nickname
    super.gsub('6', '7')
  end
end

user = User.new
user.nickname = 678
user.nickname # => '778'

While this code involves some fairly crazy metaprogramming which would be too much for a simple example like the above, I think libraries like CouchPotato can still benefit, as the application code can become cleaner by not having to do any metaprogramming but resort to standard object oriented ways of programming.

Upstream goes San Francisco!

2009-08-31T00:00:00+00:00

Today is our last day in the Berlin office before this year’s remote office. After the first iteration last year, where we stayed in beautiful Oslo for a week, we have extended the time and distance for this year: 3 weeks of San Francisco.

We’ll be working there (mostly) as usual during the days and then explore the city in the nights. For the first week we rented a beautiful apartment from where we can work and start exploring. We are still waiting for some other great place to come to us for the rest of our stay and we’ll also try out all the cool and hip coworking spaces. Apart from that we want to meet geeks and normal people, go to an apple store for the first time, maybe attend some user group events, meet with some CouchDB folks, drive across the bay … whatever comes up.

If you want us to buy you a beer, have some spare desks for a few days in your office where we could work, any hints for places/events we have to visit or know someone who might rent out an apartment please let us know in the comments. You can also follow and contact us on twitter.

Kickstart Rspec with Spork

2009-07-27T00:00:00+00:00

For doing Red/Green TDD it is essential to have fast running tests. But no matter how fast your tests are, there are always these seconds when you and your pair are sitting in front of the screen waiting that the test run actually starts. That is the time when Rails, your code, gems and plugins are loaded. In a rather small project, which I use as an example here, this cumulates into a start up time of around 5s before EACH test run. This becomes even longer when your project grows.

..... Finished in 0.127571 seconds 5 examples, 0 failures real 0m6.456s user 0m4.682s sys 0m1.681s

Although 5s, as in my example, aren’t such a long time, it sums up to a significant amount when doing Red/Green TDD. So everything that helps you to cut down test startup times, increases your productivity. AutoSpec or Rspactor approach this on the user site by running your test as soon as you change something. But they won’t cut down the technical implied start up time. Spec-server tried to approach the load time issue first by loading the Rails stack including the libraries of your environment upfront in an extra process. It uses Drb to get your test code to the pre-loaded testing process. Although the idea behind spec-server is the right one, it never worked for me in day to day usage. I came in situations in which tests failed or code wasn’t reloaded properly at all. But recently I discovered spork thanx to Rany (@purzelrakete). Spork takes the idea from spec-server and learned from its mistakes. E.g. it has its own class loading mechanism that takes care of not preloading your models at startup, instead of relying on rails auto loading. Beside working more reliable, this is also an option for other frameworks but Rails. It just works! Running the same test code than before with spork (spec paht_to_spec --drb), cuts the startup time into half.

..... Finished in 0.066628 seconds 5 examples, 0 failures real 0m3.588s user 0m0.469s sys 0m0.108s

When doing 200 test startups a day, a number which is not unrealistic, consider doing it Red/Green, this a 10 min time win :) You probably won’t need more than 10 min to set up spork.

Install spork

gem install spork

Bootstrap your Rails app

cd /path/to/project/root spork --bootstrap

This command adds some instructions and code to spec/spec_helper.rb. Basically two blocks, one to tell which code to preload and one to tell which to reload on every test run. Code outside these blocks is run at both times.

Modify spec/spec_helper.rb
Modify spec/spec.opts

Add --drb to spec/spec.opts so that rake spec uses drb to communicate with spork. Set TM_RSPEC_OPTS --drb in TextMate so that the rspec TextMate bundle uses drb as well.

Fire up spork for rspec

spork rspec

Kickstart your RSpec Tests

If you find out that some of your code wasn’t reloaded properly, spork -d shows you which classes are loaded and where from. There shouldn’t be any of your app/classes in it. If that’s the case, the reason for this can be either a eager plugin (e.g. older versions of thinking_sphinx) or you used your code in initializers or the environment. Put such code in spork blocks like these in your spec_helper for the testing environment, this will prevent wrong preloading.

Here is an example how the spec_helper could look like:

require 'rubygems'
require 'spork'

ENV[&quot;RAILS_ENV&quot;] = &quot;test&quot;

Spork.prefork do
  require File.expand_path(File.dirname(__FILE__) + &quot;/../config/environment&quot;)
  require 'spec'
  require 'spec/rails'
  require 'machinist/active_record'
end

Spork.each_run do
  require 'spec/blueprints'
end

Oh and btw, you can use spork for Cucumber as well, just modify cucumbers env.rb (same as you did with your spec_helper) and then launch spork with spork cucumber

And one last thing, as we are talking about faster test cycles, take a look at parallel _specs, to run your specs on multiple cores.

Happy TDDing!

Upstream Agile Project Checklist (updated)

2009-06-04T00:00:00+00:00

updated: added sven’s point from the comments

Explaining agile to customers is hard, sometimes more, sometimes less so. In order to improve communication and make it clear what to expect and what not to I came up with the following checklist of what agile projects are about:

You DO get a working, maintainable, production ready, 100% quality piece of software in a fixed amount of time.

You do get a first working, maintainable, production ready, 100% quality version of that software after a very short time (usually 1-2 weeks).

You DO get to launch your software in time.

We DO pair program all the time to ensure that quality and it doesn’t slow us down.

We DO write automated tests to ensure that quality and it doesn’t slow us down.

You DON’T necessarily get all the features you wanted after an iteration.

That’s why it’s important that you DO prioritize what’s most important for you.

We DO estimate how long each feature will take us to implement. But it’s only an estimate.

We DON’T accept a functional specification you bring in as a basis for our work.

We DON’T believe this would work. We DON’T believe it is possible to completely specify a software upfront with tolerable effort.

We DO use your functional spec as a basis for a discussion where we will write down stories for each feature together with you. Those form the basis of our work.

We DO embrace changing requirements by working in short iterations.

We DO deliver successful software projects when working like this.

Signature Customer

Anything I forgot? The comments are open.

Conference Triathlon - Euruko, Ruby on OS X, RailsWayCon

2009-05-28T00:00:00+00:00

Wow, this was the month of conferences. First we visited Barcelona for the Euruko: a great conference taking place every year in a different city. We attended it the 3rd time. The talks ranged from practical like “Cooking with Chef”, over entertaining like “Fun with Ruby (and without R***s), program your own games with Gosu” to just geeky like the lightning talk about “Vimmish and how much fun gramma parsers can be”. I really liked the 2 days 1 track format and the people I met there. And you can be sure that we will be in Krakow next year too.

5 days later I visited Amsterdam for Ruby on OS X, which I already blogged about.

And a little more than a week later, until yesterday, four of us attended the RailsWayCon here in Berlin, which tries to fill the gab that the RailsConf Europe left. The first day was reserved for whole day tutorial sessions.

The second day offered a lot of advanced topics to choose from in 3 tracks. I chose to hear more about Asynchronous Processing from Mathias Meyer and the nitty gritty details about Events from Lourens NaudÃ©. The keynote about the Present and Future of Programming Languages by Ola Bini was also very interesting.

Upstream also took an active part at the conference: Alex gave his talk about CouchDB Frameworks for Ruby and CouchApp (using his new presentation tool boom_amazing) and I introduced MacRuby, the Ruby that plays nice with Objective-C.

On the third day Yehuda Katz revealed some more details about Rails 3. The talk by Michael Koziarski about Rails Performance was good for a reality check. In the afternoon everybody was tired after 3 days of conference and the talks lost quality. Still a very good conference with potential.

Notes from the Ruby on OS X conference

2009-05-18T00:00:00+00:00

Edit: I updated some infos, to address things Eloy pointed out in the comment.

I had the opportunity to be at the Ruby on OS X conference in Amsterdam. As the name suggests, it was a conference focused on the state of Ruby on the OS X Platform. The talks generally gave a good background about the current status and where Ruby on OS X is going. All talks will be available in video format, according to fingertips. In the meantime, here are my conference notes for a quick overview. Though they vary very much in detail, depending on my personal interest and my ability to focus after getting to Amsterdam during the night before the conference.

The first talk was about LimeChat an IRC chat program by Satoshi Nakagawa written in RubyCocoa. Some LimeChat highlights are the compact interface, its unobtrusive operation, the always-accessible server list, the build in online paste service support and the customizable themes. Satoshi pointed out some edges between RubyCocoa and Objective-C. He mentioned necessary conversion between NSString and RubyString, because they aren’t comparable, the required NSMutableString for Unicode Characters and that the creation of Ruby threads causes interruption of the application loop. Further, he reported that the usage of ruby socket lib or net/http causes hang-ups, thus he used the Objective-C equivalent libs. On the other hand he pointed out that performance was good enough, memory usage and leaks weren’t an issue either, even during longer operation. Determining factor for memory usage was the complexity of the DOM tree in the WebView backed chat window. So no major problem here. LimeChat is currently being ported to MacRuby.

The second talk was held by Rich Kilmer who presented the state of HotCocoa and MacRuby. I watched and read similar talks by him online already, so I only wrote down some interesting bits. In the beginning he gave some insight into the relation between Ruby and Apple. For example ruby gems were changed for Apple in a way that is allow to install certain gems in an extra repository where they are protected from user access. Apple will ship the next OS X version with Ruby 1.8.7. He also stated very clearly, that RubyCocoa will still be supported by Apple in the future. Which makes not much sense to me, because since the appearance of MacRuby the development is effectively come to an halt. But, as Eloy stated in the comment, Apple promised to support it into the future, because developer have put lot of effort into applications based on RubyCocoa.

Then Rich introduced MacRuby. Laurent stated during this part of the presentation, that although memory allocation is a little slower in MacRuby, Objects don’t need more Memory than their pure Objective-C counterpart. After outlining the improvements of MacRuby, Rich came to his Cocoa-mapping layer HotCocoa. HotCocoa tries to provide sensible defaults for most Cocoa GUI Objects. It also provide short names for important, mostly rather long named Objective-C Constants. All the mapping files take care of themselves to include their corresponding framework. He than explains HotCocoas delegation feature, which allows you to provide delegate methods by defining its logic in a block. A documentation for all these mappings, which can be viewed only in lib/ruby/1.8/hotcocoa/mappings until now, will be available soon.

Further, he explained the HotCocoa command line tool. Which, like the Rails script command, generates some infrastructure to build and deploy MacRuby apps from the command line. This infrastructure will be extended soon by providing a more MVC like directory structure. Another addition will be the support of a virtual file system which allow to ship self contained applications and hide the source code. At the end of his talk he mentioned the possibility to bring MacRuby code to other platforms with the use of a ahead of time compilation.

Without further ado the talk about Rucola by Eloy Duran followed. Rucola is a CLI focused MVC framework for RubyCocoa. It provides rake tasks for compiling, testing, bundling and update feed setup. Like known from Rails, it supports multiple environments. For the ease of development it offers generators for various purposes e.g. for test stubs. It also provides libs to help with testing. Rucola will also be available in MacRuby.

After the lunch-break John Shea talked about his experience with MacRuby and game programming. He just needed 2 hours to implement Tetris in 2D with NSView. Doing this in OpenGL was a little harder. He showed different other games made by him which use OpenGL and were implemented mainly with MacRuby (99%). OpenGL can be used through CoreGraphics Interface with MacRuby. That way, he achived to display 100K polygons including textures with 58.1 Frames. Also some nice particle effects implemented with MacRuby where shown. As limiting factor when using MacRuby he pinpointed the garbage collection and the method dispatch performance. Latter should be fixed with MacRuby 0.5 as Laurent stated later.

The following talk by Manfred Stienstra was about testing Cocoa Apps with the Rucola framework. As a testing framework it uses test spec together with mocha for mocking.

One major point of the talk was how to test controllers. Rucola provides methods to generate fake outlets to test the controller against. But this didn’t work with Objective-C controller. Thus as another mean a small compiler in Rucola can be used to compile and return a controller from a nib file. Working fake outlets for Objective-C Controllers will be supported as expected on MacRuby though.

Sometimes an existing application loop is required. Manfred suggested to setup and run this loop in a separate thread in the background while testing. Also the issues of UI testing came up. A practical approach was to load the nib file and inspect the content.

A less technical, but still not less interesting talk about shipping OS X desktop applications with an interpreted language was given by Koen Bok from Sofa. At first he talked about shipping the source with the product. In his experience it is better to face the legal and commercial risk by protecting the code by law than invest time and money in obfuscation. So you will be legal prepared, if problems arise, which didn’t happen so far to Sofa. The second part was targeted specifically at people who came from web development to the Mac. You should be prepared to be still a second grade citizen on the OS X platform. As Ruby people we should use our advantages of mature frameworks and tools we know from our work with e.g. rails. Third, the user experience is a major factor on OS X with a higher baseline than web applications. Thus more work is required e.g. for Drag and Drop behavior or keyboard shortcuts. Also snappiness is an important factor for the user experience at OS X. So optimizing speed/snappiness is an optimization of the user experience. The iPhone is a good example to learn from. As for the general interface design it is a good idea to have a designer in the team, or as Koen stated it. Get a Designer!

The last official talk was about Gosu, an easy-to-use game development library and Rubenstein, a satirical re-implementation of Wolfenstein in Ruby with the help of Gosu. Julian Raschke, the creator of Gosu and Ninh Bui from phusion and the author of Rubenstein, held the talk. First Ninh explained some theory about Raycasting for rendering a 3D game. Then he looked at some optimizations, like splitting the space in discreet elements of pixels to speed up the Raycasting. At last he explained how he used Gosu for texturing in Rubenstein.

Then Julian got into detail about Gosu. The library consists only of 9 classes and has a really easy to use API. It is around for 3 years already, and the main focus stayed on the easy usability. It is available for Mac, Linux and Windows (gem install gosu). Julian showed some games created with Gosu and talked how it helps to interact with other frameworks, e.g. open-gl for 2D/3D effects or Chipmunk for physics simulation. At last both demoed Rubenstein, hilarious. That marked the official end of ruby on OS X.

At the Hackfest after lunch Laurent Sansonetti, the brain behind MacRuby, showed some internals about the currently experimental 0.5 branch, which is based on LLVM. He talked about its current state, which mile stones they reached already to make MacRuby 0.5 installable, and showed some dark corners of ruby code, which make the job so hard.

I also got a glimpse on a new iPhone game from Julian made with Gosu, which looked pretty good. But before you get too excited about the thought of ruby on the iPhone, it was made with C++ version of Gosu. The Game will be officially released soon.

Later I joined Cristi, Eloy, Julian, Laurent and others for a beer and a night out in Amsterdam till I caught my train back to Berlin early in the morning.

Ruby on OS X was totally worth the effort to get there spontaneously. This rather long day I learned a lot about Cocoa and MacRuby, met very nice people, as on every ruby conference so far, and got an impression of Amsterdam that made me want to visit the city again, but than with more time.

New Couch Potato: simple, testable, opinionated.

2009-05-17T00:00:00+00:00

After my talk about Ruby CouchDB frameworks at Scotland on Rails where I dismissed a few of of the libraries available (including my own Couhch Potato) as not fitting the CouchDB way of doing things, I have been hacking away the past few weeks working on a complete overhaul of Couch Potato.

As a first result I have just released version 0.2 of the framework. Its new goals are simplicity, embracing the CouchDB semantics and testability. In order to achieve this I had to introduce some major changes:

I disconnected models from the database - there are no more save/get/find methods in the models. Instead you can hand the models to a database object that will save/load documents for you - this allows for unit tests that are completely disconnected from the database

I have dropped associations and thrown away all the ActiveRecord like view creation/querying, replacing it with a new, more CouchDB like system. That new system is lighter, simpler and easier to extend.

The following paragraphs will show you how to work with the new Couch Potato.

Saving / loading models

As I said I have decoupled the models from the database, a model doesn’t have permanent access to the database anymore. Instead you instantiate a database object yourself and tell it to load or save a model object. This change isn’t so much about CouchDB as it is about testability. Having the database separated means you can now have true unit test of your models without talking to the database. Here is how you save/load models:

class Book
  include CouchPotato::Persistence
  property :title
end

CouchPotato::Config.database_name = 'my_db' # in Rails this is done for you

db = CouchPotato.database

book = Book.new :title =&gt; 'The Passionate Programmer' # good book

db.save! book # saves the book or raises an exception


db.load book._id # the original book

The database is responsible for running the model’s validations and lifecycle callbacks, saving the document to the database and afterwards setting the _id and _rev on the model. This way a lot of the persistence related logic is removed from the models making them more lightweight and most importantly easier to test (see below).

New Views

Overhauling of the views had two main goals:

provide a simple and extensible way for saving/querying views that works the way CouchDB works
since models don’t have access to the database anymore, find a new way to query them

Here is how you create and query a simple view:

class Book
  view :by_title, :key =&gt; :title
  property :title
end

 db = CouchPotato.database
 db.view Book.by_title # no parametters

 db.view Book.by_title(:key =&gt; 'The Passionate Programmer', :descending =&gt; true) # just use any of the parameters CouchDB accepts

The way views work is now essentially reversed (inversion of control, rings a bell?). Instead of the view method calling the database the new view method creates a specification for a CouchDB view. The view is passed to the actual database in order to query CouchDB. Again this makes testing easier (see next section) but it also gives Couch Potato a clean interface in order to make creating views easier through abstractions. There is now a hierarchy of view specification classes that you can use to more easily create and query views. The above example used the CouchPotato::View::ModelViewSpec which is the default. This spec creates a view whose map function emits one (or an array of) attribute of the model and returns full documents.

Other view specs let you create more customized views, for example the RawViewSpec, which lets you define your own map/reduce functions and return the raw CouchDB results hash.

class Book
  view :title_length, :type =&gt; :raw, :map =&gt; "function(doc) emit(doc.title.length, null)}"
  property :title
end

db.view Book.title_length # returns something like {:rows =&gt; [{:key =&gt; 25}]}

For more examples see the Documentation in the CouchPotato::View::*ViewSpec classes.

Testing

As I have mentioned repeatedly decoupling the database from the models makes testing easier. With the new Couch Potato you can unit test your models without hitting the database once. First of all that makes your test lightning fast, and secondly it allows you to more easily test for example your lifecycle callbacks because you can call them directly passing them a stub or mock database.

Here’s an example: (with RSpec)

class Book
  property :title
  property :slug
end

describe Book, 'create' do
  it 'should generate a slug' do
    book = Book.new :title =&gt; 'The Passionate Programmer'
    book.run_callbacks :before_create
    book.slug.should == 'the-passionate-programmer'
  end
end

If you don’t like calling the run_callbacks method you can also use the actual database object but without a real connection to CouchDB. Couch Potato is based on the excellent CouchRest - a more low level CouchDB framework. The CouchPotato::Database constructor expects an instance of a CouchRest database which we can replace with a stub:

describe Book, 'create' do
  it 'should generate a slug' do
    book = Book.new :title =&gt; 'The Passionate Programmer'
    db = CouchPotato::Database.new stub('couchrest database', :save_doc =&gt; {})
    db.save book
    book.slug.should == 'the-passionate-programmer'
  end
end

For more details and examples please see the README, the RDocs and watch this blog. If you want to know all about Couch Potato I encourage you to read through its source code and specs - it’s not that much code actually. Note that all this is still work in progress and time will show how well all of this works. I would be happy to hear your feedback.

Installing yum on CentOS 5.3

2009-04-30T00:00:00+00:00

We recently deployed an app on a VM with CentOS 5.3. To our delight, the packet manager yum wasn’t pre-installed. We spent some time to get it running. The Howto on http://eric.lubow.org/2008/misc/adding-yum-to-centos-5/ was a great help, but it was for CentOS 5.2. So this is an update to Eric’s blogpost.

You need to have rpm and wget already installed. Create a temporary directory, paste the following code in a file in that directory, make it executable and run it:

mkdir temp && cd temp vim yumdownload

#!/bin/bash for file in \ elfutils-0.137-3.el5.i386.rpm \ elfutils-libs-0.137-3.el5.i386.rpm \ elfutils-libelf-0.137-3.el5.i386.rpm \ expat-1.95.8-8.2.1.i386.rpm \ gmp-4.1.4-10.el5.i386.rpm \ libxml2-2.6.26-2.1.2.7.i386.rpm \ libxml2-python-2.6.26-2.1.2.7.i386.rpm \ m2crypto-0.16-6.el5.3.i386.rpm \ python-2.4.3-24.el5.i386.rpm \ python-elementtree-1.2.6-5.i386.rpm \ python-iniparse-0.2.3-4.el5.noarch.rpm \ python-sqlite-1.1.7-1.2.1.i386.rpm \ python-urlgrabber-3.1.0-5.el5.noarch.rpm \ readline-5.1-1.1.i386.rpm \ rpm-4.4.2.3-9.el5.i386.rpm \ rpm-libs-4.4.2.3-9.el5.i386.rpm \ rpm-python-4.4.2.3-9.el5.i386.rpm \ sqlite-3.3.6-2.i386.rpm \ yum-3.2.19-18.el5.centos.noarch.rpm \ yum-fastestmirror-1.1.16-13.el5.centos.noarch.rpm \ yum-metadata-parser-1.1.2-2.el5.i386.rpm do wget http://mirror.centos.org/centos-5/5.3/os/i386/CentOS/$file; done

chmod a+x yumdownload ./yumdownload This will get you all rpms you might need before be able to install yum.

Now try installing them all by typing: rpm -Uvh *.rpm This might not work, at least not for us. So try to install yum directly with rpm -Uvh yum-3.2.19-18.el5.centos.noarch.rpm and let the errors guide you. When you run into a circular dependencies, install the rpm by ignoring the dependencies. eg. rpm -Uvh --nodeps yum-fastestmirror*

This should finally allow you to install yum. Good Luck. ;)

The case of ActiveRecord vs. CouchDB

2009-03-31T00:00:00+00:00

After my Ruby sittin’ on the Couch talk about Ruby Frameworks and CouchDB at Scotland on Rails last week a bit of a debate started. In the talk I did compare the available Ruby Frameworks and discussed how well they fit the CouchDB way of doing things. In my conclusion I recommended using either CouchRest or RelaxDB for development and at the same time urged people not to use one of the ActiveRecord like libraries like CouchFoo.

As it seems to me the intention of my talk hasn’t reached all of its audience yet I’ll try to make my point again using this blog post.

The frameworks I looked at could be divided into two classes: the ones using CouchDB semantics (e.g. CouchRest/RelaxDB) and the ones trying to provide an ActiveRecord like interface for the applications (e.g. CouchFoo). The reasons I recommended not to use ActiveRecord semantics are:

CouchDB is not about relations

In ActiveRecord we have to model our domain models so they fit into a relational schema. That means flat tables and relationships between two tables, through a third and fourth table etc. To get results from the database we usually join a few tables and get back the resulting rows, nicely converted into Ruby objects for us. That is (sort of) fine for a relational database because all it has are those tables but it doesn’t work at all with CouchDB.

To use CouchDB to its full potential you need to understand and use its views

CouchDB doesn’t have a concept of tables at all. And the way you pull your data from CouchDB is fundamentally different. Instead of joining data from different tables via an SQL query you procedurally build up an index of data by providing the map and reduce functions which you then query.

In order to fully use CouchDB you have to write custom map/reduce functions and abstracting that into an API that was designed for generating SQL queries doesn’t allow you that.

I could go into more details but that really is my main point. I know from my experience that changing the wiring in your head after too many years of ActiveRecord is hard so I don’t expect anyone to immediately agree with me but I do believe that the only viable way of using CouchDB is through the interface that was designed for it and not by an abstractions that just happened to be there first.

I’m looking forward to a lively discussion, either in the comments or on other blogs.

The upstream Rails Application Template

2009-03-06T00:00:00+00:00

Ruby on Rails 2.3 is almost there and brings a bunch of pretty cool new features. One of them is called templates and it allows you to customize the bootstrapping of new applications in order to automate the initial setup of your new app and hence get up to speed faster.

I find the naming a bit confusing: since we already have view templates in Rails I will refer to this new feature as application templates from now on.

So what does an application template do? Whenever you start a new rails application you are already using the Rails default template by issuing the rails command. It creates the usual app, config etc. folders and generates a bunch of scripts and configuration files for you. Application templates simply extend this process by allowing you to add your own setup steps. This is done by writing your own template file (or grabbing someone else’s).

Pratik Naik has already written an excellent tutorial on this so I’m not going to repeat what’s already written. Basically Rails offers you a few convenience methods for adding for example routes, plugins, gems or files to your new application.

So this blog post is more about showing off my very own template that I wrote today. (For the impatient: here it is).

Of course I’m not the first and only one who has written such a template so before I started I found this collection of templates which I immediately started to ~~steal~~ borrow from. As you probably know every programmer has its own style, uses his own unique set of tools in his (or her) own way. Which is why I found all of the templates I saw didn’t really fit my very own needs. The ones I found were pretty basic (to me) so when I rolled my own I took what I needed from what was there and added a whole bunch of my own stuff:

setup of configuration files for Rspec, Cucumber and Culerity (Culerity is my plugin for driving Celerity - think Selenium but withour having to use a real browser - with Cucumber)
a working user registration/login process using authlogic, including all controllers and views
an XHTML application layout with jQuery (+ a few plugins) and blueprint CSS set up
a simple capistrano deployment script
German localization for all the built-in helpers
a Thinking Sphinx configuration file

I pretty much need all of the above for 99% of my Rails projects and it costs me at least a day every time to set this all up. Well, not anymore :) Now I can run rails my_new_app -m http://gist.github.com/75038 and immediately after that start working on the distinct features.

I’m already planning to add a few extra. First of all the user authentication needs cucumber features so I don’t break anything when extending that. I also want to add configuration and deployment for a staging server. And maybe make the whole script a bit more configurable, i.e. install thinking sphinx or not etc.

The script is already pretty large so I’m not sure how much more I would want to put into it. Since you can apply templates to exisitng apps it will probably make sense to split the whole thing up at some point, so I could put the authentication part into its own template.

If you want to get started with application templates now I suggest you simply start with mine and/or the others and grab whatever you need. Just make sure you re-publish what you added so we can all build upon everyone’s work. Thanks.

Testing PDFs with Cucumber and Rails

2009-02-14T00:00:00+00:00

On a recent Rails project we have been working a lot with PDF documents. The application generates PDF invoices and account statements, places markers and comments into existing documents and also merges multiple single page PDFs into one. We do all this with a few PDF tools:

the PDF::Writer library for Ruby to generate PDFs
pdftk - to merge multiple PDFs into one or overlay them
pdftotext part of the xpdf library, to extract texts from PDFs

On OSX all of these can be installed via MacPorts. Debian has packages as well.

While PDF::Writer is a Ruby library pdftk is a command line tool. We simple call it using Kernel.system and check the return code via the $? variable.

unless system ("pdftk #{source_pdf} ... output #{target_pdf}")
  raise PdfError("pdftk returned error code #{$?}")
end

With all this PDF processing the need for testing the contents of the generated documents arose. We have factored out all the PDF processing into a bunch of extra classes which we simply unit test with RSpec: make sure the parameters are passed to the command line correctly, that the right exception is thrown for each return code etc.

In addition to unit testing we also write customer driven acceptance tests with Cucumber, where we assert on a high level the outcome of certain actions. With HTML pages we can simply use the built-in steps that in turn use Webrat to parse the HTML like this:

Given a purchase over 200 EUR
And an invoice
When I go to the start page
And I follow "Invoice"
Then I should see "200 EUR"

Now in our case the invoice link links to a PDF but we still want to know what’s inside the document. The solution we came up with looks like this:

Given a purchase over 200 EUR
And an invoice
When I go to the start page
And I follow the PDF link "Invoice"
Then I should see "200 EUR"

What this does in the background is follow the link as usual, write the response into a temporary file, convert that to text using pdftotext and write the result back into the response. This way we can make assertions about the contents of the PDF almost as if it were an HTML page (except for tags of course). Here is the implementation:

When 'I follow the PDF link "$label"' do |label|
  click_link(label)
  temp_pdf = Tempfile.new('pdf')
  temp_pdf << response.body
  temp_pdf.close
  temp_txt = Tempfile.new('txt')
  temp_txt.close
  `pdftotext -q #{temp_pdf.path} #{temp_txt.path}`
  response.body = File.read temp_txt.path
end

Bug fighting with Test Driven Development Follow-Up

2009-02-08T00:00:00+00:00

On Thursday I gave a talk about Bug fighting with TDD and how it will help you to write better software at the Ruby User Group in Berlin. Sorry, I won’t put the slides online, although people asked about it. I think they are a pretty useless collection of keywords and pictures without the talking. Instead I like to provide a collection of links to dive into the whole testing matters.

First a collection of tools you can choose from for various testing scenarios. They all have their pros and cons, it just comes down to which will do the job best for you.

Tools for Unit Testing

Rspec
Test/Unit (part of Rails)
shoulda

Tools for Mocking

Tools for Acceptance Testing

Tools to generate test objects

Misc tools

To save you the initial searching where to start, I collected some links where you find basic information about testing in general or with certain tools.

Getting started

For everyone who wants to get more into the subject, I drilled down my feed reader to collect some interesting blog posts on testing matters.

Opinions and In-sign

So much for my two cents to make the software world better.

Does your company need help with software testing? We can help: check out our new product Scene Investigation.

Hacking CouchDB, learning Erlang and testing in Javascript

2009-02-01T00:00:00+00:00

Last week was my first Erlang gig. I paired up with Jan Lehnardt (core committer of CouchDB). Our goal was to implement a new statistics module for CouchDB within one week. Jan knew more about the CouchDB code and Erlang, I contributed my knowledge about testing and pair programming.

Integration Testing in JavaScript

Since the testing infrastructure in CouchDB left some room for improvement we decided to introduce some new concepts. The existing tests were written in JavaScript penetrating the database via its HTTP/JSON interface. The tests were really long and coarse grained which lead to a problem: When one of the tests broke it wasn’t very easy to figure out which one. We didn’t want to replace or add too much to the existing, self-made test framework so we just added a bit of sugar: BDD style specs with meaningful names:

var response_count_tests = {
  'should show the number of 404 responses': function(name) {
    // Given

    restartServer();

    // When

    CouchDB.request('GET', '/some_nonexistant_url');

    // Then

    TEquals(1, CouchDB.stats_request('httpd', 'not_found'), name);
  }
};

var all_tests = [response_count_tests, ...];

for(var i in all_tests) {
  test_group = all_tests[i];
  for(var j in test_group {
    test_group[j](j);
  }
}

The TEquals function is just our extension of an existing function T which is responsible for reporting errors when the assertion doesn’t match. By passing the name of the ~~test~~ spec we can now display that name along with the error message:

Error in ‘should show the number of 404 responses’: ‘CouchDB.stats_request(‘httpd’, ‘not_found’)’ should be ‘1’ but was ‘0’.

Unit Testing in Erlang

In addition to those integration tests we wanted to add some unit testing on a lower level. Turns out the only unit testing framework for Erlang seems to be EUnit, which is a very barebones (at least when you come from something like RSpec) xUnit type of tool. You define a test by appending “_test” to your function name and then you have a bunch of assert statements at your disposal. Well that’s at least something and it worked. Here is an example:

should_return_zero_if_nothing_has_been_counted_yet_test() ->
    test_setup(fun() ->
        Result = ?MODULE:get({httpd, average_request_count}),
        ?assertEqual(< <"0.00">>, Result)
    end).

We added the test_setup for starting and tearing down processes needed for our tests. Then we pass a fun with the actual test code. We got through the week pretty ok with this but I really missed a few things:

The test output was not red/green - ok I could live with this, at least for a while, if the output was at least formatted in a way that would allow me to spot the problems more easily. The error output of erlang seems to be a big mess in general.

No mocking/stubbing framework. I don't even know if this is possible at all, at least I'm now aware of any way to change the behavior of Erlang code after compilation. (would that contradict Erlang's share nothing philosophy?)

contexts for tests - I want to be able to group my tests, not sure how this would work in Erlang syntax as there are no nested namespaces or things you can use to group methods (except for modules, but I want something smaller and you can't nest them). You would probably end up using lists of funs or something.

Erlang

I’ve <a href=”“http://upstream-berlin.com/2007/07/24/programming-erlang-functional-programming/>read the book, I’ve watched numerous screencasts but this was my first real life project in Erlang. Many people complain about its syntax and while this isn’t the most beautiful language I’ve ever seen I have to say it’s not really a big problem. I got used to it on the first day.

What is really powerful in Erlang is pattern matching. Instead of simply assigning a value to a variable you can always match against the structure of that data to extract parts of it:

Data = {"Hans", "Wurst", {born, 1980, 10, 23}}.
{FirstName, _, {born, Year, _, Day} = Data.

This example lets you pull out numerous details from the tuple in a single statement. You can do the same for case statements or when overloading functions. So this pattern matching is a powerful concept seen everywhere in Erlang code. It makes the code you write pretty dense which is cool on the one hand but sometimes also makes it hard to read so you have to be careful how much you want to cram into a single line of code.

CouchDB is built using make - which felt like 1990. I’ve never used make much myself so I’m not really qualified to talk about it but we did spend a good amount of time in our MakeFile because of some whitespace problem, and we still haven’t managed to integrate our new tests into the build process. I’d love to see something like Rake being used instead.

All in all this have been very pleasant first steps though. Pairing up for this turned out to be a very good idea, I have learned a lot about Erlang and am eager for more now.

Culerity = Full Stack Rails Testing with Cucumber and Celerity

2009-01-28T00:00:00+00:00

Since the day we started upstream testing was our obsession. When we began all we had was the Rails built-in unit, functional and integration tests. When we learned about RSpec the world became a brighter place. When the RSpec team released their StoryRunner we were thrilled. We started writing stories in English and were able to run them. Whoa. It soon turned out the first version had some serious issues with staying DRY. More versions were released, we rewrote our stories, step by step StoryRunner got better and we got better, too.

Then Cucumber came out and it was a huge leap ahead again. With the integration of Webrat we could write much shorter stories, but more importantly we could test that our links and forms were working as well. Everything was working perfectly and the world was a happy place - as long as we didn’t use any JavaScript/AJAX…

Welcome to Culerity. Culerity is a small gem that integrates Cucumber with Celerity, a wrapper around HtmlUnit which in turn is a Java library that parses HTML and runs the embedded JavaScript code. And unlike Watir or Selenium this all happens in headless mode: without hijacking your browser.

The problem with Celerity has been that it only works in a JRuby environment, which means you either had to run your application in JRuby as well (which might not even work with certain libraries and plugins) or somehow work your way around it by running your tests in JRuby and your application in whatever Ruby you wanted to use and somehow glue it together. Culerity now fills this gap. After installing it you run your Cucumber features as usual. Celerity now spawns a Java process in the background, sends all the Celerity calls to this process and evaluates the results back in the original Ruby environment” everything works (almost) as if you were running just in your single Ruby process. For an easier start Culerity comes with the same set of Cucumber step definitions that are provided by a default Cucumber/Webrat installation. This means you can reuse your step definitions written for Webrat.

To get started you can either clone the GitHub repository or simply install the gem (langalex-culerity). Then follow the instructions in the README. If you find something is not working for you or could be improved please feel free to fork the source code and show it some love. The codebase is really small right now and fully spec’d.

Does your company need help with software testing? We can help: check out our new product Scene Investigation.

Upstream went to Maine

2009-01-22T00:00:00+00:00

We went to Maine” well, close. To evaluate the past year and make plans for 2009, the Upstream crew recently spent a weekend together. We stayed in a holiday home right on the North Sea coast where we enjoyed the frosty weather and a cosy fireplace. Everyone brought about three cameras on average. This resulted in heaps of pictures with “naturey crap” (the amateur is speaking). See by yourself on Flickr, Tumblr and KommtMit.

Our CEOs Alex and Thilo gave a presentation about Upstream’s past year. Then, our aim was to outline what Upstream should do in 2009 and how each of us wants to get involved: How much we want to work and for how much, which technologies we want to learn, does it make more sense to attend conferences or to get involved in various open source projects? We collected our ideas and discussed some of them further.

Next to talking business, we wanted to get to know each other better, so everyone had also prepared a presentation about a non-technical topic of interest:

Alex introduced us to Sailing, "the most expensive way of travelling inconveniently".
Matthias talked about Photography, what else!
Katrin gave a talk about how to recycle old consumer items by turning them into furniture, showing examples from ecoble, WebUrbanist and Lockengeloet, Hamburg.
Thilo shared his Pen and Paper Roleplaying Game experiences by reading from his hero's diary.
Frank talked about Screamo, his favorite kind of Hardcore/Punk Music, with many sound snippets!
I presented Rhythms of Resistance, a network of political samba activists.

After all those fun sessions and walking around at the beach taking pictures of every grain of sand it wasn’t so easy to get back to work but we managed” As a first result we are today relaunching the upstream website presenting our new software development/consulting products: Lift Off, Scene Investigation and Turbo Boost. More on this in another blog post.

For further reading about our weekend see what Mathias wrote.

Seven Things

2009-01-22T00:00:00+00:00

Hey a new meme, gotta be part of it. We got tagged by Jan and Till about “seven things”

These are the rules:

Link your original tagger(s), and list these rules on your blog.
Share seven facts about yourself in the post” some random, some weird.
Tag seven people at the end of your post by leaving their names and the links to their blogs.
Let them know they have been tagged by leaving a comment on their blogs and/or Twitter.

So here we go:

[caption id=”attachment_512” align=”alignright” width=”290” caption=”Our old Logo”][/caption]

The name upstream was suggested by friend. After having spent months in search for a company name we just accepted it out of desperation. Wasn't such a bad choice was it?
When seeing our old logo most people told us it looked like a tennis ball (it was supposed to be a river) - so good we did an overhaul last year.
We never ever hire anyone permanently - everyone who works for us is a freelancer. We don't want anyone to be dependent on us and we don't want to be responsible for anyone.
we pair program all the fucking time (ppatft) .. mostly
If my GTD tool wouldn't remind me every week our office plants would have died long ago
We have hired programmers and scored gigs over twitter.
We can write PHP but don't tell anyone.

I’m tagging Mathias, Gregor, Martin, Marius, Pat, Kristina and Christian.

Upcoming Event: Screencast Week @ Cockpit (updated)

2009-01-15T00:00:00+00:00

We’ll be hosting another of our Cockpit Nights. This time instead of listening to talks we’ll be watching Screencasts. Wednesday Jan 21st is Erlang, Thursday 22nd is Objective-C day. More details in our public wiki.

update: Screencasts courtesy of The Pragmatic Programmers. We thank them very much for providing them. Check out their whole series of screencasts at http://pragmatic.tv.

Want to stay up to date on events? Follow @upstream_agile on Twitter or subscribe to the iCal Feed.

upstream goes to Maine

2009-01-07T00:00:00+00:00

Well, sort of. We are going to the north sea over the next weekend. Following our heroes at 37signals we will be spending two days at the countryside discussing the future of our little company. Of the 6 people going everyone will be doing 2 15 minute talks: one about upstream and his/her involvement, one about a random topic of interest. The goal of this trip is to find out what upstream should stand for and do in 2009 (and to take as many pictures as possible, cook and eat lots of yum food and relax at the fireplace). I'll blog about the results after the weekend, stay tuned :)

Speaking at Scotland on Rails

2009-01-07T00:00:00+00:00

Scotland on Rails will take place On March 26-28 2009 in Edinburgh. The organizers have just published the list of speakers. Pretty nice lineup including Jim Weirich, Yehuda Katz, Pat Maddox, Chad Fowler and more. And the best thing: I will be speaking about using CouchDB with Ruby/Rails, starring live coding and couch potato - my own little persistence layer for CouchDB.

Keep your exceptions inbox clean

2008-12-17T00:00:00+00:00

When Hoptoad came out we were really excited. No more hundreds or even thousands of emails in case of some server failure or gasp even a bug deployed to the production server. Instead a nice looking website with a clean list. Nice.

Within days we had switched all projects from exception_notifier to Hoptoad - and then that was it. Occasionally Hoptoad would send an email with an exception but most of the time it would be some crawler requesting an invalid URL so we read and forgot about it. The problem was that with Hoptoad to forget about those errors had become be really easy - too easy. So last week a client told me the signup form on our project was showing an error page and nobody had been able to sign up for days.

Wait. We are monitoring the app and get notified about every exception. Of course we did but over the months we had accumulated more than 50 exceptions in Hoptoad and nobody was really paying attention to that big pile of false alarms caused by some crawler anymore. So yes, we did miss that positive alarm.

As a consequence we spent the same day going through all our exceptions in all projects and killed them all.

A big part of the exception came from crawlers requesting non-existent actions and stuff like that. With Rails 2.2 this is relatively easy to fix. In our routes.rb we have declared most of our routes as restful resources:

  map.resources :posts

Now the PostsController does not implement the index action but we do get requests on /posts from time to time which result in an Unknown Action exception. The solution is to remove the index route:

  map.resources :posts, :except => :index

In other cases we had to improve our guards on parameters a little. Passing an empty string for the page parameter to a paginate call seems to be a favorite pastime for the google crawler.

So, lessons learned: keep your exception inbox clean. Even if it’s just a bunch of bots.

DevHouseBerlin FTW!

2008-12-10T00:00:00+00:00

The DevHouseBerlin weekend is over, but don’t worry if you missed this one. This first one was such a success that we will do more of them.

Thanks to all of those who came, helped, organized and shared their knowledge to make this such fun. I’m not lying when saying that everything turned out to be better than expected.

The upfront organization was minimal. Special thanks to Jan and Alex! We even got some unexpected PR beforehand thanks to an online article about the DevHouse. In response we bumped up the available slots to 20. Which were all filled up soon.

The location Box 119 which houses the offices of upstream, finnlabs and rocket rentals was an ideal location for such an event with its medium large office rooms and a bigger foyer. Box 119 had a solid infrastructure, the WiFi and Internet didn’t let us down once. We even had a decent coffee machine, major factor!

Saturday was packed. I guessed more than 30 people at once filled the DevHouse. Everybody was very open and eager to share. The presentation slots for the evening were filled up already by the early afternoon. I even lost my planned slot because of getting out of bed a bit later than planned.

Right from the beginning major hacking was going on. In the afternoon the talks started. In parallel there was tons of time to chat, eat and hack at will within a very relaxing atmosphere.

PHP raised in my opinion again, after I learned through Falko’s Meta Programming PHP talk that it also has a method_missing equivalent. Nico gave some helpful user experience insights by analyzing start pages on the projector. It’s amazing how many - in retrospective obvious things - you can do wrong on your start page. With Fabian we had a non technical discussion about Economy, after his talk about Fixing Money, which unfortunately left out the solution. After that talk I heard some creepy stories from The Enterprise at dinner.

At 2am at least a dozen people were still in the house. I left at 3:30am still full awake thanks to the optimal Club Mate supply.

Sunday wasn’t that busy. People dropped in later, I had plenty of time to get a presentation slot this time. ;) Some realtime programming battle was going on in the afternoon. You can find Gregor’s battle tank A.I. on Github. Filip showed some quick code kung fu by building a poll app with Django in his talk. This little python Framework is worth a look, if you need to get up small apps fast. My first talk on Bug Fighting with TDD went well. People seemed to be very interested in the subject. I had some time to play with JRuby and was able to get HtmlUnit to run with Webrat, although my solution is still sort of hackish.

CouchDB was all over the place of course with Jan around. He helped interested folks to get it up an running and helped out with problems related to CouchDB. Alex used the time to enhance his CouchDB persistence layer couch_potato. It will make the transition for rails people much easier.

People who stayed late this sunday - which excludes me - witnessed as the very first the rise of cloudplayer the most beautiful online music player in the world wide interweb, that Erik & Henrik gave the last touches at the DevHouse.

These where just some of the talks, projects and peoples that I get to know that weekend. There was so much more as you can see from the photos and the wiki.

Wow, it was just great. And next time will be even better. We will tell you more soon on this ;)

Introducing DevHouseBerlin

2008-11-13T00:00:00+00:00

We are very proud to announce DevHouseBerlin.

DevHouseBerlin is a fun-packed weekend of hacking and sharing knowledge. We open the Box119 office for a weekend and invite hackers of all sorts to join us working on projects, sharing code and ideas and just hanging out among fellow geeks.

DevHouseBerlin runs on December 6th & 7th, 2008 and it is a free event. You can sign up on the wiki.

We are highly influenced by the SuperHappyDevHouse:

SuperHappyDevHouse is a non-exclusive event intended for creative and curious people interested in technology. We’re about knowledge sharing, technology exploration, and ad-hoc collaboration. Come to have fun, build things, learn things, and meet new people. It’s called hacker culture, and we’re here to encourage it.

Find out more about DevHouseBerlin.

Cocoa for Ruby(ists) with MacRuby

2008-10-28T00:00:00+00:00

Apple’s Application Framework Cocoa gained popularity recently, also due to the iPhone. Like many others we started to learn Objective-C to play around with the iPhone a couple of months ago. The basic syntax is fairly easy to learn, also because lots of sources exist to get you up and running. But in my opinion the real challenge is to understand how the interface builder works

The past

But as a Rubyist the C like syntax of Objective-C was quite a pain. So I didn’t enjoy working with Objective-C. The existing Ruby bridge RubyCocoa has its own drawbacks. Especially moving parameter names into the method name to mimic Objective-C’s keyed arguments doesn’t appear future proof to me. This all kept me from getting deeper into the subject.

The future

In March this year MacRuby made its appearance as a successor for RubyCocoa. MacRuby is a port of Ruby 1.9 on top of the Objective-C runtime and garbage collector, which aims to fix all of the RubyCocoa drawbacks and bring the full power of Ruby to the Cocoa framework. With the 0.3 release a couple of weeks ago, MacRuby now supports the Interface Builder, thus making it a full fledged member of the OS X Development tool chain. Since then MacRuby gained serious popularity. An official Apple Developer Connection Tutorial for “Developing Cocoa Applications Using MacRuby” was published recently. More good tutorials can be found on the web, so I just link to them instead of writing another.

During the weekend I went through the first chapters of “Cocoa Programming for Mac OSX” with MacRuby instead of Objective-C to get familiar with the Cocoa library and Interface Builder. It went super smooth so far, I really had fun. Here is just a short example of the very first app out of the book to see how the code compares to objective-c.

class Foo
	attr_writer :text_field

	def seed(sender)
		@text_field.StringValue = "Generator doesn't need to be seeded ;)"
	end

	def generate(sender)
		@text_field.StringValue = (rand(100) + 1)
	end
end

^ Ruby vs. Objective-C v

@interface Foo : NSObject
{
  IBOutlet NSTextField *textField;
}

- (IBAction)gerneate:(id)sender;
- (IBAction)seed:(id)sender;
@end

@impelemtation Foo

- (IBAction)generate:(id)sender
{
  int generated = (random() % 100) +1;
  [textField setIntValue:generated];
}

- (IBAction)seed:(id)sender
{
  srandom(time(NULL));
  [textField setStringValue:@"Generator seeded"];
}
@end

As you can see you don’t need a header file and much less special characters. It even gets better, because you don’t need to call [[NSMutableArray alloc] init] but Array.new or just [] to initialize an empty Array. Isn’t Ruby beautiful? And with MacRuby you can have it for your Cocoa Apps too. So jump on board.

Couch Potato unleashed - a couchdb persistence layer in ruby (updated)

2008-10-27T00:00:00+00:00

Update: the gem is now available, see the installation instructions below.

After several weeks of incubating on my computer it’s finally time to get real: I have just open sourced Couch Potato under the MIT license. You can get Couch Potato on github now. For an introduction to CouchDB and ruby please read my previous blog post A CouchDB primer for an ActiveRecord mindset. The following is a very short introduction into using Couch Potato. If you want to know more you can start with the README.

The goal of Couch Potato is to create a migration path for users of ActiveRecord and other object relational mappers to port their applications to CouchDB. It therefore offers a basic set of the functionality provided by most ORMs and adds functionality unique to CouchDB on top.

Installation

Couch Potato is available as a gem from http://gems.github.com, so you can just do

$ sudo gem source --add http://gems.github.com # if you haven't alread

$ sudo gem install langalex-couch_potato

$ irb
>> require 'rubygems'
>> gem 'couch_potato'
>> require 'couch_potato'
>> CouchPotato::Config.database_name = 'name of the db'

Alternatively you can download the sources from github. If you are using rails just copy the files into vendor/plugins, create a RAILS_ROOT/config/couchdb.yml file (see the README for the format) and you are ready to go. For other applications you will have to require the lib/couch_potato.rb file and then set the database name by calling CouchPotato::Config.database_name = 'name of the db'.

As Couch Potato is still very young you can expect its feature set to grow quite a bit in the near future. What you can download now is the very core together with a few features giving you a glimpse of what is about to come:

Persistence

Create a new class and make its instances persistable by including the Persistence module. As there is no schema in a CouchDB you have to declare the properties you want to persist:

class User
  include CouchPotato::Persistence

  property :name
end

Now you can save your objects:

user = User.new :name => 'joe'
user.save # or save!

Properties:

user.name # => 'joe'

user.name = {:first => ['joe', 'joey'], :last => 'doe', :middle => 'J'} # you can set any ruby object that responds_to :to_json

user._id # => "02097f33a0046123f1ebc0ebb6937269"

user.created_at # => Fri Oct 24 19:05:54 +0200 2008

You can of course also retrieve your instance:

User.get "02097f33a0046123f1ebc0ebb6937269" # => < #User 0x3075>

Associations

As of now has_many and belongs_to are supported. By default the associated objects are stored in separate documents linked via foreign keys just like in relational databases.

class User
  has_many :addresses, :dependent => :destroy
end

class Address
  belongs_to :user
  property :street
end

user = User.new
user.addresses.build :street => 'potato way'
user.addresses.first # => < #Address 0x987>

user.addresses.create! # raises an exception as street is blank

user.addresses.first.user == user # => true

When saving an object all associated objects are automatically saved as well. All these save operations are sent to CouchDB in one operation which means the whole process is atomic across all objects saved, plus only one database roundtrip is required making it much faster.

As CouchDB can not only store flat structures you also store associations inline:

class User
  has_many :addresses, :stored => :inline
end

This will store the addresses of the user as an array within your CouchDB document.

Callbacks

Couch Potato supports the usual lifecycle callbacks known from ActiveRecord:

class User
  include CouchPotato::Persistence

  before_create :do_something_before_create
  after_update :do_something_else
end

Versioning

Couch Potato supports versioning your objects, very similar to the popular acts_as_versioned plugin for ActiveRecord. To use it include the module:

class Document
  include CouchPotato::Persistence
  include CouchPotato::Versioning
end

After that your object will have a version that gets incremented on each save.

doc = Document.create
doc.version # => 1

doc.save
doc.version # => 2

You can access the older versions via the versions method.

doc.versions.first.version # => 1

When passing a version number the version method will only return that version:

doc.versions(1).version # => 1

Ordered Lists

Couch Potato supports ordered lists for has_many relationships (with the :stored => :separately option only), very similar to the popular acts_as_list plugin for ActiveRecord. To use it include the module:

class PersistenArray
  include CouchPotato::Persistence
  has_many :items
end

class Item
  include CouchPotato::Ordering
  belongs_to :persistent_array
  set_ordering_scope :persistent_array_id
end

array = PersistenArray.new
item1 = array.items.create!
item1.position # => 1

item2 = array.items.create!
item2.position # => 2

You can move items up and down simply by changing the position:

item2.position = 1
item2.save!
item1.position # => 2

Conclusion

Couch Potato is very young and it’s open source, so if you find any bugs (you most definitely will) please go to the bug tracker and file a ticket. If you want to contribute please create a fork on github. If you have ideas for improvements please email me or comment on this post.

Barcamp Berlin 3: looking back

2008-10-20T00:00:00+00:00

Last weekend was Barcamp Berlin 3 and I was there, too. This year’s location was the Deutsche Telekom Haupstadtrepräsentanz - an awesome place.

Anyway, the purpose of this post is to summarize the sessions i attended and found worthy to write about. So here we go. Slides are mostly posted in the barcamp wiki.

How to promote your Web 2.0 Application

This first session was held by Michael Sliwinski of nozbe.com - a very energetic talk presenting a couple of tips and tricks to turn the visitors of your web tool into caring users and finally paying customers. Michael has built nozbe for himself and only later started turning into and app for others. His idea(l)s are heavily influenced by the 37signals guys - looks like we operate on the same level as he does :)

Back to his talk, some of his points were: show people how to use our site, show them the benefits hey will have from using your product (not your features, their benefits), tell them why to use this app and not the competition, testimonials are important even though nobody reads them, email still works - send newsletters and special offers - much higher impact than blog posts, people still don’t use or even understand RSS. And most importantly: love your app and show that love to others.

doingtext

We skipped the next session in order to ~~prepare~~ go through our own presentation titled “getting things written” one last time (ok seriously we had started working on it the day before). We ended up with around 70 slides for a 30 minute talk. I had added a presentation feature to doingtext the week before so we did the entire show straight from the website. We had decided to make this an entertainment show so we started explaining what paper was and how to use it and then applied the principles people use to work on texts with paper (show text to others, comment, highlight, talk about it, view history, merge changes) to the various online tools available (sending ms word documents by email, google docs, writewith, adobe buzzword), then text editos with collaboration features (gobby, subethaedit) and finally presenting doingtext and its communication focused approch to text collaboration.

It looks like the talk was good - we had around 60 people attending and staying until the end of a longer discussion following our presentation.

Today I repeated the same talk at the Webmontag Berlin. Having only 8 minutes of time I ended up rushing through the 70 slides and almost rapping the words in super high speed - earning quite a bit of praise - thanks everyone.

books for freaks

This wasn’t really a presentation but more a everyone talk about their favorite books thing. Here comes everybody (great book about how people gather on the internet, how communities work, how the economy shifts from limited production capabilities to limitless production and much more), but also fictional books. Charles Stross was mentioned as the author of a couple of weird (in the positive sense) sci-fi books (The Atrocity Archives) - will buy one of those soon.

jquery tips & tricks

The presentation itself was actually a bit lame (sorry guys) but it reminded e that I really wanted to try out jQuery. On the next project - promised. $('.item').show().siblings().hide() - pretty slick eh?

our favourite tv series

Very good session for leaning back and watching funny scenes from a couple of tv series. They even mentioned Top Gear - the funniest and most overproduced car magazine ever.

death to the ipod

This one was mostly about a pretty cool gadget called the pacemaker - basically a mobile dj tool that fits in your hand. Comes with 2 decks, crossfader, equalizer, effects, beatcounter, prelistening via a separate headphone out etc. - pretty cool, I even got to test it out for a couple of minutes.

How to get resource route methodes for custom templates in rails

2008-10-08T00:00:00+00:00

If you write a custom template handler, e.g. for pdf templates you might want to use the rails resource route methods e.g. new_users_path. Here is the way how to get this working for a imaginary PDF template handler.

class PdfRender

    attr_reader :controller
    #delegate the url_for call to the controller

    delegate :url_for, :to => :controller

    def initialize(action_view)
      @action_view = action_view
      @controller = action_view.controller
    end
    ... # more handler code

end
#install routes for the handler

ActionController::Routing::Routes.named_routes.install PdfRenderer

Product Launch: doingtext.com

2008-10-01T00:00:00+00:00

We have just launched the closed beta of a new product: doingtext.com.

Doingtext is a text collaboration tool. After entering your invite key you will be able to start a new discussion about a text that is already there or to be written. You can send the URL of this discussion to anyone you want (they don’t need an account or invite) and they can comment on every line of your text and suggest changes.

If you are working with texts please head over and get a beta invite and help us turn doingtext into a great product. We rely on your feedback.

We are posting regular updates on twitter and the doingtext blog.

A CouchDB primer for an ActiveRecord mindset

2008-09-25T00:00:00+00:00

That couch was our intro picture at the workshop.
(picture from gizmodo.com)

Monday was the first event at our new upstream office, starring @janl and me presenting an introduction to CouchDB - including a new hands on examples part - and afterwards an overview of what can be done with CouchDB in Ruby so far. The talks were followed by a discussion that gave (hopefully not only) me a couple of new insights I want to share here - after summarizing the evening for the people who couldn’t make it. There will also be a video recording with synchronized slides of both talks be available in the next days (thanks @klimpong).

What is CouchDB

CouchDB ist the new cool kid on the block. It’s a document oriented database that has replication built in, can scale massively and uses an HTTP REST interface to query it. Documents are stored as JSON constructs and can be queried with views that are built using Map-Reduce (a smaller company called google has had a bit of success with that recently). Oh and it’s written in Erlang. Jan has given a number of talks on numerous events already, so there are already a couple of <a href=”“http://mwrc2008.confreaks.com/10lehnardt.html>videos</a> and slides available - not from the hands on part though :) For that you should watch his blog I guess.

Help me get rid of SQL

So as promised in the title of this post I am now going to concentrate on the need of people like me: people who have been using some sort of SQL based database for ages and things like ActiveRecord for not so many ages but enough to wash their brain to think in relations and joins and columns and all that. Btw here are my slides.

My talk starts with a couple of random thoughts (that are more clear to me now after talking them through with some smart people). With relational databases we used to have tables, columns, rows, joins and SQL queries. We were mapping complex objects onto flat tables and since that doesn’t work so well used multiple tables and helper tables to make it all happen. With CouchDB pretty much all that is gone. Instead of rows in tables with a fixed schema we now have schema-less documents that can store pretty much any kind of data structure. Instead of SQL with its known power and problems we now have those map-reduce views that work completely differently. On the one hand that means more freedom on how to do stuff, on the other it means more questions on how actually do it. To make a long story short: the essential problem here is to get rid of our relational mindset. That applies for designing databases as well as for designing tools and libraries for working with CouchDB. (apparently a book called document design will help with this, can’t find it on amazon though) Which leads me to the next topic of my talk.

CouchDB in the Ruby world

I have picked three frameworks that are currently available as open source: CouchRest, ActiveCouch and RelaxDB.

CouchREST is a thin wrapper that provides a simple API for getting, putting, posting and deleting JSON documents to and from the CouchDB HTTP interface. I don’t actually have much more to say about it. It works, it doesn’t do a lot and that’s all great. Use it. No objections from my side.

ActiveCouch is - as the name suggests - inspired by ActiveRecord. It provides an ActiveCouch::Base class you can inherit from. After that you have to define the properties you want to store (you can’t obviously get those from the database schema anymore as with ActiveRecord) and then you can start creating, saving and finding objects. They have also integrated migrations that allow you to create (very simple) views using a ruby syntax. Without going too much into detail I think these guys haven’t come so very far in moving from the relational to the document mindset. CouchDB just doesn’t fit very well with migrations and records. Plus there is hardly any documentation to find about it at all. Sorry, not going to use it as it is now.

RelaxDB at first looks very similar to ActiveCouch. Inherit from base class, define properties, create, save, find by id. But: they don’t have migrations (one step futher away from SQL, one step closer to CouchDB). Instead they automatically generate a view when you call a find method for the first time. Sounds like a good idea to me. RelaxDB also supports basic associtations, so you can declare a has_many relationship between two classes and they get stored in separate documents. Definitely useful but still the same as ActiveRecord.

CouchPotatoe

CouchPotatoe now is my own take on a Ruby persistence layer for CouchDB. Since I only have been coding on anything CouchDB related for 3 days I am not claiming for it to be anything close to useful or better than any other framework. For now its main purpose is to help me learn Couch (as we long time CouchDBers say ;) ) and its principles. But that might change in the future, who knows…

So anyway, before letting you into the huge set of features let me stress this again: the problem to solve here is not to clone ActiveRecord to work with Couch (we don’t have records anymore, we have documents) but to come up with new concepts that help writing apps on top of documents.

So the first and currently only features (the CouchPotatoe::Persistence Module) doesn’t actually do much else than the other frameworks (but a bit :) ). It converts a Ruby object into JSON and stores it into CouchDB and loads and converts it back into a Ruby object. The difference between a record and a document is that a record consists of a flat list of attributes while a document can have a deeper structure, e.g. a tree, which can have lists or subtrees and all that. Potatoe (as we long time CouchPotatoe devs say) helps you …. (damnit, i’m just reading on wikipedia that “Potatoe” is actually considered incorrectly spelled and it should be “Potato” … oh well) … to build these more complex objects by providing a couple of helpers. The first is called property and lets you simply define an attribute of any value and type that then gets stored as is into Couch. The second is called has_many. Although that might sound familiar it’s not the same as the ActiveRecord version. With has_many you can chose to either store a list of sub objects inline into you document or as separate documents. In both cases everything is stored and retrieved automagically for you. So you can either do sort of join multiple documents like you would in a relational database or you can create more complex documents. I’m planning to add more of those helpers as time and ideas progress (trees, inline versioned attributes… if you have any ideas please tell me).

The second feature that deals with retrieving stuff. It’s only a concept for now so nothing is implemented yet. During the workshop the idea came up that instead of providing some magic finder methods like in ActiveRecord to call and generate views, those should be handled as first class objects. I think I have only scratched the surface of what you can do with Couch views but I have heard people who know what they are talking about use the word crazy to describe it. So basically that sounds just like an idea very well worth trying and I’ll implement something along the lines of mapping a Ruby class to a view and provide helpers to generate, update and query that view. Whatever it may look like.

As soon as those basics are implemented I will put the project up on my GitHub account and that hope to be able to raise a bit of interest. So be patient a couple of weeks maybe. So long, I’ll be posting updates here when I have more to tell.

Let the text fields grow

2008-09-18T00:00:00+00:00

I find it rather hard to choose the right size for text boxes. They are either to small so that you feel like caged in while writing into them or they are so big that you feel lost in that great white emptiness. These times are past since someone came up with auto expanding text areas (e.g. facebook uses them).

For one of our rails apps we use John R. Wulff’s ‘text_area_with_auto_resize’ plugin. The plugin will give you auto growing text boxes for all text_area calls. I wrote a little extension for prototype’s (1.6.x) in-place editing text area that makes these text areas auto growing as well. So we can have auto growing text areas all over the place. ;)

CouchDB-Workshop

2008-09-17T00:00:00+00:00

Am 22.9. 2008 gibt es einen Workshop zu CouchDB mit Jan Lehnardt (CouchDB maintainer) und mir. Nach einer kurzen Einführung wird der Fokus vor allem auf hands on experience liegen, d.h. es wird Beispiele und Code geben. Nach Jans CouchDB-Teil werde ich einen Überblick über die derzeitigen Ruby-Bibliotheken geben und wie damit der schnelle Einstieg zu schaffen ist.

“Leider” sind bereits alle Plätze ausgebucht, da wir nur begrenzt Stühle zur Verfügung haben. Wer trotzdem kommen will trägt sich am besten in die Warteliste ein und begnügt sich dann eventuell mit einem Platz auf dem Boden. Sicherlich wird das nicht der letzte Workshop bei uns sein. Mal sehen was uns demnächst noch so alles einfällt…

Rails Camp Denmark - Back to nature the ruby way.

2008-09-07T00:00:00+00:00

If you ever meet a bunch of people sitting with their laptops in front of a tent in the middle of nowhere you might have stumbled across a rails camp. The rails camp is a new unconference where ruby hackers meet to exchange ideas and have fun in an open and inspiring environment. And what could be more open and inspiring when being torn out of your everyday internet/office/city routine by camping. Being cut off from the internet will bring up new approaches and insights to overcome programming challenges as well as bring the members of the ruby community more in touch with each other. Of course it’s also a hell of fun when you’re off from your daily routine with a lot to laugh, drink and play.

That’s why we will attend the rails camp in Denmark to meet our fellow rubyists (eg. from Oslo or the last Euruko) and might code some crazy ideas. The Rails Camp Denmark will likely take place at the ~~end of October - the final date isn’t set yet -~~ 24th October just outside Svendborg on the Danish island of Funen. If you want to join the rails camp you should invest 75 Euro and be prepared for camping over a weekend. More details and discussions can be found in the google group.

Hold your thumbs that the temperatures at the End of October don’t drop below zero so we won’t have to keep our mac books running to heat our tent. ;)

cucumber - next generation rspec story runner (updated)

2008-08-26T00:00:00+00:00

yesterday while skimming through my Twitter timeline I stumbled across aslak hellesoy’s cucumber. cucumber is a rewrite of the rspec story runner, allowing you to write user stories in plain text and make them executable by adding a bunch of ruby magic.

the features of both - story runner and cucumber - are basically the same. cucumber just gets rid of all these little glitches that disturb your work flow when working with story runner and adds that little bit of polish that makes the difference. but see for yourself:

first of all it replaces those 5 lines of infrastructure code you needed for every story with a set of conventions. (require helper, require and run story text file, load additional steps files). stories are now called features and have a corresponding .feature file extension. the steps are located in *steps.rb files as before but don’t require any more steps_for or with_steps calls.

also improved is running stories: there’s a script called cucumber that can run a feature directly by calling cucumber my_feature.feature and there’s a rake task (rake features) to run all of them - say hello to my build server.

the wording has also changed. instead of "as a <role> I want to be able to <feature> so that <outcome>" we now have "in order to <outcome> a <role> should be able to <feature>. this might seem like a minor twist but I think it’s an important one. now you (your customer) have to think about which outcome you want even before you talk about features. if you can’t come up with a sensible outcome don’t even finish that sentence. hope that will end the "oh we still need a 'so that' clause for that feature" discussions.

adding before and after hooks has now become easier. instead of adding listeners you now just create a Before do .. end or After do .. end block.

another addition is the inclusion of a couple of webrat helpers. some people might argue that this something an application neutral framework like rspec (after all you can test anything with rspec, not just web apps) shouldn’t care about, but honestly, i think at least 80% of the people using rspec use it for web apps, so helping these people write better specs should be worth it. and webrat is definitely a big step in the right direction if you want to be ~~testing~~ specing webapps. if you don’t need it just delete the common_webrat.rb file. if you do use rspec for web testing take a look at that file. with this you can start test driving your web app without writing any ruby code by clicking links and submitting forms.

  When I go to "/"
  And I follow "New Entry"
  And I fill in "my first blog entry" for "Body"
  And I press "Submit"
  Then I should see "Blog entry was saved"

Simple as that. Your (first?) cucumber feature spec. If you want more source code take a closer look at the examples on github.

Last but not least the output has been enhanced. By default all successful steps are now in green, pending steps in yellow and failures in red - and the backtrace can be found next to that failed step instead of at the end as it used to be.

~~What’s still missing is a decent TextMate bundle:~~

~~Unfortunately I’m really busy these days so I probably won’t be the one making this feature shine green. Hopefully someone else will soon.~~

Update There is a TextMate bundle now an it’s working. It highlights your feature’s syntax and lets you run a single scenario from within TextMate. Great. Get it from GitHub while it’s hot. (thanks Paul)

Functional Testing awesomeness with Webrat

2008-08-09T00:00:00+00:00

I recently started using Webrat in one of our project’s test suites and fell in love with it immediately. Webrat is a functional testing library for Ruby on Rails which you can use in your rails integration tests (or better: the RSpec Story Runner) to let your tests walk through your application. The big difference between Webrat and a “standard” Rails Integration Test is that with Webrat you click on actual links and submit actual forms, instead of just sending requests to you application.

This closes an important gap to testing in a real browser (like with Selenium) and now gives us much more confidence in our tests - we now not only know that our models and controllers integrate, but we also know that they work with our views. That had been an unpleasant problem for a while. All our tests were passing but we still had way too many problems with invalid links and forms submiting to the wrong action. No more.

Webrat provides a sort of DSL (everything is a DSL nowaday isn’t it? :) ) for interacting with all the HTML elements you would find in a HTML document like links, checkboxes, radio buttons, text fields, selects and submit buttons. A test case might look like this:

visits '/'
clicks_link 'Sign up'
fills_in 'username', :with => 'joedoe'
selects 'Male'
checks 'Terms of Service'
clicks_button

Webrat now fetches the start page of the application, parses it and follows the first link labelled with “Sign Up”, gets the corresponding page, parses the form, fills in the values and submits it. Whenever a response comes back with anything else than a 200 (or 302, in which case that redirection is followed automatically) or an element on the page can’t be found, Webrat complains with an error and you know there’s probably something wrong in your view.

In Order to get this working with Story Runner I have wrapped most calls into When calls, for example:

When 'I click on $link' do |link|
  clicks_link link
end
When 'I submit the form' do
  clicks_button
end
When 'I select $option' do |option|
  selects option
end

And now my Story looks like this:

Scenario: successful signup
  When I go to the start page
  And I click on Sign up
  And I select Male
  And I check the Terms of Service
  And I submit the form
  Then the page should show Thank you for signing up

I’m not sure yet if it makes sense to release those wrappers as a plugin, we’ll see. After all it’s just a few lines of trivial code.

By the way, Webrat is hosted on awesome github, so after using it for 30 minutes (seriously) I forked my own version and added support for the Rails date helpers (selects_date Date.today, :from => 'signup_date' instead of selects 'December', :from => 'signup_date_2i'; selects '2008', :from => 'signup_date_1i'; selects '01', :from => 'signup_date_3i') and Emails (e.g. clicking the activation link in a signup email). (Bryan Helmkamp, if you read this: Why haven’t you pulled this (and in fact any other fork) into the main line yet?)

All in all the experience has been fantastic. Webrat is on all my Rails projects now. The only - conceptual - problem so far is that I can’t use it to test AJAX. But that’s the topic of another post. And there’s a fork on github that experiments with celerity, which can do AJAX and everything.

Warum ich selbständig geworden bin?

2008-07-23T00:00:00+00:00

Ich kann arbeiten wann ich will.

Ich kann arbeiten wo wo ich will.

Ich kann arbeiten mit wem ich will.

Ich kann arbeiten woran ich will.

Ich kann mir meine Kunden aussuchen.

Ich kann arbeiten womit ich will.

Ich kann arbeiten wie ich will.

Ich kann mir mein Büro aussuchen und einrichten wie ich will.

Ich kann mir aussuchen, wieviel Geld ich verdienen will.

Ich lerne jeden Tag mehr dazu, weil ich nicht nur Programmierer, sondern gleichzeitig noch Projektmanager, Geschäftsführer, Administrator, Sales-Fuzzi und sonstwas bin.

Um das mal zusammenzufassen: ich habe die freiheit zu tun was ich will. Und wenn ich irgendwann mal keine Lust mehr habe kann ich mich immer noch anstellen lassen. Oder nach Hawaii gehen. Darum.

(Hab ich noch was vergessen? Kommentare…)

Oslo we're coming

2008-07-18T00:00:00+00:00

So, finally everything is prep’ed: flights booked, accommodation confirmed, host office organized and cat fed. Tomorrow, very early in the morning, off we go to Oslo to live and work there for one week ;)

It’s just a short period of time but we’ll try to get the most out off it. We got in contact with the local ruby user group, starting off with a couple of beers on sunday and hopefully exchanging some great wisdom. Maybe we can also initiate a speaker exchange with the berlin ruby user group.

Thanks to shortcut who have been kind enough to let us stay at their office for the week, we are mighty excited to get a glimpse on their work culture. And in the evenings and weekends we will surely check out the vicinity of Oslo.

I’m sure this will be cool work holidays ;) If anyone in Oslo wants to meet us just send an email. We’ll also be posting some photos once we’re there.

RSpec Story Runner auf deutsch

2008-07-14T00:00:00+00:00

Wir sind gerade mit einem neuen Projekt gestartet - alles richtig “nach Lehrbuch”: kurze Iterationen, Iteration Planning zusammen mit dem Kunden, Behavior Driven Design, User Stories, automatisierte acceptance Tests - all die schönen Dinge die man in so einem modernen agilen Softwareprojekt haben will.

Während der Planung der ersten Iteration haben wir zusammen mit dem Kunden User Stories geschrieben, um sie anschließend direkt in den RSpec Story Runner zu werfen, also As a User I want to upload my photo So that everyone can see my smiley face. Da unsere Kunden deutsch sprechen war das Meeting und dementsprechend auch die Stories auf deutsch, also Als Benutzer will ich mein Foto hochladen können damit alle mein tolles Grinsegesicht sehen können. Damit der Story Runner damit umgehen konnte mussten wir ihm eine neue Sprache beibringen. So geht’s:

In diesem Ticket gibt’s einen Patch für den Story Runner, womit die Schlüsselwörter in den plain text stories konfiguriert werden können. (Wer wie ich mit Kommandozeilenpatchen nicht klar kommt, TextMate hat ein schönes Diff Bundle mit dem man den Patch auf einen RSpec (z.B. 1.1.4) checkout anwenden kann).

Die Konfiguration kann dann einfach in der stories/helper.rb landen:

Spec::Runner.configure do |config|
  config.with_story_template_words({
    :story          => "Story",
    :scenario       => 'Szenario',
    :given          => 'Gegeben',
    :given_scenario => 'Gegebenes Szenario',
    :when           => 'Wenn',
    :then           => 'Dann',
    :and            => 'Und'
  })
end

Was diesem Patch allerdings noch fehlt ist die Übersetzung der zu jeder plain text gehörenden steps. Man füge einfach noch folgendes Snippet mit ein:

module Spec
  module Story
    class StepGroup
      alias_method :Wenn, :When
      alias_method :Gegeben, :Given
      alias_method :Dann, :Then
    end
  end
end

Und schon kann man seine Stories auf deutsch schreiben (sorry das Beispiel ist irgendwie grauenhaft sinnlos, es ist schon spät):

Story: Als ich will ich ich sein damit ich mal so richtig ich sein kann
  Szenario: ich bin ich bin ich
    Gegben ich
    Wenn ich ich bin
    Dann soll ich auch ich sein

Und dann in den zugehörigen steps:

require File.join(File.dirname(__FILE__), '../helper')


with_steps_for :common do
  Gegeben 'ich' do
    @me = create_me
  end
  Wenn 'ich ich bin' do
    @me.be_me
  end
  Dann 'soll $expected auch $result sein' do |expected, result|
    expected.should == result
  end

  run File.dirname(__FILE__) + '/ich_story.txt', :type => RailsStory
end

Ach ja, one more thing: Um im Story Runner so eine Art setup/teardown zu haben, einfach noch ein wenig Code in den helper:

class MyStoryListener
  def scenario_started(story, scenario)
    # hier setup code

  end

  def method_missing(method, *args)
     # no-op

  end
end

Spec::Story::Runner.register_listener(MyStoryListener.new)

Moving from Rails 2.0.x to Rails 2.1 [Updated]

2008-06-04T00:00:00+00:00

We are in the process of moving our production code from autoki to the shiny new rails 2.1 and thanks to our unit tests/(r)specs we encountered some gotchas and wows we’d like to share.

Here are some things that change in rails you shold be aware of. If you are using custom callbacks for you rails observer you now have to tell rails about these callbacks first. Simply add a define_callbacks :your_callback, :your_other_callback in models with custom callbacks.

class GroupMembership &lt; ActiveRecord::Base

  belongs_to :member, :foreign_key => :user_id, :class_name => 'User'
  ...
  define_callback :after_confirm # required since 2.1

  ...
  def confirm!
    update_attribute :status, 'active'
    callback :after_confirm # custom callback

  end
end

class GroupMembershipObserver &lt; ActiveRecord::Observer
  def after_confirm(group_membership) # will be called from confirm! in the GroupMembership

    some code here
  end
end

The call to register custom template handlers in the enviroment has changed from ActionView::Base.register_template_handler to ActionView::Template.register_template_handler. With a custom handler you can output pdfs for example.

#register a tempalte handler in enviroment.rb or as an initializer

# pdf renderer

require 'pdf_render'
ActionView::Template.register_template_handler 'rpdf', ActionView::PDFRender

Consequently the ActionController::MissingTempalte Error was moved to ActionView::MissingTemplate.

With the new Caching in place the config call changed from fragment_cache_store to cache_store to reflect the new unified caching infrastructure.

If you have included the Observable mixin in one of your ActiveRecord models, like we did before we learned to harness the power of ActiveRecord::Observer, it’s time to abandon these because the changed method overwrites the new changed of ActivRecords dirty tracking. So if your stack trace look similar to this…

NoMethodError: undefined method `each' for true:TrueClass
method attributes_with_quotes in base.rb at line 2574
method update_without_lock in base.rb at line 2479
method update_without_dirty in optimistic.rb at line 70
...

.. it’s Observable overwriting the ActiveRecord changed method.

What also comes with dirty tracking is, that a save call does nothing unless someting really changed. Although this is the desired behavior, it might break logic where you rely on the old behavior, e.g. when you just want to update the time stamps of an object in the database. You can tell the object with attr_name_will_change! that an attribute should be written to the database.

class User &lt; ActiveRecord::Base
  def visit #should set updated at to current time if visited

     updated_at_will_change! #tell the user that updated_at field will change

     self.save
   end
end

Another ActiveRecord change we found is that :dependent => true isn’t supported anymore although I don’t remember a deprecation warning. Just use :dependent => :nullfiy|:delete|:destroy.

class User &lt; ActiveRecord::Base
 has_one :profile, :dependent => true #this will break, use :nullfiy|:delete|:destroy instead

end

But now let’s come to some nice stuff we discovered.

ActiveRecord now complains with a stack level to deep if you have circular dependency behaviors defined, eg. User has_one :profile, :dependent => :destroy and Profile belogs_to :user, :dependent => :destroy. Such logic can just be the result of isolated late night coding, so an error is the desired behavior in my opinion.

Let’s get back to the aforementioned changed method. Because it returns an array of changed field names, it is a good way to fire after save actions only if a specific field changed. For our autoki application it is a good way to fire some image processing task only if the auto picture changed.

class Auto &lt; ActiveRecord::Base
  after_save :build_new_image
  private
  def build_new_image
    regenerate_widget if changed.include?(:image) # only do this when the image was changed

  end
end
end

At the end comes the real nasty stuff. Rails 2.1 really wrecks havoc under the plugins we are using. As you might guess the most have to do with caching.

While the fragement_cache_test plugins error uninitialized constant ActionController::Caching::Fragments::MemoryStore can be fixed relatively easy by changing the base class for the TestStoreclass in frament_cache_test.rb to ActiveSupport::Cache::MemoryStore and the calls to reset have do be changed to clear the timed_fragment_cache plugin and the extended_fragment_cache are rendered useless until updated, which we don’t expect to happen, so we put together the local_cache_plugin as a resplacement for both.

Will_paginate requires an update from git, and it will play nice again. It resides on github now and can be installed with script/install git://github.com/mislav/will_paginate.git thanks to the install script git support.

The paginated search plugin for ferret will require an update too, because of the changes in will_paginate. But it’s still maintained so just grap the new code from the opensoul blog.

So that’s all for now. (Wow this post got big)

We aren’t finish yet so we will post more while we find other noteworthy changes.

Dealing with legacy databases - multiple models per table

2008-06-04T00:00:00+00:00

On a recent Rails project I had to use an old database schema that had been designed years ago for some php app. The two main entities in the application were companies and their projects. For whatever reason the designer had created one huge table called organizations with all the companies and projects inside. A row in the database marked a company when the is_company column was 1 and a project when the is_project column was 1.

|----------------------------------------------------|
| id | name                | is_company | is_project |
|----------------------------------------------------|
| 1  | google              | 1          | 0          |
| 2  | world domination    | 0          | 1          |
|----------------------------------------------------|

In my rails app I created two ActiveRecord models: Company and Project and set the table name for both to organizations:

class Company
  set_table_name 'organizations'
end

class Project
  set_table_name 'organizations'
end

Now the real problem was his: when I did a Company.find :all I got all the companies and all the projects, so I had to do this instead: Company.find :all, :conditions => {:is_company => true} - throughout the project, and whenever I forgot the condition somewhere all the projects showed up in the wrong place. So what I needed was a generic solution.

To make a longer story short I played with scope_out defining a projects scope on the Company model, so I could at least do Company.find_all_companies and didn’t have to specify the conditions over and over again - but that still wasn’t DRY at all. One morning short after waking up (the time I usually have the best ideas) it dawned on me how simple the solution actually is:

class Company
  def self.find(*params)
    with_scope(:find => {:conditions => {:is_company => true}}) do
      super
    end
  end

  def self.count(*params)
    with_scope(:find => {:conditions => {:is_company => true}}) do
      super
    end
  end
end

I added a scope around all find and count calls of my model and now I can just do whatever find operations I want and don’t have to think about scopes and tables ever again. Sweet object encapsulation.

New plugin: totally restful authorization

2008-06-03T00:00:00+00:00

We have again released a new plugin. Continuing the series of completely original names it’s called totally restful authorization.

The exec summary: you can declaratively add permissions to your (ActiveRecord) models for creating, viewing, updating and destroying them. A set of before filters automagically checks all incoming requests on your restful controllers for the permission and grants or denies access based on the permissions declared on the respective model.

How to install

Now with Rails 2.1 out all you have to do is script/plugin install git://github.com/langalex/totally-restful-authorization.git.

How to use

Include he PermissionCheck Module into the controllers you want to be checked or simply into the ApplicationController to secure your entire application.

class ApplicationController &lt; ActionController::Base
  include PermissionCheck

  ...
end

Second, declare permissions on your model using the built in domain specific language.

class User
  updatable_by :admin # updatable if updater.admin? return true

  updatable_by :self, :except => [:admin] # special role self, allow all attributes except some to be updated

  updatable_by :newbie, :only => [:description] # only allow some attribute to be updated


  viewable_by :anyone # special role, includes nil


  destroyable_by [:admin, :root] # declare multiple roles at once

end

That’s it. From now on all requests will be checked against your model permissions and be blocked if the authorization fails. For more details see the README and the unit tests. (Btw. if anyone has a good idea on how to replace the controller tests with RSpec specs, i.e. get controller specs working in a plugin please tell me)

MySql - Teil 1: Den richtigen Index finden

2008-06-03T00:00:00+00:00

Die Datenbak MySql ist bei Webanwendungen sehr verbreitet, unter anderem weil sie kostenlos, relativ schnell und etabliert ist. Auch bei uns verrichtet MySQL zuverlässig seinen Dienst. Grund genug mal unsere Erfahrungen in ein paar Blogeinträge zu gießen und damit das rumlavieren in der sehr umfangreichen offiziellen Dokumentation auf echte Härtefälle zu beschränken.

Zuerst schauen wir uns mal an welche Werkzeuge MySQL von Hause aus mitbringt, um beim Erstellen von sinnvollen Indizes zu helfen.

Mit EXPLAIN vor einem SELECT in der MySQL Shell lässt sich erst einmal herausfinden, wie einzelne Abfragen ausgeführt werden. So lassen sich gut Informationen über verwendete und fehlende Indizes sammeln. Das Ergebnis von EXPLAIN wird standardmäßig als Tabelle ausgegeben. Hier kurz was die Angaben bedeuten. Die Spalte table gibt den Namen der abgefragten Tabelle an. Unter type steht die Art des Lesezugriffs. Steht dort ALL wird ein Table Scan durchgeführt. Unter possible_keys stehen die Namen der verfügbaren Indizes. Die Spalte key enthält die bei der Abfrage verwendeten Index. Es kann nur ein Index verwendet werden. Unter extra stehen die Einschränkungen nach denen die Zeilen ausgewählt werden. Steht unter type ALL, aber in den Bedingungen ‘using where’ fehlt mit Sicherheit ein geeigneter Index. Ein Grund, warum selbst ein vorhandener Index nicht verwendet wird, ist, dass die WHERE-Bedingung zu weit gefasst ist. Unter diesen Umständen ist die Verwendung eines Indizes aufwendiger als ein Table Scan. Ein gutes Beispiel für eine zu weit gefasst Abfrage ist, alle Frauen aus einer Nutzertabelle zu laden, da etwa die Hälfte aller Nutzer Frauen sind, ist der Aufwand für den Einsatz von Indizes viel zu groß und wird daher von MySQL ausgeschlossen. Als Richtwert gilt, wenn 20% oder weniger Zeilen durch eine Bedingung erfasst werden ist der Einsatz eines Indexes sinnvoll. Oder anders herum gesagt, ein Index sollte eine gute Selektivität besitzen. Mit EXPLAIN tablename lässt sich herausfinden, welche Spalten einer Tabelle bereits mit einem Index versehen sind.

Aber nicht nur die Selektivität eines Index ist entscheidend, denn jeder Index kommt zu dem Preis, dass er bei einer Änderung der Tabelle mit geändert werden muss und so INSERTS, UPDATES und DELETES länger dauern. Also ist auch darauf zu achten wie viele Spalten der Index umfasst und wie lang die Schlüssel sind. Darum ist es gut ersteinmal zu ermitteln, welche Abfragen am meisten von einem Index profitieren würden. Um diese im Live-Betrieb zu ermitteln ist der Loggingmechanismus von MySQL ein praktisches Werkzeug. Das Logging lässt sich in der Konfigurationsdatei my.cnf (bei *nix Systemen in /etc/mysql/ zu finden) des Servers einrichten. Um Abfragen, die besonders lange dauern, mitzuloggen, muss dies mit log_slow_queries = /var/log/mysql/mysql-slow.log aktiviert werden. Mit dem Parameter long_query_time = 2 kann in Sekunden angegeben werden, ab welcher Ausführungsdauer eine Abfrage mitgelogged wird. Um Abfragen ins Log zu schreiben die keinen Index verwenden kann log-queries-not-using-indexes in die Konfigurationsdatein eingetragen werden. Vorsicht, diese Einstellung kann ordentlich viel Output erzeugen.

Dass waren jetzt die einfachen Bordmittel, die dabei helfen können, sinnvolle Indizes zu vergeben. Das Letzte Wort zu Indices ist damit aber noch nicht gesprochen, denn beim Locking, womit sich der nächste Post befasst, spielen sie auch eine wichtige Rolle.

New Rails Plugin for making ActionMailer asynchronous

2008-05-19T00:00:00+00:00

We have just released a new rails plugin called workling_mailer. This plugin provides a module which - when included into an ActionMailer subclass - pushes all emails that would normally be delivered synchronously into a queue for asynchronous processing. For queuing it relies on the workling plugin which can use for example twitter’s starling as a queue server.

We use this on autoki which sends a lot of email (thanks to our social feed plugin :) ).

Recently we got errors from the mailserver complaining about too many concurrent connections. This did not only cause emails not being delivered and our users getting an error page, it also caused long running transactions in the database: if you send an email within an after_create hook in a model this happens inside of the database transaction. That means that if you have a 60s timeout on your mail server you end up with database transactions taking 60s to finish (or being rolled back in this case) which essentially locks up your tables after a short period of time.

Now that we are sending our emails from a queue one at a time, the mailserver is happy and our database transactions are again as short as they should be.

You can get the sources at github.

New Rails Plugin: social feed

2008-04-25T00:00:00+00:00

This is our latest and also largest plugin for ruby on rails so far. After the installation it adds a social feed as seen in the picture to your rails application. The sources have been extracted from autoki which has had a social feed for a couple of months now, so rest assured we have put some thoughts into it over time. :)

Features so far: As a user I can decide what kinds of events I want to see on my social feed and also wether I want to be sent an email when an event occurs. I can also decide wether others will receive a notification on their social feed concerning my own actions.

The plugin includes model extensions for the user, a controller and views for viewing the feed and editing settings as well as a generator to easily create new event types so, getting started only takes a couple of minutes. For more info check out the README or get the sources from github.

Git commits in the blog sidebar

2008-04-25T00:00:00+00:00

As the careful reader might have noticed, we have added another RSS feed to the blog sidebar. Under the keiala product blog you can now see our commits to all our open source repositories on github. To make this possible I have used yahoo pipes for the first time. The pipe consumes the RSS feeds of all our open source projects on github, merges these into one, prefixes the titles with the project names and also truncates them, before wordpress consumes that pipe’s output and displays it in a sidebar widget. That’s what I call a real mashup. :D If anyone is interested, here’s the pipe.

Hiding attachment_fu from your controllers

2008-04-21T00:00:00+00:00

Yesterday I was using attachment_fu to attach photos to a couple of models. Since attachment_fu requires you to create an extra model for the photo, you usually have a one to one relationship between the model and the photo.

class Project
  has_one :project_photo
end

But now that you have two models you also have to deal with these in your controller and views, something along the lines of this:

class ProjectsController
  def new
    @project = Project.new
    @project.build_project_photo
  end

  def create
    @project = Project.new params[:project]
    @project.project_photo = @project.build_project_photo params[:project_photo]

  end
end

Sort of ugly if all you want is to add a photo to the Project model. But fear not, after adding the following piece of code to your model, you can transparently assign the photo to your project model in the view:

The tweaked model:

class Project
  has_one :project_photo, :dependent => :destroy # the attachment_fu model


  # a virtual attribute that saves a new photo in the photo model only when there's a new file uploaded

  def photo=(photo)
    self.project_photo = build_project_photo :uploaded_data => photo unless photo.to_s.blank?
  end
end

Now you can use a standard view:

< %- form_for(:project) do |f| -%>
  < %= f.file_field :photo %>
< %- end -%>

And a standard controller:

class ProjectsController
  def new
    @project = Project.new
  end

  def create
    @project = Project.new params[:project]
    @project.save
  end
end

(credits to this railscast for the idea of using a virtual attribute, I only added a bit of sugar for attachment_fu)

Rails reporting: dead simple reports now supports excel directly

2008-04-12T00:00:00+00:00

dead simple reports is a Ruby on Rails plugin that allows you to create reports from your application data within minutes. As of now it can not only generate HTML tables and CSV files but also M$ Excel spreadsheets. This is possible through the use of the spreadsheet-excel gem. You can grab a copy from the git repository or just download it from there.

Moving to GitHub

2008-04-11T00:00:00+00:00

GitHub is out of beta so now we have moved our open source projects there. Actually they already have been there for a while but we have now split them into their own repositories. The upstream subversion will still be online for a while but new commits will only go to git. Apart from being hyped alot these days git can do some pretty cool things like committing only parts of a file or changing previous commits - well worth checking it out I’d say.

Monitoring the internals of your Rails application with Nagios

2008-04-10T00:00:00+00:00

autoki has recently grown into a more and more complex application. Besides two clusters of mongrels and the mysql database we have a memcached server, ferret and starling plus clients for asynchronous processing. WIth so many services running (and sometimes not running) the need to monitor all these grew. We decided to set up nagios on one of the servers - it’s ugly but it lets you monitor all sorts of stuff pretty easily via remote agents that run on each monitored server.

With nagios we have access to quite a number of monitoring plugins, e.g. for monitoring TCP ports (e.g. for checking memcached is still alive), HTTP, server load, free disk space etc. Yesterday I came to a point where I wanted to monitor something nagios couldn’t: When a user uploads a bunch of photos, the task of creating copies of the photos in different sizes is put in a starling queue for asycnhronous processing. If something with that processing goes wrong and the queue gets too big I want nagios to pick this up and notify me. Time for my own nagios plugin.

Nagios plugins are actually very simple. All you have to provide is something that can be executed in a shell and that returns either 0, 1 or 2 for an OK, Warning or Critical state of the monitored service. So here’s the source code for monitoring the number of photo uploads in the queue (RAILS_ROOT/lib/check_photo_uploads.rb):

#!/usr/bin/env ruby


# load rails

RAILS_ENV = 'production'
require File.dirname(__FILE__) + '/../config/environment'

# print out a warning if the queue has 10, critical when 20 entries

warn_count = 10
fatal_count = 20
actual_count = PhotosUpload.count
error = 0

print "photo_uploads "

if actual_count < warn_count
  print "OK"
elsif actual_count >= fatal_count
  print "CRITICAL"
  error = 2
elsif actual_count >= warn_count
  print 'WARNING'
  error = 1
end

puts " #{actual_count} uploads in queue"
exit error # exit with the error code that is then interpreted by nagios

At autoki the server that runs the asynchronous processes is called jobs1 and this is also where the queue should be checked, so I added this to the /etc/nagios/nrpe.cfg file (the config file for the remote nagios agent):

command[check_photo_uploads]=/usr/bin/ruby /var/www/production/current/lib/check_photo_uploads.rb

Then I had to add the service to the nagios configuration on the monitoring server:

define service {
        host jobs1
        service_description photo_uploads
        check_command check_nrpe_1arg!check_photo_uploads
        use generic-service
}

Well, that was it, and this is how it looks - ugly but it works :)

(I recently signup with scout - looks much prettier and the setup is much easier than nagios, plugins are written as ruby classes and it comes as a ruby gem - sweet concept so far, could have been my idea, more on that later)

4-Tage-Woche und Kreditkarten für alle

2008-03-10T00:00:00+00:00

Das klingt so gut, das machen wir auch: 37signals haben beschlossen, im Sommer nur 4 Tage die Woche zu arbeiten - nach 3 Tagen Wochenende ist man einfach entspannter, glücklicher ergo produktiver. Kreditkarten für alle und finanzielle Beteiligung an den Hobbys der Mitarbeiter - klingt auch nicht schlecht, aber so richtige Mitarbeiter haben wir ja gar nicht, da müssen wir uns was anderes überlegen.

upstream auf github

2008-03-04T00:00:00+00:00

Nach kurzer Wartezeit habe ich heute meine Einladung zum Betatest von GitHub bekommen und das erst gestern eröffnete public upstream subversion ins public upstream git repository kopiert. Man kann unser großartiges dead simple reports plugin jetzt also auch über git ziehen.

Der Einstieg in github war schonmal sehr angenehm. Um das erste Repository anzulegen bekam ich erstmal die benötigten Kommandos angezeigt und konnte die gleich in mein Terminal kopieren. Von den diversen git tutorials und screencasts ist bei mir leider nicht viel hängen geblieben, sodass ich git wohl erst im Laufe des Praxiseinsatzes richtig kapieren werden. Ein sehr spannender Ansatz bei github ist auf jeden Fall die Möglichkeit, bestehende Repositories zu forken und dann darin zu entwickeln. Statt also irgendwelche Patches an Open Source-Projekte zu senden, forkt man einfach nur noch deren Repository, werkelt darauf herum und kann dann ggf. die Änderungen in das Main-Repository hochladen.

upstream goes open source: dead simple reports

2008-03-03T00:00:00+00:00

Na endlich. Nach all den Jahren des nur-Geldverdienens und Open Source-Ausnutzens haben wir es geschafft, ein paar erste Zeilen Code in die Freiheit zu entlassen: dead simple reports, ein Rails-Plugin, das Reports generieren kann. Mehr dazu im neuen Bereich Open Source

Delete considered harmful

2008-02-28T00:00:00+00:00

We just encountered a bug in one of our rails applications where we had accidentally used some_mode.delete instead of some_model.destroy. I think delete should not be a public method on ActiveRecord. After all it’s just some evil means to circumvent all those rails callbacks and stuff to gain some more performance when deleting objects. It should be a protected method or something so that requires some effort (i.e. model.send(:delete)) to call it so you don’t use it accidentally when you actually want to call destroy.

TextMate Snippet Power für gettext

2008-02-17T00:00:00+00:00

Es ist super wenn die Werkzeuge, die man verwendet mit einem wachsen. Deshalb bin ich von Ruby und TextMate begeistert.

Für die, die TextMate nicht kennen: TextMate ist ein TextEditor für OS X, der sich in meinen Augen durch zwei Besonderheiten als Entwicklungswerkzeug hervortut. Erstens die leistungsfähige Live-Suche, die alle Mausklicks zum Navigieren im Projekt überflüssig macht. Zweitens die wirklich mächtigen Skriptingmöglichkeiten, die ich wohl noch lange nicht erfasst habe.

Jedenfalls musste ich Rahmen der Internationalisierung eines Projekts alle Strings in eine Übersetzungsmethode von gettext wrappen. Also ggf. Texte in den Views in Inline tags und in doppelte Anführungszeichen zu packen. Das gewünschte Ergebnis sieht etwa so aus: <%= _("Zu übersetzender Text") %>. Die ganze Arbeit ist ziemlich stupide, ließ sich aber mit TextMate auf simples Markieren des entsprechenden Textes und das Drücken des Tastenkürzels ctrl+alt+t reduzieren.

Dazu hab ich im Rails Bundle ein neues Snippet erstellt, das den markierten Text entsprechend einfasst. Der Code dafür sieht so aus <%= _("${TM_SELECTED_TEXT}") %>. Die Variable TM_SELECTED_TEXT steht einfach für den markierten Text. So weit so trivial.

Da manchmal im Text bereits doppelte Anführungszeichen vorhanden waren, haben diese leider die Strings etwas zu früh beendet. Hier kommen die regulären Ausdrücke ins Spiel um diese Anführungszeichen zu escapen. Reguläre Ausdrücke lassen sich auf die Variable wie folgt anwenden ${Variable/regexp/format/option}. Mein Snippet Code sah dann so aus <%= _("${TM_SELECTED_TEXT/"/\\"/g}") %>. Damit war das Anführungszeichenproblem kurz und schmerzlos erledigt.

Blieb noch der Fall, dass der Sting bereits als Inline Code eingefaßt ist und nur _() um den String noch nötig ist. Dazu habe ich einfach ein weiteres Snippet angelegt, das so aussah: _("${TM_SELECTED_TEXT/"/\\"/g}"). Da ich mir aber nicht ein extra Tastenkürzel dafür merken wollte, habe ich mir die Scoping Eigenschaft von Snippets zu Nutze gemacht. Scoping in TextMate bedeutet, dass das Snippet nur in einem bestimmten Kontext angewendet wird. In welchem Kontext sich der Cursor gerade befindet, ist durch das Tastenkürzel shift+ctrl+p zu sehen. Der Kontext kann auch aus einer Hierarchie bestehen. Also dem ersten Snippet den Kontext text.html.ruby für rails html templates zugewiesen und dem zweiten den string.quoted.single.ruby|string.quoted.double.ruby (Verkettungen sind auch möglich) und schon stehen beide Snippets bequem per ctrl+alt+t zur Verfügung. Cool ;)

EURUKO 2008

2008-02-13T00:00:00+00:00

Die Konferenzsaison fängt an. Kurz vor der re-publica geht’s am 28./29.3. nach Prag zur European Ruby Conference. Letztes Jahr in Wien war ziemlich lustig. Damals hatten wir in der Nacht zwischen den zwei Konferenztagen ein Twitter-basiertes Rätselspiel (twizzer) geschrieben.

Falls jemand auch da hinwill, wir hätten evtl. noch Plätze im Auto frei.

re-publica die zweite

2008-02-08T00:00:00+00:00

An Alle: Vom 2. 4. - 4. 4. 2008 findet in der Berliner Kalkscheune wieder die re-publica statt. DIE Konferenz über die Blogosphäre, Politik, Kultur, Technik und alles was sonst noch so dazugehört. Hinkommen. Wir sind jedenfalls auch da.

Somewhat safer passwords for all your 95 social networks and web x.0 services

2008-01-24T00:00:00+00:00

The general advice: We are all supposed to use a separate password that has at least 25 letters, numbers and special characters in it for everything we use on the net, because if we don’t, one compromised site gives an attacker access to all your other accounts using the same password. Think of a hacked twitter account resulting in all your emails on gmail made publicly available.

The reality: We are all lazy bastards so most uf us have one single password they use for all their accounts (am I not right?). Some of us have 2-3 passwords for different levels of trust, so the bank account and credit card sites get one, personal email gets one and all the other sites get another. A bit better already.

My suggestion: One password per site (hey I’ve heard this before). But here’s the trick (and the catch at the same time, because the passwords are similar and can hence be hacked more easily). I will use the following convention to generate a separate password that is (for me) easy to remember for each site:

Take some random sentence:

This is my personal very secure password for [...].

Say you need a password for twitter, this sentence becomes:

This is my personal very secure password for twitter.

The password will be the first letter of each word in the sentence, so for this example it’s Timpvspftw. I’m actually using the first two letters of the site’s name so the convention works for twitter and t*** (uhm. insert name of another website starting with t). To make things a bit more secure, you should change your scheme, e.g. use the last letters, or alternating between last and first.

Using more or less random letters from a sentence to generate a secure password is nothing new. It actually has been recommended for years (decades?). My only addition to this is to use the name of the service in that sentence, so you can have separate passwords and still remember them easily. And they should be fairly secure, as long as your scheme of choosing the letters and your sentence are random enough (I’m still using something different for my bank account though).

Of course the whole would be much easier with somethingh like OpenID everywhere, but until then go and make up some funny sentences for your passwords.

Pregenerating assets in Rails 2.0 revisited

2008-01-24T00:00:00+00:00

In a previous post I wrote about how we wanted to pregenerate our packed assets simply by firing up a mongrel on deployemnt and request a page so it would generate the assets by calling the stylesheet_link_tag ... :cache => true and javascript_include_tag ... :cache => true methods in our application.rhtml.

It turned out starting and stopping a mongrel during development wasn’t really a good idea. We ran into all sorts of problems with deployemnts not shutting down the mongrel or not deleting its pid file which made the next deployemnt fails and stuff like that. Plus, to be honest, this really felt like a hack from the beginning.

So what we ended up doing was this: Write a small class that calls the methods to generate the all.js/all.css files and simply call that from our capistrano script. This is how it looks (config/assets.rb):

module Assets
  def stylesheet_filenames
    ['screen', 'screen_profile', 'game', 'screen_classes', 'screen_forms', 'screen_pages', 'redbox']
  end

  def javascript_filenames
    ['prototype', 'effects', 'controls', 'upload_progress', 'prototype_extensions', 'application',
      'redbox', 'dragdrop']
  end
end

The Assets module simply contains our list of stylesheet and css files which we had to pull out of application.rhtml so that we could generate the files without making any requests through the rails stack in order to call the methods in application.rhtml. After including the Assets module into ApplicationHelper the calls in the layout now look like this:

<%= stylesheet_link_tag *(stylesheet_filenames + [{:cache => true}]) %>
<%= javascript_include_tag *(javascript_filenames + [{:cache => true}]) %>

A bit ugly but it works. We have also added a class AssetsGenerator to our assets.rb file:

class AssetsGenerator
  include Assets
  include ERB::Util
  include ActionView::Helpers::TagHelper
  include ActionView::Helpers::AssetTagHelper


  def generate_assets
    stylesheet_link_tag(*(stylesheet_filenames + [:cache => true]))
    javascript_include_tag(*(javascript_filenames + [:cache => true]))
    path = File.join(RAILS_ROOT, 'public', 'stylesheets', 'all.css')
    css = File.read path
    File.open path, 'w' do |f|
      f &lt;&lt; css.gsub(/url\(("?|'?)\/images\//) do |matches|
        "url(#{$1}#{compute_asset_host(rand(1000).to_s).to_s}/images/"
      end
    end
  end
end

This does two things: first it calls the stylesheet_link_tag and javascript_include_tag methods from the AssetTagHelper module to generate the all.js and all.css files. After that it reads in the css file and replaces all ocurrences of url('images/*') with the url of an asset host - it’s easier than teaching our CSS guy to do it by hand, plus we don’t have any hard coded urls in our css files.

We’ve been running this setup for 2 weeks now withour any problems. You could argue that it might have been easier to stick with the old asset_packager plugin but with this solution we have now one less plugin that can cause trouble and we are re-using more of the built in rails stuff than before.

Using haproxy with multiple backends a.k.a. content switching

2008-01-09T00:00:00+00:00

We have been using haproxy as a load balancer for autoki (rails) for a while. The setup was relatively simple. We had two servers full of mongrels and one haproxy was simply throwing requests at them using round robin.

Now our requirements have gotten bigger. We have extracted the game into a separate application that is being deployed on its own server independently. The problem that we want to solve with this is that while we want to deploy changes to our servers whenever we want to, the game application has to keep running all the time without interruptions by a deployment and the problems that might be caused by it. Oh, and at the same time we didn’t want to change any urls. The game has been sitting in the external_games controller, so the url was autoki.com/external_games and since we have given this url to people we wanted it to stay the same.

So now we have a new server called external_games.autoki.com with 10 mongrels running on it and we want all requests to /external_games go there and everything else still go to the old servers. The solution to this is to let the load balancer decide which servers to forward the requests to. Since version 1.3 haproxy has implemented exactly this - and calls it content switching. To get thsi to work you have to tweak the haproxy.cfg file a bit. This is the old listen section with one pool of mongrels:

listen rails *:80
  server rails-1 app1.autoki.com:7000 maxconn 1 check
  server rails-2 app1.autoki.com:7001 maxconn 1 check
  ...

First we have split up our single proxy configuration into a frontend and a backend. We will actually be using two backends - one for gaming and one for the rest, and the frontend will decide which backend to use.

backend rails_game
 server game-1 app2.autoki.com:7000 maxconn 1 check
 server game-2 app2.autoki.com:7001 maxconn 1 check

backend rails_production
  server rails-1 app1.autoki.com:7000 maxconn 1 check
  server rails-2 app1.autoki.com:7001 maxconn 1 check

frontend rails *:80

The rule to decide which backend to use is implemented using an acl. The following line creates a rule with the name game that evaluates to tue, if the url contains the string external_games. The haproxy documentation has a whole list of keywords you can use to match against all the contents of a http packet.

acl game url_sub external_games

Now all we have to do is tell the frontend when to use which backend. We do this by defining a rule for the gaming backend and set a default backend for all requests not matching our acl.

frontend rails *:80
  acl game url_sub external_games
  use_backend rails_game if game
  default_backend rails_production

That’s it. Now when we go to http://autoki.com/haproxy?stats we’ll see the two backends up and running. (only after adding stats enable to both backends actually)

Letzte Worte

2008-01-07T00:00:00+00:00

surrrrr….. RRRRRR ….. zzzzzZZZZZ …… kreisssssssch zzzzZZZZZ &â‚¬&@@&&& stille plop “Neiiiiiinnnn”

so klang es vorhin, als der eine Lüfter meines Laptops seine letzten Umdrehungen nahm. Jetzt ist Ruhe. R.I.P. Warten auf den 15. Januar - dann ist Macworld San Francisco und Steve beschenkt die Welt mit neuer Hardware. Bis dahin bleibt mir nur, Bücher zu lesen.

24C3: AnonAccess

2007-12-28T00:00:00+00:00

Gestern Abend habe ich als letztes den Vortrag zum anonymen Zugangskontrollsystem AnonAccess gehört. AnonAccess ist eine Eigenentwicklung die aus dem Bedürfnis entstand ein günstiges und vor allem bei Bedarf anonymes Zugangskontrollsystem für den eigenen Hackerspace zu haben.

Das Ziel eines günstigen Zugangskontrollsystem konnte recht einfach durch entsprechende Hardware realisiert werden. Die Hardware besteht grob aus Chipkarten - ca. 73cent pro Stück -, Mikrocontroller - 10 Euro - und kleinere Komponenten deren Kosten auch bei 10 Euro lagen.

Dem Ziel eines anonymen Systems wurde sich langsam angenähert. Als erste Idee wurden Pseudonyme auf den Karten hinterlegt und Zugangstoken von einer Datenbank vergeben. Pseudonyme sind allerdings nicht wirklich anonym. Um richtige Anonymität zu erreichen wurde die Idee so abgeändert das satt eines Pseudonym, eine zufällige ID dem der Token zugeordnet wird und diese ID auch immer mit auf der Karte geschrieben wird. Damit ist kein Rückschluss mehr möglich welche Karte welche ID hat und somit auch keine Information mehr über dessen Besitzer. Das Modell ist zwar anonym aber absolut nicht wartbar, etwa wenn jemand seine Karte verloren hat, die entsprechenden Token zurückzuziehen.

Erst die Kombination der beiden Konzepte ermöglicht einen anonymen Zugang der sich auch verwalten lässt. Dazu wird eine Zweite Datenbank verwendet die Informationen über die ID der Karten und den hinterlegten Nutzerinformationen enthält. Dies ermöglicht eine dreistufiges Identitätsmanagement. Namentlich bekannt Nutzer, ein Pseudonym und geteilte Pseudonyme. Damit die Daten nicht einfach durch den Angriff auf eines der Teilsysteme kompromittiert werden können, etwa durch auslesen der Karte, Mithören der Kommunikation oder auslesen der Datenbanken kommen entsprechende Hashing- und Verschlüsselungstechniken zum Einsatz. Als Authentifizierungmöglichkeit für den Nutzer wurde vor kurzem die Verwendung einer PINs implementiert. Das System befindet sich immer noch im Entwicklungsstadium. Wie die Verschlüsselung im einzelnen Abläuft und wie die Datenbank aussieht kann in den Slides nachgelesen werden.

In der anschliessenden Diskussion wurden einige Kritik hinsichtlich der Architektur geäussert. Unteranderem wurde die Betreibssicherheit hinterfragt, da die Karte bei jeder Verwendung mit neuen Daten beschreiben wird und ein vorzeitiges herausziehen der Karte, diese unbrauchbar macht. Ein Problem dem durch den Einsatz von Einzugslesegeräten aus dem Weg gegangen werden kann. Grundlegender waren die Bedenken, dass das System nicht inhernt anonym ist, sondern auf das System vertraut werden muss.

24c3: Terror und Bücher

2007-12-28T00:00:00+00:00

What is terrorism?

Meine letzte und gleichzeitig beeindruckendste Session des ersten Tages hatte den Titel What is Terrorism?. Dabei berichtete Anne Roth von ihrem Leben als Freundin eines Terrorismusverdächtigen. Ihr Freund wurde eines Tages ohne Vorwarnung von der Polizei wegen Terrorverdacht festgenommen. Seitdem und auch schon davor stehen die beiden plus Freunde und Verwandte unter ständiger Überwachung durch die Polizei. Der Vortrag war voller irgendwie schon lustiger Anekdoten (“Ich habe meine Mutter angerufen, um den die Leitung abhörenden Polizisten zu sagen, dass ich jetzt gern fernsehen würde, die Wanze aber den Empfang stört”), zum Anderen kamen aber auch Details zutage, die einen an den Fähigkeiten der deutschen Polizei zweifeln lassen. Das Gute ist, dass das Ganze inzwischen in Annes Blog verfügbar ist. Sehr lesenswert.

Elektronische Dokumente und die Zukunft des Lesens

Der 2. Tag begann mit einer Session auf deutsch. Steini berichtete über die neuesten Entwicklungen im Bereich der e-Books. Klassische Displays hatten bisher vor allem das Problem, zum Einen in sehr heller Umgebung (Sonnenlicht, ahhh) schlecht lesbar zu sein, zum Anderen meist nach ein paar Stunden ihr batteriebetriebenes Leben auszuhauchen, bis ihnen an einer Steckdose wieder neues eingehaucht wurde. Seit einigen Jahren versuchen nun die Hersteller, durch sogenannte eInk Displays diese Probleme zu lösen, indem diese statt selbst zu leuchten das Umgebungslicht reflektieren und bei ausbleibender Energieversorgung ihren Inhalt nicht verlieren. Die Probleme dieser Displays sind zur Zeit noch eine recht lange Umschaltzeit (teilweise 500ms) sowie eine kurze Lebensdauer (ein paar tausend Umschaltzyklen). Diese Probleme scheinen in den Labors aber langsam gelöst zu werden, sodass in 1-2 Jahren mit tatsächlich brauchbaren eReadern zu rechnen ist.

Neben den technischen Neuerungen gab es Hinweise auf mögliche wirtschaftloche und gesellschaftiche Veränderungen. Dem Buchmarkt könnte mit einem Durchbruch von eReadern etwas ähnliches wie der Musikindustrie blühen. Die Verlage verlieren an Macht, da die Buchverkäufe zurückgehen, Autoren werden entlassen, Buchläden gehen pleite. Eine mögliche Entwicklung wäre, dass die nun ~~arbeitslosen~~ wieder Zeit habenden Autoren in Eigenregie ebooks herausgeben könnten. Wie in der Musik würden die Verwertungsketten wesentlich kürzer werden (Autor -> Leser statt Auto -> Verlag -> Händler -> Leser), und damit die Preise fallen könnten. Ein weiterer interessanter Hinweis: zur Zeit muss ein Sachbuch min. 300 Seiten dick sein, damit es von einem Verlag teuer verkauft werden kann. Dies zwingt die Autoren mitunter dazu, ein Buch mit eigentlich 50 Seiten Inhalt auf die geforderte Größe aufzublähen. Durch ebooks und die entsprechend geringeren Kosten wäre es auch möglich, 50seitige “Mini-Bücher” herauszubringen. Da würde man endlich mal ein paar mehr davon schaffen.

Durch die verkürzten Produktionssyklen (kein Druck, Distribution in Läden usw.) wird es weiterhin denkbar, Bücher in kleineren Teilen herauszugeben. Als Beispiel wurden Fortsetzungsromane genannt, von denen jede Woche ein weiteres Kapitel erscheinen würde, das sich ohne Probleme on demand an die Leserschaft verteilen ließe.

Zum Schluß gab’s noch den obligatorischen Hinweis auf die “illustrierte Fibel für die junge Dame” aus Diamond Age. In diesem Roman spielt ein vernetztes, interaktives eBook eine große Rolle, das ein kleines Mädchen bei seiner Entwicklung begleitet und ihm eine Art elektronische “Kindersitter” ist.

24c3: Design patterns for running a hacker space

2007-12-27T00:00:00+00:00

Von heute an findet in Berlin am Alexanderplatz der 24. Chaos Communication Congress (24c3) statt. Kurz gesagt handelt es sich bei dem ganzen um eine jährlich stattfindende Hackerkonferenz, auf der es neben der eigentlichen Technik und deren hacken um damit im Zusammenhang stehende gesellschaftlcihe und politische Themen wie z.B. die Vorratsdatenspeicherung geht.

In meiner ersten Session ging es um den Eigenbau von Flugdrohnen. Im wesentlichen wurden Moellflugzeuge um Rechner, Sensoren und Kameras erweitert und das ganze mit einer Software versehen. Schon konnten wir im Sall live verfolgen, wie auf einem Acker irgendwo in Frankreich eine Drohne startete und nach unseren Anweisungen Schleifen flog und uns dabei Videobilder der Umgebung lieferte. Wirklich sehr nett das ganze - mir ist nur noch nicts eingefallen, was ich damit überwachen wollen könnte. Vielleicht die Staße vor meinem Haus, um zu entscheiden, ob ich fürs Fahrradfahren Regensachen anziehen muss oder nicht? Hm. ;)

Die zweite Session hieß “Design patterns for running a hacker space”. Das Thema Orte aufbauen ist für mich vor allem im Zusammenhang mit dem 9to5-Projekt (kein upstream-Projekt) interessant, bei dem es um den Aufbau von Orten geht, in denen (vor allem) Selbständige in Zukunft ihrer Abeit nachgehen können. Mehr dazu später in diesem Blog bzw. vorerst im 9to5-Wiki.

Eine weitere Session, die für 9to5 interessant sein dürfte findet heute abend statt: 22 Uhr, AnonAccess - Ein anonymes Zugangskontrollsystem. Wie der Name verät, geht es dabei im ein System, um Mitgliedern zu einem Ort Zugang zu gewähren, ohne mechanische Schlüssel zu verteilen gleichzeitig aber den anonymen Zugang zu gewährleisten, d.h. das System weiß nicht, wer wann gekommen und gegangen ist. Ich werde leider nicht da sein können. Falls also jemand hingeht und mitschreibt, wäre ich für einen Link dankbar.

Im Folgenden jedenfalls meine Notizen zu besagter Session, eine Liste der vorgestellten Patterns, die sich mit dem Aufbau und Betreiben eines Ortes für Hacker befassen. Der erste Punkt beschreibt jeweils das Problem, die nachfolgenden die vorgeschlagene Lösung. Vorbemerkung: Patterns sind keine Anleitung, über ihre Anwendung sollte jeweils am konkreten Fall entschieden werden. (die interessantesten Dinge hab ich mal fett gemacht)

sustainability patterns

infrastructue pattern

first infrastructure or projects?
first infrastructure
infrastructure-driven - ideas come out of infrastructure, even stuff you didn't think of

grace hopper pattern

is this the right time? have you thought about all the problems?
sure it is time
it's alwasy easier to ask for forgiveness than it is to get permision
when in doubt, do it
problems will be solved (don't think about it now)

community pattern

how should your group communicate
3 problems: realt time communictation, discussion, documentation
mailing list, wiki , irc (ich würde da noch Sachen wie IM (in meiner Umgebung z.Z. meist Skype), evtl. twitter, Blog hinzufügen)

critical mass pattern

how many people?
2 + 2: one partner to get kicked of, 2 more to get the work done
aim for 10 for a start

strong persomalities pattern

nothing gets done
need people who have authority, strong personalities, not people who use authority and get laughed at
need people with experience with building up spaces

independence patterns

landlord and neighborhood pattern

weird landloard, picky neighbors
talk to neighbors, need cool neighbors, they might call the cops too often
you don't live a majority livestyle - better if your neighbors don't either

roommate anti-pattern

guests are fine but don't have anyone live in your space

separee pattern

want to chill/work in small groups, but main room is occupied, want to smoke without disturbing anyone
smaller, separate rooms instead of one big
use curtains, doors
smokers room

kitchen pattern

you need food and caffeine at odd times
have a kitchen (cooking together - socializing)
have fridges
sell soft drink - income (most spaces 50% of income)
have a dishwasher (!)
freezer for pizza etc.

coziness pattern

bring in couches (also for sleeping), sofas, comfortable chairs, pillows, tables, ambient light, stereo, projector, videop game consoles
bringing in plants didn't work

shower pattern

people start smelling funny after hacking for too long
have a bathroom with a shower (also for people staying over night, guests from other cities)
washing machine for towels if you wanna get fancy

membership fees pattern

need money for rent, utilities, projects
collect fees regularly
choose appropriate amount, then make no exceptions ever
student discounts
have a totalitarian treasurer
always have 3 months of rent on your account

the sponsoring anti-pattern

never depend your space on external sponsors - companies go broke
space should not be located at company/university - you exclude people not in that culture ( non-students e.g.)
donations are ok
room must be independent

regularity patterns

plenum pattern

internal conflicts, have democratic decision-making
have a regular meeting with all members
have an agenda and set goals
make people commit themselves to tasks

tuesday pattern

every weekday sucks for a meeting, there's alwasy someone who has an appointment
pick tuesday. end of discussion. (for all spaces - you always know when everyone is meeting at every space)

open chaos pattern

you want to draw in people, have an interface to the outside world
have a monthly public and open lecture, talk or workshop for everyone
then invite interesting people to regular meetings, don't tell the weirdos :)

u23 pattern

your space needs fresh blood
recruit young people through a challenge, course that spans several weeks
let them solve problems in teams
tutor them, give them room

sine curve pattern

enthusiasm goes away after some time
enthusiasm has a sine curve with a cycle duration of 4 years
keep it running, don't give up

conflict resolution patterns

consensus pattern

need a group decision make sure no one gets left behind
don't take votes - discuss until everyone agrees

democracy pattern

discussion doesn't lead to goal
do take votes, majority wins

command pattern

nobody cleans up etc.
order people, *always participate*, yell if neccessary

sudo leadership pattern

uou start with a nie group but you suddenly find yourself in a dictatorship run by a single person
do not have ranks
use leadership only temporarily

responsibility pattern

don't have time anymore for a task you volunteered for
take pride in your volunteer work
it you can't do it anymore hand it over

debate culture pattern

everybody's yelling at a meeting and nothing gets done
make people with actual socail skiils lead the discussion (e.g student council)
learn not to interrupt others

bikeshed anti-pattern

you suggest building a new bikeshed, but now everybody discusses about the color
if you suggest something everybody will discuss it
identify useless discussions and end them
www.bikeshed.com
if it's complicated people will not discuss, it it's trivial and they can contribute they will

private talk pattern

someone causes a problem that cannot be resolved in the group
let someone experienced talk to the person, don't expose them in front of the group

old hadware pattern

space filled up with junk/old hadware, can't bring in new
create a pile of old/unused hardware, let everybody take from it
anything left should be thrown away
announce it, with an escalation system (3 times)
http://wiki.koeln.ccc.de/index.php?title=Hacker_space/Hardware-Gesetz

creative chaos patterns

key pattern

space shoyld be accessible all the time
hand out keys, track who owns keys, collect deposit for it
have a goof lock
or electronic locking system (creates problems)
other presentation at 24c3 about it

club mate pattern

need to raise funds
buy a pallette of club mate
didn't really get that - people only productive with club mate?!

Am Ende wurden noch URLs zu bestehden Hacker Spaces genannt: das-labor.org, c-base.org, netzladen.org.

Pre-generating cached Stylesheets and Javascripts with Rails 2.0

2007-12-21T00:00:00+00:00

We just switched autoki over to Rails 2.0.2 in the last days. While the transition we replaced the asset_packager plugin with the new Rails 2.0 :cache => true option in stylesheet_link_tag/javascript_include_tag.

As our assets (public/*) are served from an apache webserver that has its own checkout of the project, we can’t let Rails generate the all.css/all.js files on the fly when the first request comes in. Instead we have to pre-generate them at deployment. For asset_packager we had a capistrano task that did just that. Rails doesn’t have that. Instead the logic to generate the files is built right into the link/include methods in the AssetTagHelper(!). So now we are simply doing this in our capistrano recipe:

desc "generate combined css and javascript files on web"
after 'deploy:symlink', :roles => [:web] do
  run "cd #{release_path} && mongrel_rails start -d -e production -p 3001 && curl http://localhost:3001 > /dev/null"
  run "cd #{release_path} && mongrel_rails stop"
end

Fire up a mongrel, curl the start page, bring it back down. Feels a bit overkill but it works. (at least I hope it will when I’ll first try it later today)

Running a skipped Rails Migration

2007-12-12T00:00:00+00:00

Sometimes it happens that you skip a migration. Imagine you were working on a branch and are now merging back into trunk. In the meantime someone committed the migration 303 into trunk but you also had a few migrations in your branch. Now your development database is on 306 and you missed 303. Instead of migrating back to 302 and then again up to 306 you can also do the following in your rails console:

>> require 'db/migrate/303_create_comments_table'
>> CreateCommentsTable.up

There goes your migration.

RSpec 1.1 with StoryRunner coming?

2007-12-12T00:00:00+00:00

The latest update from the rspec trunk changed the version.rb file indicating a 1.1 Release Candidate 1 version. Will there finally be a stable RSpec version that has StoryRunner? That’d be a great present for christmas.

Grafikeffekte im Terminal

2007-12-11T00:00:00+00:00

Nie mehr Langeweile auf der Shell, einfach iTunes Visuals in den Hintergrund gelegt und schon pulsiert die Kommandozeile. Es lebe die Erfindung des halbtransparenten Fensters:

Weihnachtsstöckchen 2.0

2007-12-05T00:00:00+00:00

Das Knallrosa Tagebuch hat einen ganzen Wald von Stöckchen quer durchs Internet geworfen, und einer ist auch hier in diesem Blog angekommen. Da das hier aber ein Tech-Ruby-Web2.0-Startup-Blog ist, werden die Antworten wohl auch etwas anders ausfallen. Zu den Fragen:

Heißt es nun "zu" oder "an Weihnachten"?

Definitiv to, denn das Web2.0 spricht Englisch und to klingt auch schon wie two point 0. Und es heisst auch nicht Weihnachten sondern Weihnachtsgeschäft, und in dem wird dann richtig Umsatz gemacht. Jedenfalls bei den großen Web-Shops. Wir (noch) Kleinen wundern uns derweil, was unsere User die ganze Zeit machen.

Welcher Geschenketyp bist Du - Lastminute-StresseinkäuferIn oder GedankenspielerIn im Herbst?

Semi-Last-Minute-Online-Mass-Customization-Kunde. Die Frage, ob dann auch noch pünktlich geliefert wird, verleiht dem ganzen zusätzliche Spannung gegenüber dem klassischen Offline-Weihnachtsshopping.

Leider haben sich digitale Geschenke noch nicht durchgesetzt. Sonst könnte man Web Communities oder Programme verschenken. Ich erzähl ja immer wieder gern die Geschichte von dem Jungen, der seiner Freundin eine selbst programmierte, animierte Kerzenflamme geschenkt hat (nein, das war nicht ich).

Der ideale Wunschweihnachtstraum?

Ein ultraportables Macbook Pro mit 20 Zoll Display das man aber auf A5 zusammenklappen kann, mit Multitouch-Display übers ganze Gerät, dazu WLAN und UMTS, 512Gb Solid State Disc, 24h Aukkulaufzeit, 0.5cm dickes Gehäuse, wasserdicht und stoßfest, Wahnsinns Grafik- und CPU-Leistung und jemand, der mir dann Heiligabend ab und zu was zu essen und zu trinken vorbeibringt :)

Ach ja, um erstmal auf der Geschenkschiene zu bleiben: Ruby on Rails 2.0, Ruby 1.9, Rubinius 1.0, Mysql 6.0 mit Falcon Engine oder doch besser gleich CouchDb 1.0, TextMate 2, Firefox 3…

Falls es das alles nicht gibt könnte ich mir auch vorstellen, auf einem Schlitten sitzend durch eine winterliche Schneelandschaft zu fahren, in der Ferne klingen die Glocken, ein paar Sternschnuppen huschen über den Himmel, im Hintergrund lauschige Musik (aber bitte nichts weihnachtliches), und meine Abendbeschäftigung wäre es, die Schneeflocken zu zählen, die vom Himmel fallen. Aber das ist natürlich alles komplett unrealisitsch, zurück zum Macbook Pro…

Die Realität?

Rails 2.0 ist sogar halbwegs realistisch, wenn man den Machern glauben kann. Das Macbook Pro kommt dann wohl erst im Januar, und ich habe gar keinen Schlitten. Ganz klassisch zu Hause sitzen und Kartoffelsalat mit Wiener Würstchen mampfen dann wohl.

Bei welchem Geschenk würdest in hysterisches Lachen verfallen, weil Dir sonst nichts anderes einfällt und Du von Heulkrämpfen Kopfschmerzen bekommen würdest?

Das Buch Windows Vista Home. Konfiguration, Kommunikation, Lösungen.. Oder ein Computerbild-Abo. Oder einen exklusiven Premiumaccount Account bei Netmoms.de.

Welchen Namen würde euer Baum tragen, wenn Du Dir über eine solche Frage Gedanken machen müsstest?

Ich würde natürlich den Web2.0 Namensgenerator fragen: Agizu.

Wie klingt Heilig Abend?

Auch dafür gibt es im Web natürlich eine Lösung: soundbadge.net - Ein RailsRumble-Projekt von u.A. Jan Krutisch. Und nächstes jahr wird uns diese Frage hoffentlich dann soundcloud beantworten können.

Wie riecht Heilig Abend?

So frisch wie eine große, von mir aus 566 Billiarden Möglichkeiten persönlich zusammengestellte Portion MyMüsli. Zur Zeit auch mit Weihnachtsbassis-Müsli erhältlich. Dazu dann vielleicht noch gebrannte Mandeln und Pfefferkuchenkrümel?

Was ist Deine erste Erinnerung an Weihnachten?

Das war zu Zeiten als es noch kein Google (1998), kein Ebay (1997), kein Amazon (1999), kein Web2.0 und nicht einmal AOL (1996) oder Compuserve (1999), (hoffentlich lehn ich mich nicht gerade zu weit aus dem Fenster) ja nicht einmal autoki (3/2007) gab. Da kann also nichtmal das Internet Archive weiterhelfen. Aber das ist auch gar nicht Webzweinullig. Jedenfalls gab es bei diesem Weihnachten einen richtigen Weihnachtsmann mit eine gelben Kiste voller Geschenke, Gedichte aufsagen, Lieder singen und all das was es als Kind halt noch gibt.

Wer war früher der Weihnachtsmann?

Ich glaub mein Opa, aber so richtig weiß ich das nicht. Vielleicht war’s ja auch so einer - ach nee das gab’s ja da noch nicht.

Wie lange hast Du an den Weihnachtsmann geglaubt?

Ich hoffe immer noch auf ihn. Und darauf, dass er mir 2 Wochen Urlaub mitbringt.

Wie oft muss "Drei Haselnüsse für Aschenbrödel" sein?

Was?! Das da? Nie gesehen.

</param></param></embed>

Kannst Du glauben, dass es tatsächlich Leute gibt, die diesen wunderbaren, supertollen kulturellen Kulturschatz nicht kennen? (kleiner persönlicher Seitenhieb;-) )

Nee, jetzt wo’s das auf youtube gibt.

Noch Fragen? :) Mit fällt gerade niemand so richtig zum Weiterwerfen ein. Wer immer will möge sich angesprochen fühlen.

Random links: smalltalky ruby & quietbacktrace

2007-12-04T00:00:00+00:00

Patt Maddoxx recently blogged about a ruby hack to avoid the dreaded nil-if, e.g. if user.comments.first; user.comments.first.title;. instead you can now do: user.comments.first.if_not_nil?{|c| c.title}. http://evang.eli.st/blog/2007/11/22/smalltalky

dan croak has written a precious little ruby gem that filters the noise from you exception backtraces: http://giantrobots.thoughtbot.com/2007/12/3/shhh-your-test-unit-backtraces-are-too-noisy

Implementation Patterns emerging

2007-12-03T00:00:00+00:00

I recently read Kent Beck’s new book Implementation Patterns - it was a nice read and I guess it somehow strengthened my senses towards patterns a bit. Unfortunately it is written with Java in mind, and as we all know Patterns are language specific. The following can’t really be applied to Java as far as i know (having stopped writing in Java with 1.5). So I discovered a kind of implementation pattern I had used a few times in the last weeks. Maybe everyone has already written about it but I want to stress that I discovered it myself :) Here it is:

The Selector Pattern

When to use it: When you have a complicated if-elseif-elsif-else or switch clause and want to replace it with something more elegant.

When not to use it: It works well for limited complexity. I things get more complex you may want to use class inheritance instead.

What it does: Put the each condition and the according logic into a hash. Then iterate through the conditions to find a match and execute its logic.

Example

Suppose you have a class Video that represents a video on youtube or some other hosting provider. Its purpose is to render the neccessary HTML tags to embed it into a document. The generated HTML differs depending on the hosting provider, e.g. youtube.com or vimeo.com. The code could be expressed like this:

class Video
  def initialize(url)
    @url = url
  end

  def html
    if @url.include?('youtube.com')
      "<embed>player.youtube.com/#{@url}</embed>"
    elsif @url.include?('vimeo')
      "<embed>vimdeo.com/player/#{@url}/1</embed>"
    end
  end
end

Using the selector pattern it might look like this:

class Video
  @@renderers = {
    'youtube.com' => lambda{|url| "<embed>player.youtube.com/#{url}</embed>"},
    'vimeo' => lambda{|url| "<embed>vimdeo.com/player/#{url}/1</embed>"}
  }

  def initialize(url)
    @url = url
  end

  def html
    select_provider.call(@url)
  end

  private

  def select_provider
    @@renderers.select{|provider, code| @url.include?(provider)}.first.last
  end
end

With only 2 providers this doesn’t make too much sense but with something like 5 your elsifs start to get really ugly and the selector pattern offers a more ligtweight way to replace these than creating an entire hierarchy of subclasses for every case.

Euruko 2007 Review

2007-12-03T00:00:00+00:00

Last weekend (10. to 11. November) we were at the Euruko, the European Ruby Conference in Vienna. The conference wasn’t nearly as big as the Railsconf Europe, but the ad hoc character adapted very well to demands of the attendees and the topics were more academic. Here is a round up of the presentations I attended.

The Keynote was about the future of ruby with a look back to its beginnings by Hal Fulton. About the upcoming ruby 1.9 he said, that it wouldn’t change that much, but will gain performance by having its own virtual machine. Another aspect of his talk was the shift to enterprise use.

The following talk was about the Database Library Dbrb, it is basically a wrapper for ruby dbi to make it easier to work with it. Interesting if you have to do data crunching closer to the relational domain.

David Anderson shows a structured wiki, where you can build simple Applications with forms and so on.

Ramine Darabiha, Sven C. Köhler show their Javascript based content distribution/mashup system MySit.es which is still under heavy development.

After the lunch break Christian Neukirchen showed Rack, a simple and modular web server interface specification. It basically wraps the status, headers, and body in a array. Following the rack specification you can add middleware between your server and the web framework, e.g. log file generation or URL rewriting.

An alternative to watir, a in browser testing framework, was shown by Kingsley Hendrickse, the Ruby IE Scripting System (RIESS) and AutoGUI.

The powerfull, but hard to learn, web site scraping framework scRUBYt was introduced by Peter Szinek.

The last talk of that day was about the use of Domain Specific Languages and how easy it is to build one, held by Martin Grund.

In the evening we had an excellent dinner in the ‘Universitätsbräu’. My personal dish recommendation ‘Schöpfsteak’.

We took the opportunity and input from the conference to try out some new stuff in a late night coding session, although I had had just 4 hours of sleep the night before. But alex will write some more about our twitter mashup twizzer.

On Sunday we missed the first talks. So i can’t tell you anything about it ;)

The first I joined again was from Tim Becker, who showed DTrace. DTrace is an integrated tracing framework for debugging applications or the OS itself. It is integrated into Solaris and the latest Mac OS 10.5 ‘Leopard’.

What Trees are good for, was explained by Paul Battley. For example they are good for faster best match search using the Levenshtein distance. But I have to admit, I didn’t followed that talk very well.

Then Stephan Kämper talked on Quality in code, and shows some exaple of good and bad code.

Hal Fulton filled the little gap before the lunch break with his proposal for an in operator in Ruby.

After the lunch it got even more academic with Ragel, a state machine compiler, presented by Ry Dahl

Then we showed our results from the coding session.

The last presentation of the Euruko was held by Sasch Schlegel about the ebXML Messaging System called Hefeweizen. I liked the MAMAs and PAPAs analogy ;)

Before we caught our flight home, we had another dinner at the ‘Universitätsbräu’.

I enjoyed the Euruko very much, I will try to attend next year again. Maybe in Prague then.

Berlin Barcamp 2 - Was können Social Communities von Spielen lernen?

2007-11-20T00:00:00+00:00

Barcamp2 Berlin, meine erste Session. Holger Dieterich von moviepilot.de projiziert die Mechanismen, nach denen klassische Spiele funktionieren, auf social Communities im Web. Anhand von 10 Beispielen zeigt er, wie eine Community für ihre User spannender und unterhaltsamer (mehr Traffic) werden kann, indem sie von Spielen lernt.

Folgene Annahmen macht Holger über Spiele:

sie machen Spaß
die Spieler werden in eine Art Flow-Zustand versetzt (vergessen die Zeit)
Spiele sind ein soziales Erlebnis (z.B. Fußball)
werden immer wieder gespielt

Genau das, was man als Community Builder auch von seinen Usern sehen will :) Die Herausforderung ist nun, die Mechanismen, die Spieler ihren Spaß bringen und sie immer wieder spielen lassen, auf Web Communities zu übertragen. Ich hab das mal für autoki versucht:

Sammeln: in manchen Spielen, z.b. Magic oder Pokemon, entsteht ein regelrechter Sammelwahn, in diesem Beispiel um die Karten, die zum Spielen benötigt werden. Man kann den Spielern also einen gewissen Sammlertrieb nachsagen, den sie im Spiel befriedigen können. Bei autoki gibt's dafür die Merkliste (auf dem Userprofil), auf der man seine Lieblingsautos sammeln kann. Ist aber noch ausbaufähig, man sollte anschließend mit diesen Autos evtl. Autoquartett spielen können o.ä. Auch das Anlegen mehrer Listen wie z.B. bei iTunes die IMixe wäre evtl. spannend.
Feedback/Status: Die Spieler sollten über den Status des Spiels sowie ihren eigenen bescheid wissen. Beim Fußball gibt's z.B. gelbe Karten und einen Punktestand. Die Analogie in Social Networks sind Statusupdates z.B. über Aktivitäten der Freunde und des Users selbst, welche wir bei autoki in unseren Social Feed auf der Startseite nach dem Login abbilden. Auskunft über die eigene Aktivität gibt der Aktivitätsindex auf dem eigenen Profil.
Hindernisse und begrenzte Resourcen: In vielen Spielen stehen mit nur begrenzte Resourcen zur Verfügung, z.B. die Anzahl der Spielkarten oder Spielsteine. Dadurch wird der Wert der entsprechenden Resource gesteigert. Wäre mal eine Überlegung wert, z.B. das Anhupen zu begrenzen. Zur Zeit hupen einige User recht wahllos ihre ganze Umgebung voll, was auch mal nerven kann. Wenn man nur 3 Leute pro Tag anhupen könnte, wäre der Wert dieser Geste gleich ein ganz anderer.
Angepasster schwierigkeitsgrad: Mach es dem Neuling leicht, aber fordere den Profi immer noch heraus. In Max Payne z.B. greifen die Gegner weniger hartnäckig an, wenn der Spieler zu oft daneben schießt. Im Bereich der Social Community könnte das bedeuten, Neulingen die Features erst schrittweie beizubringen, um sie nicht zu überfordern. Bei autoki gibt's inzwischen zu einigen Features Einführungstexte, die erklären, wie was funktioniert. Wenn man's kapiert hat lassen diese sich auf Nimmerwiedersehen wegklicken. Die Frage ist, ob man so weit gehen wollte, einige Features Neulingen zunächst nicht anzubieten, um sie nicht mit zu vielen Information auf einmal zu erschlagen. Eine generell bessere Strukturierung für Neulinge und Profiuser scheint mir aber sinnvoller zu sein.
Customisierung: in Jedem Rollenspiel Kann ich als Spieler meinem Charakter bestimmte Fähigkeiten verleihen, ihm schicke Sachen anziehen und oft auch den Augenabstand und die Haarfarbe einstellen. Im Web macht myspace vor, was an customizing alles möglich ist - das Ergebnis ist meist eher erschreckend ;). Die große Frage ist, was der eigenen Community wichtiger ist - personalisierte Profile bis zur Unkenntlichkeit oder Lesbarkeit und Konsistenz. autoki geht im Moment den zweiten Weg, mal sehen, ob sich da nochmal was in die andere Richtung tut.
Wettbewerb: Klar, in den meisten Spielen geht es darum, größer, schneller oder reicher zu werden als die anderen - der Wettbewerb stellt wohl mit die größte Motivation dar, Spiele überhaupt immer wieder zu spielen. Nicht fehlen darf eine Highscoreliste, auf der die Champions ihre Portion Ruhm abbekommen. Autoki bedient diesen Punkt ganz einfach durch die Integration von Spielen, vor allem des Autoquartetts. Um die ersten Plätze auf der Monats- und Gesamtspiel gibt es regelmäßig Kämpfe. Weiterhin gibt es Dinge wie bestbewertete Fotos und Autos - wird in Zukunft gerade noch weiter ausgebaut.
Minispiele: Kleine Spiele, die nur einige Sekunden/Minuten dauern, immer wieder mal zwischendurch gespielt werden können und im Idealfall süchtig machen. Da lassen sich wohl sämtliche Handyspiele einordnen, oder auch kleinere Einlagen in eigentlich komplexeren Computerspielen, wie z.B. einem Spielautomaten, der in einem Adventure rumsteht oder ähnliches. In dem Sektor ist autoki mit dem Quartettspiel ganz weit vorne. Dauert nur 15 Runden und man kann sowohl gegen Computer als auch menschliche Gegner spielen. Trotzdem gibt's auch hier noch Verbesserungspotential, vor allem für neue User, die das Spiel nicht kennen.
Glück im Spiel: Zufällig auftretende Ereignisse wie Bonuspunkte oder Gewinne motivieren, immer mal wieder zu spielen bzw. in der Community aktiv zu werden, da jederzeit die Chance auf ein solches Ereignis besteht. Mit steigender Aktivität steigt natürlich auch die Wahrscheinlichkeit eines zufälligen Gewinns. So ungefähr funktionieren bei autoki die Herausforderungen bzw. das Bewerten von Autos. Man kann sich per Zufallsbutton von einem Auto zum nächsten durchklicken und jeweils seine Stimme abgeben. Wenn man Glück hat trifft man auf ein richtig cooles Auto.
Belohnung: Wer viel spielt soll auch belohnt werden. In Rollenspielen z.B. erhält der Spieler neue Ausrüstungsgegenstände nachdem er eine Aufgabe gelöst hat. Je weiter das Spiel voranschreitet, desto größer werden auch die Belohnungen. Auf autoki gibt es dazu erste Ansätze, wie z.B. den Status "Mr. Autoki", der auf dem Profil des vom Girl's Club Auserwählten prangt.
Letzter Punkt: Zuschauer. Zuschauer sind neben Traffic für die Website gleichzeitig Belohnung für die Aktiven. Wenn ich ein Video hochlade und sich nach 2h 500 Leute das angeguckt haben bin ich motiviert weitere Videos hochzuladen. Wenn ich in einem Fußballstadion vor 40.000 Leuten ein Tor schieße, gibt mir das eine höhere Befriedigung als vor leeren Rängen. Dementsprechend zeigen wir auf autoki die Anzahl der Besucher auf Profilen, Fotos etc. an. Und wer die meisten hat landet ganz oben auf der entsprechenden Top-Liste.

Die Slides gibt’s übrigens bei slideshare.

Using and Testing ActiveRecord/Rails Observers

2007-10-27T00:00:00+00:00

We recently introduced a new feature in autoki called the social feed. It’s basically a yellow box displaying any events on the platform relevant to the current user, like a friend has posted a new photo, or a new interesting car was uploaded. The data model behind this is pretty straightforward, we have a FeedEvent class and all kinds of subclasses, e.g. a MessageReceivedEvent. Each event belongs to a user and an event source, in this example the user would be the user who received the message and the event source would be the message itself. For each user, we simply display all the events that belong to him or her.

Now the question was this: How do we create these events? The most straightforward way would probably have been to create them in the models, so the Message model would have an after_create callback that created the event. What we didn’t like about this solution was that we would put a whole bunch of logic into the models that didn’t really belong there. Why would a Message care if there was some kind of event feed? Plus these events would be all around in our unit tests and make the bloated and probably sloooow (again). So we wanted to use the observer pattern to remove the creation of the event from the models.

For observers in a Rails app you basically have two choices waiting there for you: the ruby Observable mixin and the ActiveRecord::Observer class. We didn’t have much time (as usual) and only took a very short look at both and quickly decided to go with the ruby Observable. ARObservers seemed to only allow the usual before/after create/update/save/destroy callbacks and looked much more heavyweight than the tiny Observable module. So we did this: (sorry new example, this time with comments.)

Ruby Observable Mixin

Integration Test

def test_create_comment_creates_event
  myuser = create_logged_in_user
  user = create_user
  FeedEvent.delete_all
  post "comments/create", :comment => {:content => 'test', :comment_owner_id => user.id, :comment_owner_type => 'User'}
  assert_equal(1, FeedEvent.count)
end

Controller Test

def test_create_attaches_observer
  user = new_logged_in_user
	c = new_challenge
	post :create, :comment => {:content => 'test', :comment_owner_id => c.id, :comment_owner_type => 'Challenge'}
	assert_equal(1, assigns(:comment).count_observers)
end

Controller Implementation

def create
	@comment = Comment.new params[:comment].merge(:user_id => logged_in_user_id)
	@comment.add_observer CommentObserver.new
  @comment.save
end

Model Test

(using mocha)

def test_create_notifies_observer
  user = new_user
  observer = stub
  comment = Comment.new :comment_owner => user, :content => 'x'*20, :user => new_user
  observer.expects(:update).with(:create, comment)
  comment.add_observer observer
  comment.save!
end

Model Implementation

require 'observer'

class Comment
  include Observable
  after_create :notify_comment_observer

  ...

  private
  def notify_comment_observer
    changed
    notify_observers :create, self
  end
end

Observer Test

def test_update_creates_create_event
  comment_observer = CommentObserver.new
  comment = new_comment
  comment.receiver = new_user
  CommentCreateEvent.expects(:create!).with(:user => comment.receiver, :source => comment)
  comment_observer.update :create, comment
end

Observer Implementation

class CommentObserver
  def update(event_name, comment)
    CommentCreateEvent.create! :user => comment.receiver, :source => comment unless comment.user == comment.receiver || comment.receiver.blank?
  end
end

Wow. That’s a whole lot of code for a tiny little yellow box with one sentence in it. And it’s not even complete because you have to handle deletion of the comments and some other stuff as well. (no event if user comments on his own objects).

Anyway, for some reason we implemented this for 8 types of events or so: Integration test, attach observer in controller, call changed and notify_observers method in model, create event in observer. It was a real pain because we were implementing almost the same thing again and again. Especially attaching the observer in the controller seemed too much work. We thought about doing some meta ruby magic to be able to do the same as rails does with cache sweepers. Instead of attaching the sweeper to the model directly you simply declare that you want this cache sweeper to be active in this action:

class MyController &lt; ApplicationController
  cache_sweeper :comments_sweeper, :only => :create
end

It didn’t really work out because we couldn’t figure out the right magic to do the right things when the model was loaded or created without letting the model do it, which would make the whole use of observers pointless.

ActiveRecord Observers

Finally after a couple of days we took a closer look at the ARObservers. They don’t require you to add them to a model instance every time you want to use them. All you have to do in order to use them is to configure them in your environment:

config.active_record.observers = :message_observer

You then derive you observer class from ActiveRecord::Observer and from then on get all your before/after create/update/save/destroy events for free, delivered right to your observer method with the corresponding name:

class MessageObserver &lt; ActiveRecord::Observer
  def after_create(message)
    MessageReceivedEvent.create! :user => message.receiver, :source => message
  end
end

Now we still had the problem that we wanted custom events, e.g. an after_read event. After some digging deep down in the rails sources we found out how:

class Message
  def read
    self.mark_as_read!
    <strong>callback :after_read</strong>
  end
end

That’s it. Now we implement an after_read method in the observer and we were done.

Now the testing part.

Any idea how the rails guys got the observers attached to every model instance in the app automagically? Well, they attached the observer to the class, not the instance. Sounds like a brilliant idea doesn’t it? But wait. We said one big reason for us to use the observers was that we didn’t want our unit tests to be concerned with this. And now the observers were attached to the classes and all our tests were carrying them as well. Our first solution was to move the observer configuration from environment.rb to the development.rb/production.rb files but this only solved one problem while creating two others: We now had the observer configuration in multiple places and, more importantly, our integration tests didn’t have the observers as well, hence, were now failing.

We ended up doing this: moved the observer configuration back into environment.rb, removed the observers from the model classes before running the unit tests, attached them back before running the integration tests:

# test_helper.rb


# remove all activerecord observers

Dir.glob File.join(RAILS_ROOT, 'app', 'observers', '*.rb') do |file|
  File.basename(file, '_observer.rb').camelize.constantize.delete_observers
end

(We have a separate directory for our obserers in app/obserers)

For our integration tests we created a new file integration_test_helper.rb which we now require instead of the test_helper.rb:

require "#{File.dirname(__FILE__)}/test_helper"
# add observers in case they have been removed by unit tests

Dir.glob File.join(RAILS_ROOT, 'app', 'observers', '*.rb') do |file|
  clazz = File.basename(file, '_observer.rb').camelize.constantize
  clazz.delete_observers
  observer_clazz = File.basename(file, '.rb').camelize.constantize
  observer = observer_clazz.instance if observer_clazz.respond_to?(:instance)
  clazz.add_observer observer if observer.is_a? ActiveRecord::Observer
end

With this setup we now have working unit and integration tests. No need for controller tests and implementation, model tests and implementation only if we have a custom event (such as after_read). What’s left are integration tests and a unit test and implementation of the observer, which is mostly trivial:

class MessageObserverTest &lt; Test::Unit::TestCase

  def setup
    @observer = MessageObserver.instance
    @message = new_message
  end

  def test_create_creates_message_received_event
    MessageReceivedEvent.expects(:create!).with(:user => @message.receiver, :source => @message)
    @observer.after_create(@message)
  end
end

Mehr als der Flug zum Ballermann

2007-10-17T00:00:00+00:00

Nachdem wir kommt-mit.de schon Anfang des Jahres als Plattform und Community zur individuellen Reisevorbereitung gestartet haben, dringt der Trend auch langsam zu den großen durch. So titelt golem.de heute: Internet bei der Urlaubsvorbereitung immer wichtiger. Genau. Leider beschränken sich die Aussagen dort auf die Buchung von Flügen, Hotels usw. Der eigentlich spannende Teil liegt meiner Meinung nach jedoch beim ganzen “Drumherum” der Reisevorberitung, das sich vor allem beim Verreisen mit mehreren (soll ja vorkommen) durch das Internet viel leichter abwickeln laesst.

Deshalb geht es bei kommt-mit auch nicht darum, einfach nur den billigsten Flug nach Ballermann zu finden, sondern den ganzen Planunsgaufwand vorab zu meistern, und gleichzeitig die Vorfreude zu schueren.

Nach der Anmeldung kann man zunächst seine Mitreisenden einladen. Der Reisegruppe steht nun ein eigener Bereich zur Verfügung, in dem sie über ein Forum ihre Reisevorbereitungen abstimmen koennen. Über die Aufgabenliste kann leicht geklärt werden, wer den Grill und wer die Kohle zum gemeinsamen Campingurlaub mitbringt. Und die beste Feuerstelle lässt sich doch schonmal vorab per Satellitenbild aussuchen.

Im Anschluss können die besten Schnappschüsse des Jahresurlaubs hochgeladen und von allen bewundert werden. Es ist natürlich weiterhin erlaubt, sich auch außerhalb des Internets zum Fotogucken zu treffen ;)

Auf der Startseite gibt es ein kleines Video, was die Funktionen von komm-mit noch einmal erläutert.

Arme virtuellen Maschinen

2007-10-17T00:00:00+00:00

Hat sich eigentlich schonmal jemand gefragt, wozu eigentlich virtuelle Maschinen, also z.B. ein Linux in einem VMWare/Paralls, ihren Bildschirmschoner laufen lassen? Dabei haben sie doch gar keinen Bildschirm, sondern maximal ein paar hundert Pixel in einem Fenster, in dem sie ab und zu ihre bunten Grafiken dem geneigten Benutzer zeigen dürfen, falls der sie nicht einfach mit einem Klick ihr Fenster minimiert oder ganz komplett ausschaltet.

Arme, dumme virtuelle Maschinen. Wenn sie doch nur die geringste Peilung hätten, worin sie da eigentlich laufen. Na hoffentlich läuft die Erde nicht auch in einer virtuellen Maschine.

Morgen gleich nach dem Frühstück: mysql schneller machen

2007-09-25T00:00:00+00:00

nachricht an mich selbst: morgen frueh nochmal "quick wins with mysql" lesen und den eigenen mysql-server ueberpruefen.

Railsconf Europe 2007 Roundup 3 (rubyworks)

2007-09-24T00:00:00+00:00

rubyworks is a full rails stack put together also by thoughtworks. this time it’s all open source. it features haproxy (supposed to be much faster and light weight than apache_mod_balancer) as load balancer, mongrel as application server, monit and runit for monitoring and controlling mongrel, mysql/postgrs/oracle ruby bindings for database connectivity and also bindings for ferret, libxml, hpricot and rmagick.

everything is nicely packaged as rpms and debian packages so it can easily be installed within 5 minutes (it really works). even better, the packages at the same time pretend to be ruby gems, so you can safely install other debian packages and gems that depend on one of the libraries provided by rubyworks.

now we had already set up a cluster of 6 servers for autoki with everything set up more or less perfectly so why would we need rubyworks? answer: to steal the config files for runit/monit and haproxy. our mongrel setup has always been a bit shaky, especially when it came to restarting the mongrels after a deployment. after using the rubyworks setup with runit now everything is stable. (btw runit can run and supervise any process in *nix and is ready to be the successor of the old init which is used by most linux distros to start up all the processes. one advantage is that it starts all processes at once instead of piece by piece, plus runit handles putting a process in the background and keeping it alive there, something mongrel is especially bad at).

Railsconf Europe 2007 Roundup 2 (mingle)

2007-09-24T00:00:00+00:00

Mingle is the new agile project management tool from thoughtworks studios. it’s not open source but it’s free for up to five users - sounds like a fair offer to me. for everyone who thinks differently - it’s written in ruby and deployed using jruby, but the class file decompiler is part of the jdk. i wonder how long they can actually make people pay for it. hello open source business models?

anyway, when thilo installed mingle from one of the cds given out to all conference attendees we first were a bit disappointed. after the installation you basically get a nice looking but totally empty screen leaving you with not much more than a big question mark over your head. we didn’t have much time then so that was it for mingle until i attended the corresponding session where things got a bit clearer: mingle is a completely customizable tool. the only thing that’s hard coded are users and cards/stories/tickets/you-name-it and that everything is a wiki page - the rest is up to you. the main features are these:

define your own attributes for your cards: you can add new attributes to your cards and in place edit all of them - auto complete shows you a list of values that have been entered into that field before
filter cards by any combination of attributes and save the filter, after which you can add it as a tab on the top
card transitions - create a set of transitions where a card with a specific state can be transitioned to another state, e.g. from status open to status accepted and assigned to a person. this enables the project team to map their work flows into mingle
create graphs and tables from the cards - multiple graph types are supported for easy tracking of iteration progress or number of bugs etc.
project templates - so you don't have to sit in front of a blank screen as we did

mingle looks like it could become a really cool tool. after all, trac is getting a bit boring after all these years(?) and we’ve actually stopped using its ticketing system, because it wasn’t easy enough to change it to our needs.

Railsconf Europe 2007 Roundup 1 (rspec)

2007-09-24T00:00:00+00:00

Last week was RailsConf 2007 in Berlin. Only a couple of days have passed and it already seems so far away. Time for some blogging before everybody forgets that it even took place and nobody’s going to read this :) So here’s the interesting part of the sessions I attended:

A Half-day of Behavior-driven Development on Rails (rspec)

This was the first session on tutorial day and for me one of the most interesting. It basically gave a looong introduction to behavior driven design, how it evolved from things like TDD and who realized what while enganged in which project back in the good old times. One of the interesting parts for me was when they talked about the whole story writing/specification process. I had heard most of it before but it was a good refreshment. The central statement was to write “Software that matters”, and to achieve this, you’d have to get the specs right - as we XPers know this should be done by collecting user stories. The suggested format for such a story was this:

As a [role] I want [feature] so that [outcome]

Example:

As an administrator I want to see the list of users so that i can choose which one to delete.

Great, a template for writing story cards. Following the story, you need an acceptance test that verifies that the story has been implemented. The suggested format is this:

given [context] given [more context] when [event] when ... then [outcome] then ...

Example:

Given i am logged in as root Given a list of users When i view /users Then i see all the users

Now in the early days you would write this on a story card or on some other maybe non-digital, maybe digital surface, but you would not do this: run the acceptance test as code. (I’m aware there are some projects who’ve tried/done it but I’ve never found one too convincing and really used it). Now you can. They actually implemented a new feature in rspec trunk called StoryRunner to do exactly this and here’s how it looks:

Story 'As an administrator
I want to see the list of users
so that i can choose which one to delete.' do

Scenario 'logged in as root with 2 users in system' do

  Given 'i am logged in as', 'root' do |name|
    @user = create_user :name => name
    post /sessions/create, :name => name, :password => @user.password
  end

  Given 'a list of users' do
   @users = [create_user, create_user]
  end

  When 'i view', '/users' do |location|
    get location
  end

  Then 'i see all the users' do
    @users.each do |user|
      response.should have_text(user.name)
    end
  end

  def create_user
    ...
  end

end

end

How cool is that, but it doesn’t even stop here. You can now reuse the code blocks you have created (!!):

Scenario 'logged in admin with no users in the system'
  Given 'i am logged in as', 'root'
  Given 'there are no users' do
    User.delete_all
  end
  When 'i view ', '/users'
  Then 'i see a message that there ar no users' do
    response.should have_text 'nobody there'
  end
end

Ok, technically speaking it’s actually not much more than reusing code but the way it’s done in StoryRunner, you are almost forced to create your own, application specific testing DSL! (Did I mention how fantastic this is?). For a better example which includes a “normal” rails integration test and compares it to an rspec story implementation see the evang.ei.st blog.

Of course they also showed the “usual” rspec stuff (referred to as “classic rspec”) for specing models and controllers. I you haven’t already go back to the evang.eli.st and check out Easy Controller Tests and Expressing Intent Through Expectations - a really good (imho) introduction on writing controllers using ~~test~~ spec first.

Enough for now, I’m sure there will be more on rspec here soon.

mocha Präsentation

2007-09-06T00:00:00+00:00

Hier wie versprochen die Slides meiner Presentation zum Thema mocha bei der Ruby User Group Berlin.

Links:

Mocha Floehopper (Blog des Mocha Entwicklers) Presentation about Ruby Mocks TestingRails Blog (gelöscht :() Use stubs and mocks to convey intent Ruby: Mocks and Stubs using Mocha

Kleinere Thumbnails mit strip

2007-09-06T00:00:00+00:00

Open Source ist wirklich eine tolle Sache, jeder kann den Code für seine Zwecke anpassen. Das haben wir auch schon ein paar mal gemacht, hier kommt noch einen schönes Beispiel welches für Open Source spricht.

Wir haben viele kleine thumbnails auf der Startseite von autoki, die haben wir genauso wie alle anderen Formate mit file_column generiert. file_column ist ein praktisches Plugin das eine Schnittstelle über RMagick zu ImageMagick bildet, um einfach verschiedene Formate von Bildern für ein Model zu generieren. Vor Kurzen mussten wir feststellen, dass die autoki Startseite ungewöhnlich viel Bandbreite braucht. Beim Blick auf ein einzelnes Thumbnail zeigte sich, das bereits ein kleines 53x53 Pixel Jpeg 48KB gross ist.

Die erste Vermutung war, dass die Qualituatseinstellungen für die Jpeg Kompression zu hoch ist. Ein bisschen Suchen in der RMagick Dokumentation brauchte das attribut quality hervor, mit dem sich die Qualitätseinstellungen regeln lässt. Etwas ausprobieren brachte dan auch einen Teilerfolg. Indem den schlüssel :attributes ein Hash übergeben für die einzelnen Versionen übergeben wird liess sich die qualität steuern. Aber das brachte nur ein paar Kilobyte.

Also musste das Problem wo anders liegen. Nächste Ansatzpunkt war die Kompression. Auch hier half die RMagick Doku weiter. Mit dem key :compression läst sich die Kompression festlegen. Aber auch hier kein Erfolg, wie erwartet wird automatisch die Kopression verwendet die die Dateiendung vorgibt.

Also die Datei mit TextMate geöffnet um vielleicht einen Hinweis auf das Problem zu erhalten. Dort fand sich neben dem erwarten Zeichensalat XML tags. Diese sahen nach EXIF-Informationen aus. Im EXIF-Format werden Meta-Daten über die Aufnahme wie Zeitpunkt, Kamera, Belichtungsinformation etc. gespeichert. Dazu fanden sich noch weitere XML Blöcke die offensichtlich von Photoshop stammen. Das Problem war also das die Metadaten aus der Datei nicht entfernt wurden.

Jetzt wo das Problem bekannt war, konnte auch die Suchmaschiene dazu befragt werden, wie man diese Metadaten loswird. Die Antwort ist strip. Die Rmagick methode entfernt alle Nichtbilddaten. Wird bei file_column eine Bildvariante mit :thumbnail satt :version erstellt wird strip ebenfalls aufgerufen. Doch die Thumbnail Varianten lassen sich nicht mit crop noch zusätzlich beschneiden. Aber genau das machen wir um gleichmässig erschneinende thumbnails zu erstellen. Darum muss die strip Methode an geeigneter Stelle aufgerufen werden. Da die skalierung und das beschneiden in der mehtode transform_image in der FileColumn Klasse geschiet ist dort ein guter Ort strip aufzurufen. Ob nun am Anfang oder am Ende ist eigentlich egal. Da wir all unsere anderen file_column Hacks auch ans ende dieser MEthode gepack haben, kommt der strip aufruf noch dazu.Und nachdem die Bilder mittels regenerate_images neu erstellt waren, ist die Grösse der tumbnails auf etwa 4KB (ein Zehntel!) gesunken. Jetzt baut sich die Startseite viel schneller auf und der Bandbreitenbedarf ist erheblich gesunken.

Also um kleine Thumbnails zu generieren kann man entweder :thumbnail statt :version in file_column verwenden, wenn man die thumbnails jedoch zuschneiden will sollte man am besten strip in der methode transform_images aufrufen um die Metadaten wie z.B. EXIF Informationen aus der Datei zu entfernen. Das ganze kann sich auch lohnen wenn viele grosse Bilder gespeichert werden müssen.

Mnemonic for Duck Typing.

2007-08-26T00:00:00+00:00

I’m currently reading the Ruby Cookbook and found a good explanation for duck typing. You know, the concept to treat objects according to their methods rather their classes/modules. It might be well known to you, but i like to share it anyway.

Duck Typing means:

"... if an object quaks like a duck, just go ahead and treat it like a duck"

P3P - Idiotie

2007-08-20T00:00:00+00:00

P3P steht für Platform for Privacy Preferences und ist ein vom w3c erfundenes dingens, mit dem ich als Webseitenbetreiber eine Art maschinenlesbare Datenschutzerklaerung abgeben kann - ich kann z.B. angeben, ob ich Daten über meine Besucher speichere und wenn ja was ich mache und wem ich sie gebe usw.

Das klingt alles irgendwie ganz nett, auch wenn mir so richtig noch kein guter Anwendungsfall einfallen will (ist mit eigentlich auch ziemlich egal gerade), aber zum Glück haben die Jungs aus Redmond schonmal vorgearbeitet und das ganze in ihr Browserflaggschiff namens Internet Explorer eingebaut.

Und jetzt ratet mal wofür. Um den Benutzern ein sichereres und besseres surfen zu ermöglichen vielleicht? Um Websitebetreibern irgendeinen Vorteil zu bieten? Genau, haben sie nicht. Stattdessen haben sie folgendes tolles Feature eingebaut:

Wenn ich auf eine Seite surfe, z.B. diese(s/n) Blog, und da zum Beispiel zum Artikel über’s Autoquartett, dann ist besagtes Quartettspiel über einen IFrame in die eigentliche Seite eingebunden, d.h. Inhalte eines fremden (des autoki)-Servers werden auf der Seite von upstream (übrigens seit heute GmbH i.G. :) ) angezeigt. Und was macht der Internet Explorer mit diesen fremden Inhalten? Er verbietet ihnen, Cookies zu speichern. Cookies sind das, in dem sich das Autoquartett u.a. dem Punktestand merkt. Ohne Cookies keine Punkte und kein Quartettspiel. Und jetzt kommt der Clou: Mit Hilfe von P3P kann man das ganze jetzt wieder abschalten, indem man nach mehrstuendiger Suche diesen Artikel, ebenfalls in Redmond entstanden, findet. Und da steht dann geschrieben, dass die Worte P3P: CP="CAO PSA OUR" diesen Prachtbrowser wieder zu Vernunft bringen, also den Normalzustand ohne P3P herbeiführen. Toll. Ich bin begeistert. Mehr solche innovativen Funktionen, dann steigt der Marktanteil bestimmt auch bald wieder. (bei autoki liegt übrigens <a href=http://getfirefox.com”>Firefox</a> vorne).

Ich hab mich nicht allzu lange mit dem Kürzel beschäftigt, aber sowas wie Wir speichern keine Daten und sind ganz lieb dürfte es schon sein. Von wegen ;)

P.S. jaja, es gibt bestimmt auch eine total tolle Verwendung für das Ganze, bei der es egal ist, dass der Websitebetreiber über seinen Datenschutz behaupten kann, was er will. Die Kommentare sind offen :)

9to5 - bald geht's los

2007-08-20T00:00:00+00:00

Vom 9to5-wir-nennen-es-arbeit-festival-camp gibt’s jetzt zum einen diese ansehnlichen Banner, zum anderen ~~den Fahrplan~~ das Programm. lauter Lesungen, Hörspiele, Filme und DJs. Na wenn das nicht nett wird. Da belibt bestimmt noch genug Zeit zum plaudern und an lustigen Projekten arbeiten. Oder Zeit, sich meinen Vortrag zum Thema Startup-Gründung für Anfänger anzuhören. - nach derzeitigem Plan Samstag 18 Uhr im Raum Bremen.

Oder zusammen etwas sinnvolles oder sinnfreies mit meinem Nabaztag, diesem kleinen Plastikhasen mit WLAN (bestimmt auch ein digital bohemian), Stimme und Ohren, zu machen, der bis dahin hoffentlich eingetroffen ist.

Mehr zu alldem dann spätestens Donnerstag, oder wir sehen uns einfach da. Eintritt tagsüber übrigens frei, von 9to5 dann 60 EUR für 3 Tage.

Captchas with rails and multiple servers

2007-08-17T00:00:00+00:00

On autoki.com we have this “Tell-a-friend” functionality, where people can enter the email addresses of their friends and have the link to a cool photo or car sent to them. Until recently this was only accessible to logged in users because we were afraid of spam bots using it to send emails to everyone using our servers. At some point we realized that this function could be much more useful if everyone would be able to use it - member or not. The solution we chose to fight the spam bots was pretty standard: Captchas - Completely Automated Public Turing tests to tell Computers and Humans Apart. (I just love this name)

There are a couple of plugins for rails to implement captchas. Most of them display an image with distorted letters and numbers and ask the user to enter these in a form field. The alternative is to ask the user a simple question that a human can easily answer but a computer (as in spam bot) can not. The latter has two advantages: they work for blind people / text browsers and we don’t have to generate and store images - so we went with this one. validates_captcha (sorry couldn’t find a working link) is a plugin we have used before and it supports both - images and questions (so called logic captchas).

Now the only problem was this: autoki runs on multiple servers and the captcha validation process involves multiple http requests, which can each go to a different server. Validates_captcha stores the captchas in a local file, so if the generation of the captcha occurs on one server but the validation on another, that other server can’t find the captcha. The solution we chose is actually pretty simple: rewrite validates_captcha to use memcache instead of the local file.

To accomplish this we only had to change one file, captcha_challenge.rb. All we had to do was replace all the occurrences of store[:captchas] with memcache. The modified file now looks like this:

require 'digest/sha1'

module FleskPlugins #:nodoc:


  # This is an abstract class. Use one of its subclasses.

  class CaptchaChallenge

    include CaptchaConfig
    extend CaptchaConfig

    DEFAULT_TTL = 1200#Lifetime in seconds. Default is 20 minutes.


    attr_reader :id, :created_at
    attr_accessor :ttl

    @@types = HashWithIndifferentAccess.new

    def initialize(options = {}) #:nodoc:

      self.id = generate_id
      options = {
        :ttl => config['default_ttl'] || DEFAULT_TTL
      }.update(options)
      self.ttl = options[:ttl]
      @created_at = Time.now
      self.class.prune
    end

    # Implement in subclasses.

    def correct? #:nodoc:

      raise NotImplementedError
    end

    # Has this challenge expired?

    def expired?
      Time.now > self.created_at+self.ttl
    end

    def ==(other) #:nodoc:

      other.is_a?(self.class) && other.id == self.id
    end

  private

    def generate_id #:nodoc:

      "captcha_#{self.id = Digest::SHA1.hexdigest(Time.now.to_s+rand.to_s)}"
    end

    def id=(i) #:nodoc:

      @id = i
    end

    def write_to_store #:nodoc:

      CACHE.set self.id, self, self.ttl
    end

    class << self

      #Get the challenge type (class) registered with +name+

      def get(name)
        @@types[name]
      end

      #Register a challenge type (class) with +name+

      def register_name(name, klass = self)
        @@types[name] = klass
      end

      # Find a challenge from the storage based on its ID.

      def find(id)
        CACHE.get id
      end

      # Delete a challenge from the storage based on its ID.

      def delete(id)
        CACHE.delete id
      end

      # Removes old instances from PStore

      def prune
      end#prune

    end#class << self

  end
end#module FleskPlugins

With this, all captchas now get stored on the central memcache server and all the application servers have access to it. And it’s even better than the original, because memcache supports time to live, so old captchas get removed from the store automatically.

Note: We are using the MemcacheClient gem to access memcache from rails. The CACHE constant is declared in the environment files, so in production.rb we have this:

require 'memcache'

memcache_options = {
   :compression => false,
   :debug => false,
   :namespace => "autoki_#{RAILS_ENV}",
   :readonly => false,
   :urlencode => false
}
memcache_servers = [ 'cache.autoki.com:11211']

cache_params = *([memcache_servers, memcache_options].flatten)
CACHE = MemCache.new *cache_params

For development and testing we use a simple hash based stub:

class TestCache

  def initialize
    @items = {}
  end

  def set(key, object, ttl = nil)
    @items[key] = object
  end

  def get(key)
    @items[key]
  end

  def [](key)
    @items[key]
  end

  def []=(key, value)
    @items[key] = value
  end

  def delete(key)
    @items.delete key
  end

  def clear
    @items.clear
  end
end

CACHE = TestCache.new

Now the exercise left would be to factor out the storage code so the choice of captcha storage can be configured in the validates_captcha configuration, anyone? :)

FileColumn: regenerate images

2007-08-12T00:00:00+00:00

If you change the images dimensions or introduce a new version for models using file column you need a way to update the existing ones. Here is a script that will regenerate all file column images and its version. We still look for a way to generate only new versions.

# recreates all file_column images in all needed sizes

class ImageUpdater

  def update_all
    update_model(:user, :photo)
    update_model(:auto, :photo)
    update_model(:photo, :file)
    update_model(:group, :photo)
  end

  private

  def update_model(class_name, attribute_name)
    eval(class_name.to_s.capitalize).find(:all).each {|instance|
      begin
        dir = instance.send(attribute_name)
        unless dir.blank?
          instance.send("#{attribute_name}=", upload(dir))
          instance.save!
        end
      rescue Exception => e
        puts e.message
      end
    }
  end

  # copied from file_column/test_case.rb

  def upload(path, content_type=:guess, type=:tempfile)
    if content_type == :guess
      case path
        when /\.jpg$/ then content_type = "image/jpeg"
        when /\.png$/ then content_type = "image/png"
        else content_type = nil
      end
    end
    uploaded_file(path, content_type, File.basename(path), type)
  end

  def uploaded_file(path, content_type, filename, type=:tempfile) # :nodoc:

    if type == :tempfile
      t = Tempfile.new(File.basename(filename))
      FileUtils.copy_file(path, t.path)
    else
      if path
        t = StringIO.new(IO.read(path))
      else
        t = StringIO.new
      end
    end
    (class << t; self; end).class_eval do
      alias local_path path if type == :tempfile
      define_method(:local_path) { "" } if type == :stringio
      define_method(:original_filename) {filename}
      define_method(:content_type) {content_type}
    end
    return t
  end

end

Using Mocha - A review

2007-08-01T00:00:00+00:00

For aprox. 3 months we have been using mocha now. And we promised to get back to this subject when we passed our 2000th revision.

In this post I’d like to share some expirence we made while using mocha. But first a short introduction to mocha taken from the rubyforge page.

Mocha is a library for mocking and stubbing using a syntax like that of JMock, and SchMock. One of its main advantages is that it allows you to mock and stub methods on real (non-mock) classes and instances.

We started to use mocha on an existing test suite and changed our test code in place when we wrote a new test or had to update old ones. The transition was mostly painless, there were three things where we stumbled.

Avoid any_instance, we got some strange interactions with instances in other tests (Maybe changed with the 0.5 release, which is great btw.). You can get along very well without any_instance by stubbing .new. The second challenge was more the change in the way how to write tests. We were used to push all the required objects in the database, then run the code and examine the result afterwards.

def test_index_shows_logged_in_user_in_highscore
  5.times { create_user :game_points =&gt; 10 }
  u = create_logged_in_user :game_points =&gt; 3, :first_name =&gt; 'gamer'
  get :index
  assert_select '#hall_of_fame td', u.name
end

This example shows an old functional test. As you can see we hit the database hard. We built some helper methods like create_user to create often used database objects, but we didn’t think about stubbing so far.

With mocha in place, we could test the expected behavior, which must be expressed beforehand. We also had to build new test helpers to provide us with stubs instead of full fledged objects, or at least real objects with stubbed behavior.

def test_index_shows_logged_in_user_in_highscore
  user = new_logged_in_user :game_points =&gt; 3, :first_name =&gt; 'gamer'
  Game.expects(:all_time_rank_of_user).with(user).returns(1)
  get :index
  assert_select '#hall_of_fame td', user.name
end

This code tests the same as the test above but this time with mocha. This test (almost) doesn’t hit the database and is more readable with the mocha syntax.

The last thing we had to learn. Every unexpected call to a mock object will cause your test to fail. This is good in a way, because you lessen the risk to miss some dependency in your code. But in the beginning it was hard to find the origin for the missed call, because the type is always ‘Mock’, so that you get errors like the following: #<Mock:0x4cf5bf8>.find(:first, :order => 'created_at DESC'}) - expected calls: 0, actual calls: 1 Similar expectations: find('11') find('11') Only the call parameters and the stack trace can reveal the culprit.

If you get confused about all this mocking and stubbing. You can find a good explanation between the difference of mocking and stubbing and the different flavors in these slides from james mead

The reward for all the work are much faster running tests and a decoupled testsuite, because you can mock/stub dependencies.

Here some real figures.

Before: 424 unit tests with 637 assertions run in 319 seconds. 326 functional tests with 646 assertions run in 322 seconds

After: (still some potential) 659 unit tests with 1177 assertions run in 165 seconds. 574 functional tests with 1285 assertions run in 171 seconds.

The nubers speak for themselves 8)

Bilder aus dem Netz mit file_column runterladen

2007-07-29T00:00:00+00:00

Hier also ein weiteres Post zum Thema file_column Hacks.

Vor kurzem mussten wir die Anforderung umsetzen, dass Bilder, die im Netz liegen, wie eine vom Client hochgeladene Datei speichern zu können. Warum wir sowas machen wollen? In diesem Fall, um Daten von einer Anwendung über http nach autoki zu importieren. Dazu gehörten auch Bilder, die über eine URL erreichbar waren.

Dafür haben wir zuerst die Zuweisung der file_column Instanz in file_column.rb so angepasst, dass zwischen einer hochgeladenen Datei und einem String, der die URL enhält, unterschieden werden kann. Diese Zuweisung erfolgt in der generierten Methode für das file_coloumn-Attribut. Passenderweise heißt die Variable für den define_method-Aufruf attr.

Die Anpassung ist im Folgenden zu sehen: Wenn ein String statt ein File-Objekt übergeben wird, rufen wir zuerst unsere Methode auf, um die Datei herunterzuladen.

define_method "#{attr}=" do |file|
  if file.is_a?(String) &amp;&amp; !file.blank?
    file = send "#{attr}_download_file", file
  end

  state = send(state_method).assign(file)

  instance_variable_set state_attr, state

  if state.options[:after_upload] and state.just_uploaded?
   state.options[:after_upload].each do |sym|
   self.send sym
  end
end

So weit so trivial. Nun mag man denken, dass das Problem so gut wie gelöst ist, denn eine URL lässt sich Dank uri-open wie ein File-Objekt behandeln. Ruby ist es nämlich herzlich egal, ob ein Objekt die richtige Klasse hat, solange es die gerade benötigten Schnittstellen bereitstellt. Dieses Konzept wird in Ruby duck typing genannt. open-uri stellt die von File-Objekten benutzten Methoden, open und read, für URLs breit.

Das Problem ist aber, dass das von Mongrel erzeugte Objekt eine Instanz von CgiFile ist, die noch einige andere Methoden bereitstellt, die von file_column aufgerufen werden. Hier machen wir uns eine weitere Eigenschaft von Ruby zu nutze. Wir definieren die noch benötigten Methoden für das File-Objekt, indem wir das Bild mit instance_eval zur Laufzeit zwischenspeichern . Unsere dynamisch erzeugte Download-Methode, die ein File-Objekt bereitstellt, das sich wie eine durch einen Upload erzeugt CgiFile verhält, ist nachfolgend zu sehen.

define_method "#{attr}_download_file" do |url|
  file = Tempfile.new('autoki_file_column_download')
  remote_file = URI.parse(url)
  file.write remote_file.read
  file.close
  file = File.new file.path, 'r'
  file.instance_eval "def original_filename
    \"#{url.split('/')[-1]}\"
  end"
  def file.content_type
    "image/#{original_filename.split('.')[-1]}"
  end
  file
end

Wichtig ist es, die erzeugte temporäre Datei zu schließen und erneut als Read-Only zu öffnen, denn die Temporärdatei ist write-only beim anlegen.

Programming Erlang 3: funs, list comprehensions, guards, records, if/case

2007-07-28T00:00:00+00:00

I started reading the book Programming Erlang and while I read I blog along what I find most interesting/important. In the end, this will hopefully be some kind of erlang tutorial, for your pleasure and my reference. :) I also started creating an Erlang Cheat Sheet with all the syntax on it. Here are the previous posts and now comes the third:

funs

Funs are anonymous functions. Funs can be assigned to variables and, more importantly, passed as arguments into functions and also be returned by functions - these would be called higher order functions.

1> Double = fun(X) -> X * 2 end.
2> Double(2).
4

One of these higher order functions is lists:map. It takes a list and a fun and applies that fun to every element of the list, returning again a list with the results. This way we could rewrite our list processing example from the previous post like this:

1> List = [1,2,3].
2> lists:map(fun(X) -> X*2 end, List).
[2,4,6]

The implementation of map is also pretty easy:

map(_, []) -> [];
map(Fun, [H|T]) -> [Fun(H) |  map(Fun, T)].

List comprehension

List comprehensions are a special construct for processing lists. They take an expression and one or more generators and filters.

1> List = [1,2,3].
2> [X*2 || X <- List].
[2,4,6]

X * 2 is the expression. It is applied to every element returned by the generator X <- List which simply pulls all the elements from the list.

A filter would, well, filter elements before handling them over to the expression:

1> List = [1,2,3].
2> [X*2 || X <- List, X > 1].
[4,6]

Here, the filter X > 1 removed the first element from our list.

Guards

Guards are a kind of extension to pattern matching in that they are more powerful. One way to use them is for defining functions:

1> greater_than_5(X) when X > 5 -> true,
1> greater_than_5(X) -> false.
2> greater_than_5(10).
true

Guards can also be used at any place you would use an expression, where they evaluate to either true or false. By the way, true and false are nothing more than atoms, so nothing special about them. Erlang knows about the difference between true and false, though. :)

You can also chain guards together. Separated by semicolons you then have a guard sequence, which ORs all the guards. If you use commas instead, Erlang ANDs all the guards.

1> true; false.
true
2> true, false.
false

The above doesn’t actually work. Line gives a syntax error - I don’t know why. Anyone? (Hey book, tell me.) Anyway, you can also use boolean expresion:

1> true or false.
true

Records

Records are some syntactic sugar above tuples. Now you can name the elements instead of having to remember their positions. First you have to define the record:

-record(book, {
  title,
  price,
  status = new
}).

Here we define a record Book with three fields title, price and status where status is set to "new" by default. (Note: record definitons can’t be typed into the shell but must be included into modules.)

Now we can create a record like this:

1> MyRecord = #book{title = "Programming Erlang", price = 19.99}.

#book{title = "Programming Erlang", price = 19.99, status = new}

To update do this:

2> MyRecord2 = MyRecord#{status = old}.

#book{title = "Programming Erlang", price = 19.99, status = old}

The above code actually creates a copy and leaves the original untouched. To read from a record we again use our precious pattern matching:

3> #book{title = Title} = MyRecord.

4> Title.
"Programming Erlang"

.. or simpler:

5> MyRecord#book.title.

"Programming Erlang"

if/case

Erlang also has the usual if and case clauses:

1> if
     X > 1 ->
       "greater";
     X <= 1 ->
       " less or equal";
   end

There’s no explicit else, instead you just keep adding guards.

The case clause also looks pretty familiar:

1> Language = "Erlang".
2> case Language of
     "Erlang" -> "yay";
     "C#" -> "get out";
   end

The next chapter is about exception handling, then comes “advanced sequential programming” (sounds exciting, doesn’t it? :) )

Programming Erlang: Functional Programming

2007-07-24T00:00:00+00:00

I started reading the book Programming Erlang and while I read I blog along what I find most interesting/important. In the end, this will hopefully be some kind of erlang tutorial, for your pleasure and my reference :) Here’s the first post and now comes the second:

Yesterday I started reading the next chapter called “Sequential Programming”. It actually is about functional programming: functions, anonymous functions, higher order functions, predicates, pattern matching for calling functions a.s.o. - I had to read most of it twice to actually understand it and haven’t even finished the chapter yet. Anyway, let’s start. By the way I started creating an Erlang Cheat Sheet with all the syntax on it. Being used to the very clear Ruby syntax, things started to get a bit confusing in this chapter.

functions

Note: You can not enter function definitions in the erlang shell erl. see the modules section below to learn how to run this code.

A function in erlang is a function as in any (?) language. It receives parameters and returns a value (well, there’s more, read on). The definition looks like this:

area({rectangle, Width, Height}) -> Width * Height;

This defines the function area which takes a tuple containing the atom rectangle (to identify the type of shape) and its with and height. It of course returns the area of the rectangle by multiplying width and height.

parameter matching

We can now define the same function again with different parameters, e.g. like this:

area({circle, Radius}) -> Radius * Radius * 3.14 .

As you might have guessed already, this now computes the area of a circle. The cool thing is, that when you now call area, erlang works out which of the function to call, depending on the given parameters. If you pass a tuple with the atom rectangle as its first element and two values, the first definition matches ({rectangle, Width, Height}). With circle and one value, the second one matches.
Note: the order of the function definitions id important, erlang starts with the first and goes down the list until it finds a match. But this is only important if you call a function where multiple definitions will match your parameters.

List recursions

one thing you can do with this pattern matching is processing lists recursively. for example, you could define a function that takes a list of numbers and returns the negated values. you do this by defining two functions: one takes a list with at least one element in it. this function does the computation with the head (first element) and then calls itself recursively with the tail (rest) of the list. the second definition expects an empty list and simply denotes the end of the recursion. take a look:

negate([Head|Tail]) -> [Head * -1] ++ negate(Tail);
negate([]) -> [].
negate([1,2,3]). % [-1, -2, -3]

The first definition computes the negated value of the head. It then puts the result in a list (the angle brackets around the Head * -1) and concatenates the result of the computation of the tails - which is again a list of values.

Modules

Modules are as usual collections of functions. A module definition starts with -module(my_module).. after that you have to export all the functions you want to use from outside of the module using -export([my_function/1]). where the /1 denotes the arity of the function (the number of aruments).

-module(my_module).
-export([hello_world/0]).
hello_world() -> "hello world".

If you want to run any of the functions above, you have to put them into a module and save it as a .erl file. You can then compile and use the module from the shell:

c(my_module).
my_module:hello_world() % "hello world"

So that was the easy part of this chapter (chapter 3 in the book) but enough already for this post. Stay tuned for more.

Programming Erlang - Chapter 1: Getting Started

2007-07-22T00:00:00+00:00

I just started reading the latest book from the pragmatic programmers: Programming Erlang. In this series of articles, I’m going to summarize and comment on every chapter I read. Enjoy. (The reason I’m doing this in English is that I simply feel like writing some English again and that I hope to reach a larger audience :) )

What is Erlang and why read that book?

Erlang is a programming language which has things like parallel processing and functional programming features built right into the language. Software written in Erlang is therefore supposed to utilize the power of multi core processors found in today’s and future computers much better than software written in … my precious Ruby? omg. Anyway, Erlang seems to be a language with features completely different from what I’ve seen before, so that’s my main reason to start reading about it, and we’re supposed to learn a new new language every year anyway. Let’s get started.

Installation

The Erlang SDK can be installed on all the big 3, i.e. OSX, Linux and Windows. For OSX, just use MacPorts and after a sudo port install erlang we’re set to go.

Getting started

For getting started with Erlang, we can use the Erlang shell. which can be started using the erl command. Now we can start entering Erlang expressions and the shell will evaluate these. (Note: Every line ends with a period)

1> 5 + 10.
15

This is a simple addition and pretty boring so let’s move on.

Variables

Variables are the first thing Erlang handles differently than the “normal” languages. First, all variables have to start with an uppercase latter. Second and more importantly, all variables can only be assigned a value once, so they’re actually constants, after we assign them a value. Example: (The % denotes a comment.)

1> X = 3. % ok
2> X = 4. % not ok, X is already 3

The book states that having immutable variables makes our programs less error prone, because we know exactly which variable has which value - so far soo good. I feel the more important consequence from having only constants is the other advantage mentioned: it makes things much easier when it comes to parallel processing, because you can avoid sharing memory among the processes running in parallel. Makes sense, somehow.

Pattern matching

The other important feature of Erlang mentioned in chapter 1 is this: The “=” does not represent an assignment operator but is a pattern matching operator. This means that when using the “=”, Erlang doesn’t simply assign the value of the right hand side to the variable on the left hand side but it tries to match both sides of the equation.

1> X = 1 + 1. % if X gets assigned 2, the equation is true, so X gets assigned 2

This is of course just a very basic example, the more interesting stuff comes with the different data types. Read on.

Data types

Erlang of course has the basic data types such as integers, floats and strings. (where string are actually lists of integers). In addition we get tuples, which are like C-structs:

1> MyTuple = {"joe", "doe"}.

This creates a tuple with 2 values and stores it into the variable MyTuple. To read the second value of the tuple, we again use the pattern matching operator:

1> {_, LastName} = MyTuple.

Now Erlang tries to match the two tuples and hence puts “doe” into the LastName variable. The underscore represents something like /dev/null, so “joe” is not assigned to anything here.

Another basic type are lists. Lists are defined using square brackets:

1> MyList = [1, 2, 3].
2> [First, _, _] = MyList.
3> First.
1

Another way to retrieve the first value in the list is the

operator:

1> MyList = [1, 2, 3].
2> [First|Rest] = MyList.
3> First.
1
4> Rest.
[2, 3]

It stores the first value in First and the rest of the list in Rest. Sounds like good old recursive list processing, we’ll see.

That’s it for now - I hope to read at least the next chapter over the weekend. Stay tuned for more.

Weltpremiere: Das autoki-Quartett für die eigene Website

2007-07-20T00:00:00+00:00

Tadaaa, weltexklusiv nur für Besucher dieses Blogs, das nagelneue autoki-online-Autoquartett - ab Montag auch für alle anderen, dieses Wochenende nur hier:

Viel Spaß beim Spielen.

Ruby Cookbook Tipps - Arrays, inject und Hashes

2007-07-20T00:00:00+00:00

Thilo hat beim letzten Treffen der Berliner Ruby User Group gleich mal einen Buchgutschein für O’Reilly gewonnen und hat sich das Ruby Cookbook ausgesucht. Gestern ist’s angekommen und ich hab gleich mal geblättert. Hier zwei kleine Schätze, die sich auf (fast) zufällig aufgeschlagenen Seiten fanden:

Array.inject

Ich wusste schon lange, dass es diese Methode irgendwo geben musste, nur gefunden hatte ich sie noch nicht. Eine Methode, mit der man aus den Elementen eines Array einen Wert berechnen kann, z.B. die Summe eines Array voller Zahlen. Inject kann genau das, es ruft für jedes Element des Array einen übergebenen Block auf und gibt das Ergebnis an die Berechnung des nächsten Elements.

a = [1, 2, 3]
a.inject {|sum,i| sum + i}

Es geht auch etwas komplexer, z.B. aus einem Array mit Wertepaaren ein Hash aufbauen:

a = [[1,2], [3,4]]
 a.inject({}) {|hash, el| hash[el.first] = el.last; hash}

Wichtig hierbei ist, dass der Block wieder den Hash zurueck gibt, damit die nächste Operation wieder auf dem Hash läuft. Hash.[]= liefert jedenfalls nicht den Hash, sondern den zugewiesenen Wert.

Array.min / Array.max

Irgendwann brauchten wir mal die kleinere von 2 Zahlen. Ich hab ewig in der Doku von FixNum gesucht und nix gefunden und am Ende die eine Zeile selbst implementiert, unzwar für FixNum. Natürlich hatte ich an der falschen Stelle gesucht, denn um die kleinste von zwei oder mehr Zahlen zu finden, kann man einfach einen Array benutzen:

a = [1,2,3]
a.min # 1

MySql Lost Connection Error in großen Rails Anwendungen

2007-07-18T00:00:00+00:00

Seit kurzen beunruhigt uns die folgende Fehlermeldung: Mysql::Error: Lost connection to MySQL server during query. Sie tritt immer wieder sporadisch ohne erkennbare Ursache auf. Heute Nacht bin ich eher durch Zufall auf eine Erklärung und idealerweise auch auf eine Lösung des Problems gestoßen.

Kurz zusammengefasst: ActiveRecord verwendet pro Model eine Datenbankverbindung. Wenn die Datenbank unter Last steht, kann es passieren, dass die Verbindung nicht schnell genug bereitgestellt wird und der Lost-Connection-Fehler auftritt.

Über das setzen von ActiveRecord::Base.verification_timeout=14400 oder einen Wert niedriger als die MySql Server interactive_timeout -Einstellung in der environment.rb lässt sich der Timeout heraufsetzen. Um dem Problem nachhaltig zu begegnen, hat Tyler Kovacs von zvents das Pluginmysql_retry_lost_connection geschrieben, das versucht, die Verbindung erneut herzustellen, wenn es einen Timeout gab.

Ruby User Group Berlin - Präsentation zu file_column

2007-07-03T00:00:00+00:00

Ich halte am Donnerstag eine kleine Präsentation zum Thema file_column und Widgets/Banner. Wer hin will: www.rug-b.com bzw. einfach 19:30 Uhr bei /i-d media, Ohlauer Strasse 43, Berlin Kreuzberg. Man sieht sich.

Die Slides gibt’s hier.

Rails-Konferenz 2007 - Ersteindruck

2007-06-23T00:00:00+00:00

Gestern fand in Frankfurt (M) die Rails-Konferenz 2007 statt. Es folgt: kommentierte Fotos von der Veranstaltung. Es folgt später: Kommentare zum Inhalt der Konferenz:

Aus irgendeinem Grund hab es ganz viele Java-Bücher und nur ein einziges kleines Rails-Heftchen zu kaufen. Wer wohl dahinter steckt? Eine Geheimoperation der Java-Community, die Ruby kaputt machen wollen, indem sie die Entwickler wieder zurück in die alte Welt holen?

Begeleitend zu den Büchern gab es auch ein paar Zeitschriften. Neben dem Java-Magazin, dem Eclipse-Magazin und dem PHP-Magazin auch das hier abgebildete t3n - Coverstory: PHP5 mit MySQL5 (!) Was wollen die alle hier?

Es gab ganz viel Kaffe und lustige Snacks in Form von Gläsern, in denen eine Schicht Müsli von einer Schicht Joghurt bedeckt wurde. Leider zu wenige Löffel, damit man auch alle essen konnte.

Hübsche Blumenbilder an den Wänden. Rubine hätten besser gepasst, aber das Hotel hatte vermutlich gerade keine zum Abmalen da. Hoffentlich wird das auf der RailsConf im September besser.

Es gab zwei Räume mit vielen Lampen an den Decken, viel zu kleinen Beamern (also die von den Beamern angestrahlten Flächen waren zu klein, die Beamer waren okay, so von der Größe her), Stühlen im Raum und Steckdosen unter im Boden versenkten Deckeln. Das mit den Steckdosen hätte mir ja auch mal jemand früher sagen können, immerhin saß ich zwischendurch eine halbe Stunde ratlos mit leerem (Laptop-)Akku rum.

Neben vielen Stühlen (siehe voriges Bild) gab’s auch viele kleine Flaschen mit warmer Coke und Saft und Wasser drin. Weiß man in Frankfurt noch nicht, dass man jetzt Bionade (<= Links klicken => ) und Club Mate trinkt? Vor allem zum Wachbleiben wäre ich für letzeres sehr dankbar gewesen.

Und dann gab es da noch Menschen mit lustigen Hemden, die wahrscheinlich beim Bund waren. (Hände hinterm Rücken)

Leute kamen extra aus der Schweiz angereist, um uns lustige Slides mit Monstern drauf zu zeigen. (Vortrag: Offline arbeiten. Slingshot, AIR & Co. später mehr dazu)

Echten Code gab’s auch zu sehen. Wer findet den Fehler in 10 Sekunden? Hättense mal getestet…

Andere haben auch Fotos gemacht, dafür aber gleich richtig dicke Dinger mitgebracht. Naja, war auch echt dunkel da, nix für zittrige Hände.

Manche haben ganz viele Wörter mitgebracht, falls ihnen mal nichts zu sagen einfallen würde.

Noch andere waren da schon wesentlich weiter und haben selbst ausgedachte Wörter gezeigt.

Gefilmt hamse das Ganze auch. Mal sehen, wann das dann im Netz zu sehen sein wird. Laut Aussage des Filmers muss er erst noch seine Web-2.0-Vidoes-Slides-Synchroniserungs-Abspiel-Website fertig programmieren. Na dann. (gibt’s doch schon)

Rails wird verkauft.

2007-06-21T00:00:00+00:00

Auf Grund ihres überragenden Erfolgs haben 37signals beschlossen, von jetzt an nur noch am Strand rumzuliegen. Daher verkaufen sie ihr inzwischen von Millionen benutztes Web-Framework Ruby on Rails. Das ganze läuft bereits - natürlich über Ebay, jeder kann mitbieten; siehe Screenshot unten.

Text-im-Bild mit Rails, file_column und RMagick Teil 2/2

2007-06-16T00:00:00+00:00

Im ersten Teil habe ich gezeigt, wie man mit file_column in Rails automatisch andere Bilder zu einem Bild hinzufügt, um z.B. Rahmen oder Hintergründe hinzuzufügen.

In diesem Teil geht es darum, auch Text in ein Bild schreiben zu können: Hierbei ergeben sich zwei Probleme. Zum einen muss der Text vom Model in die file_column-Klassen gelangen, zum anderen muss er dort via RMagick in das Bild gerendert werden. Das Ergebnis könnte dann z.B. so aussehen (autoki-Quartettkarte):

Fangen wir von hinten an, RMagick.

Text in RMagick rendern

Die Beschreibung des Texts im Model soll folgendermaßen aussehen:

file_column :photo, :magick => { :versions =>
  {
    :badge => {:name => "badge", :text =>
      {
        :text => 'hello world',
        : x => 10,
        :y => 241,
        :color => '#363636',
        :size => 9
      }
	}
}}

Hierbei soll an die Koordinaten 10/241 der text hello world in der Schriftgröße 9 und in Dunkelgrau gerendert werden.

Um den Text in unser Bild rendern zu können, erweitern wir wieder die transform_image-Methode in der Datei magick_file_column.rb des file_column-Plugins:

if img_options[:composite]
...
end
if img_options[:text]
  if img_options[:text].is_a?(Array)
    img_options[:text].each do |text_options|
      img = add_text text_options, img
    end
  else
    img = add_text img_options[:text], img
  end
end

def add_text(text_options, img)
  text = text_options[:text]
  unless text.blank?
    img = img.annotate ::Magick::Draw.new, 0, 0, text_options[: x], text_options[:y], text do
      self.pointsize = text_options[:size] || 11
      self.font_weight = text_options[:weight] || 400
      self.fill = text_options[:color] || 'black'
    end
  end
  img
end

Das war’s eigentlich auch schon - jetzt haben wir ein hello world in unserem Bild. Teil 2…

Werte aus dem Model auf's Bild

Neben statischem text wollen wir auch dynamische Inhalte in unsere Bilder rendern. Wie auf der Autokarte oben zu sehen, rendern wir hier den Namen und die Leistungsdaten des Autos, die bei autoki aus der Datenbank kommen.

Leider gibt es dabei ein kleines Problem: Die Definition der file_column-Bildgrößen hängen nicht an der Model-Instanz, sondern an der Model-Klasse, d.h. an der Stelle, an der wir die Texte definieren, stehen die Daten aus der Datenbank noch gar nicht zur Verfügung. Folgendes funktioniert also nicht.

file_column :photo, :magick => { :versions =>
  {
    :badge => {:name => "badge", :text =>
      {
        :text => "#{self.top_speed} km/h",
        :x => 10,
        :y => 241,
      }
	}
}}

Stattdessen müssen wir einen kleinen Umweg gehen, wir verwenden Procs, die erst zur Laufzeit der Anwendung den Text generieren:

file_column :photo, :magick => { :versions =>
  {
    :badge => {:name => "badge", :text =>
      {
        :text => Proc.new {|model| "#{model.top_speed} km/h"},
        :x => 10,
        :y => 241,
      }
	}
}}

Damit das ganze funktioniert, muss file_column noch etwas erweitert werden. Zum einen muss die add_text-Methode die Procs aufrufen:

def add_text(text_options, img)
  text = text_options[:text]
  text = text.call(@object) if text.is_a?(Proc)
  unless text.blank?
    img = img.annotate ::Magick::Draw.new, 0, 0, text_options[:x], text_options[:y], text do
      self.pointsize = text_options[:size] || 11
      self.font_weight = text_options[:weight] || 400
      self.fill = text_options[:color] || 'black'
    end
  end
  img
end

Zum anderen muss die Model-Instanz file_column zur Verfügung gestellt werden. Dazu erweitern wir die Methoden transform_with_magick und create_magick_version_if_needed um eine Parameter object

def transform_with_magick(object)
  @object = object
  ..
end

def create_magick_version_if_needed(version, object)
  @object = object
  ..
end

Zuletzt muss noch der Aufruf von create_magick_version_if_needed in der Methode url_for_image_column in der Datei file_column_helper.rb angepasst werden:

def url_for_image_column(object, method, options=nil)
  ..
  if options
    subdir = object.send("#{method}_state").create_magick_version_if_needed(options, object)
  end
  ..
end

Das war’s. Wenn ich nichts vergessen habe, gibt’s jetzt dynamische Textinhalte in Bildern mit file_column. In einem nächsten Teil dann: Bilderkompression, Amazon S3 Backups.

Wie sehen uns beim Autoquartett.

P.S. Am 5.7. werde ich das ganze bei der Berlin Ruby User Group noch einmal live zeigen, kommt vorbei.

P.P.S. ich schreibe das hier gerade mit der neuen Beta von Safari 3. Vom User vergrößerbare Textareas - Gold wert mit Wordpress.

MFG Rails-Konferenz

2007-06-16T00:00:00+00:00

Wir fahren am Donnerstag Abend mit Auto(ki) zur Rails-Konferenz in Frankfurt/Main. Es sind noch 2-3 Plätze frei im Wagen, einfach Email an alex[at]upstream[minus]berlin.com

Vertikales Webdesign

2007-06-15T00:00:00+00:00

Ich poste das hier eigentlich nur, damit ich dir URL nicht verliere … gefunden bei 37signals: type that keeps the beat - wie man sich ein Raster aus Zeilenhöhe und -abstand baut und alle Seitenelemente an diesem Raster ausrichtet. Schon hat man ein stimmige(re)s Design. Dieses Bild (aus dem 37signal-Artikel) zeigt das Ganze eigentlich sehr schön:

In den Kommentaren gibt’s dann auch gleich mal ein CSS-Beispiel:

body{
  font-size: 0.75em;
  line-height: 1.25;
}

h1 {
  font-size: 2em;
  line-height: 1.25;
  margin: 1.875em 0 0.625em 0;
}

h2 {
  font-size: 1.5em;
  line-height: 1.6667;
  margin: 0.8333em 0 0.8334em 0;
}

p {
  margin-bottom: 1.25em;
}

Performance Benchmarking in Rails

2007-06-10T00:00:00+00:00

Na? Ist die Userprofil-Seite mal wieder zu langsam, nachdem beim letzten Release durch geschicktes anordnen und ausblenden noch einmal 3 Bilder und Informationshäppchen mehr draufgepackt werden konnten? Und jetzt will’s wieder keiner gewesen sein und niemand hat eine Ahnung, wo die 500ms denn nun genau hingehen? Für das profilen z.B. einer Rails-Seite stehen dem geneigten Entwickler mehrere Tools zur Verfügung.

logfiles

Erster Anlaufpunkt sind die logfiles in RAILS_ROOT/log - je nach logging-level stehen dort die Gesamtzeit für’s prozessieren der Seite, noch einmal unterschieden in Datenbank- und Render-Zeit (production environment) sowie zusätzlich die Zeiten, die für jeden Datenbank-Query und das rendern von Partials draufgehen.

benchmark do .. end

Wer keine Lust hat, sich im Logfile die entscheidenen Zeiten herauszusuchen, kann auf die von Rails angebotene benachmark-Methode zurückgreifen. Dieser wird einfach ein Block mit dem zu profilenden Code übergeben und schon steht die laufzeit im Logfile. Funktioniert auch im Model.

<%- benachmark 'user details' do -%>
  <%= render_lots_o_stuff %>
<%- end -%>

RubyProf

Wer es ganz genau wissen will, kann RubyProf verwenden. Wie es der Name vermuten lässt, handelt es sich dabei um einen “richtigen” Profiler - man bekommt also nicht nur die Gesamtlaufzeit der durchgemessenen Codes sondern das ganze aufgeschlüsselt nach Methoden, den ganzen Ruby-Stack herunter. Um RubyProf zu installieren, reicht erwartunsggemäß ein einfaches gem install ruby-prof. Der eigentliche Profile-Aufruf erfolgt wieder inline und könnte so aussehen:

<% result = RubyProf.profile do %>
  		  <%= render :partial => 'shared/pinboard', :object => @user, :locals => {:posts_comments => @posts_comments}%>
  		<% end %>
  		<%- printer = RubyProf::GraphHtmlPrinter.new(result)
  		file = File.new('/Users/alex/Desktop/profile.html', 'w')
      printer.print(file, 0)
      file.close -%>

Als Belohnung liegt dann nach kurzer Zeit eine riesige HTML-Tabelle auf dem Desktop, mit den Laufzeiten aller Methodenaufrufe innerhalb des gewählten Blocks, und es stellt sich heraus, dass irgendjemand (wer war das nur wieder?^^) einen fiesen after_initialize-Callback an den User gebaut hat, der für jeden geladenen User ein paar Milisekunden Zeit verbrät…. after_initialize ist gefährlich, don’t do this at home ;)

gruenderszene.de bei autoki.com

2007-06-10T00:00:00+00:00

Jan von gruenderszene.de war letzte Woche bei uns und hat ein Interview über autoki.com gemacht.

Link: sevenload.com

Wir haben übrigens noch viel mehr Features außer den Gruppen… :)

Mountain West Ruby Conference Videos

2007-06-01T00:00:00+00:00

confreaks.com zeichnen Vorträge auf Konferenzen auf Video auf und stellen sie anschließend ins Netz - praktischerweise so, dass man gleichzeitig sowohl Slides als auch Vortragende sieht. Und der Flash-Videoplayer hat auch noch einen schicken Vollbildmodus. Und als erste Konferenz haben sie sich die Mountain West Ruby Conference ausgesucht. Hab mir gerade die Rails Code Review Session angesehen - sehr schöne Tipps, saubereren Code zu schreiben. Fast schon ein Klassiker zum Beispiel, statt:

if a.nil?
  b = 'blah'
else
  b = a
end

einfach:

b = a || 'blah'

Hi, I'm a MacRuby on Rails

2007-05-20T00:00:00+00:00

Die großartigen Apple-Ads kennt man ja inzwischen, jetzt gibt’s auch die Rails-Variante. Toll:

</param></param></embed> </param></param></embed> </param></param></embed>

TextMate tuning

2007-05-11T00:00:00+00:00

Über die Monate ist TextMate, das wir für die Entwicklung von autoki verwenden, ziemlich langsam geworden. Das betraf vor allem drei Bereiche: projektweite Suche, Dateien finden mit Apfel-T sowie von einer anderen Anwendung zu TextMate wechseln.

Ich war schon kurz davor, dem TextMate-Programmierer Geld zu bieten, damit er das Ding schneller macht. Vorher noch mal kurz gegoogelt, ob es noch andere Leidtragende gibt, und siehe da: einfach mal das 600MB große Test-log in log/test.log gelöscht und TextMate fliegt wieder. :)

Bild-im-Bild und Text-im-Bild mit Rails, file_column und RMagick

2007-05-09T00:00:00+00:00

Update: Teil 2 ist fertig.

Gerade haben wir bei autoki Badges eingeführt - kleine Banner, die sich auf Blogs, Foren etc. einbinden lassen, und auf denen neben dem jeweiligen Auto des Mitglieds die URL zu seinem oder ihrem Profil zu sehen ist:

Im folgenden Posting werde ich erklären, wie wir das ganze umgesetzt haben. Ziel war es, einfach einzubindende Grafiken zu generieren. Eine Alternative wäre Flash gewesen, jedoch lässt sich Flash an vielen Stellen (vor allem als Signatur in Foren) nicht einbinden.

Wir brauchten also eine Lösung, die uns für jedes Auto mehrere Banner mit jeweils einem Foto und Text darauf automatisch erstellt. Für die Skalierung der Bilder benutzen wir das file_column-Plugin für Rails, das auf RMagick , den Ruby-Bindings für ImageMagick, aufsetzt,. Damit lassen sich sehr einfach skalierte und beschnittene Versionen einer Grafik erzeugen, indem man in einem Rails-Model, nennen wir es mal Auto, ein Attribut als file_column-Attribute markiert:

class Auto
  file_column :photo
end

Um die verschiedenen Bildgrößen zu erzeugen, genügt es, mittels :version die gewünschten Größen anzugeben:

file_column :photo, :magick => { :versions =>
  {
	:large => "480x480",
	:thumb => "238x"
  }
}

Dieser Code erzeugt zwei Versionen: :large passt unter Beibehaltung der Seitenverhältnisse in einen Rahmen von 480x480 Pixeln, :thumb hat eine Breite von 238 Pixeln und eine daran angepaste Höhe.

Alles ganz schön und nett, aber um damit Banner zu produzieren, kommen wir nicht umhin, file_column ein wenig zu ~~hacken~~ erweitern. Unser Problem lässt sich in 3 Schritte aufteilen:

das Autobild herunterskalieren, so dass es auf den Banner passt,
das verkleinerte Bild in ein Banner-Template-Bild kopieren,
die URL des Autoprofils in das Bild schreiben

Schritt eins können wir bereits lösen:

file_column :photo, :magick => { :versions =>
  {
	:badge_1_online => {:crop => "52:46", :size => "52x46!", :name => "badge_1_online"}
  }
}

Mit Hilfe von :crop wird das Bild zusätzlich beschnitten, damit es garantiert 52x46 Pixel groß ist.

Um nun das Bild in das Banner-Template zu platzieren, verwenden wir folgende Syntax:

file_column :photo, :magick => { :versions =>
  {
	:badge_1_online => {:crop => "52:46", :size => "52x46!", :name => "badge_1_online",
  :composite => {:x => 3, :y => 3, :file => 'badge_1.gif'}
  }
}

Soll heißen: Das Autobild wird an der Stelle 3/2 in das Bild badge_1.gif kopiert. Wie schon erwähnt, funktioniert das so erstmal nicht. Nachdem wir die Methode transform_image in der Datei magick_file_column.rb des file_column-Plugins folgendermaßen angepasst haben, aber schon:

if img_options[:size]
 ...
end
if img_options[:composite]
					composite = ::Magick::Image.read("#{RAILS_ROOT}/public/composites/#{img_options[:composite][:file]}")[0]
					img = composite.composite!(img, img_options[:composite][:x]+1, img_options[:composite][:y]+1, ::Magick::OverCompositeOp)
				end

Diese Modifikation setzt voraus, dass unser Banner-Template badge_1.gif in public/composites/liegt. Mittels der RMagick-composite-Methode wird das Auto mit zwei Zeilen Code in den Banner kopiert.

Sieht doch schonmal ganz nett aus oder? Und wie das mit dem Text funktioniert, erkläre ich im nächten Post :) Das ist nämlich etwas komplizierter….

Mit grauen Schwaden gegen den Spam

2007-05-02T00:00:00+00:00

Eigentlich hatte ich mich schon damit abgefunden: zuletzt erreichten mich taeglich ca. 150 Spam-Emails pro Tag, in denen mir die bekannten Aktienpakete, Pillen und anderen Schätze versprochen wurden. Neulich musste ich dann für einen Kunden ein paar Emailaccounts bei webmail.us (Email-Provider) einrichten. Also schnell Adressen eingetragen und noch kurz eine Testmail versandt und … kam nicht an. Also mal kurz ins Logfile des eigenen Mailserver geschaut und siehe da: temporarily rejected RCPT … greylisted (see http://en.wikipedia.org/wiki/Greylisting).

Greylisted? Black lists, white lists, klar. Aber grau? Schnell nachgesehen und herausgefunden: method of defending electronic mail users against e-mail spam. A mail transfer agent which uses greylisting will “temporarily reject” any email from a sender it does not recognize. If the mail is legitimate, the originating server will try again to send it later, at which time the destination will accept it. If the mail is from a spammer, it will probably not be retried - coole sache, nur … Mailserver umkonfigurieren? Au weia. ich bin froh, dass der überhaupt funktioniert. Glücklicherweise gibt’s für Exim auf Debian ein Paket namens greylistd. Einfach apt-get install greylistd und anschließend ein greylistd-setup-exim4 add und voila - greylisting a la Debian. Und das beste, ich habe in den letzten zweit Tagen tatsächlich nur eine einzige (!) Spam-Email erhalten.

Wenn mir jetzt noch jemand erklärt, wie ich diese Zeitverzögerung beim senden von Emails von dynamisch verteilten IP-Adressen aus wegbekomme….

Perian: DivX & Co. für Quicktime

2007-04-19T00:00:00+00:00

Perian ist eine freie Quicktime-Komponente für OSX, die Formate wie DivX, XviD abspielen kann. Zusammen mit Flip4Mac kann Quicktime somit nun endlich die meisten Filme und Formate abspielen. Das ging zwar bisher mit VLC auch, aber durch Quicktime ist das ganze doch wesentlich besser integriert: der Finder hängt sich beim Versuch, einen DivX-Film in der Vorschau anzuzeigen, nicht mehr auf und endlich kann man die ganzen Streifen auch mit Fernbedienung und Front Row genießen.

Spaß mit Rails

2007-04-13T00:00:00+00:00

Einer der großen Vorteile des Pair Programming ist das Vorhandensein einer zweiten Person, mit der man zusammen über ~~Blödsinn~~ Geistreiches lachen kann. Wie z.B., wenn man zum zehnten mal pust statt puts schreibt, um irgendwo einen Wert auf der Konsole auszugeben.

Fall zwei: Mit rake die Tests laufen lassen. “Na dann rakeln wir uns mal wieder”, “Rake dich nicht so auf”, oder “Da raket sich was.”

Wir amüsieren uns jedenfalls köstlich beim programmieren :)

re:publica: mister wong und powerpoint-karaoke (videos!, updated)

2007-04-13T00:00:00+00:00

prolog

muesste es nicht langsam mal keynote-karaoke heissen? wer macht denn heute noch powerpoint? der herr lobo hatte jedenfalls wie sehr viele andere auch einen dieser apfel-rechner dabei.

mister wong

ich wusste nur, dass es sich um einen sponsoren der re:publica und die deutsche variante von del.icio.us handelt. erwartet habe ich eine werbeveranstaltung für <a href=http://mister-wong.de”>mister wong</a>, und im endeffekt war es auch eine, aber so schlimm war das gar nicht. ich habe naemlich zum ersten mal erfahren, dass man social-bookmarking-dienste als suchmaschine benutzen kann, da ja die häufigkeit der “verbookmarkung” und das vertaggen zusammen so eine art pagerank ergeben. warum bin ich da nie drauf gekommen? seit jahren pflege ich meine bookmarks bei del.icio.us, aber auf die idee, das als suchmaschine zu benutzen bin ich aus unerfindlichen gründen nicht gekommen.

schön (an mister wong) fand ich, dass man seine bookmarks aus anderen diensten wie eben del.icio.us sehr einfach und auf einen schwubs importieren kann. und auch exportieren, wenn dann das next big thing im social bookmarking kommt ;)

auf meine frage, warum denn nun mister wong besser als del.icio.us ist, und ich da nun hin soll, gab’s leider keine besonders ueberzeugende antwort, ausser der, dass es da mehr deutschen content gaebe - na dann.

eher amuesant fand ich die wiederholten einwuerfe aus den hinteren reihen (billigere plaetze?), dass dieser komische chinese als logo einen ernsthaften einsatz, z.b. in unternehmen, ja wohl komplet ausschliessen wuerde, und dass die zielgruppe ja wohl eher teenies seien. und ueberhaupt, haette man nicht einen deutschen oder franzosen nehmen koennen, und kommen die chinesen mit mister-wong.cn ueberhaupt klar, so von wegen da so ein landsmann in der ecke? antwort: in china gibt’s gar keine wongs, hoechstens wangs. schoen.

erwähnung fand noch visual orgasm pixel town - hach wie niedlich, huebsch anzusehende pixelhaeuser, die man auf zu erwerbenden grundstuecken bauen kann. gab’s zwar irgendwo schonmal aber egal. machen wir auch mit, fuer unsere reisecommunity. ein strandhaus oder sowas, mal sehen. (ideen anyone?)

powerpoint keynote karaoke

da gibt’s nicht viel zu sagen, ausser, dass die vortraege teilweise recht witzig waren und das publikum zu begeisterungsstuermen hinreissen konnten, aber seht selbst.. (ach ja, wer’s nicht weiss: beim keynote karaoke muss jemand (aus dem publikum) zu einem ihr vorher nicht bekannten thema eine prasentation (mit vorgegebenen beamer-slides) halten).

</embed>

republica powerpoint karaoke: Stilgeschichte des Möbels on Vimeo

Update: hier sind die anderen zwei videos:

</embed>

re:publica powerpoint karaoke: Ã–kosystem Pansen on Vimeo

</embed>

republica powerpoint karaoke: dorst on Vimeo

re:publica, Guten Morgen im Metaverse

2007-04-13T00:00:00+00:00

Wir sind wieder am Start. Jetzt gleich starten wir mit einen Talk zum Thema, Metaversen (Kurzerklärung: Immersive 3D Welten wie z.B. Second Life).

Der Vortrag startet mit einem Screenshott des ersten grafischen Metaversum: Habitat(Lucas Filmgames). Dies soll zeigen das Metaversen ihren Ursprung in Computerspielen hat. Denn werden die in metaversen verwendeten Begriffe erklärt, wie, woher der Begriff Avatar kommt. Die Geschichte der Metaversen geht wie gesagt bei den Spielen los, M.U.D. war der erste Schritt in diese Richtung, dann folgte ein weiterer Meilenstein mit TinyMUD. Dann schön anschaulich, ein noch laufendes MUD (Aardwolf).

Der nächste Schritt war Habitat. 3D zog 1995 mit Worlds Chat ein. Es folgt eine kleine Zeitreise in Active Worlds an einen Ort den es vor 12 Jahren gab und dann wie diese Welt sich bis heute verändert hat. (Sehr nett.)

Der grosse Komerz (neben Second Life) folgt mit Entropia Universe, ein Spiel mit einem Ausgefeilten Wirtschaftssystem. Ohne Kreditkarte kommt man hier nicht weit. Waffen kosten Geld und verfallen. Kosten 0,5 - 1,5 $ pro Stunde. Leider gibts Problem beim Anmelden für eine Live Demo. Das provozierte gleich den Kommentar aus dem Publikum: “Mit Open ID wäre das nicht passiert”. Es wird auf das Problem der Geldwäschemöglichkeiten bei SecondLife eingegangen, Entropia steuert dem mit einem bankähnlichem Cash Card System entgegen, um mehr Tranzparens zu schaffen.

Die Diskusion verlagtert sich auf Probleme des Eigentums und wie lassen sich die online Besitztümer sichern? Und eine viel grundlegende Frage wurde eingeworfen, was hat man von virtuellen Besitz. Denn gibts ein interessanten Einblick in die Hochzeitsplanner Industrie in Second Life. Diese Industrie ist recht beliebt, da niemand den Gestalltern reinredet und sämtliche Rechte an ihren Werken halten. Die Meinung des Vortragenden ist, dass alles was von Firmen in Second Life betrieben wird nicht nachhaltig ist, sondern nur zur Erprobung neuer Marketingstrategien dient. Die Kosten für die eigene Online Welt können so gespart werden.

Dann machen wir den letzten Schritt und schauen in die Zukunft. Hier für steht Croquet ein offenes und dezentralles Mehrweltensystem. Wir sehen Nutzungsbeispiele für Bildung, Unterhaltung und auch Militär. Am Ende fällt noch der Begriff Web 3.0, das Web als immersives 3D Erlebniss. Ein interessanter Blog zum Thema Metaversen gibts von Markus Breuer. Ein neues Metaverse mit dem Namen Smeet mit Schwerpunkt auf Kommunikation wird noch aus dem Publikum eingeworfen.

Jetzt noch schnell dass Posting überarbeiten, dann gehts weiter.

re:publica, wir sind auch schon da

2007-04-11T00:00:00+00:00

So, mittlerweile sind wir auch auf der re:publica eingetroffen. Da jetzt eine kleine Pause entstanden ist, nutze ich diese gleich einmal. Es sind doch viele bekannt Gesichter vom barcamp 06 zu sehen. Aber mehr Frauen.

Der gerade gehörte Workshop zum Thema Interaction Design Patterns for Communities, war dann doch etwas unkonkreter als ich gehofft habe. Aber immerhin hat man jetzt einige Anhaltspunkt zum nachsurfen (hier, hier und hier).

Noch ein kleiner Wermutstropfen. Den Talk zu Blogmythen hab ich leider verpasst. Der war schon heute Vormittag. Aber genug andere werden wohl die Nachricht in die Welt hinaus bloggen: “Blogger nicht gleich Nerds, Blogs sind relevant und Blogs sind keine Gegenöffentlichkeit”

re:publica (wir sind auch) generation mash-up*

2007-04-11T00:00:00+00:00

So mal ein kleiner Live-Bog-Versuch (mit Nachbearbeitung). Der Talk zum Thema Mash-Ups mit Tim: Moderation, Goerg: Startup, Bernhard: Free Software, Joachim als Google. Erstmal die Anfangsfragen: Wer kennt/nutzt Mash-Ups. Die meisten wissen hier, was Mash-Ups sind, aber weniger als die Hälfte wollen und haben schon Mash-Ups gemacht.

Georg erzählt, was Mash-Ups für ihn sind und bedeuten. Tim meint, die Blogosphähre ist ein Mash-Up (Interessante These). Google entwickelt die APIs, um seine Softwareentwicklung in den Griff zu bekommen. [Dann etwas Microsoft-Vista-Bashing: Softwaretechnik von vor 20 Jahren].

Tim fragt: Warum macht Google das Material frei? Bernhard sieht Mash-Ups als Inhalte und Arbeitserleichterungen und aber auch als Bindung/Beeinflussung der Nutzer an die Inhalteanbieter. [Es folgt eine Diskussion darüber, ob Kathrina-Aufnahmen von Google zensiert wurden (Wenn doch die Welt nur so einfach wäre, wie bei dieser Diskussion dargestellt)].

Jetzt mal wieder Fakten von Georg. Es werden erstmal die Anbieter, die da sind, für zusätzliche Dienste genutzt, z.B. Buchsuche über die Amazon API. Flickr wurde durch offene APIs lebendig. Auf jeden Fall bieten APIs Mehrwert für eine Plattform. Einwurf von Tim: Nicht mehr hacken der Communities sondern einfach APIs nutzen.

“Was hat die Freie-Software-Szene von Mash-Ups?” geht als Frage an Bernhard. Fazit: Kommerzielle Interessen laufen den freien Inhalten entgegen. Oder umgekehrt, z.B. genetischer Code ist frei, wird aber kommerziell besser verwertet. Google sagt dazu: Wir sind keine Pfadfinder. Nichts ist frei: Nutzer zahlt nicht, sondern durch Werbung wird bezahlt. [Interessant: So im Beispiel genant. Werbung in den Karten durch Pins.] Georg meint dazu: APIs sind auch Strategien zur Markenbildung. Bernd wird ideologisch: Er bezahlt indirekt mehr, wenn Google frei ist. [Nette Ablenkung durch die SMS-Blasen auf der Leinwand] Joachim/Google glaubt, dass Mash-Ups den Desktop ersetzen werden (Wollen wir das?). Zur Frage von Vertrauen kommt eine Analogie zu Bank, die unser Geld verwaltet. Tim kennt Leute, die Gold kaufen und lagern (das glaube ich).

[Jemand neben mir holt die Diskussion zu den Faken zurück, mit einer Frage an Georg]. Welche APIs muss man kennen und welche werden kommen? Wichtig ist ihm die S3 API, um IT auszulagern. Allgemein ermöglichen APIs, dass andere die Nutzungsmöglichkeiten einer Anwendung erweitern. Es fehlt ihm noch viel Funktionalität für’s Handy. Tim gibt zu Bedenken, dass die Abhängigkeiten durch andere Services zunimmt. Dadurch stellt sich in einer kleinen Diskussion heraus, dass durch Service-Level-Agreements Geld verdient werden will.

[Aus dem Publikum, zurück zur Ideologie]. Banken werden durch öffentliche Behörden überwacht und kontrolliert, Google aber nicht. [Dann wieder zurück zu den Fakten.] Google möchte immer vorne dabei sein, aber de facto Standards bleiben. [Es folgte eine Interessante Formulierung:] Alte Google Web API wird eingestellt, da nicht so genutzt, wie Google sich das gedacht hat. Es folgen Details zum Umgang mit den API Keys. Lizenzgeber wollen wissen, wo das Kartenmaterial genutzt wird, darum Google Maps nicht in Intranets verwendbar. Das Statement ändert sich in: Kartenmaterial darf nicht anonym rausgegeben werden.

Dann stellt Bernhard einige Fragen, um etwas vom Google-Thema wegzukommen: Wie einfach kann man denn nun real zwischen APIs wechseln? Was sind gute Mash-Ups? Wie werden die Mash-Ups finanziert? Lassen sich diese Standardisieren? Georg differenziert zwischen APIs und Formaten, die diese verwenden. Es driftet doch wieder zu Google. Google möchte transportierbaren “elektronischen Schatten” mal anbieten, (nach eigenen Regeln natürlich). Blick in die Zukunft von Google. Programmiersprache für das Web, um Daten und Mash-Ups selbst zu bewegen.

Dann sind die 90 Minuten um. Letzte Worte. Georg freut sich auf Open ID etc. Bernd möchte Antworten. Joachim promoted die bereits ausgebuchten Google Developer Days (Gut das wir heute früh schon gebucht haben).

re:publica: Interaction Design Patterns

2007-04-11T00:00:00+00:00

Nachdem Thilo sich gleich negativ zum dem Vortrag über Interaction Design Patterns (Folien sowie weiteres sollen demnächst online sein) geäußert hat, will ich nochmal kurz meinen Senf dazugeben. (Was ist eigentlich Interaction Design? - ) Ja, es ging sehr viel um Grundlagen und sehr wenig um praktische Beispiele oder konkrete Tips, aber da ich noch nie davon gehört (oder es wieder vergessen) habe (wer weiß das schon so genau? :)), war ich mit den gegebenen Anstößen eigentlich schon ganz zufrieden.

Nochmal kurz zum Inhalt: Die Idee der Interaction Design Patterns wird, wie auch bei den Programmier-Patterns auf die Architekten zurückgeführt, die sich schon Ende der 1960er damit beschäftigt haben. Der Grundegedanke ist, für immer wieder auftretende Probleme eine Lösung zu finden, die sich in der Anwendung bewährt hat, und diese zu dokumentieren. Trifft nun jemand auf so ein Problem, muss er theoretisch nur in einer Art Pattern-Library das passende Pattern finden und anwenden. Theoretisch deshalb, weil es zum einen gerade bei den Interaction Designern noch keine richtig umfangreichen Bibliotheken gibt, zum anderen, weil diese Pattern sehr abstrakt sind und erst auf die eigene Lösung angepasst werden müssen.

Ein Beispiel war, dass man, um die Registrierung/Login von Usern einer Community zu ermöglichen, einen Registrierungs-Link möglichst direkt unter den Login-Button setzt; ein anderes, dass man, will man Content von Users bewerten lassen, dazu das von Amazon bekannte 5-Sterne System verwenden solle.

Im Vortrag wurden 3 Links zu bestehenden Pattern-Libraries genannt. Gemerkt habe ich mir nur die Yahoo-Library. Die anderen sollten dann hoffentlich bald in den Folien zu finden sein. (Thilo hat sie, sehe ich gerade.)

Nach dem Vortrag hat mir Colin noch eine Flickr-Collection anvertraut: dort legt ein gewisser factoryjoe jedes mal, wenn er ein Problem besonders gut gelöst sieht, einen Screenshot davon ab. So eine Art visuelle Inspirations-Library also. Sehr schick.

Es wird mitgelauscht.

2007-04-11T00:00:00+00:00

Gerade gabs hier ein kleinen Hinweis von Tim, von wegen mitlesen von POP3-Passwörtern und Logins. Mal schauen, ob ich auch Mail von mir selbst bekomme.

We are hiring

2007-04-05T00:00:00+00:00

Wir suchen übrigens noch Verstärkung für’s autoki-Team:

Zwei Rails-Entwickler, Freelancer oder Angestellte, Teil- oder Vollzeit. Pflicht: Kenntnisse in Rails und verwandten Technologien (SQL, Javascript, HTML usw.), wohnen in Berlin, Lust auf TDD/XP. Kür: Erfahrungen mit TDD, XP, umfangreiche Rails-Kenntnisse, Javascript-Guru, Linux, OSX
Einen Produktmanager. Pflicht: AuskennerIn im Web

Wir bieten: Die ultimative Erfahrung beim Aufbau einer WebZwoNull-Seite von (fast) ganz am Anfang bis zur Skalierung auf Millionen von Userm; Fame, Geld, ein tolles Team, Selbstbestimmung, ein Buero an der Sonne uvm.

Bei Interesse: Mail an kontakt(at)autoki(punkt)com.

autoki launch

2007-04-04T00:00:00+00:00

Nach einigen Monaten Entwicklungszeit ist Autoki nun offiziell online gegangen. dazu Ami im nagelneuen Autoki-Blog:

"autoki ist der Treffpunkt für Autos mit Geschichte. Eine Plattform, bei der man interessante Autos vorstellen, anschauen, diskutieren und bewerten kann. Ein weiteres Anliegen ist es, dass wir Autobesitzer vernetzen wollen und wirklich Wert auf Autos mit Geschichte legen"

Na denn, wir sehen uns beim Autoquartett.

Mingle: Das agile Projektmanagement-Tool, auf das wir alle seit Jahren gewartet haben?

2007-04-01T00:00:00+00:00

Ich benutze seit Jahren Trac für Projektverwaltung und Dokumentation. Bugs und Requirements zu den Tickets, Milestones anlegen, ein paar custom Reports, alles andere ins Wiki. Alles ganz schön und gut, aber das Gefühl, dass da noch mehr geht, wollte nie so richtig weichen.

Ein Tool, das agile Prozesse direkt unterstützt oder gar .. quasi einfordert. Das mir all die Zahlen liefert, die ich schon immer haben wollte. Ein Tool mit eingebauter Planning Game-Unterstützung. Was auch immer.

Und nun kündigen ausgerechnet Thoughtworks, die Altmeister des Agilen, so ein Tool an. Und es soll Mingle heissen. Außer Werbegefasel gibt’s noch nix zu sehen, aber ich bin hochgradig gespannt.

kommt-mit.de - das video

2007-04-01T00:00:00+00:00

Endlich fertig, der erste Screencast zu kommt-mit.de - unserer letzte Woche in public beta gegangen Reisecommunity. Das Video zeigt die ersten Schritte: Anmelden, eigene Reise anlegen, Freunde einladen, Organisatorisches und noch einiges mehr… viel Spass.

</embed>

(zum Video im eigenen Blog etc. einbetten: <embed src="http://www.vimeo.com/moogaloop.swf?clip_id=163065" quality="best" scale="exactfit" width="400" height="385" type="application/x-shockwave-flash"></embed>)

Endlich wieder grüne Unit Tests (updated)

2007-03-31T00:00:00+00:00

Ich bin in den vergangen Tagen über mehrere Blogeinträge gestolpert, die sich mit dem Thema Farben in Unit Tests (also, bei deren Ausführung) beschäftigt haben. RedGreen ist etwas unelegant, da man statt einfach rake einzutippen rake | redgreen schreiben muss.

Mit diesem kleinen Patch und dem colored Gem geht dann doch alles vollautomatisch. Toll.

Update: gerade ist mir aufgefallen, dass die Tests, wenn man sie innerhalb von TextMate ausführt, recht unansehnlich werden, da TextMate die entsprechenden Codes plain ausgibt, statt die Schrift farbig werden zu lassen. Bisher hatte ich oben genannten Patch in eine Datei colorized_tests.rb gepackt und diese mittels require File.expand_path(File.dirname(__FILE__) + '/colorized_tests')am Ende von test_helper.rb eingebunden. Um die Farbcodes in TextMate abzustellen, habe ich noch ein unless ENV["TM_PROJECT_DIRECTORY"] dahintergehangen. Jetzt sehen die in TextMate ausgeführten Tests wieder wie gewohnt aus, und bei rake gibt’s trotzdem Farben. Cool.

Wir machen unseren Tests Beine

2007-03-30T00:00:00+00:00

Da unsere vielen Tests mittlerweile ein wenig lange brauchen, haben wir uns am heutigen Forschungstag mal wieder mit dem Problem Testing auseinandergesetzt. Das Ergebniss diesmal schon vorne weg: unsere erfolgreichste Entdeckung ist Mocha. Mit diesem Framework lassen sich Stubs und Mocks mit realen Objekten verwenden. Es folgt ein kleiner Auszug aus unserem bisherigen Testcode.

def create_activated_user(attributes = {})
email = generate_email_address
User.create!({:gender =&gt; 'm', :first_name =&gt; 'joe', :last_name =&gt; 'doe', :email =&gt; email,
:password =&gt; 'test', :password_confirmation =&gt; 'test', :city =&gt; "berlin", :status =&gt; 'active'
}.merge(attributes))
end

def create_auto(user = nil, attributes = {})
user = create_user unless user
Auto.create!(auto_hash(user).merge(attributes))
end

def test_activity_returns35_for_user_with_picture_and_auto_with_picture
user = create_activated_user :photo =&gt; '1.gif'
create_auto user, :photo =&gt; '2.gif'
assert_equal(35, user.activity)
end

Dauer: 795ms

Nach kurzem Studium des Mocha Cheat Sheets haben wir dann den Test wie folgt geändert.

def test_activity_return35_for_user_with_picture_and_auto_with_picture
with_photos = stub :with_photos =&gt; [Auto.new(:photo =&gt;'test.gif')]
user = User.new :photo =&gt; '1.gif'
user.stubs(:autos).returns(with_photos)
assert_equal(35, user.activity)
end

Dauer: 49ms

Na wenn das keine signifikate Verbesserung ist. Auf Mocha gestoßen sind wir über den Testing Rails Blog, auf dem leider nichts Neues mehr zu passieren scheint. Eine kleine Übersicht von Stub-Frameworks mit problemspezifischen Einschätzungen fanden wir auf dem Blog von Jay Fields.

Wir werden Mocha in der nächsten Zeit in unseren Test integrieren, und bei der 2000ten Revision sagen wir euch wie lange unsere Test Suite läuft.

1000. commit!

2007-03-30T00:00:00+00:00

Wir haben wieder einen Grund zu Feiern. Die 1000ste Revision. Hier mal ein paar Zahlen über dieses Jubiläum. Man beachte die Code to Test Rate :)

+----------------------+-------+-------+---------+---------+-----+-------+
| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Controllers          |  1588 |  1336 |      30 |     175 |   5 |     5 |
| Helpers              |  1335 |  1146 |       3 |      97 |  32 |     9 |
| Models               |  2566 |  2068 |      40 |     258 |   6 |     6 |
| Libraries            |  1693 |  1416 |      10 |      61 |   6 |    21 |
| Components           |     0 |     0 |       0 |       0 |   0 |     0 |
| Integration tests    |   179 |   157 |       2 |      16 |   8 |     7 |
| Functional tests     |  3496 |  3034 |      61 |     389 |   6 |     5 |
| Unit tests           |  8551 |  6355 |      43 |     448 |  10 |    12 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total                | 19408 | 15512 |     189 |    1444 |   7 |     8 |
+----------------------+-------+-------+---------+---------+-----+-------+
Code LOC: 5966     Test LOC: 9546     Code to Test Ratio: 1:1.6

kommt-mit: Reisen von früher

2007-03-29T00:00:00+00:00

Auf vielfachen Wunsch kann man ab sofort bei kommt-mit das Reisedatum auch bis zu 3 Jahre in der Vergangenheit angeben - endlich können also auch die schönsten Reisen von früher mit korrektem Datum versehen werden.

rails hat einen song

2007-03-28T00:00:00+00:00

ganz schoen nerdig, aber naja, trotzdem nett. hier isser:

[audio:http://media.libsyn.com/media/coderpath/railin_away.mp3]

(via CoderPath)

Launch: kommt-mit.de - Die Reisecommunity

2007-03-27T00:00:00+00:00

Endlich ist es so weit - nach nunmehr 3 Monaten Entwicklungszeit geht kommt-mit.de online. KommtMit ist die erste deutsche web 2.0-kompatible Community für Reisen und Ausflüge. Plant vom nächsten Strandurlaub über die Fahrradtour, die Klassenfahrt, den Traumurlaub irgendwann mal, die Dienstreise, das Gruppenseminar bis hin zur Weltumrundung alle eure Reisen. Auf KommMit könnt ihr:

alle Freunde einfach zu euren Reisen einladen
mit der Aufgabenliste im Voraus klären, wer denn nun den Dosenöffner nicht vergessen darf oder wer für's Picknick zuständig ist
auf der Pinnwand den Abfahrtstermin diskutieren
im Kochbuch Rezepte heraussuchen und euch eine Einkaufsliste zusammenstellen lassen
ein Reisetagebuch führen: für die zu-Hause-Gebliebenen und die Erinnerung
Fotos hochladen (und sogar angucken ;) )
das Reiseziel anhand von Satellitenbildern und Fotos erkunden

.. und noch so einiges mehr. Wir freuen uns über jeden Besucher und auch über Feedback, denn noch ist die Seite im Betatest. Neuigkeiten ab sofort dann immer hier im Blog.

Rails Fragment Caching - Testing and time based expiry

2007-03-23T00:00:00+00:00

in the last days i started implementing caching for autoki.com. my first stop was this excellent rails caching tutorial over at railsenvy.com.

basically, rails offers 3 ways of caching page content:

page caching: an entire page gets stored on the hard disk and can then be served by apache instead of rails - very fast but almost useless for us, as every page has some dynamic element in this. we still consider it for some ajax calls.
action caching: also caches the entire page, but it's still served by rails, which means before_filters for stuff like authentication still work - still not for us, see above
fragment caching: as the name implies caches fragments of a page - yay, sounds good

fragment caching

the basics are really easy. first, you surround the parts of the page that you want to cache with a <% cache do %> ... <% end %>. this writes the fragment to a file in /tmp/caches and from now on, rails serves this part of the page from the cache.

this alone doesn’t get you any performance improvements yet, because your controller is still loading all the data from the database before rendering the view. to avoid this, you wrap your data loading code into unless read_fragment 'name' ... end blocks - now you page should be running lightning fast already.

the third problem to tackle is to clean the cache at the right time. this can either be done in the controllers by calling expire_fragment 'name' or by using so called sweepers. they are basically observe your models and clean the right fragments on events like after_create - so when you add a new user to the system, you can clean the list of users from the cache.

time based expiry

sometimes, it’s not very efficient to clean the cache every time the underlying data changes. we have some site wide statistics for example, that require a load of processing power to calculate, and it’s enough if they get recalculated, say, every hour. rails doesn’t come with time bases cache expiry, but luckily, the timed_fragment_cache plugin comes to the rescue. it allows you to add an expiry time to your cache blocks in the erb templates and adds another method when_fragment_expired to you controllers, which allows you to test if a fragment has expired before running your data loading code.

testing it

we have some fairly complex pages which results into a multitude of cache blocks and sweeper calls on many many models, so i wanted to make sure to get it all right - how else could i do this than by using test first? rails doesn’t offer any support for testing its caching functionality so i had to use another plugin: the page cache test plugin. it basically offers two assertions: assert_caches_fragments makes sure that i have inserted a cache block in my template and that the fragment gets saved into the cache. assert_expire_fragments makes sure that my fragment gets removed from the cache when i hit a certain action. that’s all nice and good but a few things were missing: first, i wanted to test that my cache logic in the controller was working (i.e. i wasn’t loading data when i didn’t have to) and second, i wanted to test my time based expiring fragments as well.

testing caching in the controllers

my controller test does two things: first it hits the page and checks if the action has loaded a certain @variable - then it hits the same page again and checks that this time, the variable has not been loaded, i.e. the cache was hit instead. my custom assertion for that looks like this:

def assert_caches_fragments(*fragments, &amp;block)
  yield
  fragments.each do |fragment|
    assert_not_nil(assigns(fragment[1]), "Fragment '#{fragment[0]}': the variable @#{fragment[1]} doesn't get set at all")
  end
  yield
  fragments.each do |fragment|
    assert_nil(assigns(fragment[1]), "Fragment '#{fragment[0]}' is not cached")
  end
end

now i can do this:

assert_caches_fragments ['/home/index/best_rated_auto', :best_rated_auto],
  ['/home/index/best_game_auto', :best_game_auto] do
    get '/'
  end

this calls the url ‘/’ and checks that the variable @best_rated_auto is filled with data. then it calls it again and makes sure rails now uses the ‘/home/index/best_rated_auto’ fragment instead. same for the second pair.

testing time based cache expiry

# time based fragment expiry cache test
# asserts that the given fragments are enclosed in when_fragment_expired calls
# fragments - 2 element arrays of [fragment_name, variable_name]
# variable_name - the name of @variable that gets set when the cache has expired
# &amp;block - the call that hits the action, e.g. get '/'

def assert_expires_by_time(*fragments, &amp;block)
  yield
  fragments.each do |fragment|
    assert_not_nil(assigns(fragment[1]), "Fragment '#{fragment[0]}': the variable @#{fragment[1]} doesn't get set at all")
  end
  yield
  fragments.each do |fragment|
    assert_nil(assigns(fragment[1]), "Fragment '#{fragment[0]}' is not cached at all")
    set_fragment_date fragment[0], 1.minute.ago
  end
  yield
  fragments.each do |fragment|
    assert_not_nil assigns(fragment[1]), "Fragment '#{fragment[0]}' is not time cached"
  end
end

def set_fragment_date(name, time)
  ActionController::Base.fragment_cache_store.write(name+'_meta', YAML.dump(time))
end

this code adds a bit more logic to the assertion in the last paragraph. after the second hit to the url it sets the fragment’s time so it is expired and hits the page a third time to check if the data comes from the database again.

plugin hacking

one last thing. in order not to fill my functional tests even more i decided to use integration tests for the cache testing. the fragment cache plugin has one limitation: in integration tests, it expects the fragment name to be a has, and that hash to contain a :controller => 'my_controller' pair. to work around this, i hacked the plugin the following way: in fragment_cache_test.rb i changed the first line of the check_options_has_controller method to this: if option = options.detect { |option| option[:controller].nil? } && !@controller - now i could keep my fragment names and still use integration tests. all i had to do was to define @controller = MyController.new in my test.

welcome to fully tested rails caching :) now comes all the boring stuff - implementing it everywhere.

re:publica anmeldung ist offen

2007-03-11T00:00:00+00:00

schnell anmelden, unter https://ssl-id1.de/re-publica.de/akkreditierung/

Rails: FileColumn schneller testen durch Mocks

2007-03-11T00:00:00+00:00

FileColumn ist ein wirkliches zauberhaftes Plugin fuer Ruby on Rails. All unsere Bilder-Uploads haben wir bisher damit umgesetzt. FileColumn bietet Helper, um Datei-Uploads zu bauen und kann die hochgeladenen Bilder natuerlich auch wieder anzeigen. Das eigentlich schoene ist aber die Integration mit RMagick, einem Ruby-Binding fuer ImageMagick, wodurch sich Bilder on the fly skalieren sowie beschneiden lassen und die verschiedenen Versionen dann automatisch im Dateisystem landen. Alles, was man dazu braucht, ist eine Zeile Code im Model, z.B.

class User
  file_column :photo, :magick => { :versions => {
	:normal => "238x",
	:tiny => "26x35!",
	:small => {:crop => "1:1", :size => "73x73!", :name => "small"},
  }}
end

Dieser Code erzeugt neben der Originalversion 3 Kopien: Eine 238 Pixel breite und entsprechend des Bildformats hohe, eine gestauchte, genau 23 x 35 Pixel grosse und eine genau 73 x 73 Pixel grosse, beschnittene Version.

So weit alles schoen. Das einzige Problem war, dass unsere Unit-Tests durch FileColumn ganz schoen langsam wurden. Bei jedem Testdurchlauf, der irgendwas mit Bilder zu tun hatte, wurden jedes mal mehrere skalierte Versionen erzeugt, was natuerlich Rechenzeit kostete. Die Loesung haben wir letzte Woche dann endlich mal zusammengehackt, und sie sieht so aus: Ein Modul, was von FileColumn installierte Callbacks ausser Gefecht setzt, wird in ein Mock des Models eingebunden, aber der Reihe nach. Hier das Modul, gelegen in test/file_column_mock.rb:

module FileColumnMock

  def mock_file_column(attr)
    define_method "#{attr}=" do |value|
      if value.is_a?(File)
        write_attribute attr, value.path
      else
        write_attribute attr, value
      end
    end

    define_method "#{attr}_after_assign" do
    end

    define_method "#{attr}_after_save" do
    end

    define_method "#{attr}_state" do
      DummyState.new self.send(:read_attribute, attr)
    end

    define_method "#{attr}_after_destroy" do
    end
  end

  class DummyState

    def initialize(file_name)
      @file_name = file_name
    end

    def has_magick_errors?
      false
    end

    def absolute_path(*args)
      @file_name
    end
    def relative_path(subdir = nil)
      @file_name
    end

    def temp_path
      @file_name
    end

    def assign_temp(temp_path)
      nil
    end
  end
end

In unserem Beispiel wollen wir die Klasse User von ihrer FIleColumn-Last befreien. Dazu legen wir in test/mocks/test eine Datei user.rb an und fuellen sie mit folgendem Inhalt:

require 'models/user'
require File.dirname(__FILE__) + '/../../file_column_mock'

class User
  extend FileColumnMock
  mock_file_column :photo
end

Der Effekt des ganzen ist, dass die Original User-Klasse aus app/models nochmals geoeffnet und mehrere Methoden, die normalerweise automatisch beim Setzen des photo-Attributs loslaufen, umd die skalierten Versionen zu erzeugen, durch Ueberschreiben ausser Gefecht gesetzt werden.

Das war’s. Und schon fliegen die Tests wieder mit voller Geschwindigkeit.

Forschungsergebnisse: Besser testen

2007-03-11T00:00:00+00:00

Da wir uns diese Woche etwas um die Laufzeit unsere Tests gekümmert haben, war das Thema unseres letzten Forschungstages “besser testen in Ruby on Rails”. Da Testen bereits Thema bei Peepcode war, war der Screencast Pflicht. Der brachte leider kaum neues. Doch ganz zum Schluss wurden dann doch noch zwei praktische Tools vorgestellt. Mit TestTimer (gem install test_timer) läst sich die Laufzeit aller einzelnen Test messen. So lassen sich echte Langläufer ausmachen. Ein weiteres sehr praktisches Tool ist FakeWeb. Damit lässt sich Http-Kommunikation simulieren - perfekt, um die Testumgebung von externen Seiten zu entkoppeln ohne den Produktionscode zu ändern. Hier mal ein Beispiel aus der Dokumentation.

def test_request FakeWeb.register_uri('http://example.com/test_me', :string => "Hello World!") content = Net::HTTP.get(URI.parse('http://example.com/test_me')) assert_equal "Hello World!", content end

Bei unserer Rechereche im Netz stießen wir zu guter Letzt noch auf einen kleiner Helfer, der automatisch jede Seite auf gültiges HTML prüft, jedem Link folgt, jeden Ajax Link aufruft und Formulare mit zufälligemn Daten füttert. So lassen sich Routing-Fehler, fehlende Seiten und falsche Formulare leichter aufspüren. Dieses Plugin hat den passenden Namen SpiderTest. Auch hier ist die Verwendung kinderleicht.

def test_spider get '/' assert_response :success spider(@response.body, '/') end

SpiderTest befindet sich noch in der Entwicklung.

So das waren unsere Ergebnisse mal kurz zusammengefasst. Über hilfreichreiche Tipps und Kritik freuen wir uns, genauso wie über jeden netten Kommentar :)

Endlich: CruiseControl.rb - continuous integration a la Ruby

2007-03-02T00:00:00+00:00

Vorwort: Continuous Integration (CI) bezeichnet das kontinuierliche, automatische integrieren aller Softwarekomponenten eines Projekts. Die geschieht ueblicherweise bei jedem Commit ins Versionskontrollsystem (bei uns Subversion). Das CI-Tool macht dabei einen frischen Checkout des Projekts und laesst die automatischen Tests laufen. Im Fehlerfall wird Alarm geschlagen, was den sofortigen Entwicklungsstillstand, bis der Fehler behoben ist, zur Folge haben sollte. Angewendet wird CI vor allem bei Freunden der agilen Softwareentwicklung, insbsondere bei den extreme programmern (also uns). Machen sollte es aber eigentlich jeder Entwickler, der von sich behauptet, professionell zu sein.

Hauptteil: Gestern gefunden, heute im Rahmen unseres Forschungstages installiert: CruiseControl.rb ist nach Java und .Net die dritte CruiseControl (CC) Implementierung, diesmal als Ruby on Rails-Anwendung.

Was sogleich angenehm auffaellt: Im Gegensatz zur Java-Variante benoetigt CC.rb so gut wie keine initialie Konfiguration. Nach dem Download genuegt ein einfaches ./cruise add projektname --url svn-url und anschliessen ./cruise start und schon startet ein Webserver und das builden kann losgehen.

Ein kleines Problem gab es dann doch: CC.rb kommt anscheinend nicht mit einem deutschen Subversion klar (undefined method `number’ for nil:NilClass). Bevor es also ans cruisen geht, noch schnell (unter *nix bash) unset LANG && unset LANGUAGE eingegeben und schon geht’s.

OpenID gewinnt an Fahrt.

2007-03-01T00:00:00+00:00

Einige der Grossen arbeiten noch daran, viele viele andere haben es schon. Trotzdem gelang OpenID der entscheidene Durchbruch erst vor gut einer Woche, also ich mir meine persoenliche OpenID zugelegt habe, aber der Reihe nach:

Die Kurzfassung

OpenID ist ein dezentrales Framework zur Authentifizierung von Benutzern im Internet.

Die Langfassung

Das Problem: Als fleissiger Internetnutzer habe ich bestimmt 50 Accounts bei verschiedensten Diensten. Bei jedem dieser Dienste musste ich mich anmelden, meine Emailadresse eingeben, mir ein Passwort ausdenken und meine 30 Lieblingsfilme, Hobbys und Buecher eingeben. ~~Falls~~ da ich faul war, haben die meisten dieser 50 Firmen das gleiche Passwort, und sobald bei einer dieser Firmen die Datenbank ins Netz entfleucht (ok, angeblich waren die Passworter verschluesselt), kann der neue Besitzer sich damit bei all den anderen Firmen in meinem Namen anmelden und allen moeglichen Unfug anstellen. Im Wesentlichen habe ich also zum einen ein Komfortproblem, indem ich immer wieder die gleichen Daten eingeben muss, zum anderen ein Sicherheitsproblem, da ich verschiedensten Unbekannten meine Daten anvertraue.

Ein frueherer Loesungsansatz: vor einigen Jahren hatte ein grosser Konzern aus Redmond mal ein System namens Passport in Umlauf gebracht. Dieses war als sogenanntes Single Sign On-System gedacht, d.h. ein Benutzer richtet sich einen Passport-Account ein, und kann sich damit fortan bei allen Seiten, die Passport unterstuetzen, mit diesem einen Account anmelden. Die Sache hatte einen kleinen Haken: man sollte ausgerechnet Microsoft seine Daten anvertrauen, dem grossen boesen Konzern, der die Weltherrshchaft an sich reissen wollte. Das hat sich inzwischen ja erledigt, da sich ein Nachfolger gefunden hat, aber damals…

Jetzte endlich: OpenID: wie in der Kurzfassung angekuendigt, ist OpenID dezentral, d.h. es gibt keinen zentralen Provider mehr, der meine Identitaet verwaltet, sondern es gibt viele davon, und ich kann mir einen aussuchen. In meinem Fall war das MyOpenID.com. Nachdem ich mich dort angemeldet habe (ja, wieder Email, Passwort), bekomme ich meine eigene OpenID in Form einer URL, in meinem Fall http://langalex.myopenid.com/.

(Kurzeinschub: gerade laeuft das Fritz-Chaosradio Nr. 121 zum Thema digitale Identitaet, und just in dieser Minute schwenkt der Fokus auf OpenID, also im Podcast dann so bei Minute 78. Gerade wiederholen die Jungs so ziemlich genau das, was ich hier geschrieben habe. seufz Zum Glueck hab ich ja noch mehr.)

Also zurueck: ich habe jetzt also meine MyOpenID (oder Yahoo-OpenID oder AOL-OpenID oder von wem auch immer). Nun moechte ich zum Beispiel auf LiveJournal mir einen Blog einrichten. Normalerweise muesste ich mich dazu zuerst bei LiveJournal anmelden, also wieder Email, Passwort usw. Stattdessen kann ich dort einfach meine OpenID-URL eingeben und mich damit anmelden. Damit sich nicht jeder, der meine OpenID kennt, damit ueberall anmelden kann, muss nun noch sichergestellt werden, dass auch wirklich ich vor dem Rechner sitze. Dazu fragt LiveJournal nun bei meinem OpenID-Provider, also MyOpenID, ob ich’s nun bin. Um das zu beweisen, muss ich auf der MyOpenID-Website mein Passwort eingeben, und ich bin drin, also bei LiveJournal.

Das Ergebnis: Ich habe nur noch eine Identitaet, naemlich die OpenID-URL und ein Passwort, und dieses Passowrt bekommt niemand, ausser mein OpenID-Provider zu sehen. Mit dieser Identitaet kann ich mich nun ~~ueberall~~ schon bald bei immer mehr Diensten einloggen.

Goodies

Ein Restrisiko habe ich natuerlich immer noch, da ich immer noch meinem OpenID-Provider vertrauen muss. Um auch das zu umgehen, kann ich einfach meinen eigene OpenID-Server aufsetzen. Leider habe ich gerade den Link verloren, sorry.

Nettes Gimmick: ich kann relativ einfach eine OpenID mit meiner eigenen Domain erzeugen, also sowas wie alex.upstream-berlin.com. Dazu melde ich mich zunaechst ganz normal bei einem beliebigen OpenID-Provider an. Zusaetzlich muss ich unter alex.upstream-berlin.com eine HTML-Seite bereitstellen, die ueber ein paar Meta Tags auf die eigentliche OpenID-URL verweist.

Implementierung: das Entscheidene ist natuerlich, dass nun moeglichst viele Sites das ganze unterstuetzen. In absehbarer Zeit wird das in unseren Projekten der Fall sein. Eine Ruby-Library scheint es ja schon zu geben.

Immer wieder gerne: der Hinweis auf Dick Hardts Prasentation Identity 2.0, bei der das Prinzip nochmal toll erklaert wird.

Nochmal zum Thema Rails-CMS

2007-02-25T00:00:00+00:00

Nachdem ich vor einiger Zeit ueber diese Liste Rails-basierter Content Management Systeme gestolpert war, habe ich mir heute zur Vorbereitung eines neuen Projekts diese Liste einmal etwas genauer angesehen.

Meine Anforderungen waren eigentlich ganz einfach (dachte ich):

eine Art Mini-Blogsystem fuer News
Fotos, MP3s hochladen und auf Seiten darstellen
WYSIWYG-Editor fuer alle Seiten
Bedienung auch durch Nicht-Geeks (!)
Unterstuetzung von RSS-Feeds und Kommentaren
Ruby / Rails-basiert

Die meisten der Anforderungen klingen eher nach einer Blog-Software. Wordpress waere eigentlich wie geschaffen. Neben Blog-Features unterstuetzt es auch statische Seiten und die Administrationsoberflaeche ist einfach grosssartig, durch Plugins lassen sich MP3-Player und anderes einbauen. Leider ist das Ding in PHP geschrieben, und da ich mindestens am Layout ein paar tiefgreifende Aenderungen vornehmen werde muessen, von PHP-Code aber fuerchten muss, Augenkrebs bekommen, faellt Wordpress erstmal weg.

Zunaechst also Typo und Mephisto, zwei Rails-Blogsysteme:

Mephisto

(Uberigens “The best blogging system ever”. Das erinnert mich irgendwie daran. Na dann.) Nach der Installation stehe ich vor einer leeren Admin-Oberflaeche, in der ich irgendwelche Sections anlegen kann. Dabei darf ich jeweils zwischen Blog-like Sections (Posts pro Seite) und Statischen Seiten (ein Post = die ganze Section) waehlen. Oben prangt auch ein Assets-Button, mit dem man Bilder hochladen kann. Tja, und dann kommt irgendwann der Editor zum Erstellen eines Blog-Eintrags. WYSIWYG is nich und wie man die Assets ohne HTML zu Coden in den Post bekommt, erschliesst sich mir leider auch nicht. Ueberhaupt ist das ganze etwas unuebersichtlich. Besagte Sections dienen offensichtlich als Kategorien fuer saemtliche Posts, wobei ein Post in beliebig vielen Sections auftauchen kann. Das macht fuer ein reines Blog Sinn, aber irgendwie fehlt mir hier im Konzept die Trennung zwischen statischem Content und Blog. Nee, Mephisto geht gar nicht.

Typo

Die Typo-Website besteht nur aus einem Trac.Na hoffentlich haben die Entwickler die Zeit, die sie an der Website gespart haben, in die Usability des eigentlichen Produkts fliessen lassen. Aber kann jemand, der nicht einmal die Praesentation seiner Software halbwegs schick gestalten kann, denn eine Software schreiben, bei der Worte wie Endnutzer, Usability und Komfort im Vordergrund stehen? Wir werden sehen …

20 Minuten spaeter: Nachdem die standardmaessige Datanbankkonfiguration mit Sqlite zunaechst nur Fehler produziert hat, laeuft Typo jetzt auf Msyql. Leider bestaetigt sich meine Befuerchtung: eine furchtbar haesslche Administrationsoberflaeche begruesst mich nach dem Login mit einer bestimmt 5 Bildschirme langen Seite, in der ich irgendwelche wild zusammengewuerfelten Einstellunge vornehmen soll. Nagut, weiter zum Erstellen eines Artikels. Ein Paar verdammt grosse, leere Textfelder mit Namen wie extended content (was ist das denn?!), Kategorien auch als Textarea. Ganz unten gibt’s auch was zum Datei hochladen. Aber nix, womit ich die Datei dann in die Seite einbinden koennte. Waaah. Raus hier.

Tja, das waren die beiden Rails Blogsysteme. Ziemlich ernuechternd so weit. Mit denen geht schonmal gar nix. Werden die “richtigen” CMSe vorbeiziehen koennen? Lesen Sie weiter…

Radiant

Radiant nenn sich selbst “no fluff” und “simplified” und “flexible”. Klingt schon einmal etwas bescheidener und sympatischer als das beste Blogsystem ever, koennte aber auch “noch keine Zeit fuer mehr Features gehabt” heissen. Nach dem Login habe ich einen Baum von Seiten vor mir. Es lassen sich neue Seiten einfuegen sowie bestehende bearbeiten. Leider gibt’s auch hier kein WYSIWYG sondern nur plain text bzw. Textile (wiki-like syntax) und Co. (hatten die anderen auch alle). Seiten lassen sich Layouts zuordnen, in denen dann wiederum Snippets die einzelnen Bereiche einer Seite fuellen. Weiterhin gibt es sogenannte Behaviors, mit denen sich einer Seite Programmlogik beibringen laesst.

Mit Hilfe kleiner Tags laesst sich die Logik dann in die Seite integrieren. Ein simples Blog liesse sich zum Beispiel so erzeugen: Man erstellt eine Seite, die die Liste der Eintraege anzeigen soll. Die Eintraege haengt man dann unterhalb dieser Seite als eigene Seiten ein. Ein Tag iteriert durch alle Kind-Seiten und zeigt deren Inhalt an.

Ab dem naechsten Release soll es Extensions geben, mir denen sich dann neue Features hinzfuegen lassen. Der aktuelle Release Candidate hat allerdings nicht funktioniert. Klingt wirklich alles sehr schoen und einfach, gefaellt mir wirklich gut das Konzept. Aber was soll ein Normal-User damit anfangen? Leider gar nix. Also weiter.

Rubricks

Rubricks hat auf seiner Website Demos und Videos des Systems. In dem Video sieht man grosse bunte Icons in allen Bonbonfarben. Und wie jemand mit einem Klick ein komplettes Forum erzeugt. Und alles ist animiert und man kann die Seitenelemente mit der Maus verschieben. Ich habe Angst. Ich will keine eierlegende Wollmilchsau. Ich will was kleines, beherrschbares. Zum Glueck sehe ich nach der Installation nur einen Errror 500. Puh. Schnell wieder loeschen.

Streamlined

Streamlined ist eigentlich kein CMS, wird aber im eingangs beschriebenen Artikel neben ein paar weiteren Tools als Startpunkt fuer die Entwicklung eigener Systeme empfohlen. Im Grunde ist Streamlined die Maybach-Version des Scaffold Generators von Rails. Aber Fotos hochladen kann man damit auch nicht. Und WYSIWYG. Naechster.

Spread CMS

Hach wie schoen. Der Besuch der Spread-Website beschert mir sogleich ein warmes Gefuehl ums Herz. Ein tolles Web2.0-Logo mit der obligatorischen Spiegelung nach unten, eine aufgeraeumte Seite, schoene Farben und Screenshots der Admin-Oberflaeche, auf der Seiten mit eingebetteten Fotos zu sehen sind. Leider ist mein Franzoesisch etwas eingerostet, sodass sich mir die kleinen Texte neben den Bildern nur teilweise erschliessen, aber egal. Das Ding will ich sofort ausprobieren, und hier kommt der Haken: das ganze steckt noch oder ist schon vor langer Zeit in der Konzeptionsphase haengengeblieben. Neiiin.

Im Projekt-eigenen Trac finde ich noch einen Artikel Making a better CMS, in dem sich jemand ueber die eklatanten Maengel einiger untersuchter CMSe auslaesst. Danke, ich bin anscheinend nicht allein.

Ich geh mir jetzt mal den Code von Wordpress angucken. Vorher noch ein paar Tomaten gegessen, Gemuese soll ja vorbeugen gegen (Augen)krebs. Seufz.

Deine Tests wissen die Antwort.

2007-02-25T00:00:00+00:00

Nachdem wir vor kurzem auf Rails 1.2.2 umgestiegen sind, verbrachte ich letztens einige Stunden damit die Ursache für folgenden Fehler zu finden, der beim starten von mongrel auftrat: 'remove_const': cannot remove Object::ModelTranslation (NameError) Die Logfiles zeigten, dass es Probleme beim Laden von Abhängigkeiten für Globalize gab. Allerdings war die Ursache völlig unklar. Ein Aufruf unserer Unit Tests mit rake lieferte dann die banale Lösung: der Datenbankserver lief nicht.

Das lehrt mal wieder, immer die Tests laufen lassen, wenn es Probleme gibt!

Tag der Forschung.

2007-02-24T00:00:00+00:00

Eine der Faszinationen der Informatik fuer micht ist, dass es eigentlich jeden Tag etwas neues zu entdecken, zu lernen und auszuprobierne gibt. Sei es eine neue Technologie, eine bisher unbekannte Art, etwas zu benutzen oder eine neue “best practice”, die einen dem unerreichbaren schwarzen Guertel naeher bringt. Dummerweise (?) folgert daraus aber fuer alle Fachbeteiligten ein gewisser Zwang, sich moeglichst vielen dieser Neuerungen wenigsten kurz zu widmen, will man vor der Konkurrenz spielen und auch da bleiben. Nur woher die Zeit nehmen? Nach acht Stunden vor dem Bildschirm sitzt die Motivation fuer weiteres Bildschirmgeflimmer oder papierne Fachlektuere im untersten Kellergeschoss. Wochenende? Nee.

Deshalb: ab sofort gibt es jeden Freitag Nachmittag den upstream Forschungs(halb-)tag. Freitag ab 14 Uhr ist - ausser in Notfaellen (es musste einen Haken geben) - das regulaere Arbeiten an Projekten untersagt. Stattdessen werden Buecher gewaelzt, Screencasts rezipiert oder das Netz durchforstet. Oder gebloggt. Ganz im Namen der Weiterbildung, der Forschung und des freien Experimentierens. Oder Austausch mit anderen, sollte sich das mal ergeben. Zur Unterstuetzung einer forschigen Atmosphaere werden wir des oefteren einen Ortswechsel anstreben. Mal sehen, wie das so funktioniert. Erfahrungsberichte dann an dieser Stelle.

Unser User-Interface ist gut.

2007-02-21T00:00:00+00:00

Auch wenn es die Kunden vielleicht nicht sofort glauben, aber sagen können wir es auf jeden Fall schonmal - meint die BVG Berlin und vermerkt in ihren Fahrkartenautomaten nach Klick auf den Hilfe-Button:

Die Bedienung der Automaten ist leicht. Folgen Sie der Bedienung intuitiv. Sie werden erfolgreich sein.

Jetzt wissen wir endlich, was wir auf die Startseiten unserer Produkte schreiben sollen. Danke.

Testing helpers in Ruby on Rails 1.2

2007-02-21T00:00:00+00:00

We are just migrating one of our projects from Rails 1.1.6 to 1.2.2. One of the problems we encountered was, that Recipe 44: Write tests for your helpers from Rails Recipes (by the way Advanced Rails Recipes will come out in Augut 2007) is not working anymore. The line

@controller.send :initialize_current_url

resulted in a

can't clone NilClass

After some messing around, we found out that the following setup method solves this:

def setup
  @controller = MyController.new
  request = ActionController::TestRequest.new
  <strong>@controller.instance_eval { @url = ActionController::UrlRewriter.new(request, {}) }</strong>
  @response = ActionController::TestResponse.new
end

Happy testing :)

Caching in Rails

2007-02-21T00:00:00+00:00

Peepcode haben wieder einen ihrer exzellenten Rails-Screencasts veröffentlicht. Thema heute: Caching. Na das schauen wir uns doch auch mal an. Steht ja schon auf der to-do Liste.

Trac-Alternative

2007-02-18T00:00:00+00:00

Momentan setzen wir für Tickets, Dokumentation und Release-Planung auf Trac. So richtig glücklich sind wir damit aber nicht. Denn die Entwicklung von Trac schreitet in letzer Zeit langsam vorran, eine Unterstüzung mehrer Projekte ist in naher Zukunft nicht in Sicht, auch lief das Setup nicht immer ohne Probleme ab. Also hab ich mich mal nach einer Alternative umgeschaut. Und bin mit Retrospectiva fündig geworden. Es liefert die wichtigen Funktionen: Issue tracking, Milestone/Release Management, Code Browser, Suche, Changeset und Revision Management, Wiki. Und das ganze auch für mehrer Projekte. Zum größten Teil basiert Restrospektiva auf Collaboa, ebenfalls ein Trac -Konkurrent, aber ohne Wiki. Beide wurden mit userem aktuellen Lieblingframework Rails entwickelt - und das ziemlich rasant. Naja Collaboa hatte zwischenzeitlich mal ein Jahr Pause eingelegt.

Wir werden mal einen testweisen Umstieg wagen. Unsere Erfahrungen lest ihr dann hier.

re:publica in berlin

2007-02-17T00:00:00+00:00

vom 11.-13.4. 2007 findet in berlin die re:publica statt. dabei handelt es sich um eine konferenz rund um’s “web 2.0” (logo :) ). soziale netzwerke, blogs, podcasts usw. besser erklaeren koennen das aber die macher (spreeblick und newthinking) selbst.

allem anschein nach wird das ganze wohl kostenlos sein, ich bin auf jeden fall da. um teilzunehmen, trage man sich auf der wiki-seite ein.

Schicke Schriften jetzt auch im Web

2007-02-16T00:00:00+00:00

der aesthetische weg ist klar: abgerundete ecken, farbverlaufen, grosse buchstaben, web 2.0-standardfarbschemata.^^ was bisher fehlte, waren mehr als die vier web-standard-schriftarten. schluss damit macht sIFR: einmal eingebunden, steht dem stolzen website-besitzer, blogger oder community-homepage-designer ploetzlich die welt der schriften offen.

technisch nett geloest: die seite enthaelt zunaechst ganz normales html. ein javascript tauscht bei vorhandenem flash-plugin die standard-schriften gegen flash-filme mit entsprechender formatierung aus. so haben auch nicht-flasher und such(maschinen) noch was von der seite.

wie’s aussieht, sieht man auf der beispielseite, wie’s geht, steht im doku wiki

trotzdem sei an dieser stelle der vorsichtige gebrauch angemahnt - bei 200 flash-filmen pro seite macht auch der staerkste rechner schlapp ;)

10 Wege in die Erfolglosigkeit

2007-02-16T00:00:00+00:00

guy kawasaki hat in der kiste gekramt und einmal die zehn duemmsten wege, mit einer Webanwendung zu scheitern, niedergeschrieben. darunter finden sich aergernisse wie zu lange urls, usernahmen ohne @-zeichen oder sofortiger registrierungszwang, aber auch absolute blocker wie “internet explorer 5.5 only” oder nicht lesbare captchas. hm, und eigentlich sind es auch vierzehn punkte, und nicht zehn, aber egal. mir sind auch noch ein paar eingefallen (unsortiert):

bugs in production

nichts ist erfrischender, als eine error 500-seite bei der anmeldung oder nicht funktionierende submit-buttons. beliebt sind auch inkonsistente informationen, z.b. startseite: du hast 3 neue nachrichten, klick auf den entsprechenden link, postfach: keine nachrichten vorhanden. je nachdem, wie weit solche fehler in die zentralen funktionen einer anwendung reichen, war dieser besuch dann unter umstaenden der letzte. konkurrenz-sites erleichtern den wechsel natuerlich ungemein.

formulare

“bitte fuellen sie vor der anmeldung dieses formular mit 30 feldern aus. die als eingabe erlaubten werte unterliegen diversen beschraenkungen, die aber erst nach dem absenden bekanntgegeben werden”. geht gar nicht. ein einzelnes textfeld ist da gleich viel sympathischer.

langweiliges design

naja, kommt auf den zweck der seite an, aber das design ist nunmal aushaengeschild einer site. einer seite mit gelbem hintergrund und pinker schrift traut man einfach nix zu, sorry. mit runden ecken, farbverlaeufen und tollen schriften ist man auf jeden fall auf der sicheren seite ;)

featuritis

was bei haushaltsgeraeten und kuechenmaschinen bestimmt toll ist (der waschkuehlende toaster mit eingebauter zahnbuerste), taugt im digitalen gut zur abschreckung. angenehme freiraeume auf den seiten muessen mit jedem hinzugewonnenen feature immer kleiner und gequetschter werdenden links weichen, bis es dann irgendwann so aussieht.

lange ladezeiten

mit dem erfolg kommt die serverlast. minutenlange ladezeiten und timeouts nerven ganz gewaltig, und man sieht sich schnell nach alternativen um.

ach ja. ausnahmen bestaetigen wie immer die regeln. studivz hat’s schliesslich trotzdem geschafft :) und demnaechst koennt ihr uns die genannten punkte dann bei unseren eigenen projekten um die ohren hauen. und ein paar mehr punkte lassen sich bestimmt auch noch finden.

Zahlungsunwillige Kunden?

2007-02-15T00:00:00+00:00

kleiner tipp fuer selbstaendige, deren kunden nicht so gerne ihre rechnungen bezahlen: 5% rabatt geben, falls eine rechnung innerhalb von 10 tagen bezahlt wird. das hat wohl schon massiv geholfen. und die zahlenden kunden fuehlen sich auch noch gut, weil sie einen rabatt bekommen haben. und von der schleichenden preiserhoehung um 5% merkt ja keiner was ;)

(via flo (allerdings offline))

web 2.0 erklärt

2007-02-13T00:00:00+00:00

dieses wunderbar gemachte video erklaert dem interessierten nutzer einmal die unterschiede zwischen papier, web 1.0 und web 2.0. sowohl technisch als auch in bezug auf die konsequenzen, die sich daraus fuer die (netz-)welt so ergeben. habe ich erwaehnt, dass das video ganz wunderbar gemacht ist?

(video nach dem klick)

</param></param></embed>

(via techcrunch)

essen im friedrichshain: hot * dog * soup

2007-02-13T00:00:00+00:00

ein gar wunderbarer ort zum schnellen und gesunden mittagessen zwischendurch. gelegen in der gruenberger str. (kurz hinter ecke gabriel-max-str.) bietet sich dem hungrigen nebenstehendes bild.

neben bestimmt 10 verschiedenen hot dogs (daenisch, brasilianisch, hot chili …) gibt es ein reichhaltiges angebot an ziemlich abgefahrenen suppen (cocos-spinat, apfel-curry, gorgonzola-lauch) - angebot staendig wechselnd. ach ja, alles auch in vegetarisch und vegan.

keine ahnung, ob’s in dem laden ein wlan gibt, aber fuer laengeres verweilen ist’s dort eh zu klein. auf jeden fall haben sie ein abo der sueddeutschen und des spiegels.

10 von 10 punkten.

rails cms

2007-02-12T00:00:00+00:00

habe gerade einen artikel über content management systeme, die in rails geschrieben sind, gefunden. sowas kann man bestimmt immer mal gebrauchen, also poste ich das hier, um ihn spaeter wiederfinden zu koennen.

Hallo Welt...

2007-02-12T00:00:00+00:00

… wir sind upstream aus Berlin und wir entwickeln Anwendungen für das Web ab Version 2.0. An dieser Stelle werden wir über Themen wie Start Ups, unser derzeitiges Lieblingsframework Ruby on Rails, technische Spielzeuge, die digitale Boheme, Mittagessen im Friedrichshain, Programmierkniffe, agile Softwareentwicklung, interessantes und amüsantes aus der Netzwelt sowie natürlich unsere eigenen Aktivitäten bloggen.

Unsere Hauptbeschäftigung liegt derzeit bei autovc.com - man könnte es das StudiVZ für Autofahrer nennen. Im Moment noch public beta - wir freuen uns über jeden Besucher, jedes neue Mitglied und auch Kommentare.

Wir, das sind zur Zeit zwei unternehmergewordene Programmierer, sesshaft in Berlin Friedrichshain: Thilo Utke und Alexander Lang.

Ziel dieses Blogs ist es, neben der Kommunikation unserer Projekte und Produkte, Beiträge zu Fragen und Themen der Netzgemeinschaft zu leisten sowie unser Netzwerk um interessante und fähige Menschen zu erweitern.

Bei Fragen wendet euch per Email an alex[at]upstream-berlin[punkt]com bzw. thilo[at]upstream-berlin[punkt]com oder hinterlasst einen Kommentar.

Alex und Thilo.

Das Internet zerstört den Einzelhandel.

2007-02-12T00:00:00+00:00

neulich in einer Einzelhandelsfiliale einer nicht ganz unbekannten Computereinzelhandelskette:

König Kunde: Hallo Verkäufer: … (über seinen Rechner gebeugt) König Kunde: (steht inzwischen am Tresen, Verkäufer dreht ihm den Rücken zu) … (fängt an, mit den Fingern möglichst laut auf der Tischplatte zu tippeln) Verkäufer: … (konzentriert auf den Rechner starrend) König Kunde: räusper Verkäufer: … König Kunde: Ich hätte gern ein Verlängerungskabel für einen Beamer Verkäufer: (ohne aufzublicken) ham wa nich, Montag wieder König Kunde: (etwas verdutzt ob der unerwarteten Antwort) Hm, schade. Ich bräuchte aber heute eins. Verkäufer: … (nach wie vor dem Rechner zugewandt) König Kunde: (entdeckt ein Regal mit Monitor-/Beamerkabeln und geht darauf zu) Verkäufer: (schaut gebannt auf seinen Rechner) Kunde: Das sind doch hier Monitor-Verlängerungskabel oder? Verkäufer: Nee, nur Anschlusskabel. Kunde: Naja, müste doch auch gehen oder? Ich hab so einen Adapter vom Laptop aus und da müsste man doch so ein Kabel zum Beamer… Verkäufer: (zuvorkommend) Nee geht nich. Kunde: (verunsichert) und so ein DVI-Kabel? Ich hab ja DVI am Rechner. Und dann der Adapter… Verkäufer: Würd ich nich machen. Kundenarsch: (entdeckt ein VGA-Anschlusskabel in der richtigen Länge.) Damit müsste es doch gehen oder? Verkäufer: … Kundenarsch: Das nehm ich. Verkäufer: Neunzehn Fuffzich Kundenarsch: Danke. Verkäufer: … Kundearsch: Tschüss. Verkäufer: … (tippt auf seinem Rechner rum)

Zu Hause angekommen, bemerkt der Kunde, dass das gekaufte Kabel eigentlich ein Verlängerungskabel ist, also genau das, was er kaufen wollte. Er freut sich wahnsinnig über den gelungenen Kauf und dankt dem Himmel, dass es den freundlichen Einzelhandel mit dem guten Service und der persönlichen Beratung noch gibt, sonst hätte er vermutlich ewig im Internet nach dem besten und güntigsten Kabel gegoogelt und es durch einen aufwändigen Mausklick bestellen müssen, woraufhin ihm das ganze am nächsten Tag nach Hause geliefert worden wäre. Glück gehabt. Danke JE Computer in der Frankfurter Allee Filiale Berlin Friedrichshain.

upstream agile - software

Über Tests: fast, isolated integration tests for distributed apps

12 Rules for Writing Web Apps in 2012

Business Models

1. Models are plain old Ruby objects.

2. Your models do not persist themselves.

3. All execution flow is explicit.

4. Models do not expose their properties publicly.

5. Each model encapsulates its persistent data.

Persistence

6. Use whatever database or service is best for the current job.

Front-end

7. Your front-end is a RESTful JSON API.

8. Your app provides OAuth2 authentication.

Testing

9. Your API is fully tested with integration tests.

10. Your business models are unit tested.

11. Your tests run within two minutes.

Frameworks

12. Prefer small and simple

The bottom line

Testability

Scalability:

Flexibility

Happiness

Conclusion

Http Caching assets in Rails 3.1

Ruby TDD Workshop

Sublime Text 2 with RVM on OSX

JavaScript Workshop for Designers 2

Testing the AnyTime date picker with Cucumber/Selenium

Euruko 2011 call for papers

JavaScript under scrutiny at Plat_forms 2011

Diplom thesis: a distributed application with CouchDB and Javascript

Upstream Summer Party

Transactions in CouchDB

Experimenting with node.js/mite

Making multple API requests and still be fast

Storing account data securely

Nice and fast GUI

Conclusions

Capybara - Mix and match acceptance testing

Product Launch: cobot - coworking space management service

From Journeyman to Client

Case Study

Categories

Tags

Related books

Reviews/Ratings

Licensing

Internationalization (I18n)

Statistics

Conclusion

Git meets Tracker, a possible Workflow

Upstream spreading to Scandinavia

Time-out

Upstream on the Move

Awesome presentations with boom amazing

What is behind it?

Generating SVGs on OSX…

Using it

Hacking

Conclusion

Unit Testing CouchDB Views with Couch Potato

How it works

Testing Couchapps with Cucumber and Culerity

DevHouseBerlin^2

Ruby Metaprogramming: Dynamically Inherited Methods™

Upstream goes San Francisco!

Kickstart Rspec with Spork

Upstream Agile Project Checklist (updated)

Conference Triathlon - Euruko, Ruby on OS X, RailsWayCon

Notes from the Ruby on OS X conference

New Couch Potato: simple, testable, opinionated.

Saving / loading models

New Views

Testing

Installing yum on CentOS 5.3

The case of ActiveRecord vs. CouchDB

CouchDB is not about relations