Vendoring rails using a symlink

Posted by Andrew Premdas Wed, 22 Jul 2009 17:53:00 GMT

Been meaning to do this for ages!

First of all clone rails from git hub into a suitable folder

git clone git://github.com/rails/rails.git rails.git

I like to give repositories like this a .git extension so can easily differentiate from local things, but you can call the folder whatever you want.

After the repository is cloned be sure to checkout the branch you want, in my case git checkout origin/2-3-stable

Now you can vendor your rails projects by going into the root folder of the project and running

ln -s /path_to/rails.git vendor/rails

Adjust the path as required.

Updating Passenger

Posted by Andrew Premdas Wed, 22 Jul 2009 17:18:00 GMT

Updating the passenger gem is a bit of a pain. Its dead easy to just to the gem update and forget to do the other stuff you need to do, especially if you update lots of gems at the same time.

Anyhow after updating passenger be sure to run passenger-install-apache2-module as root. Follow the instructions and update the passenger config (in/etc/apache2/other/passenger.conf on my work box)

Searching for empty form fields using rspec

Posted by Andrew Premdas Tue, 21 Jul 2009 10:05:00 GMT

Took me far to long to work out how to do this!

To find an empty input field we can look for

"input[type=text][value='']"

e.g.

response.body.should have_tag("input[type=text][value='']")

This is useful when filling in forms badly to make sure your bad values still remain when you return to the form to edit them.

Contextual Controllers

Posted by Andrew Premdas Fri, 03 Jul 2009 04:25:48 GMT

I've been struggling alot recently with controllers and with code written by my colleagues. During this period I think I've been making quite a few mistakes. One of the rules I generally like to make about things particularly in software is that if its stupid or doesn't work its probably your fault. This is a really important rule which is real easy to apply to anyone but yourself. So its really easy for me to spot others making mistakes by not applying this rule. When I hear people say Selenium doesn't work, or SASS is rubbish I can comfortably be a wise old owl and think, "ah they should just take a little more time and they will realise that these judgements are made hastily.

Another rule I like to apply is that we (human beings) are mostly ignorant about everything. Socrates (who at the time was considered the wisest of the Greeks) used to tell people seeking his wisdom that he knew nothing. This wasn't some glib bit of rhetoric to reinforce his social status. All his years of study had made him fundamentally aware of how vast knowledge is, how much depth any subject can have, and how little he really did know and understand. This ignorance goes far deeper than just a lack of factual knowledge and goes from the global to the intimate. Using this rule is really dangerous, its so easy to apply it to others and to forget to apply it to yourself, doing this results in smug ignorance. In fact the only value of this rule is self application. It is perhaps a more dangerous expression of the buddhist concept of a "begginers mind"

So what does this have to do with controllers

Well I've been struggling with existing code in our application and in particular with trying to keep code out of controllers. I don't want code in controllers for a number of reasons

  1. They seem to be much harder to spec than models
  2. It seems real easy for various bits of business logic to end up in them - where it is really hard to test
  3. Its real easy for important things like error messages to end up in controllers, and again it feels that this is the wrong place for them
  4. I'm just not comfortable with them, they don't feel like good objects.

Now I've recently been dealing with alot of existing code in controllers that I've not been happy with. In our application we are now treating addresses as a polymorphic nested entity that other entities can have. In addition we have introduced the concept of NamedAddress, which are special in the fact that they have a name and also in the way they persist. Both our controller and views; and our features and steps have struggled to isolate the addressable bit from the relationship with the particular parent.

During this process I've been learning lots, improving the code and generally working really hard on improving quality. Trouble with working really hard is that its really easy to work really stupidly and I've been doing that too. I've also been assuming that existing code is stupid. Take the following code in the index method of the OrdersController

def index
  @pending_orders = @customer.pending_orders
  @completed_orders = @customer.completed_orders
end

I broke this code (well actually the view it shows) by the following refactoring of routes

 map.resources :customers do |customers|
   customers.resources :addresses
   customers.resources :orders
 end

to

 map.resources :customers, :has_many :addresses, :shallow => true 
 map.resources :customers, :has_many :orders, :shallow => true

This is a really stupid refactoring. Why?

So back to this index method which I'm now having to look at because my features are broken. Well its obviously wrong, because its an Order controller with customer code in it. I've just spent ages making sure that my Address controller doesn't have code that refers to a specific parent, and refactored lots of features and in particular steps that should have been Addressable not Customer/Address or Order/Address. And now on top of that there are all these Order features and in particular steps that have @customer in them. So now I'm peeved and am about to start to work even harder to get this stuff sorted when ... I have to go home.

Time passes ... and I realise that "its my fault" and that "I am stupid"

Remember "if its stupid and it doesn't work its probably your fault". Remember that stupid refactoring! and the fact that your added shallow routes to the customer order relationship at the same time as to the customer address relationship. Understand that applying shallow routing will change/break numerous named routes and url generation methods. Realise how stupid it is to do this in for two relationships at the same time. Realise that doing this refactoring for one relationship should be a seperate issue and commit - and here I am trying to do this for two relationships in the middle of doing something else!! Realise that I shouldn't even be looking at the OrdersController never mind making flawed judgements about the code in it. Understand that in the 'shop' part of this application that orders fundamentally belong to a customer and cannot exist without them. Realise that my colleagues implicitly get this possibly due to their greater experience with ecommerce. Realise that if the index method in the OrdersController is flawed its only very minor (perhaps just the customer needs to made available to the view). Realise that controllers are more contextual than models; and that the implied one to one relationship between controllers and models is a figment of my misunderstanding. Realise that a simple one line before filter can express a controllers context and allow a controller to fulfill a significantly different role.

There is lots more, but I'm really tired now.

So once again programming provides that wonderful opportunity to learn, and reinforces the need for my to keep my little rules and diligently apply them to myself.

Using semantic meaning in features and user interfaces

Posted by Andrew Premdas Fri, 24 Apr 2009 05:59:00 GMT

WORK IN PROGRESS

This explains why I need to fork webrat, and why I believe features should rely upon css classes and id's.

I believe that the CSS class is generally the best mechanism for expressing the semantic meaning of a particular piece of html (combined with DOM location and html tags). For example if somehow in viewing an order we want to reorder the order then I would first produce

.order
  .reorder
    # re-order goes here

This clearly establishes the semantic meaning - that I can reorder an order. Now the actual UI could be a number of things

  • Instructions: Reorder by phoning 0800 555 5555 and quoting order#
  • a button [reorder]
  • a link click <here> to reorder

Now as far as my business requirement and feature is concerned I don't care about these implementation details, and these details are certainly volatile. However the semantic layout is fixed. Here I see a representation of an order and can reorder the order.

I can now refine this feature to state that I should be able to tell the order to reorder. When I do this I should see some response in the UI which indicates whether my command has been successful.

When I reorder
Then I should see ...

In web applications you can do this in two ways

  • follow a link
  • submit a form (by pressing a button)

Again my feature does not care how I reorder unfortunately with webrat I now have to choose - I can either 'click' or 'press'.

.order
  .reorder
    %a{:href => order_path(:id => order.id)}= h(order.name)

above I am 'clicking' a link, whilst below I am pressing a button

.order
  .reorder
     -form_for @order do |f|
       ... 
       f.submit "reorder"

Now as far as my feature is concerned I don't give a damn how the re-ordering is done I just want to reorder. However I have to change my scenario (yes I know I can hide this change to a degree by re-using steps and having fancy step definition).

Scenario: Reorder
Given there is an order
When I view orders 
And I click reorder
Then ...

     OR 

Scenario: Reorder
Given there is an order
When I view orders 
And I press reorder
Then ...

Webrat compounds this frustration by making it really hard to press or click. Currently it will click only using text, id or an href, and its even more restrictive when it comes to pressing. So my first improvement was to extend click link to also be able to use classes. Now at least I can do

click link reorder

Although I have to change my implementation code

.order
    %a{:clases => reorder, :href => order_path(:id => order.id)}= h(order.name)

My Fork of Webrat

Posted by Andrew Premdas Fri, 24 Apr 2009 05:41:25 GMT

Webrat has become a very important part of my coding toolkit, to the point where its omissions and flaws cause great annoyance. However with the rise of Github we can just fork webrat, fix things and then issue pull requests and hope to get things back into the original.

This is what I have done with Webrat.

However my pull request has been ignored and now has been rejected ticket ? Here again Git provides a solution, I just keep maintaining my current version of the plugin pulling changes from the original. To do this I have to overcome a couple of problems

  • gem generation
  • differentiating my version of the plugin from the original

Gem Generation

Webrat uses a rake task to generate its gem, and does not want github to generate gems. I do want github to generate my gem so I've added 'jeweler' tasks to the rakefile so that it is easy for me to pull changes and regenerate a .gemspec for github to use. As a gemspec contains a list of every file in the gem, it is very important to update the gempspec every time I pull otherwise my version of the gem will end up missing files.

So I installed jeweler

sudo gem install jeweler

Then added following to rake file

require 'jeweler'
...
Jeweler::Tasks.new(spec)

This adds loads of rake tasks rake -T, and I can use rake gemspec to build my gem.

Differentiating my Version

With a correct gemspec in my repository. Github should do the business and build diabolo-webrat. Now peeps can use this with the same convenience as the original.

I discuss in another article my reasons for forking webrat.

Writing Classes why its so easy to do it wrong in Ruby

Posted by Andrew Premdas Tue, 07 Apr 2009 06:01:18 GMT

Working in Rails and dealing with Rails code and various bits of Ruby in plugins etc. it is easy to forget some OO fundamentals. This is partly due to the power of Ruby and partly due to specialist classes like controllers, models and specs distracting me away from the fact that all these things are still classes.

There used to be a rule of thumb used in Java that no method should be more than 10 lines long. Personally I took this down to 5 lines. Ruby is so powerful, that in 10 lines we can create something seriously complex, in fact we can do this in five lines. The problem with these rules of thumb is that they seem pretty stupid unless you know the underlying reasoning behind them. So lets explore this a bit further and remember how to write classes properly

Public, Private

The methods a class responds to are its public interface. This interface should document how the class works. The code that implements this interface has two purposes

  1. To further document intention, i.e. to describe what we are doing
  2. To do stuff

Surprise, surprise - the first is far more important than the second.

Learning from Java

When writing in clunkier languages like Java it becomes particularly obvious that public methods should actually delegate doing stuff to private methods. Well written java classes will have a small number of public methods, supported by a large number of private methods which actually do the work.

This is an old bit of java I wrote some time ago. The good thing about this code is that the public method clearly explains what is happening

public class Parser {

  ... // twenty lines of code defining state in private variables - including out

  public StringBuffer parse(File in) throws FileNotFoundException, IOException, ParserException {
    this.in = in;
    checkParseArgs();
    checkFileExists();
    openFile();
    initialiseStorage();
    visitedFiles.add(in);
    setCurrentWorkingDirectory();         
    while (getNextInclude()){
      processInclude();
    }
    return out;
  }

  ... // 230 lines of code and comments defining 21 void private no arg methods that actually do the work
}

Of course we don't want to go back to the verbosity of java, but in becoming much more line efficient in our code, we shouldn't forget the importance of clarity of public methods. If I turn this into English I get

Parser parses a file by,

  • checking the parse args
  • checking the file exists
  • opening the file
  • initialising the storage
  • adding the file to the visited file list
  • setting the current working directory
  • finally getting each include and processing it

Now this isn't perfect - I don't know what an include is (yet), but its fairly close to self documenting code, and its easy to find out things by just navigating to the private methods

The private methods in this class either look like the public method, or do one thing. I'll show a few of them below

private void checkParseArgs(){
    if (in == null) {
        throw new IllegalArgumentException("Parser.parse() will not except null arguments");
    }
}


private void processInclude() throws IOException, ParserException{       
    getIncludeFile();
    parseIncludeFile();
    replaceIncludeWithFile();
}  

private void getIncludeFile() throws IOException, ParserException{
    fReader.mark(tagStartInclude.length() + 255);
    readToStartOfPath();
    readPath();
    readEndTag();
}

private boolean isComment() throws IOException {
    int len = tagStartComment.length();
    char [] buf = new char[len];
    fReader.mark(len);
    fReader.read(buf,0,len);
    String s = new String(buf);
    fReader.reset(); // back to start of tag
    return tagStartComment.equals(s);
}

OK so in Ruby terms this is all very dull and long winded and we can do things much quicker and terser. But what I'd like to keep from this code is the idea that even in Ruby public class methods should clearly document their intention and in general delegate to private methods to do stuff. I'd also like to keep the idea that a method either does something or tells you how something is done - but not both. Finally I can remove all complex conditionals from my ruby code using these techniques. No need for case statements and definitely no need for nested ifs!

By the way, I wrote this parser class 7 years ago. Coming back to it, it took me about 5 minutes to work out what it does and how it does it, and this reminded me of why I wrote it. It would be nice if the ruby code I write today would be so clear in 7 years time!

Ruby Classes

With Ruby its very easy to forget about private methods and just implement 10 line public methods. From one of my Rails models I have

def weight_in_kg=(weight)
  if weight.nil? || weight == ""
    self.weight_in_grammes = weight if self.weight_in_grammes.nil?
  else
    strip = weight.strip[@@strip_regex]
    split = strip.split('.',2) unless strip.nil?
    if split.nil? || strip.nil?
      self.weight_in_grammes = weight if self.weight_in_grammes.nil?
    else
      kg = (split.first).to_i
      gms = split.size > 1 ? (split.last).to_i : 0 
      self.weight_in_grammes = kg * 1000 + gms
    end
  end
end

Now this code is just so much harder to read than the java code above. At 12 lines its not long and in Ruby its just so easy to write! And with my unit tests I can get away with this code. But I wrote this about 9 months ago - and I have less idea of what it is doing than the java code I wrote 7 years ago! Create a few hundred methods like this and you'll soon have no idea of what your code is doing.

Features for a Resource

Posted by Andrew Premdas Tue, 07 Apr 2009 04:33:32 GMT

REST is great, and resources are a great design tool. If we design our rails application around resources we can get alot of things for free. If we apply the same idea to features can we create a standard set of features to give us a quick start for each resource we identify?

Resources - Object and Collection

With standard rails routing we will get various named routes which we can use either directly in our features or indirectly in our step definition

For example if our routing table has

     new_admin_product GET    /admin/products/new
 destroy_admin_product GET    /admin/products/:id/destroy
    edit_admin_product GET    /admin/products/:id/edit
         admin_product GET    /admin/products/:id

              products GET    /products
               product GET    /products/:id

Then we can easily write

When I view products

and match this with

When /^I view product$/ do 
  visit products_url
end

In a similar way we can do things like

Given there is a product with name widgit
When I view the product

matched with

Given /there is a product with name (\S+)$/ do |name|
  Product.generate!(:name => name)
  @x = Product.find_by_name(name)
  @x.should_not be_nil
  @x
end

When /^I view the product$/ do 
  # use the variable x from above
  visit product_url(@x)
end

Its fairly straightforward to extend these features and steps to match the other actions, and with a bit of ruby and regex fu we should be able to generalise these features so we can use them with any resource

Resource Views

The simplest way to do this is to use CSS classes. I take this approach because it is robust. Object are represented in a div with the singular name as a class and collections use the plural. e.g. in Haml

.product 
  product details go here

.products
  list of products go here

Resources - Info (A Basic Data Definition)

I think it might be a nice idea to describe a resource on its list page using some text in an div#info. e.g.

#info
  Products are things that we actually sell. They come from wholesale_products which are supplied to us.
.products

This is just a little convenience which will be replaced/hidden when we start doing specific work with the resource.

Composing Features

Posted by Andrew Premdas Thu, 19 Mar 2009 02:20:24 GMT

Composing Features

Writing good features is an art-form, a very new art-form that people are just beggining to explore. Of course people have been specifying how things should work throughout history, and people have been trying to specify what software should do (with varying degrees of failure!) since software has been written.

You can compare writing features to a new school of art e.g. pontillism. Pontillism built itself on the existing foundations of painting and art - in particular colour theory - and applied these using a new technique to create something a little different. In our new school of features we produce something a little different (a plain text executable) with a new tool (Cucumber) and the application of old and new techniques. To do this well we have to learn what works and what doesn't.

Asking Questions

When we write a feature we need to ask lots of questions. A feature is an exploration of something we want to do. Starting a feature is focused on two things, loosely describing the feature and asking the most important question.

  • Why do we want to do this?

The first temptation when writing features is to skip this question! After all we know we need the feature, otherwise we wouldn't be writing it in the first place. And we want to get on with things! What's the point in wasting time on this question?

Well your about to commit a substantial amount of your time writing this feature and a substantial amount of other peoples time in implementing this feature. If it takes you 20 minutes to answer this question properly and you discard one in fifty of your potential features you will still have saved loads of time and effort.

To help you answer this question you can consider the following questions

  • Whats the point?
  • Can it wait?
  • Is there something more important we need to do first?

Writing Scenarios

Once we get to the stage of writing scenarios, we should be fairly confident that we are not going to be wasting our time writing the scenarios. In addition to specifying what should be done, writing scenarios help us to discover things about our feature. As we write our feature we should be open to the following ideas

  • We need to do something else first!
  • We don't need to do this now!
  • This is pointless!
  • We don't need this!

Hopefully the point is made now!

Writing Scenarios (the details)

As you write a scenario its a good idea to ask a few more questions.

  • What am I assuming?
  • Where does this come from?
  • Is this a compound statement?
  • Do I know what this really means? And even if I do can anyone else?
  • Who can do this?
  • Who should not be allowed to do this?
  • Is this name ambiguous, clumsy, accurate, intuitive?

To write scenarios efficiently we need to develop the skills to ask the write questions at the right time. A warning though - not asking questions, guarantees you will write rubbish, be innefficient, waste other peoples and never develop the skills to write scenarios quickly. Remember this is an art form you have to really work at it to get good.

I'll try below to give some illustrations of applying these questions

Examples (ECommerce Products)

NOTE: Just sketching out things here for the moment

Scenario: A product should

Illustrate composition using VAT Ask the questions

Where does the Supply Price come from? Is it more important to write about this first? Where does the VAT rate come from.

Stop writing this feature and write the feature that sets the VAT rate. As your doing this ask

How many VAT rates? What happens when the VAT rate changes? - to new sales, to old sales How do we change the VAT rate

  • write the feature

Feature As a site admin I need to change the VAT rate for all products In order to comply with the national changes to VAT caused by the credit crunch

Do we need to keep the VAT rate with each purchase?

Is the VAT rate

If the supply price comes from

Feeds and Shit

Posted by Andrew Premdas Fri, 13 Mar 2009 01:31:02 GMT

Feeds and shit

Feeds could be a strong competitive edge for an ecommerce platform. So we should do them properly and automate the process. If we do this then we can standardise the administration functionality to

  • monitor feeds
  • deal with errors
  • deal with conflicts
  • summarize and report on changes

Feed

has one source has one supplier has many runs has one last run has one Adaptor

Adaptor

Consumes run.feed.source produces run.log, run.errors, run.output has many runs

Run

has one output has errors has datetime has one log has one adaptor has one feed has one last-run

Product

has many updates

Update

has one Product has one Feed

For each new feed we will have write and Adaptor. We can use ActiveMerchant for guidance as it has 'Adaptors' for each payment engine. Of course we can use OO and existing adaptors to make this job easier.

Example Adaptors

Excel Spreadsheet Adaptor

Adaptor checks if Acme's spreadsheet has been changed since last time. Then it calls a ruby script that converts the spreadsheet into XML. Then it processes the XML.

Funky Web Service Adaptor

Adaptor connects to BeCool's Product Update Service. It pulls from it all the changes since the last run into some funky YAML temporary file which it processes ...

Duper 2 way Adaptor on Demand

2UptoDate.com connects to our Adaptors web service and says heh my products have changed. Adaptor connects back and gets changes in funky YAML etc.

Older posts: 1 2 3 ... 5