Metaphysical Developer

Closures, Collections and some Functional Programming

Posted in Languages by Daniel Ribeiro on May 2, 2009

Collection libraries are quite common today in almost all languages, and we are very glad that every non trivial piece of software does not require us to define again what a List is, neither what a Tree is. Also, we do not have to keep on reimplementing basic algorithms such as sort. Some languages even provide literal ways to define the most commonly useful collections, such as lists and maps.

Closures (also known as blocks and lamba expressions) are also common on several languages (Java and C++ being notable exceptions), even though it only started becoming more mainstream with Ruby. Besides being useful on creating DSLs , they allow us to easily define callback procedures on GUIs, declare transactions, readily craft simple implementations of Visitors and Commands patterns, enable controlled lazy evaluation on languages that do not natively support it, refactor complex switch statements to a simple map leading to Closures, and so on.

Even though closures and collections are useful on their own, when combined they allow us to abstract iteration mechanisms. Smalltalk is notable for blending closures and collections in a category of methods that form a protocol called enumeration protocol. For instance (examples in ruby, that also features an enumeration protocol, similar to smalltalk), we can rewrite this:

novels = []
for b in books
  if b.novel?
    novels << b
  end
end

Into this:

books.select {|b| b.novel?}

This way the code gets clearer, more concise and more simple, while encapsulating the actual iteration algorithm, which allow us to reuse the very same filter whether the books are coming from a list, from a file, from a database, or from a SOAP request. Not only that, but it allows us to refactor several filters of books into one line methods:

class Books
  def novels()
    return @books.select {|b| b.novel?}
  end

  def older_than(year)
    return @books.select {|b| b.age > year}
  end

  def name_starts_with(str)
    return @books.select {|b| b.name[0, 1] == str}
  end
end

But abstract iteration mechanisms are more than just filtering. Common enumeration protocols (such as those from Ruby and Smalltalk) features several methods to detect elements, to create new maps with a function, sort elements, get the maximum and minimum elements (using a closure as a definition of comparison) and so on. Many of these are inpsired by functions orginally defined on functional programming languages, such as fold and map.

The encapsulation of iteration mechanism is also usefull when we want to start using coarse-grained parallelism, such as proposed by Fork Join (which, ironically is written in a language that does not have closures) . That is: you had a sequential code, and with very little alteration, you can get a parallel/distributed one (while being cautious of side effects). In our example: Books#novels could be very well selecting the novels in separate threads, and joining them all together. It would only be a matter of changing the implementation of @books.

Concluding: Closures and Collections allows us to improve code by:

  • making it more concise
  • making it clearer
  • making it more suitable to refactoring
  • making sure you Don’t Repeat Youself
  • enabling you to easily turn sequential code into parallel

Considering that all of these points are too good to let go, and considering that java 7 will not get closures and that not only are anonymous inner classes really closures but also too verbose, I started a little project called Fluent Java. This project, among other things, brings up the enumeration protocol to java, while still keeping it terse, and easy to use (Fork Join integration is planned as well).

Edit: Java might get simplified closures after all (at least, for now, it is planned). But still too early to say regarding how this was already pulled off once, and Oracle has some issues to work through with Sun, which may push JDK7 to being released early.

Edit (22 Sep 2010): Yes, it got pulled away. From the Chief Architect of the Java Platform Group: It’s time for … Plan B. Scala has them though…

About these ads

7 Responses

Subscribe to comments with RSS.

  1. ben said, on May 25, 2009 at 8:03 am

    I’ve been using my own closure and functional style java for years now. It works really well.

    The thing is, the ruby examples are nice for simple things, but when you have a predicate that is very complex, multiple predicates, or boolean logic with predicates you need to abstract things out.

    The only real thing that prevents us writing great code in plain old java I have discovered is the lack of method and field literals.

    Consider this class:

    public class Test {
    private String name;
    private int id;
    }

    and the following [generated] class, which should be trivial to do for most programmers:

    public class TestMeta {
    public static StringField name = new StringField(“name”);
    public static IntegerField id = new IntegerField(“id”);
    }

    Now you have power:

    List tests = …;
    List filtered = select(tests).where(name.startsWith(“c”).and(id.lessThan(100)));

    When you generate literals, those literals can provide type safe predicate generation, closure generation etc.

    Another more complicated example can be:

    List ids = select(tests).where(name.startsWith(“c”)).get(id).sortBy(name);

    As you can see, this dramatically reduces programmer error, little if any anonymous classes are needed, and everything can be easily covered with a test suite.

    If you want, you can have different generators that output things such as sql-specific operations.

    Similarly you can have method based calls:

    public class TestMeta {
    public static StringField name = new StringField(“name”);
    public static IntegerField id = new IntegerField(“id”);
    public static IntegerMethod<Collection> sum = new IntegerMethodTest<Collection>(“sum”);
    }

    And can be used similarly:

    int total = select(tests).where(name.startsWith(“c”)).fold(sum);

    Thoughts?

    • danrbr said, on May 25, 2009 at 8:59 pm

      I showed similar on Fluent Java’s examples, as it can be seen here:

      Set result = books
      .findAll(hasProperty(“year”, greaterThan(1999)))
      .map(“getAuthors”)
      .flatten()
      .select(hasProperty(“firstName”, startsWith(“M”)))
      .toSet();

      On these examples I reused pre-made functions from Hamcrest, Google Collections, Fork Join, and easy to use getter functions and ways to craft functions from methods.

  2. ben said, on May 25, 2009 at 8:04 am

    Btw it stripped out my generics in my comment above. Everything is type safe.

  3. ben said, on May 25, 2009 at 10:16 pm

    Dan, that’s great, but in my opinion unless its refactorable you shouldn’t use it. Ive been bitten too many times by code like you prescribed in production sysems where reflection just breaks.

    Code generation is the happy medium. It ensures that you can’t make a mistake like this:

    Set result = books
    .findAll(hasProperty(”years”, greaterThan(1999)))
    .map(”getAuthos”)
    .flatten()
    .select(hasProperty(”firstNme”, startsWith(”M”)))
    .toSet();

    If you are happy to do this, then you may as well just use ruby or scala.

    • danrbr said, on May 26, 2009 at 9:36 pm

      It can be refactorable using dynamic proxies, while taking the fluency down a bit (or even by using the utterly verbose annonymous inner classes) Of course, writing such dynamic code takes the need of tests (or better yet, TDD) to a whole new level. If you do it, then you should be fine.

      Regarding scala and ruby: I almost agree with you. But the talk about this is kinda lengthy, and it will have to wait, as I have some more pressing issues to talk about before (possibly in two weeks I’ll address this issue more carefuly).

      But it is good that people start feeling the inconveniences that lack of Closures can be to Java (or any other language for that matter).

  4. ben said, on May 26, 2009 at 11:54 pm

    Agreed, this is getting long.

    I personally think that the strong points of java are compile time checking + refactoring + tooling. If you are going to throw away compile type checks (and thus refactoring), you may as well use a more dynamic language as you are only left with tooling, but at a reduced effectiveness. Plus, you end up writing tests for all your dynamic code.

    So, in my opinion it’s code generation or leave.

    And, when you can use code completion, refactoring and compile time checks for the code I presented above, life is pretty sweet. I don’t understand why it’s not used more often myself.

    • Epo said, on August 10, 2009 at 6:03 pm

      Hi Ben

      This sounds very interesting.

      Is there a way you can provide us your framework or more sample ?

      I also use code generation and this meta-capabilities would add a lot.

      Thanks in advance

      Epo


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: