Metaphysical Developer

Idiomatic Clojure with LightTable

Posted in Clojure, Software Development by Daniel Ribeiro on November 18, 2013

LightTable, the famous innovative IDE (slash reactive work surface, as described by the author) recently got some pretty amazing additions: user defined plugins and user defined custom expressions.

Custom expressions allow users to define functions that get any block of text and replace with the result, stream information back to LightTable, or display the result inline:


The clojure project Kibit immediately came to mind when I saw the new addition. Kibit is a static code analyzer for Clojure that allows you to find a more idiomatic way of writing a given block of text.

So I wrote this short LightTable integration for it. With it, LightTable can replace any s-expression with its more idiomatic version. For instance, this code:

Screen Shot 2013-11-17 at 5.53.01 PM


Screen Shot 2013-11-17 at 5.53.21 PM

To see how Kibit did, here is the before and after shots of the expression broken down in its components:




In order to use Kibit with LightTable, you just need to:

1. add lein-kibit it to your project.clj:

Screen Shot 2013-11-17 at 5.56.49 PM

2. add the integration function to your usermap:

Screen Shot 2013-11-18 at 10.54.40 PM

The sample project can be found on GitHub.

Tagged with: ,

Comments Off on Idiomatic Clojure with LightTable

Downgrading RubyMotion

Posted in Software Development by Daniel Ribeiro on May 31, 2013


RubyMotion is a great way of writing native iPhone apps using Ruby rather than good old Objective-C.

The excitement around it has not stopped at 37 signals’ post about Why I loved building Basecamp for iPhone in RubyMotion: many libraries creating very convenient APIs for doing all sorts of things were created. A non exhaustive list include BubbleWrap, Teacup, Futuristic, Motion-Xray.

As with any new piece of fast moving and new technology, deprecation moves fast. Recently I found myself picking up a open source library that was very useful, but did not keep-up with the platform’s pace, and therefore it would no longer build with the latest version of RubyMotion (which was a major number upgrade to version 2). Needing to build the original app in a few hours, I looked into how to downgrade my RubyMotion version.

I could not find anywhere how to do it, but thankfully RubyMotion’s support was quick to help me with it:

$ sudo motion update --force-version=1.35

And that was it. Back to having a lots of fun with IOs development.

Tagged with: ,

RubyUnderscore: A bit of Arc and Scala in Ruby

Posted in Languages, Software Development by Daniel Ribeiro on October 31, 2010

A few months ago I’ve mentioned one thing that bothered me in ruby was No way to create simple blocks”. This is in contrast to other languages, such as Scala, Clojure and Groovy’s underscore, percent and “it”, respectively, shortcut notations. There are other languages with equivalent mechanisms as well. Even newer languages like Coffeescript have considered adding it. As James Iry mentioned, such constructs are in fact related to delimited continuations.

However ruby has Syntax Tree manipulation (via Parse Tree gem). Using it I created the RubyUnderscore project, which brings this ruby, using the underscore symbol (just like Scala and Arc). With it, it is possible to refactor the following:

    classes.reject { |c| c.subclasses.include?(Enumerable) } { |d| d.greater_than(old_date) } { |x| x.invoke }


    classes.reject _.subclasses.include? Enumerable _.greater_than old_date _.invoke

The last case can also use symbol to proc coercion (appending & to symbol): &:invoke

However, the proc coercion is not flexible enough to allow arguments or invoke a method chain. Which I think brings a small increase in readability and code quality, not to mention that by making closures easier to declare, it fosters them to be used more. I find this to be a good thing, specially when you start to refactor your loops into maps, selects, rejects, group_bys and reduces

This also highlights another issue I mentioned in Improving Ruby: that Syntax Tree manipulation is too important to be supported only on MRI, and not throughout the implementations, like JRuby and Rubinius. Python has this built in into its standard library (through ast module and inspect.getsource), and Lisp can also do syntax tree manipulation with its macro system. The importance of such capability was mentioned by Paul Graham (one of the creators of Arc, which is a dialect of Lisp):

Letting people rewrite your language is a good idea. You, as the language designer, can’t possibly anticipate all the things programmers are going to want to do with it. To the extent they can rewrite the language, you don’t have to.

However, syntax tree manipulation in ruby is not only unsupported in most implementations, but it is also poorly documented (even though PostRank’s founder Ilya Grigorik‘s post on the subject is a very good introduction) and a bit awkward to use: the visitor from sexp-processor gem embraces side effect (mutating all the tree nodes while processing them) and the tree nodes are just arrays, unlike Python’s modules where there is a class for every node type. It is important to note that if you are willing to pre-process your ruby code, you can use ruby2ruby to generate the equivalent and regular ruby code, which will work all over.

These techniques are expected to be fixed as more people realize the gains they bring, and these improvements find their way into YARV and eventually other ruby implementations. Ruby is a very nice, clean, productive and elegant language, and it would be shame if we stopped making it even better.

Tagged with: , , ,

Scrapbooking Startup Essays

Posted in Software Development, Startups by Daniel Ribeiro on April 30, 2010

I have been talking a bit about startups recently. In these conversations it is common that I bring up some essay from Paul Graham, as he wrote a lot of very insightful articles on the topic (not to mention invested in). However, there are too many good essays, and I usually have a hard time remembering the highlights among all of them, or pointing them to friends. Therefore I have assembled the parts that I personally found to be the most insightful, clever, intriguing  and/or just plain controversial.

Note: I have published a Greasemonkey script called Paul Graham Topics that adds a topic sidebar on the essays, to make it easier to read them.

For those who dislike Greasemonkey, or just wanna see if it is worth installing it, I’ve uploaded here a Sinatra app on Heroku (which incidentally was funded by Paul Graham) that adds the script. The source of the project can be found here.

Edit (24/Jun/2010): Added Paul Graham’s definition of startup  (from How To Make Wealth), just to contrast with this one given by Eric Ries (who is getting a lot of attention lately with his Lean Startup movement and Startup Lessons Learned conference): “A startup is a human institution designed to deliver a new product or service under conditions of extreme uncertainty.”

Disclaimer: I do not claim ownership of any of the text below.

Ideas for Startups
If 98% of the time success means getting bought, why not be open about it?

Perhaps letting your mind wander is like doodling with ideas. You have certain mental gestures you’ve learned in your work, and when you’re not paying attention, you keep making these same gestures, but somewhat randomly. In effect, you call the same functions on random arguments. That’s what a metaphor is: a function applied to an argument of the wrong type.

How to Make Wealth
Startups usually involve technology, so much so that the phrase “high-tech startup” is almost redundant. A startup is a small company that takes on a hard technical problem.

(…) it is, as Edison said, one percent inspiration and ninety-nine percent perspiration.

One way to put up barriers to entry is through patents. But patents may not provide much protection. Competitors commonly find ways to work around a patent. And if they can’t, they may simply violate it and invite you to sue them. A big company is not afraid to be sued; it’s an everyday thing for them. They’ll make sure that suing them is expensive and takes a long time. Ever heard of Philo Farnsworth? He invented television. The reason you’ve never heard of him is that his company was not the one to make money from it. The company that did was RCA, and Farnsworth’s reward for his efforts was a decade of patent litigation.

The mere possibility of being interrupted deters hackers from starting hard projects. This is why they tend to work late at night, and why it’s next to impossible to write great software in a cubicle (except late at night).

Big companies can develop technology. They just can’t do it quickly. Their size makes them slow and prevents them from rewarding employees for the extraordinary effort required. So in practice big companies only get to develop technology in fields where large capital requirements prevent startups from competing with them, like microprocessors, power plants, or passenger aircraft. And even in those fields they depend heavily on startups for components and ideas.

A Student’s Guide to Startups
That is one of the most distinctive differences between school and the real world: there is no reward for putting in a good effort.

One of the things employers expect from someone with “work experience” is the ability to get things done, with no excuses

So class projects are mostly about implementation, which is the least of your problems in a startup.

What Startups Are Really Like

If ideas really were the key, a competitor with the same idea would be a real threat. But it’s usually execution that matters.

How to Start a Startup
No matter what kind of startup you start, it will probably be a stretch for you, the founders, to understand what users want. The only kind of software you can build without studying users is the sort for which you are the typical user. But this is just the kind that tends to be open source.

It’s easier to make an inexpensive product more powerful than to make a powerful product cheaper

Why Smart People Have Bad Ideas
Work people like doesn’t pay well, for reasons of supply and demand. The most extreme case is developing programming languages, which doesn’t pay at all, because people like it so much they do it for free.

Why to Not Not Start a Startup
That’s ultimately what drives us to work on Y Combinator. We want to make money, if only so we don’t have to stop doing it, but that’s not the main goal. There have only been a handful of these great economic shifts in human history. It would be an amazing hack to make one happen faster.

It’s exciting to think we may be on the cusp of another shift like the one from farming to manufacturing. That’s why I care about startups. Startups aren’t interesting just because they’re a way to make a lot of money. I couldn’t care less about other ways to do that, like speculating in securities. At most those are interesting the way puzzles are. There’s more going on with startups. They may represent one of those rare, historic shifts in the way wealth is created.

Hackers and Painters
I’ve never liked the term “computer science.” The main reason I don’t like it is that there’s no such thing. Computer science is a grab bag of tenuously related areas thrown together by an accident of history, like Yugoslavia. At one end you have people who are really mathematicians, but call what they’re doing computer science so they can get DARPA grants. In the middle you have people working on something like the natural history of computers– studying the behavior of algorithms for routing data through networks, for example. And then at the other extreme you have the hackers, who are trying to write interesting software, and for whom computers are just a medium of expression, as concrete is for architects or paint for painters. It’s as if mathematicians, physicists, and architects all had to be in the same department.

Great Hackers
After software, the most important tool to a hacker is probably his office. Big companies think the function of office space is to express rank. But hackers use their offices for more than that: they use their office as a place to think in. And if you’re a technology company, their thoughts are your product. So making hackers work in a noisy, distracting environment is like having a paint factory where the air is full of soot.

Succinctness is Power
I wrote about this in On Lisp. A complex macro may have to save many times its own length to be justified. If writing some hairy macro could save you ten lines of code every time you use it, and the macro is itself ten lines of code, then you get a net saving in lines if you use it more than twice. But that could still be a bad move, because macro definitions are harder to read than ordinary code. You might have to use the macro ten or twenty times before it yielded a net improvement in readability.

The Future of Web Startups
Is seed funding not merely national, but international? Interesting question. There are signs it may be. We’ve had an ongoing stream of founders from outside the US, and they tend to do particularly well, because they’re all people who were so determined to succeed that they were willing to move to another country to do it.

Knuth: Computer Programming as an Art
Language designers also have an obligation to provide languages that encourage good style, since we all know that style is strongly influenced by the language in which it is expressed. The present surge of interest in structured programming has revealed that none of our existing languages is really ideal for dealing with program and data structure, nor is it clear what an ideal language should be. Therefore I look forward to many careful experiments in language design during the next few years.

In this sense we should continually be striving to transform every art into a science: in the process, we advance the art.

“The science without the art is likely to be ineffective; the art without the science is certain to be inaccurate.”

What Business Can Learn from Open Source
There’s a name for people who work for the love of it: amateurs. The word now has such bad connotations that we forget its etymology, though it’s staring us in the face. “Amateur” was originally rather a complimentary word. But the thing to be in the twentieth century was professional, which amateurs, by definition, are not.

As in software, when professionals produce such crap, it’s not surprising if amateurs can do better. Live by the channel, die by the channel: if you depend on an oligopoly, you sink into bad habits that are hard to overcome when you suddenly get competition.

Be Good
You can see how there would be. When you’re small, you can’t bully customers, so you have to charm them. Whereas when you’re big you can maltreat them at will, and you tend to, because it’s easier than satisfying them. You grow big by being nice, but you can stay big by being mean.

You get away with it till the underlying conditions change, and then all your victims escape. So “Don’t be evil” may be the most valuable thing Paul Buchheit made for Google, because it may turn out to be an elixir of corporate youth. I’m sure they find it constraining, but think how valuable it will be if it saves them from lapsing into the fatal laziness that afflicted Microsoft and IBM.

From what we’ve seen, being good seems to help startups in three ways: it improves their morale, it makes other people want to help them, and above all, it helps them be decisive.

Rarely-Asked Questions
Couldn’t you add something equivalent to Lisp macros to languages like Perl or Python?
(…) But it would be hard to do that without creating a notation for parse trees; and once you do, your language has become a skin on Lisp

Why YC
The real reason we started Y Combinator is neither selfish nor virtuous. We didn’t start it mainly to make money; we have no idea what our average returns might be, and won’t know for years. Nor did we start YC mainly to help out young would-be founders, though we do like the idea, and comfort ourselves occasionally with the thought that if all our investments tank, we will thus have been doing something unselfish. (It’s oddly nondeterministic.)

The 18 Mistakes That Kill Startups
The reason we tell founders not to worry about the business model initially is that making something people want is so much harder.

The Power of the Marginal
I think that’s one reason big companies are so often blindsided by startups. People at big companies don’t realize the extent to which they live in an environment that is one large, ongoing test for the wrong qualities.

The eminent, on the other hand, are weighed down by their eminence. Eminence is like a suit: it impresses the wrong people, and it constrains the wearer.

So if you’re an outsider you should actively seek out contrarian projects. Instead of working on things the eminent have made prestigious, work on things that could steal that prestige.

If you really want to score big, the place to focus is the margin of the margin: the territories only recently captured from the insiders. That’s where you’ll find the juiciest projects still undone, either because they seemed too risky, or simply because there were too few insiders to explore everything.

So that, I think, should be the highest goal for the marginal. Be inappropriate. When you hear people saying that, you’re golden. And they, incidentally, are busted.

Five Questions about Language Design
(…) designing programming languages is like designing chairs: it’s all about dealing with human weaknesses.

If you look at the history of programming languages, a lot of the best ones were languages designed for their own authors to use, and a lot of the worst ones were designed for other people to use.

When I first learned Lisp, what I liked most about it was that it considered me an equal partner. In the other languages I had learned up till then, there was the language and there was my program, written in the language, and the two were very separate. But in Lisp the functions and macros I wrote were just like those that made up the language itself. I could rewrite the language if I wanted. It had the same appeal as open-source software.

Another thing that might turn out to be useful for server based software, surprisingly, is continuations. In Web-based software you can use something like continuation-passing style to get the effect of subroutines in the inherently stateless world of a Web session. Maybe it would be worthwhile having actual continuations, if it was not too expensive.

It’s traditional to think of syntax and semantics as being completely separate. This will sound shocking, but it may be that they aren’t. I think that what you want in your language may be related to how you express it.

Is it necessary to take risks to design a good language though? Many people might suspect that language design is something where you should stick fairly close to the conventional wisdom. I bet this isn’t true. In everything else people do, reward is proportionate to risk. Why should language design be any different?

The Failure of Code Reuse

Posted in Software Development by Daniel Ribeiro on September 30, 2009

Using pre-made solutions as parts of solving a bigger problem is a common motif on several areas of human knowledge. At first, it makes perfect sense not to waste valuable time making something that was already done. Encompassing this ideal, a lot has been said about doing this with software, what is commonly referred as code reuse.

There are three canonical types of reusing code, as listed by Erich Gamma and Ralph Johnson: software libraries, Design patterns and Frameworks. There are other kinds of code reuse such as software platform (similar to both a framework and collection of libraries ), services or even using a complete application and building on top of it. Each and every one of these types of code reuse differ in its learning curve, in its flexibility (in the sense of how many different new applications, libraries, platforms or frameworks can be built from it) and the rate of functionality/design embedded in it.

But things are not so simple as putting Lego bricks together. Whenever looking for reusable piece of software, how to pick the most suitable one? Even if there is only one piece of one kind, you still have to figure out if it is suitable, or how much work it will be needed to make it suitable (which is the most common case).

This is because the piece can be too complex, not well documented, not fulfill all the desired properties, not be robust enough, without binds to the language/platform you are using, not have a permissive enough license, not be robust enough or make trade-offs that are not really what you would expect (such as choosing CP instead of CA on CAP), not be supported by a big company, and so on. And this gets a lot more troublesome when you have lots of piece of software to choose from. Since evaluating these things take time, a more agile approach is to make the trade-off of how much time to spend evaluating such pieces in order to gain information on them, in order to reduce the risk of making an inappropriate decision (always weighted by how bad can it be to make one, somewhat similar to swot). Even then, on your core domain, reusing big pieces of code is hardly advisable.

However not all software is reusable. At least not easily reusable. The reasons lies on the difficulties of having a reusable design (such as those stated by Uncle Bob Martin in his SOLID series, or by Neil Bartlett on his Component Models Tutorial) and the fact that making it easy for others to use requires a bit more support (such as documentation, tutorials and/or screencasts). Not to mention that most business are not about launching frameworks or application libraries, and therefore, are not interested in this work (which can be good or bad, depending on the cost of making it reusable, and the money made/saved of doing it). Which ensues that a lot of software is not reusable. All of this could suggest that making reusable bits of software requires a Big Design Up Front. However, agile practitioners suggest making code reusable as it is actually needed (a last responsible moment approach). This has actually worked well in practice, as the framework Ruby on Rails was extracted from other projects from 37 Signals, and Apache Hadoop was extracted from another project: Apache Nutch.

Code reuse is not a simple matter, and should not be taken lightly. It involves decisions that can be more relevant from a economical/business perspective than from a purely technical perspective. In the end, the real failure of reuse is the failure to realize this.