Lisp: Show me the Beef.

This is the wendy's moment.

God do people like talking about lisp. How amazing and transcendent it is.

Now, I tried lisp. For one semester of Scheme in my programming languages class, unfortunately taught by a teacher who was unable to communicate to humans, and not because he was foreign.

I never went back. Don't even think about going back. Not even a little. Of course I bombed out of the Math department, and most lisp people are math people.

Still, lisp is beyond hype. It is the most mythical and legendary thing in computers, which is bizarre since usually the subject of such things has passed in antiquity. Lisp is right there. Go ahead, there's a dozen implementations. Even a JVM one. The legendary power, right there.

Here's the catch. And this applies to haskell too: WHERE ARE THE APPS? I've heard ludicrous stories of lisp hackers bragging 100x productivity over other languages.

Okay.

Where are the apps?

C begat the UNIX operating system, DOOM, databases, and many office suites. That is a serious software resume.

What has lisp given us? Emacs? Anything else? Anything?

Java doesn't have as amazing a resume as C does, but it does run huge numbers of web sites. Those are killer apps in their own right, transforming your life. How many sites are written in Lisp? That should be some low-hanging fruit. Nope, ruby beat you to that.

What is apparent after decades, let me repeat, _decades_ of existence, through the expansion of computing power by a factor of 1000, lisp has not delivered on its legend. Not even minimally.

Instead, what is apparent is that it is a cult language. This comparison is so appropriate, since the cult of lisp subdivides into even more subcults, even down to the individual. Between the non-infix notation, parentheses, macros, individually produced programming paradigms and features, one man's lisp code is unreadable by another.

If lisp was the most freedom in programming possible, then it used its freedom to shoot itself.

I'll grant the lispers this: if you write lisp, your IQ and ability are measurably above other programmers. And that is the problem with lisp. For every story about the lone lisper beating an entire competing programming department's output in some circumstance, I can't distinguish between the lone genius producing a lot of software in any language/ecosystem from the language/ecosystem itself.

Everyone knows that a single smart programmer can often beat the output of 5-10 average people having to write specs, communicate, have status meetings, status reports, agree on interfaces, deal with different communication vocabularies, and different abilities.

But it just doesn't scale. Lisp doesn't scale. Lisp programmers don't scale. Lisp programs don't scale.

And that's why the crowning achievement of Lisp is a text editor.

Weeds, Flowers, and Fruit Trees

Oh, you have a new language that will revolutionize the world?

JVM languages are sprouting. It's getting a bit ridiculous. Fun question for someone with more time than me: are there now more than 100 JVM languages?

Please, grant me the power of a weed whacker. Almost all these new languages are nothing but weeds.

See, you have your staple crops in the garden: wheat, corn, potatoes, etc. This is what Java-the-language (and software ecosystem) is. Not flashy, but it has many compilers, a full runtime, massive IDE support, libraries, and accessible to programmers of many abilities. Just like developed crops are fairly easy to grow, yield well, and basically get used to make white bread. It has decompilers, debuggers, monitoring, tracing, and tons of other ecosystem features.

Then there are the weeds. Sure they may look different or do a few things better than Java, but for the most part they lack IDE tooling, decompilers, monitoring, lots of good examples. Basically, not worth your time, but even worse, they are taking up cropland, water, fertilizer, and other resources that better plants and plant ecosystems could produce.

There are flowers. Oh, these languages look pretty, do amazing things. But you can't eat them, or make textiles, or animal feed. They have all the minuses of the typical weed language, such as crappy IDE support and ecosystems, but you keep them around since you can pluck it occasionally for fun, or for some specific purpose.

Fruit trees would be the best, lots of food, persist year-to-year, solid foundation, but beautiful.

Not a lot of fruit trees.

Anyway, the bar needs to be raised. Your compiler and syntax description isn't enough. A new language isn't a new language until it has:

- compiler

- IDE tooling:

- syntax highlighting

- code completion (assuming it has sufficient static typing)

- syntax error dynamic detection

- line-by-line debugging

- runtime variable/stack/etc debugging during stepthrough

- clean integration with existing java libraries, even if you have your own runtime

- a good set of examples that demonstrate the feature set of the language

That's just the bar for announcing your practical arrival on the scene. That's not even the beginning of being somewhat mature:

- decompilers, well, if the language becomes common enough

- web frameworks, enterprise frameworks, GUI frameworks, etc.

- web services, templating, and a host of other libraries that are specific to your language's style

- unit test frameworks, build frameworks, etc etc etc

- lots of programmers

- I could go on a long time. Basically, look at Ruby's rise and all the things that were necessary for that to not be a toy language: webserver that didn't constantly crash, optimizing VM (JVM basically does that for you), web framework (Rails), templating frameworks, and lots of other things.

And you know what? That's not even good enough. How does your language change the craft of programming? You know what? Ruby didn't do squat for programming. Ruby on Rails changed the craft of programming. Ruby + Rails was a legitimate new language.

My fundamental bitch with new languages is that they fundamentally don't solve problems any better than Java. They don't make it faster, they don't really change most day-to-day programming. Most make programming more confusing to people. Even groovy basically requires you to know java programming before you can learn to groovy. It's IDE plugins still suck (well, under eclipse). I think the typical JSP plugin is more useful, and that's pretty damning.

Java itself is careening toward C++ hell. There is so much in C++, it might as well be impossible to learn all the features. Considering the compilers barely implement all the features at a bare minimum of support (but pray, don't expect, for good compile error reporting). The features are bolted on. Annotations have basically become C macros.

A lot of what made Java great had nothing to do with the language. The language was a C-like dialect engineered solely for being easily understood by the majority of programmers. Java's killer features: JVM, cross-platform, cross-platform runtime library, internet support and applets, namespacing, unicode support, garbage collection, and FREE.

Compared with C programs, you didn't need an unreadable soup of macros for "cross platform" adaptation, didn't need to worry about the actual size of ints, eliminated lots of memory leaks, eliminated a ton of security holes with non-pointer math strings. It was so much easier coding in java, even if the language was boring.

The new languages are really just shifting around chairs on the deck of the cruise ship JVM and it's optimizing runtime, garbage collection, security features, libraries. The JVM provided a

Most languages are just feature-loading more inheritance options (multiple inheritance, mixins) and doing syntactic sugar (closures, autoboxing, type inferring).

What we really need is new ecosystem features: packaging and dependency management (OSGi, ruby gems, linux packaging methods, maven, namespace version tagging), cross-language compiling (coffeescript), monitoring (JMX++), features for modularizing code so these huge dependent codebase package trees don't grow out of control, a new messaging infrastructure for XML/JSON/whatever with integrated parse/query/path lookup, url-based classpath (not filesystem paths, urls, with a local cache), maybe integrate a RESTful storage paradigm like sling and JCR/jackrabbit for code and configuration storage, better dynamic reloading of code support, fundamental support for varied sizes of devices (micro/super-low footprint, smartphone footprint, desktop/server footprint), good support for scaled execution, RMI, multithreading.

Get me a new ecosystem, not a new language.

I hate game theory questions

http://www.techinterview.org/post/526325766/pirates

I hate this question, at least it's statement. A lot of these game theory questions are bandied about, but they are loaded with unspoken assumptions about human behavior behaving in perfectly logical rigidity to unspoken assumptions.

The unspoken assumptions here are:

- the pirates will obey the rules (uh, they are pirates, why would they consistently obey) over time

- pirates will not bargain or collude, except in the way that is laid out in the explanation

The strange thing about this "puzzle" is that it assumes perfect adherence to proposals and rules, but assumes no one can communicate alternatives.

If the result (98 - 0 - 1 - 0 -1) is so inevitable, then why wouldn't the pirates (0 - 1 - 0 -1) countercollude to kill the leader in exchange for equal distribution, under the express conditions of giving up the right of decision of division based on rank?

What we have here are five agents. The prioritization only matters for the first case if the other four agents (actually, three can collude, leave the least popular one out in the cold, and kill the top man). If the deconstruction is that the initial collusion leads to the four situation, then the remaining three can collude to drop it to equal division amongst them and kill the top man, and so on. Since the number one priority (unstated) is living over being rich, this collusion strategy would force the divider to think of his future and be more equitous.. Under this collusion strategy, the first leader to not be greedy lives, and keeps money, with security.

In my opinion, the outlined strategy would only provide a tentative peace until the others determined a collusion strategy. The fact that there exist deconstructions of the puzzle for n=1,2,3,4 doesn't matter as much. The fact you START with five as the initial playfield invites the opportunity for collusion that wouldn't exist at simpler n=1 or n=2. Essentially, the decision tree is not well expressed for the more complicated situations.

In reality, this is nothing more than a minimal resource allocation problem dressed up in a sexy way that highlights current income inequality and distribution in modern capitalist economies. It really invites too many messy gray outcomes.

IT projects: it's all about the people

http://www.infoq.com/news/2011/10/risky-it-projects

Well, still no progress on large IT project implementation in the last decade or so. Which compounds on the lack of progress in the previous decades. Of course it's practically impossible to measure projects on the basis of scope, requirements, complexity, etc across the ages. But they can measure the perceived rate of failure and cost.

Software development, delivery, maintenance, and interoperability have not produced major gains, and this study underlines it. I'd argue that the only real broad-scale efficiency success stories have been defined around widely accepted standards, especially HTTP, HTML, and other web standards.

After a couple decades in IT myself, it's pretty apparent to me what gets in the way, and it's not particularly controversial: it's the people, stupid. And that's highlighted by the fact we added several zeroes of capability to the hardware our software runs on, from tens of megahertz to multi-gigahertz processors, from 250 megabytes of storage to 2.5 terabytes of storage (!!). From 4MB RAM to 16 GB of ram. Mother of god those are amazing engineering accomplishments.

First off, lets start with the business people, who control the purse strings and decisions. it astonishes me that still, STILL, to this day, dealing with business people in systems specification at every stage from incubation/prototyping/formal design, indicates fundamental ignorance of IT delivery complexity, concerns, lifecycle, delivery times, etc. They don't even have an approximate understanding. Considering your typical business person has been through the process of getting an MBA, and thus have been "baselined" in the standard practices of business administration, this is pretty telling.

About the most complex thing MBAs learn universally is accounting. IT knowledge is still siloed into the "MIS" degree. And there we have the problem. What business function isn't intimately related to an IT system? Accounting, budgeting, production, supply chain, HR, recruiting, marketing, customer care, I can't think of ANY MBA-concerned business function whose efficient operation isn't tightly coupled to IT systems.

And yet, the basic process, scale, risk, and results from IT projects to integrate, install, configure, adapt, and deliver IT capabilities is treated with absolute ignorance on all these MBAs. Hell, you still, STILL hear about CEOs that proudly state they can't even turn on their computers and have their secretary print out their emails and do dictation. For every dinosaur like that, there are hundreds of people with other stages of malignant IT ignorance, running major companies whose very operation is fundamentally dependent on their IT systems.

I don't get it. You want a competitive edge in MBA land? Have basic-level IT management knowledge. Know about technology at a basic level. Know about the "great divide" between the detail necessary to specify a system expected by non-IT people, and IT people. Know that large-scale IT projects succeed when they represent an evolutionary process, not a dramatic huge system rollout. There's a ton of MBA-level IT knowledge that can be gleaned without knowing how to write a java class, what CORBA is, or any of the other limitless technical complexities and low-level protocols are involved. But your typical business person (as in: 99% of them) are utterly clueless about this.

The second major people problem is access. I''m not just talking about defending your systems from script kiddies and external threats. It's the more insidious beauracracy-generating internal access processes set up by IT organizations. Need to install software? Need to deploy from dev to test to stage to prod? Need access to a machine? Need access to a database? Need a machine to install a server?

Yeah, typical IT organizations you need to navigate a metric crapton of apathetic ticketing systems, signoffs, okays, and manual people decisions to do even basic things. And why? Cover Your Ass. It's all it is. All these managers with oversight over their specific concern (servers, databases, deployments, desktops), rather than enable access and freedom and innovation, would rather lock it down. Really tight. As in nothing goes in or out without them knowing about it.

What causes cross-business systems development failure? Inability to integrate. If you need access to some data (even read-only), you can't just look at the database, slap a quick web service gateway, and then enable the data access. Nope, you'll need to request access to the database, then request access to a new server, request access into the server server, request permission to install applications on the server, request permission on the internal firewall ports, etc etc etc. Sound simple? Typically those are queued behind formal request systems and take days if not weeks to review and complete. Massive time and efficiency losses.

Unfortunately, this is an area I can only complain about, I don't have any good solutions. Perhaps systems need better design from a "future data access" concern. Many/most systems are developed as "service oriented", but rarely are the specifications for the access well documented. Of course most services are probably still written in SOAP, which makes the service convoluted and inaccessible, destroying all the advantages of service orientation.

The third people problem is the IT churn of old-hat new-hat software, protocols, terminology, frameworks, and the like. A lot of these problems are due to the lack of evolved standards and practices that became prevalent over a couple centuries like accounting.

Almost every concept, problem, and concern in IT is represented by a half-dozen terms that largely mean the same thing. Even technologies around well-agreed standards like the internet have far too much terminology variation. This represents a fundamental communication problem even among IT people, who can't communicate concerns, architectures, approaches, and practices as efficiently as occurs in science, law, and accounting.

The fourth problem is the goddamn sales people. This is the most insidious source of problems in IT, since the software sales people talk to the business people, and I can't tell you how many "hey look I built your app in 10 minutes" demoware presentations that completely pull the wool over the businesspeople. Then they get angry when "10 minutes" becomes "10 months".

Businesspeople love being told that all the work done by their underlings that seems so inefficient but usually isn't that inefficient, could be done 100x faster, and they're just morons. The magic bullet. Businesspeople love this, since if the magic bullet works, they get promoted. And all they care about is getting closer to stock options.

There's dozens of these sales jobs underway in large IT organizations at each time, each subsuming and disrupting things right and left. Each basically not solving or enabling anything new.

Because you know what really produces progress? Not magic bullets, but boring evolutionary changes like gradual service enablement, stabilized systems, virtualization.