Tag Archives: spark

FYI: creating a new language does’t make your writing better

Remember back in the the day when the some kids would invent their own language like Pig Latin and then talk it amougst themselves?  And if you didn’t talk to them in Pig Latin you just weren’t cool enough to be part of the conversation.  I was more like the “why bother” kid.  Not that I couldn’t learn Pig Latin, I just didn’t see the point.  Seriously, not that much changes.

Don’t get me wrong, I’m being hard on Berkley out of love.  Several of my relatives went there during the People’s Park era.  I get that having your own language like Swift or Go provides artificial culture creation and built-in revenue protection.  That’s fine if you want to program to the Google or Apple cool-aid culture and plug right in.  Not my thing, but it takes all kinds.  It’s just that I’m not getting this whole Scala thing.  Why it exists.

Ok so I get that for a lot of people the Java music died when Scott McNealy finally sold out to Oracle.  And the whole Bill Joy thing and how that was handled…  Shameful, all.  Oracle’s attempts to profiteer and build an empire off Java with things like the OpenOffice purchase: ridiculous.  But the funny thing is, Oracle has taken a beating from their industry customers (like me) and has actually realized that being a great Java stakeholder is the best chance they have of preserving market share.  Of course they would never admit that publicly but that’s what I love about Oracle: they spend a lot more time delivering than talking about it.  They’re kind of like the quiet doer kid who thinks he’s Iron Man.

The thing I don’t get is that if you want to go functional, why not go all the way?  Why create a Java hybrid, and then keep backing it even when the parent language already added functional constructs?  Lisp has been around forever.  Heck back in the 90s I did functional programming in C with void pointers.  And Java *gained* more than twice the market share than Scala ever had between 2015 and 2016.  Talk about a moving target.

So lets take a look at a shining example of how Scala is such a more elegant and maintainable language than Java: https://github.com/apache/spark/blob/branch-1.6/core/src/main/scala/org/apache/spark/SparkContext.scala

scala-spark-spagetti
Spaghetti alla Spark e Scala del primo

So a couple of points about the above, which sits at the core of the Spark tool which is driving most of this Scala noise:

  1. I’ve seen better spaghetti code written in C using goto statements
  2. No knowledge of and/or respect for Gang of Four patterns, Enterprise Integration Patterns, or any other principles self-respecting programmers follow
  3. Hard coding Hadoop dependencies in because that’s for sure how I’ll store data when everything is on Non-volatile RAM

It’s not really about the language in the end: it’s about who’s writing it and the quality and integrity of what is written.

So I just want to say it clearly and definitively for all to hear: Twitter is dead wrong, Nathan Marz is right, and the coolest kids are alive and well and speaking Clojure just because they love it.  Nothing personal.