Beginning Scala

justify to myself why my method should exceed a single statement. Keeping methods short allows you to encapsulate a single piece of logic in a method and have methods that build upon each other. It also allows you to easily under- stand the logic in the method. 19897ch09.fm Page 276 Thursday, April 23, 2009 4:33 PM C H A P T E R 9 ■ S C A L I N G Y O U R T E A M 277 Refactor Mercilessly In the beginning, you can write your Scala code as you would your Java code. It’s a great place to start. Then, start applying the above rules. Let’s go back to our validByAge example from Chapter 3 (see Listing 3-3). We’ll start with the imperative code: def validByAge(in: List[Person]): List[String] = { var valid: List[Person] = Nil for (p <- in) { if (p.valid) valid = p :: valid } def localSortFunction(a: Person, b: Person) = a.age < b.age val people = valid.sort(localSortFunction _) var ret: List[String] = Nil for (p <- people) { ret = ret ::: List(p.first) } return ret } Turn your vars into vals. def validByAge(in: List[Person]): List[String] = { val valid: ListBuffer[Person] = new ListBuffer // displaced mutability for (p <- in) { if (p.valid) valid += p } def localSortFunction(a: Person, b: Person) = a.age < b.age val people = valid.toList.sort(localSortFunction _) val ret: ListBuffer[String] = new ListBuffer 19897ch09.fm Page 277 Thursday, April 23, 2009 4:33 PM 278 C H A P T E R 9 ■ S C A L I N G Y O U R T E A M for (p <- people) { ret += p.first } ret.toList } Turn your mutable data structures into immutable data structures. def validByAge(in: List[Person]): List[String] = { val valid = for (p <- in if p.valid) yield p def localSortFunction(a: Person, b: Person) = a.age < b.age val people = valid.sort(localSortFunction _) for (p <- people) yield p.first } Make your method into a single statement: def validByAge(in: List[Person]): List[String] = in.filter(_.valid). sort(_.age < _.age). map(_.first) While you can argue that this is too terse, we can refactor another way: def filterValid(in: List[Person]) = in.filter(p => p.valid) def sortPeopleByAge(in: List[Person]) = in.sort(_.age < _.age) def validByAge(in: List[Person]): List[String] = (filterValid _ andThen sortPeopleByAge _)(in).map(_.name) Either of the refactoring choices you make, the business logic of your code is a lot more visible. The refactoring also moves you toward thinking about the transformations in your code rather than the looping constructs in your code. Compose Functions and Compose Classes In the previous example, we composed filterValid and sortPeopleByAge into a single function. This function is the same as this: (in: List[Person]) => sortPeopleByAge(filterValid(in)) 19897ch09.fm Page 278 Thursday, April 23, 2009 4:33 PM C H A P T E R 9 ■ S C A L I N G Y O U R T E A M 279 However, the composition of the two functions results in code that reads like what it does. We started by turning our methods into single statements. This makes testing easier and makes the code more readable. Next we compose a new function by chaining the two functions together. Functional composition is a later stage Scala-ism, but it results natu- rally from making methods into single statements. In Chapter 7, we explored how Scala’s traits can be composed into powerful, flexible classes that are more type-safe than Java classes. As you evolve your Scala coding skills and begin to refactor classes rather than methods, start looking for common methods across your interfaces and traits. Move methods from concrete classes into traits. Soon, you’ll likely find that many of your classes have little in them other than the logic that is specific to that class and the vals that are needed to evaluate that logic. Once you reach this level in your coding, you will likely find that your traits are polymorphic, that your traits represent logic that can be applied to a contained type, and then you can feel secure that your mind has completely warped into thinking Scala. Once you’re thinking Scala or thinking that you’re thinking Scala, you might want to go hard-core on selling Scala into your organization. The next section provides some talking points for selling Scala. It gives you the benefits of my experience selling new technologies in organizations. Please keep in mind that “because it’s cool” is not a justification for an organization to adopt a new technology. The new technology must solve a problem. However, it’s pretty safe to say that most organizations want to make their developers happier and more productive, and Scala provides a great way to achieve those goals. Selling Scala in Your Organization So, you’re convinced that Scala is the right language for writing code. You’ve done a few Skunk Works projects and gotten a few team members on board with Scala. Now it’s time to sell Scala to management. This is perhaps the trickiest sell. There will likely be people in your organization who oppose change and are fearful of new things. It happens every- where. In this section, I’m going to talk about techniques you might try and arguments you might use to get Scala into your organization. The Super-Soft Sell One interesting technique that I’ve heard about for selling an organization on Scala is to start asking people to write tests using Specs rather than JUnit or TestNG. Because this is not “production code,” there’s less organizational resistance to this idea. Further, because tests are not production code, the operations guys don’t have to worry about tests in Scala. There’s no credible argument that somehow Scala-based tests would fail to execute correctly. 19897ch09.fm Page 279 Thursday, April 23, 2009 4:33 PM 280 C H A P T E R 9 ■ S C A L I N G Y O U R T E A M So, propose that you and a couple of team members write BDD tests in Scala. The tests will run just fine alongside your existing Java tests with the existing test harnesses. But you’ll give other developers a chance to use Scala, and you’ll give the business people and management folks a chance to see the readability differences between the Java-based tests and the Scala-based tests. The Back-Office Project If you’ve got a small, not-mission-critical, back-office project in your organization, then it may also be a good fit for Scala. Such projects often have a short life span and are viewed as disposable. Thus, if you succeed with Scala, it’s great. If you fail, the business will not fail. If you succeed and then leave the company, the worst case is that the whole thing gets thrown away and rewritten in Java or something else. Pitching Scala for the back-end project should be a pretty simple sell, and ultimately it gives you the opportunity to show your coworkers and management the benefits of Scala while easing their concerns about how Scala runs and integrates with Java code. The Big Sell I’ve been a user of leading-edge technology for my whole career. In 1977, when I was 14, I sold the Federal Emergency Management Agency on using Apple ][ computers for displaying dynamic content for the visual part of emergency broadcasts and was awarded a contract to implement the software. I started using NextStep (now Mac OS X) in 1990 and launched Mesa, the first real-time spreadsheet, in 1992 on NextStep. It’s only been in the last few years that developers have discovered the AppKit for OS X and iPhone development. I was an early user of WebLogic and ran the first Java-only high-volume site on the Internet. I was the one who led the charge for using Java. I have been an early adopter of a lot of tech- nologies that are now part of the mainstream.10 It is possible that someone who wants to block the adoption of something new will escalate the issue to the CEO or even the board level. I had an interesting dinner with a board member one time because a team member escalated up to the board the decision to move to .NET and managed code. One board member had written some Commodore 64 code back in the day but was a business and deal guy. More than half the meal was spent explaining garbage collection, with the board member insisting that reference counting was really the only sure way to do managed memory. But I digress. The important thing for you is to be prepared to answer any question about Scala. Please also answer every question honestly. You gain a lot of credibility by saying, “Java developers will miss a lot of the refactoring tools that they are used to. On the 10. To be fair, I’ve had my share of bad calls, too. I was a proponent of OS/2, OpenDoc, and BeOS. Please, stop laughing. 19897ch09.fm Page 280 Thursday, April 23, 2009 4:33 PM C H A P T E R 9 ■ S C A L I N G Y O U R T E A M 281 other hand, Scala’s software composition idioms make obsolete some of the refactoring idioms.” The statement is accurate and verifiable, so when the CEO or board member asks his trusted advisor to look at Scala, the trusted advisor will likely validate that statement. On the other hand, if you say, “Scala has complete IDE support, so it will work with our existing tool chain,” it’s likely that you will lose credibility because as of this writing, Scala IDE support is not on par with Java IDE support. Some people will raise the specter of the “operational characteristics” of Scala code. Because Scala code compiles down to JVM bytecode, the operational characteristics of Scala code are no different than Java code, and stack traces in Scala or mixed Scala and Java code look just like stack traces in Java code. Addressing the Potential Risks of Scala Over the last 18 months, as I’ve sold a lot of Scala, I’ve received a lot of questions. Some are rational: • Q: If you build this project in Scala and/or Lift, who else can maintain it? A: The Scala and Lift communities have thousands of members, are growing, and contain a number of people actively seeking Scala- and Lift-related jobs and consulting. • Q: What does Lift give us? A: A much faster time to market with collaborative appli- cations than anything else out there. • Q: Why not use Rails, which has great developer productivity? A: Rails is great from person-to-computer applications, but when you’re building a chat app or some- thing that’s person-to-person collaborative, Lift’s Comet11 support is much more effective than that of Rails. Plus, if you have to scale your app to hundreds of users simultaneously using your app, you can do it on a single box with Lift, because the JVM gives great operational benefits. And there are some less rational questions: • Q: Why use new technology when we can outsource the coding to India and pay someone 5 percent of what we’re paying you? A: Because I’m more than 20 times better than the coders you buy for 5 percent of my hourly rate, plus there’s a lot less management of my development efforts. • Q: Why not write it in Java to get the operational benefits of Java? A: Prototyping is hard. Until you know what you want, you need to be agile. Java is not agile. Ruby/ Rails and Scala/Lift are agile. Choose one of those to do prototyping, and then port to Java if there’s a reason to. 11. 19897ch09.fm Page 281 Thursday, April 23, 2009 4:33 PM 282 C H A P T E R 9 ■ S C A L I N G Y O U R T E A M • Q: Will Scala be incompatible with our existing Java code? A: No. Read my lips … no. The same guy who wrote the program (javac) that converts Java to JVM bytecode wrote the program that converts Scala to JVM bytecode. There’s no one on this planet who could make a more compatible language than Martin Odersky. My Direct Experience (YMMV) My experience with building Buy a Feature and other Innovation Games for Enthiosys has yielded a lot of useful data. Buy a Feature went through prototyping, revisions, and all the normal growth that software goes through. It works. (Yeah, it still needs more, but that’s the nature of software versions.) Other developers, US-based and India-based, have joined the project. Bringing the new developers on board was not materially different than when I hired a bunch of Java developers and asked them to write C# code. There are new tools, new idioms, new libraries, but the learning curve was fairly smooth. It took between 5 and 40 hours of pairing time to get new developers up to speed. That’s not a bad investment. On the operational side, we also had an occasion to have 2,000 simultaneous (as in at the same time, pounding on their keyboards) users of Buy a Feature and we were able to, thanks to Jetty Continuations, service all 2,000 users with 2,000 open connections to our server and an average of 700 requests per second on a dual-core Opteron with a load average of around 0.24. One of the customers of Buy a Feature wanted it integrated into their larger, Java-powered web portal along with two other systems. I did the integration. The customer asked, “Where’s the Scala part?” I answered, “It’s in this JAR file.” He said, “But I looked at the bytecode and it’s just Java.” I answered, “It’s Scala, but it compiles down to Java bytecode, and it runs in a Java debugger, and you can’t tell the difference.” He said, “You’re right.” So to this customer’s JVM, the Scala and Lift code looks, smells, and tastes just like Java code. If I renamed the scala-library.jar file to apache-closures.jar, nobody would know the difference at all. Okay, but with each set of people I talk to, I hear a similar variation about the “operational risks” of using Scala. Let’s step back for a minute. There are development and team risks for using Scala. Selecting Team Members Some Java programmers can’t wrap their heads around the triple concepts of (1) type inference, (2) passing functions/higher-order functions and (3) immutability as the default way of writing code. Most Ruby programmers I’ve met don’t have those limitations. So, find a Ruby programmer who knows some Java libraries, or find a Java programmer who’s done some moonlighting with Rails or Python or JavaScript, and you’ve got a developer who can pick up Scala in a week and be very productive with Scala in two months. 19897ch09.fm Page 282 Thursday, April 23, 2009 4:33 PM C H A P T E R 9 ■ S C A L I N G Y O U R T E A M 283 Even with the limitation of weak IDE support, head-to-head people can write Scala code two to ten times faster than they can write Java code, and maintaining Scala code is much easier because of Scala’s strong type system and code conciseness. Yeah, But How Compatible? You can make the assertion that at the operational level, Scala code and Java code are indistinguishable. I wrote a Scala program and compiled it with -g:vars (put all the symbols in the class file), started the program under jdb (the Java Debugger) and set a break point. This is what I got: Step completed: "thread=main", foo.ScalaDB$$anonfun$main$1.apply(), line=6 bci=0 6 args.zipWithIndex.foreach(v => println(v)) main[1] dump v v = { _2: instance of java.lang.Integer(id=463) _1: "Hello" } main[1] where [1] foo.ScalaDB$$anonfun$main$1.apply (ScalaDB.scala:6) [2] foo.ScalaDB$$anonfun$main$1.apply (ScalaDB.scala:6) [3] scala.Iterator$class.foreach (Iterator.scala:387) [4] scala.runtime.BoxedArray$$anon$2.foreach (BoxedArray.scala:45) [5] scala.Iterable$class.foreach (Iterable.scala:256) [6] scala.runtime.BoxedArray.foreach (BoxedArray.scala:24) [7] foo.ScalaDB$.main (ScalaDB.scala:6) [8] foo.ScalaDB.main (null) main[1] print v v = "(Hello,0)" 19897ch09.fm Page 283 Thursday, April 23, 2009 4:33 PM 284 C H A P T E R 9 ■ S C A L I N G Y O U R T E A M My code worked without any fancy footwork inside of the standard Java Debugger. The text of the line that I was on and the variables in the local scope were all there, just as if it was a Java program. The stack traces work the same way. The symbols work the same way. Everything works the same way. Scala code looks and smells and tastes to the JVM just like Java code. Now let’s explore why. A Tad Bit of History A long time ago, when Java was Oak and it was being designed as a way to distribute untrusted code into set-top boxes (and later browsers), the rules defining how a program executed and what were the means of the instruction set (bytecodes) was super-important. Additionally, the semantics of the program had to be such that the Virtual Machine running the code could verify that (1) the code was well behaved and (2) the source code and the object code had the same meaning. For example, the casting operation in Java compiles down to a bytecode that checks that the class can actually be cast to the right thing and the verifier ensures that there’s no code path that could put an unchecked value into a variable. Put another way, there’s no way to write verifiable bytecode that can put a reference to a non-String into a variable that’s defined as a String. It’s not just at the compiler level but at the actual Virtual Machine level that object typing is enforced. In Java 1.0 days, there was nearly a 1:1 correspondence between Java language code and Java bytecode. Put another way, there was only one thing you could write in Java bytecode that you could not write in Java source code (it has to do with calling super in a constructor). There was one source code file per class file. Java 1.1 introduced inner classes, which broke the 1:1 relationship between Java code and bytecode. One of the things that inner classes introduced was access to private instance variables by the inner class. This was done without violating the JVM’s enforcement of the privacy of private variables by creating accessor methods that were compiler-enforced (but not JVM-enforced) ways for the anonymous classes to access private variables. But the horse was out of the barn at this point anyway, because 1.1 brought us reflection, and private was no longer private. An interesting thing about the JVM: from 1.0 through 1.6, there has not been a new instruction added to the JVM. Wow. Think about it. Java came out when the 486 was around. How many instructions have been added to Intel machines since 1995? The Microsoft CLR has been around since 2000 and has gone through three revisions, and new instructions have been added at every revision, and source code compiled under an older revision does not work with newer revisions. On the other hand, I have Java 1.1–compiled code that works just fine under Java 1.6. Pretty amazing. Even to this day, Java Generics are implemented using the same JVM bytecodes that were used in 1996. This is why you get the “type erasure” warnings. The compiler knows the type, but the JVM does not, so a List looks to the JVM like a List, even though the compiler will not let you pass a List to something that expects a List. On 19897ch09.fm Page 284 Thursday, April 23, 2009 4:33 PM C H A P T E R 9 ■ S C A L I N G Y O U R T E A M 285 the server side, where we trust the code, this is not an issue. If we were writing code for an untrusted world, we’d care a lot more about the semantics of the source code being enforced by the execution environment. So, there have been no new JVM instructions since Java was released. The JVM is perhaps the best-specified piece of software this side of ADA-based military projects. There are specs and slow-moving JSRs for everything. Turns out, this works to our benefit. Present-Day Compatibility The JVM has a clearly defined interface to debugging. The information that a class file needs to provide to the JVM for line numbers, variable names, and so on, is very clearly specified. Because the JVM has a limited instruction set and the type of each item on the stack and of each instance variable in a class is known and verified when the class loads, the debugging information works for anything that compiles down to Java bytecode and has semantics of named local variables and named instance variables. Scala shares these semantics with Java, and that’s why the Scala compiler can compile bytecode that has the appropriate debugging information so that it “just works” with jdb. And, just to be clear, jdb uses the standard, well-documented interface into the JVM to do debugging, and every other IDE for the JVM uses this same interface. That means that an IDE that compiles Scala can also hook into the JVM and debug Scala. Scala’s operational characteristics are the same as Java’s. The Scala compiler generates bytecode that is nearly identical to the Java compiler. In fact, that you can decompile Scala code and wind up with readable Java code, with the exception of certain constructor oper- ations. To the JVM, Scala code and Java code are indistinguishable. The only difference is that there’s a single extra library file to support Scala. Popping the Stack to the CEO or Board Now, in most software projects, you don’t have CEOs and board members and everybody’s grandmother asking what libraries you’re using. In fact, in every project I’ve stepped into, there have been at least two libraries that the senior developers did not add but somehow got introduced into the mix. (I believe in library audits to make sure there’s no license violations in the library mix.) So, in the normal course of business, libraries are added to projects all the time. Any moderately complex project depends on dozens of libraries. I can tell you to a 100 percent degree of certainty that there are libraries in that mix that will not pass the “Is the company that supports them going to be around in five years?” test. Period. Sure, memcached will be around in five years, and most of the memcached clients will not. Making the choice to use Scala should be a deliberate, deliberated, well-reasoned choice. It has to do with developer productivity, both to build the initial product and to maintain the product through a two-to-five-year life cycle. It has to do with maintaining 19897ch09.fm Page 285 Thursday, April 23, 2009 4:33 PM 286 C H A P T E R 9 ■ S C A L I N G Y O U R T E A M existing QA and operations infrastructure (for existing JVM shops) or moving to the most scalable, flexible, predictable, well-tested, and well-supported web infrastructure around: the JVM. Recruiting team members who can do Scala may be a challenge. Standardizing on a development environment may be a challenge as the Scala IDE support is immature. (But there’s always emacs, vi, jEdit and TextMate, which work just fine.) Standardizing on a coding style is a challenge. These are all people challenges and all localized to recruiting and development and management thereof. The only rational parts of the debate are the trade-off between recruiting and organizing the team and the benefits to be gained from Scala. Bottom line: to anyone other than the folks with hands in the code and the folks who have to recruit and manage them, “For all you know, it’s just another Java library.” Why Is Scala Different? If Scala looks like Java to the JVM, why shouldn’t a company stick with Java? Put another way, if there’s nothing that can be written in Scala that can’t be written in Java, why use Scala? These are important questions, and they are the kind of strategic questions that you will need to answer when selling Scala into your organization. What Scala gives you is aids in concurrent programming: • Syntactic flexibility • Excellent default libraries for doing concurrent programming (immutable data structures and Actors) • Pattern matching Scala’s syntactic flexibility means that you can express concurrency-related calls in a way that allows you to have a different syntax for concurrent operations. The syntax was borrowed from Erlang: gameBoard ! Chat(who, "Hey guys, this is a chat message") So when I’m coding in Scala, I know that the previous method invocation is going to send an asynchronous message to my gameBoard. It’s got the benefits of Hungarian Notation12 without the verbosity or ugliness. This call syntax also means that the new programmer isn’t going to put a call like this inside a loop. How many times have we all seen the junior developer put a remote method invocation (RMI) call inside a loop and wonder why performance sucks? 12. 19897ch09.fm Page 286 Thursday, April 23, 2009 4:33 PM C H A P T E R 9 ■ S C A L I N G Y O U R T E A M 287 Scala supports immutability by default. This means that I can pass an immutable data structure to other threads, and they can use that data without synchronization. There’s no need to worry about concurrent access to data because the data is not going to change out from under you. There are eight synchronized code blocks in the 15,000-line Buy a Feature code base, and we have not had a single concurrency-related defect. Scala’s Actor library (which is implemented entirely as a library and has no compiler support) sits on top of Doug Lea’s Fork-Join library. Actors allow for event-based program- ming. It turns out that this model works very, very well for games and for asynchronous browser-based applications. Yes, a fair number of Python game applications are based on the Twisted event library. It turns out that the event-based semantics are very similar for both Actors and Twisted, so it’s no surprise that they are both choices for multiplayer games. (There’s a French company building a massive multiplayer online game [MMO] in Scala.) Actors are threadless, stackless event handlers. They only consume stack/thread resources when there’s a message for the Actor to process. This is very handy as you can have millions of Actors hanging out waiting to do something, and they only consume system resources when there’s something to do. Additionally, it’s possible to write a custom scheduler so that you can have all one million Actors processing messages, but they only consume a bounded number of threads. Finally, Scala’s pattern matching provides a powerful way to compose event handlers (even dynamically). Anything You Can Do, I Can Do Better (in Scala) So, there’s nothing I can do in Scala that I can’t do in Java, with enough time. There’s nothing I can do in Java that I can’t do in C. However, Scala lends itself to much more developer- and machine-efficient patterns. In fact, Scala gives me the coding efficiency and flexibility that I had with Ruby along with the type safety and performance character- istics of Java and the JVM. Buy a Feature contains 15,000 Scala lines of code (LoC) and represents about one man- year of effort. Measured in this vector, there’s nothing surprising. Buy a Feature compiles down to 2,300 classes and has roughly the functionality of a 2,300-class Java program (think ClearSpace). So, if I were to tell you that two people wrote 2,300 Java classes in a year or two, you’d tell me I was nuts. Yes, simple Scala statements explode into many classes. But the fact is that the complexity embodied in the simple statement represents the complexity associated with many Java classes. Scala is to Java as C++ is to assembly language. In Scala, I can concisely express far more business logic, that is also far more type-safe, in a single line than in Java. This means that the developer productivity is much higher in Scala than in Java. 19897ch09.fm Page 287 Thursday, April 23, 2009 4:33 PM 288 C H A P T E R 9 ■ S C A L I N G Y O U R T E A M Giving Scala a Lift The Lift Web Framework has Comet support. That means that state change on the server side is immediately pushed to the browser. Lift’s Comet support makes chat applications, multiuser games, and other browser-based applications trivial to write. Here’s the entire code required to write a multi-user chat application in Lift: case class Messages(msgs: List[String]) object ChatServer extends Actor with ListenerManager { private var msgs: List[String] = Nil protected def createUpdate = Messages(msgs) override def highPriority = { case s: String if s.length > 0 => msgs ::= s updateListeners() } this.start } class Chat extends CometActor with CometListenee { private var msgs: List[String] = Nil def render = {msgs.reverse.map(m => {m})} {ajaxText("", s => {ChatServer ! s; Noop})} protected def registerWith = ChatServer override def lowPriority = { case Messages(m) => msgs = m ; reRender(false) } } There’s nothing magic about Lift’s Comet support, but it would be much harder to do in Java. 19897ch09.fm Page 288 Thursday, April 23, 2009 4:33 PM C H A P T E R 9 ■ S C A L I N G Y O U R T E A M 289 How Lift’s Comet Support Works Lift has CometActors, which represent server-side state in a section of browser real estate. The real estate is demarcated by a with a GUID. All Lift pages are rendered using Scala’s built-in XML support. After the render phase, but before the page is streamed to the browser, Lift looks through the page to see whether the page contains HTML that points to any CometActors. If yes, Lift rewrites the XML and inserts JavaScript to do Comet-style long polling. After the page is loaded, the browser opens an XMLHTTPRequest to the server with the GUIDs of all the Comet components on the page along with the version number of each of the Comet components. The server receives the request and creates an Actor for each GUID, and each Actor registers itself as a listener with the appropriate Comet component. The registration includes the version number of the component as contained by the browser. If the servlet is running in Jetty or a Servlet 3.0 container, Lift automatically invokes the container’s “continuation” mechanism so that the pending request is consuming no threads. It is consuming an NIO socket, and it’s also consuming one Actor per Comet component on the page. When the Comet component receives the listener registration, it compares the version number with the current version number. If they differ, the Comet component immedi- ately sends the Actor a message containing the diffs between the version that the Actor/ browser has and the current version of the Comet component. If the version number is current, the Comet component does nothing. If the Comet component receives a message and updates itself, it notifies the listener of the diff between the old version and the new version of the component. During the “no changes” phase, the only system resources being consumed are memory and an NIO connection. No threads or stacks are involved. When the Actor receives an update from the Comet component or after 110 seconds, the Actor creates a response to the Ajax request. It then invokes the continuation and sends the response to the browser (either JavaScript containing commands to perform the diffs or a Noop). The browser executes the JavaScript, waits 100 milliseconds, and restarts the process. I could have implemented all of this in Java. In fact, there is a Comet library that sits on top of Jetty and Dojo that has the same scaling characteristics. However, the amount of code to implement this scheme in Scala contains roughly the same number of characters as the above description. I’m sure that would not be the case in Java. 19897ch09.fm Page 289 Thursday, April 23, 2009 4:33 PM 290 C H A P T E R 9 ■ S C A L I N G Y O U R T E A M Summary Designing and building complex computer software is a very serious thing. Our liveli- hoods, and increasingly our whole society, depend on the stability and flexibility of our interconnected computer systems. Our cars and our banks and our grocery stores and our hospitals and our police departments all work better because they are interconnected by computer systems. Those systems run on the software that we write. In this book, I’ve taken a very lighthearted approach to introducing you to the Scala programming language. I’ve approached the daunting task of learning a new language and possibly a new set of programming patterns in a fun way. I hope that you have enjoyed the journey and are already thinking about new ways to reason about designing software and writing code. I want to end this journey by talking a bit about architecture. Architecture is very important in overall system performance and team performance. Scala has a lot of the tools that allow for much better architectural decisions. It’s kind of a Zen and the Art of Motorcycle Maintenance thing—you use the patterns that your language and its libraries make easiest. Scala makes it easier for coders to implement architecturally solid designs than does Java or Ruby. Thank you! 19897ch09.fm Page 290 Thursday, April 23, 2009 4:33 PM 291 Index ■Symbols -- method, 71 ~ method, 235, 198 ^^^ method, 236 ^? method, 237 ! (bang) method, 138 ++ method, 71 | operator, 235 \ (backslash) operator, 82 \\ (double backslash) operator, 82 >: relationship operator, 128 _ wildcard character, 16 ■A accept method, 135–136 access method, 146 ACID, 153 act method, 139, 146 Actor libraries, 138, 287 Actors, 287 atomic updates, 153–155 changing behavior of, 149–152 Comet, 289 composing, 164–169 concurrency and, 137–138 defining, 138–141 Erlang, 138, 140 implementing transactions using, 153–164 instantiating, 141 listener implementation with, 142–148 message processing by, 138, 148–149 protocol handler, 149–152 receiving replies from, 148–149 state, 140–141 addListener method, 183 Ajax, HTTP requests, 105 anonymous inner classes, 47 Ant, 14, 269–270 Any class, 27 AnyRef class, 27 AnyVal class, 27 apply method, 34, 35, 54, 107, 222 architects, 273 Array class, 36 ArrayList, 50, 254 ArrayList, 254 arrays of functions, 103 invariant, 208 assembly language, immutability of, 49 assign-many variables, 19 assign-once variables, 19 asynchronous message handling, 147 atomic updates, 153–155 AtomicReference, 89 Attributes, defining, 78 ■B back-office projects, using Scala for, 280 backslash (\) operator, 82 BaseType, 221 BeginXAction message, 159 Behavior-Driven Development (BDD), 263–266 best practices, 274–279 bmap control structure, 110, 112 Boolean Map method, 110 build tools, 266–271 Ant, 269–270 Buildr, 271 Maven, 266–269 Simple Build Tool, 271 buildCalc method, 107 Buildr, 271 Buy a Feature, 287 By clause, 202 ■C C language, 49 calcReact method, 151 calculator parser, 237–242 callbacks, 104 call-by-name, 31–33 call-by-name parameters, 107–109 case classes, 36–39, 120–122 for defining messages to Actors, 139 nested pattern matching in, 126–127 pattern matching and, 39, 122 case objects, 202 casting, 254 catch block, 18 chaining, 73 19897IDX.fm Page 291 Monday, April 27, 2009 3:41 PM 292 ■I N D E X changed method, 147 characters, escaping, 80–81 chat server, with dynamic message handling, 150–152 class declarations, 22–23 class hierarchies with invariant type parameters, 209 traits and, 184–190 class hierarchy, 27, 133–134 classes adding methods to existing, 175–180 anonymous inner, 47 case, 36–39, 120–122, 126–127 collections, 49, 51–53, 66–71 composing, 278–279 contravariant, 211–213 covariant, 210–211 inner, 26, 47 invariant, 208–210 Java, 45–46 Ruby, 45–46 Scala, 22–23, 26–27, 45–46 See also specific classes cnt variable, 140 code, Java vs. Scala, 56–61. See also Java code; Scala code code blocks, 31, 48 collection manipulation methods, 71 collections classes, 49, 51–53 lazy, 52 List, 53–66 Map, 67–71 mutable vs. immutable, 53 Option, 72–74 Tuples, 66–67 columns concrete, 198–199 table mapping, 197 column types, 197–198 ColumnTypes, 198 combinators, 235–237 Comet support, 288–289 CometActors, 289 command line, Scala at, 11–13 comments, 45 CommitXAction message, 160 companion objects, 178 compatibility of Scala, 283–284 with JVM, 285 compiler, 35 compiling, Scala programs, 13–14 composed functions, 105–107 composition of classes, 278–279 of functions, 278–279 concurrency, 87–91, 137–138 cons cells, 53, 123, 129 containers functions in, 103–104 contravariant parameter types, 211–213 control structures building, 109–112 for running queries, 201 covariance, 207–208 covariant parameter types, 210–211 covariants, 128 ■D data hiding, 131–136 data structures, immutable, 49–51 data types, testing, 119–120 def keyword, 16–17, 28 dependency-management systems, 266–269 dependent types, 221 domain-specific languages (DSLs), 171, 273 doSet method, 90 duck typing, 110, 138 Dwemthy’s stairs (example), 214–231 ■E ECMAScript, 242 elem method, 235 else expression, 79 Emir, Burak, 5 equals method, 37 Erlang, 138, 140 events, binding functions to, 103–104 Excel, 56 exceptions, 43, 68–69 exceptions function, 130 expressions, 105–106 evaluating, 106–108 nesting, 41–42 ■F Federal Institute of Technology, Swiss (EPFL), 5 FieldProduct class, 200, 203 FieldType class, 198 filter method, 55, 56, 71 find method, 199–201 flatMap method, 18–19, 64–65, 69–70, 275 foldLeft method, 19, 63, 66, 71, 240 for comprehensions, 41–43, 65, 76, 83, 253, 258 fromList method, 259 fsc compiler, 13–14 FT parameter, 199–200 functional languages, 97 functional programming (FP), 131–136 FunctionN trait, 213 FunctionNN trait, 97 functions, 34–35, 47 array of, 103 binding to events, 103–104 19897IDX.fm Page 292 Monday, April 27, 2009 3:41 PM 293■I N D E X bound to variables in scope, 100–102 building, from other functions, 105–107 composed, 105–107 composing, 278–279 in containers, 103–104 higher-order, 233–234 interactive applications and, 104–105 vs. methods, 97 mixing with collection operators, 55–56 as object instances, 93–94 partial, 97–99, 129–131, 149–152 passing, as parameters, 94–96 pattern matching as, 129–131 type parameters and, 99–100 writing, 57 ■G Generic Java (GJ), 5 generics, 254 get method, 68–69, 74, 274–275 getOrElse method, 74 getters, 149 global state, 50–51 globally unique identifiers (GUIDs), 104 Graceless Failures blog, 272–273 ■H Haller, Philipp, 6 handler method, 166 handleRequest method, 130 Hashmap class, 36, 67 Hashtable class, 67, 71 Haskell language, 97 Hello World program, 14–15 higher-order functions, 233–234 HTTP requests, 105 Hungarian Notation, 286 ■I if expressions, 62, 79 if/else expressions, 40–41, 115 immutability, 49–51, 75, 275–276, 287 immutable collections/data structures, 49–53 concurrency and, 87–91 List, 53–66 Map, 67–71 Option, 72–74 Tuples, 66–67 imperative coding, 57–58 implicit conversions, 173–175 dangers of, 180 defining libraries using, 175–180 implicit scope and, 180–181 implicit method, 173 implicit parameters, 202 implicit scope, 178, 180–181 import scala.io_, 16 import statements, 21–22, 26 in parameter, 16, 18 inner classes, 26, 47 inner objects, 26 inner traits, 26 input variable, 19–20 instanceof test, 119 instances, 27, 34 functions as, 93–94 Java, 45–46 methods and, 97 Ruby, 45–46 Scala, 45–46 instruction attribute, 86 Int, parsing a String to, 16–18 IntColumn, 198 interactive applications callbacks in, 104 functions and, 104–105 interactive Scala, 11–13 invariant parameter types, 208–210 Iry, James, 9 is method, 253, 258 isDefinedAt method, 129 isOdd method, 55 ■J Java anonymous inner classes, 47 classes, 45–47 compatibility between Scala and, 285 data structures in, 49–51 interfaces, 46–47 mutability in, 275 vs. Scala, 45–48, 56–58 static methods, 47 Java 1.1, 284 Java code problems with, 172–173 vs. Scala code, 56–61 Java Generics, 284–285 Java Virtual Machine (JVM), 2, 5, 34, 254 history of, 284–285 present-day compatibility, 285 synchronization, 137 JAVA_HOME variable, 6 JavaScript, 138 JavaScript Object Notation (JSON), 242 JavaTokenParsers, 238 JDBC connections, 201, 205 JDBC looping, 111–112 JSON (JavaScript Object Notation), 242 JSON parsing, 242–261 junior developers, 273 JUnit, 264–265 JVM. See Java Virtual Machine 19897IDX.fm Page 293 Monday, April 27, 2009 3:41 PM 294 ■I N D E X ■K keys, 67, 70–71 ■L lambdas, 48 language-level concurrency, 137 lazy val keyword, 30 len method, 76 libraries added to projects, 285 defining, using implicit conversions, 175–180 type-safe, 214–231 Library trait, 205 Lift Web Framework, 10, 105, 267, 274, 281, 288–289 Linux, installing Scala on, 8 List class, 53–66, 203 vs. ArrayList, 50 covariant lists, 210 creating lists, 53–54 filter method, 55–57 flatMap method, 64 foldLeft method, 63 for comprehension, 65 functions and, 55–56 get method, 74 List[T], 53–55 map method, 58–59, 234 pattern matching, 65, 123–126 reduceLeft method, 61–62 sort method, 59–60 transformations, 58–60, 60–61 listener traits creating, 181–183 using, 183–184 listeners Actor-based, 144–148 generic, 165–167 implementing, 142–148 log method, 107 logging, call-by-name parameters and, 107 loop method, 139 looping, 64, 111–112 ■M Mac OS X, installing Scala on, 8 MacIver, David, 9 mailing lists, 9–10 man parameter, 253 Manifests, 253–254 manual casting, 254 Map class, 67–71 adding elements, 71 creating Map, 67 filter method, 71 flatMap method, 69–70 get() method, 68–69 keys, 70 reduceLeft method, 70 removing elements, 70, 71 map method, 58–59, 234 match operator, 129 Maven, 14, 266–269 Maven Project Object Model (POM), 271 method declarations, 28–30 method definitions, 16–17 method invocation, 31–34 methods, 23, 71 adding to existing classes, 175–180 vs. functions, 97 Java, 47 keeping short, 276 in objects, 24 overriding, 30 partially applying, 99 Ruby, 48 static, 24 synchronizing, 44 traits and, 23–24 type parameters and, 99 mixins, 47 ML language, 97 Moors, Adriaan, 5 Multics.scala, 87–91 multiline comments, 45 multithreaded processing, 153 multithreaded programs, 87 mutable classes, 50 mutable collections, 53 mutable data structures, 87 ■N naming conventions, for classes, 27 nested operations, 64 Nil object, 53 Node class, 76 NodeSeq, 78–79 None, 27, 72 normal handler, 159 Nothing class, 27, 128 notifyListeners method, 146 now method, 77 null testing, options instead of, 274 null, 68–69, 72 number constants, 20 ■O O(log n) operations, 81, 84 O(n) operations, 81, 84 object instances. See instances Objective-C, 138 object-oriented programming (OOP) data hiding, 131 vs. functional programming, 131–136 19897IDX.fm Page 294 Monday, April 27, 2009 3:41 PM 295■I N D E X objects, 24–26, 47, 178 oddTime method, 78 Odersky, Martin, 3, 5 Option[Int], 17 Option[NodeSeq], 78 Options defined, 17 using, in place of nulls, 274–275 Option[T], 72–74 OrderBy case class, 202 orElse method, 130 Ortiz, Jorge, 9 override modifier, 29 override val syntax, 126 ■P packages, 21 parameters, 28–20 call-by-name, 107–109 contravariant, 211–213 covariant, 210–211 invariant, 208–210 passing functions as, 94–96 type, 99–100 parameter types, 16–17, 208–213 parser combinator library, 105, 235 ParseResult, 261 Parsers, 233–261 calculator parser, 237–242 combinators, 105, 235–237 JSON, 242–261 regular-expression, 242 Twitter, 255–261 parsing, XML, 81–84 partial functions, 97–99, 129–131, 149–152 PartialFunctions, 129–131, 149–152, 168 PATH variable, 6 pattern matching, 18, 65–67, 85, 106, 287 against constants, 125 basics, 38–40, 115–129 case classes, 120–122 as functions, 129–131 internals of, 128–129 Lists and, 123–126 matching any type, 117–119 nested, in case classes, 126–127 OOP and, 131–136 testing data types, 119–120 PCData, 81 Predef, 175 PreparedStatement, 197, 200 Pretty, Jon, 9 primitive types, 17, 27 -print option, 117 Procs, 48 product expressions, 238 Product trait, 198 Programming Methods Laboratory (LAMP), 5 protocol handler, 149–152 PT parameter, 202 Python, 138 ■Q queries building, 200–201 making type-safe, 201–202 Query Builder, code, 191–197 QueryParams, 200, 201 ■R Rails, 2–3, 281 Ranges, 52–53 react method, 139, 149–150 readAll method, 29 recursive programming, 65–66 reduceLeft method, 61–62, 70 refactoring, 277–278 RegexParsers trait, 242 regular-expression Parser, 242 remove instruction, 85 remove method, 55 removeIt rule, 85 removeListener method, 183 rep method, 240 repeatEvery method, 89–90 ResultSet, 197, 200 return types, 16–17 ReturnType type, 199 RewriteRule, 85–86 root class, 27 Ruby, 2–3, 138 metaprogramming, 224 methods, 48 mixins, 47 vs. Scala, 45–48, 224, 231 singletons, 47 run method, 90 RunParser trait, 237 ■S SafeMap trait, 253, 254, 258 Scala adoption of, 272–274 advantages of, 286–287 best practices, 274–279 characteristics of, 172 classes, 22–23, 26–27, 45–46 command line, 11–13 compared with Java, 45–48, 56–58 compatibility of, 283–285 history of, 5–6 implicit conversions, 173–175 19897IDX.fm Page 295 Monday, April 27, 2009 3:41 PM 296 ■I N D E X installation, 6–8 interactive, 11–13 introduction to, 1–4 potential risks of, 281–282 vs. Ruby, 45–48, 224, 231 selling, to your organization, 279–289 as statically typed language, 138 writing tests using, 279 ScalaCheck, 50–51, 265 Scala code vs. Java code, 56–61 safety of, 172 testing, 263–266 Scala community, 8–10 Scala compiler, 35 Scala.io package, 16 Scala programs compiling, 13–14 Hello World, 14–15 number printing, 15 Sum.scala, 15–20 Scala scripts, 13 Scala syntax, 20–45 call-by-name, 31–33 case classes, 36–38 class declaration, 22–23 class hierarchy, 27 code blocks, 31 comments, 45 for comprehension, 41–43 constants, 20–21 functions, 34–35 if/else statements, 40–41 import statements, 21–22, 26 method declarations, 28–30 method invocation, 33–34 object definition, 24–26 package, 21 pattern matching, 38–40 synchronized method, 44 trait definition, 23–24 try/catch/finally construct, 43–44 variable declarations, 30–31 while statements, 40–41 ScalaTest, 265–266 scala.xml.transform package, 84 scope, functions bound to variables in, 100–102 scripts, Scala, 13 sealed traits, 202–203 Seq, 18 Seq[Node], 76 sequential collections, 53–66 service variable, 159 sGet method, 255, 258 shape abstractions, 132–136 Simple Build Tool, 271 single backslash (\) operator, 82 singletons, 47 Smalltalk, 138 Some[NodeSeq], 78 Some[T], 72 sort method, 59–60 SortOrder trait, 202 Specs framework, 264–265 Spoon, Lex, 5 spreadsheets, 50 stack overflows, 144 start method, 141 state, 50–51, 140–141 static methods, 24 statically typed languages, 138 String class, 173 string constants, 20 Strings parsing to an Int, 16–18 testing, 264 structural types, 182 structural typing, 110 sum expressions, 238 sum method, 18–19 Sum.scala, 15–20 sumSq method, 66–67 superclasses, 46–47 Swing, 47 switch statement, 38–39, 6, 115 Syme, Don, 6 synchronization, concurrency without, 87–91 synchronized keyword, 137 synchronized method, 44 synchronous messaging, 144 syntactic flexibility, 286 syntax, Scala, 20–45 ■T table mapping to column, 197 defining, 191–207 defining traits, 197 Query Builder code, 191–197 Table trait, 197 tables, implementing with concrete class, 205–206 TableType parameter, 201 takeWhile method, 56, 57 team members, selection of, 282–283 team structure, 272–274 testing, 263–266 options instead of null testing, 274 ScalaTest, 265–266 Specs framework, 264–265 XML, 264 TestNG, 265 19897IDX.fm Page 296 Monday, April 27, 2009 3:41 PM 297■I N D E X throwing exceptions, 43 time attribute, 78 TIMEOUT message, 159–160 toInt method, 16–18 toList method, 56 toString method, 36–37 traits, 18, 23–24, 47, 181–190 class hierarchies and, 184–190 composing, into Actors, 164–169 creating, 181–183 defining generic, 197 sealed, 202–203 using, 183–184 transactional Actors, 155–164 transactions ACID, 153 implementing using Actors, 153–164 performing, 155–161 transform method, 85 try/catch/finally construct, 43–44 try/finally construct, 110, 112 Tuples, 66–67, 198–199, 203 Twisted event library, 287 Twitter JSON parsing, 252–261 TwitterStatus instances, 259 TwitterUser class, 259 type inferencer, 206 type parameters, 197–202 contravariant, 211–213 covariant, 210–211 functions and, 99–100 invariant, 208–210 variance and, 208 types, 190–207 column, 197–198 defining, 221 dependent, 221 implicit conversions, 173–175 structural, 182 type-safe map utility, 253–255 type-safe queries, 191, 201–202 type safety, 230, 206–207 type variance. See variance ■U Unit class, 27, 79 unityped languages, 138 Update message, 154–155 update method, 35–36 URLs, expressing as List, 130 using control structure, 109–112 ■V val keyword, 19, 30, 101 vals, 275–276 var keyword, 19, 30, 101 variable declarations, 30–31 variables assign-many, 19 assign-once, 19 call-by-name, 107–109 functions bound to, in scope, 100–102 variable types, 30 variance, 207–213 contravariant parameter types, 211–213 covariant parameter types, 210–211 invarient parameter types, 208–210 rules of, 213 vars, 275 Venners, Bill, 265 visitor pattern, 133–134 ■W Wadler, Philip, 5 Webb, Jamie, 9 while statements, 40–41 _wildcard pattern, 119 Windows, installing Scala on, 6–7 ■X XAction message, 159 XML constants, 20 escaped characters, 80–81 literals, 75, 78 parsing, 81–84 in Scala code, 75–81 testing, 264 transformations, 84–86 19897IDX.fm Page 297 Monday, April 27, 2009 3:41 PM Offer valid through 11/09. 19897IDX.fm Page 298 Monday, April 27, 2009 3:41 PM