ÜberConf

Chess Pieces or Domain Expertise? Your Choice

Johanna Rothman — Tue, 18 Jun 2013 08:36:00 CDT

Many years ago, I started a job as a contract manager, and it became clear I had a big problem. I had developers who knew one area of the code well. I had testers who knew not much of any area of the code well, even though they had worked for the organization for many years. Why? They had been shuffled from one project to another almost every month for years.

The writers had more domain expertise than anyone, because they had learned the product from end to end. What was I going to do? This project needed to finish in eight weeks, I needed to hire my replacement, and the people were shell-shocked from one manager after another. I was the fourth Director of Software Development in six months.

I decided to make a project portfolio, and rank the projects to know which projects were first, second, and third. I knew about this first project, but what was second and third? I was sure we had more crises, waiting.

Next, I asked people what they wanted to work on. I was sure that people could select their areas of responsibility, and it would work out. This is before we had agile teams. I was using a staged delivery lifecycle, so we had cross-functional teams and we worked by feature.

Next, I asked the teams to develop their deliverables in two-week chunks: what deliverables, as in features, could they deliver in the next two weeks, working as a cross-functional team?

The teams started to work together. They started to work across the code, not just in one area of the code. It wasn’t perfect, but it was working. The testers expanded their knowledge, because they were able to focus on the entire product. The developers learned the product end-to-end. The writers were happier, because people answered their questions.

About three weeks into this, a crisis happened with the second-ranked project. A senior manager wanted to yank a bunch of people off the top-ranked project and put them onto the second ranked project. He told me I had to give him people.

“No. I have no one for you.”

“But you work for me.”

“Yes, I work for the organization. But this project, the one that everyone is on, is more important than your project. So, I have no one for you, and I won’t for another two weeks. Unless I hear from the CEO that your project is more important than this one. In that case, we will stop working on this one, and everyone will work on your project.

We only have eight people. We can’t multitask. We can only work on one project at a time. Moving people off and onto projects as if they are chess pieces doesn’t make sense. This top-ranked project has two more weeks to go, and then it’s done. When it’s done, I can assign all eight people to your project, no problem. But assigning them now? Craziness.”

Now, should I have called him crazy? That was darn close to a career limiting conversation. Don’t do that! But the rest of the conversation? Useful.

And that’s the topic of this month’s Management Myth #18: I Can Move People Like Chess Pieces. When you move people like chess pieces, you deny them the opportunity to learn domain expertise. You don’t manage the project portfolio, and you decrease the capacity of your organization. It’s all around bad.

Did I want to be nice to the senior manager? Of course I did. But, good management is not about being nice to everyone all the time. Much of management is about saying no when you have to. And, I really needed to say no.

With agile approaches, I would have had more options: I could have used iterations and sequenced the projects differently—maybe. But that would not have allowed the organization to release the top-ranked project in the next few weeks. It would have made the people feel horrible, as if I’d yanked the promised dessert away from them. Yes, the project was that close to completion.

While you can move people as if they are chess pieces, it’s rarely a good idea. You want to leave people with project teams, to let them create a solid team—that’s the forming, storming, norming part. And, you want to keep people on a product long enough that they develop significant domain expertise in the product. When they become bored and ask to move, then they will tell you in a one-on-one that it’s time to move.

In the meantime, don’t move people as if they are chess pieces. People need to develop mastery over their work. They need the autonomy to select the areas of their work that they will develop domain expertise in. They need to feel as if they have a sense of purpose about the work. Gee, it sounds as if I’m parroting Dan Pink, in Drive. I guess I am. When you commit to project teams, and leave people where they are, so they can learn the product and learn how to work with their teams, and let the work flow through the teams, you create an environment that allows for autonomy, mastery, and purpose. And, that’s part of what a great manager does.

Type-Safe Web with Kotlin

Andrey Breslav — Mon, 17 Jun 2013 09:13:00 CDT

We told you about Kara Web Framework a while ago. It is written in Kotlin and relies on type-safe builders. It doesn’t have to be the only web framework for Kotlin, but the general principles seem good, so I wrote an article about these principles: Type-Safe Web with Kotlin, it gives an overview of the framework design, and then gives many details about builders, extensions and so on.

Examples used in this article are available at https://github.com/abreslav/kara-files/blob/master/src/tutorial.kt and https://github.com/abreslav/kara-files/blob/master/src/small_builder.kt

Slides from my talk given at JPoint in Saint Petersburg are also available online.
The talk was given in Russian, video accessible is here.

It’s Déjà vu All Over Again

Alan Shalloway — Thu, 13 Jun 2013 21:22:00 CDT

Several years ago I tried to discuss the need for Lean when Scrum was being used on projects with more than one team. Ken Schwaber didn’t want to hear this and eventually threw me off the Scrum Development Yahoo discussions group. I admit, I was talking outside the domain of what he wanted – the pretense was that Scrum wasn’t Lean and the Scrum Development group wasn’t the place to talk about Lean.

Times have changes – a little. We now hear how Scrum is based on Lean but I have heard little acknowledgement that Lean-thinking, used explicitly, is an essential ingredient for Scrum teams. At the Scrum gathering I heard of a successful Scrum implementation of a company that had 150-200 folks in it. I can validate that myself. Because I had been in there myself, before they were successful with Scrum across the enterprise. True, it didn’t take much to get them from successful team Scrum to successful enterprise Scrum, but it did take Lean-thinking, value stream mapping and an understanding of Lean-flow to get them there. While Scrum is a great team process I still contend it is rarely successful at scale because the framework itself does not provide the insights needed to extend it. If that’s the case, part of the Scrum discussion should be about what is needed to include in the framework.

For the last few years I have been promoting Kanban, both team-Kanban and the Kanban Method, a transition management method across the entire value stream. Team-Kanban is best used within a full-value stream view as well, but it doesn’t require the Kanban-Method’s approach. It is unfortunate that the term Kanban has been so overloaded (see De-Mystifying Kanban for a description of all the types of Kanban).

Net Objectives is not attached to any one method. I believe we are the only company that promotes Scrum, XP, Team-Kanban, Kanban Method, SAFe, and Lean, along with technical practices. Our approach is to see where people are, what their culture is, how much discipline they need, the extent they can form teams, amongst other things. If you missed our webinar, Beginning an Agile Transformation, you might want to watch it for a full list of what to look for.

Our experience enables us to cull the best approach from all of these methods to suit our clients’ needs. Sorry, I’m not trying to make this a sales pitch, anyone can take this approach. Unfortunately, relatively few consultants do. I’ll admit I'm writing this blog for practitioners and my advice is don’t decide between methods, learn from each of them. In other words, it’s not Scrum Vs Kanban, it’s how do we take the best approach for the needs at hand. This requires understanding the strengths and limitations, particularly where they best apply, of both.

Since the Lean-Kanban community has been forefront in bringing new methods to the Agile space, I had thought it would behave a bit different from my experience in the Scrum community. Unfortunately, apparently not. Because Net Objectives uses both Scrum and the Kanban Method, we’ve learned a lot from both. While the Kanban Method facilitates the creation of effective value streams when true teams can’t be created we’ve seen that teams are truly valuable (see Know the Power of Teams).

I started discussing the value of teams on the Kanban Development board and essentially ran into the same attitude as I had years ago. First it was – “teams are orthogonal to the method.” Then it was “let’s not discuss this.” Then it was “you (meaning me) are not behaving professionally.” This time I left before getting moderated or thrown off. But it’s a bit distressing to see the same attitude – “our method is correct, let’s not discuss things that aren’t defined in it.”

Tragic.

Practitioners – please don’t fall into this trap. There are many consultants out there not attached to any one method. Because of this, they often don’t have an accreditation from either the Scrum Alliance or the Lean-Kanban University. Don’t take that as a negative – it might be a plus.

Al Shalloway
CEO, Net Objectives

Author:

alshall

Slides from Exploding Management Myths Posted

Johanna Rothman — Mon, 10 Jun 2013 07:24:00 CDT

I gave a talk last week at Better Software/Agile Development, called Exploding Management Myths. This is my first talk based on some of my management myths. Yes, the ones I’ve been writing for the last 18 months. Yes, I have more in me :-)

I have posted the slides and the audio from my talk at slideshare. I still cannot understand how to properly sync the slides and audio at slideshare. I don’t talk evenly, which is what slideshare expects. I tried to separate the slides and audio. Can’t do that either. Grumble, grumble. However, it’s posted, and if you download the slides and then listen to the audio, you can manually sync the slides and the audio :-)

If you have a favorite myth that I have not yet addressed, please let me know. I will write about it.

I really need help from you. I need a title for the eventual book. Everyone talks about debunking myths, so I don’t think I want to go with debunking. I thought it might be “Exploding Management Myths.” Maybe. Doc List suggested it might be “Blowing Up the Bullshit.” Maybe :-) Do you have ideas for the eventual title for this management book? I would like at least a few more options.

See the management myths on Stickyminds here, and on my site.

Talk @ GeekOUT Tallinn: Language Design Trade-Offs (Kotlin and Beyond)

Andrey Breslav — Mon, 10 Jun 2013 06:27:00 CDT

This week I’m speaking at GeekOUT Tallin, and my colleagues Mikhail Vink and Sergey Karashevich are holding a 15-minute DEMO on Thursday, telling you about cool stuff in JetBrains’ IDEs.

The topic of my talk is “Language Design Trade-Offs (Kotlin and Beyond)“, so one can summarize it as “why designing languages is SO interesting”.

Abstract

If your programming language is small, you’re probably born before 1950, and your first computer was bigger than your present apartment. And even those languages are not so small. One spends quite some time to master a programming language. Why?

Because there are very many decisions “compressed” into the form of a language. Nothing limits a programmer’s imagination like a compiler, and nothing limits a language design like a real world with all of its “legacy”, compatibility concerns, performance limitations, generations-old habits and leaky abstractions.

This talk is about tradeoffs: why we, as language designers, do (or rather did) this and not that.

See you on Friday, 14 June 2013 11:30 @ Room 2

http://geekout.ee/schedule/andrey-breslav-jetbrains-building-jvm-languages/

UPD:

In Defense of Kanban

Alan Shalloway — Sat, 08 Jun 2013 21:03:00 CDT

As many folks know, Net Objectives does both Scrum and Kanban. Admittedly, our Scrum is very much like Scrumban (or Scrum done under the context of Lean) but it is still an implementation of Scrum. Scrum, as it normally manifests itself, has several challenges in that it requires teams to learn too much about how to apply it (much is know that hasn't been incorporated into it) and many teams try it when it really isn't the best framework for their situation.

Challenges such as an inability to form true teams, close the gap between coding and testing and the resulting unfinished stories at the end of sprints has many teams abandon the true Scrum practices of cross-functional teams that are truly time-boxed. Some in the Scrum community have called this Scrum-but, a demeaning term which I've always disliked for two reasons. First, it is disrespectful, and second, it hides the fact that most of poorly practiced Scrum is a result of Scrum the lack of most of the Scrum thought leaders not stressing the Lean principles on which Scrum is based and why it works.

The new vogue now is for Scrum teams to drop time-boxing and cross-functional teams and say they are doing Kanban. I suppose the logic is that Kanban doesn't have these. While true, Kanban is not defined by not having iterations or not needing cross-functional teams. It is defined by visibility, managing flow, explicit policies, a white-box process, and continuous improvement with PDCA. Let me be clear - if you have been doing Scrum and have stopped time-boxing and don't have devs and testers working together you are not doing Kanban. You are not even doing a shallow version of Kanban. You are not really doing any well-defined Agile method/framework. END OF STORY. (but not the end of the blog).

This actually reminds me of a company I contracted with over a decade ago. They claimed they did XP, but when I asked them how they did XP, they defined it in terms of what they didn't do: we don't document, we don't do major designs up front, we don't ... Unfortunately, they also didn't pair program, write tests up-front, do continuous integration - bottom line, they did not do XP. Methods are not defined by what you don't do alone. They are defined by what you do and what you don't do. In most cases, when a method/framework doesn't do something, there is something else it does to achieve the intent of what other methods may have practices for. For example, Kanban's explicit policies and managing WIP accomplishes what cross-functional teams and time-boxing purports to do.

What's unfortunate, however, is that many folks don't understand this. This belief, along with, oddly enough, the belief that you should start with Scrum and then move to Kanban, is widely promoted by folks who have never done Kanban but make their living with Scrum. It's not unlike Chevy dealers telling prospective customers what's wrong with Fords. My advice is that when deciding between the two, actually, not a good move, in that you should learn from both. So, perhaps I'll say "when learning about the two" learn from someone who has done both, someone who can tell you what the advantages and disadvantages are. There are lots of these folks around, btw, so I am not saying you should pick us (although that'd be a good thing, I assure you).

Al Shalloway
CEO, Net Objectives

Author:

alshall

The Differences Between Lean Manufacturing and Lean Software Development

Alan Shalloway — Sat, 08 Jun 2013 17:18:00 CDT

Since lean comes from manufacturing, many question its validity for software developers. Our own experience is that Lean in software is very important. This blog covers three areas:

The essential paradigm shift of lean and why it applies even more to software
How the physical world is different from the software world during implementation

The Essential Lean Paradigm Shift.

Lean builds on top of Deming’s systems thinking and management style. Taichii Ohno, the creator of Lean, says he added two concepts to Deming’s work: a focus on “just-in-time” and “autonomation”.

Just-In-Time basically means, do work just in time for the following step. In software it means don’t do the requirements too far before you are ready to design. Don’t do the design too far before you are ready to code. Don’t code until you are almost ready to test. Agile manifests this by having small batches or even, as in XP, doing these different tasks seemingly simultaneously. The essential shift here is to take our focus off of productivity and people or machines and put it towards time and the workflow. In other words, instead of attempting to have each stage of the work (or each person) be as productive as possible, we focus on eliminating the delays between the steps. Just-in-time can be implemented by having small batches of work and coordinating the flow via pull.

Autonomation means automation with a human touch. Essentially, things should flow until there is an error and then a person needs to step in, see what happened, and improve the system.

Both of these concepts apply to software development, even more so than in the physical world. The reason is that when one doesn’t do “just-in-time” work piles up and delays occur between work stages. This increases the amount of work to be done even when errors (e.g., bugs) don’t occur. When errors do occur, these delays dramatically causes extra work. Consider the time it takes to fix a bug when it is detected immediately. Detect it just a couple of weeks later, and even if it hasn’t escaped from the development team it will take many more hours to fix than when caught early. In other words, just-in-time in software saves more than it does in building physical world products.

How the physical world is different from the software world during implementation,

A Lean mantra is “eliminate waste.” In the physical world, one can see waste in many forms:

Materials left over on the shop floor
Inventory on-hand that isn’t needed yet
Improperly made items that are hanging around, waiting to be fixed
The amount of work that is taking place

Unfortunately, in the software world, none of these are visible, at least not directly. The mantra – “eliminate waste” is a sound good mantra that provides little guidance. However, these wastes can be made visible by focusing on tracking delays in the workflow. Virtually all wastes in software development come from one of the following types of delays:

Delay from getting information until using it (e.g., old requirements)
Delay from making an error until detecting it (e.g., long time between code and test)
detecting an error until fixing it (actually a variant of #1)
waiting on someone for information
waiting on the next person to hand the work off to
workflow interruption due to something else receiving our attention

The salient characteristic of delay is that it can be made easily visible by noting the relationship between queue size and delay. If we create a workflow board (Kanban board) that explicitly describes our work, then we can see delays in our workflow by having queue sizes being larger than they should be. Hence, while the mantra “eliminate waste” makes sense in the physical world, in the software development world it should be “eliminate delays.”

For a seven minute expose on how delays cause a lot of extra work, watch: How Delays Cause Waste, A Main Tenet of Lean one of our Lightning Webinars.

Summary

While Lean springs from manufacturing, the insights that were so relevant there are even more critical in software. The main lessons are:

We must eliminate delays in our workflow
Work in software development is not visible, but delays, which can be seen via queues, are

Al Shalloway
CEO, Net Objectives

Author:

alshall

Installing PostgreSQL 9.0 on Mac OS X 10.6.8 via MacPorts

Bruce Snyder — Fri, 07 Jun 2013 12:13:00 CDT

While installing PostgreSQL 9.0 on Mac OS X again, I had to figure out all these steps yet again. So I'm documenting this process for my own sake because I have been through this now twice on two computers recently, having to hunt down all of these commands each time. I'm hopeful that this will help others as well.

Use the MacPorts command port to install PostgreSQL 9.0, then create and own a data directory and a logs directory:


$ sudo port install postgresql90-server
$ sudo mkdir -p /opt/local/var/db/postgresql90/defaultdb
$ sudo chown -R postgres:postgres /opt/local/var/db/postgresql90
$ sudo mkdir -p /opt/local/var/log/postgresql90
$ sudo chown -R postgres:postgres /opt/local/var/log/postgresql90

Now you need to initialize the database using the data directory that was created above:


$ sudo -u postgres /opt/local/lib/postgresql90/bin/initdb -D /opt/local/var/db/postgresql90/defaultdb

I prefer to change the postgres user's shell to bash:


$ sudo dscl . -create /Users/postgres UserShell /bin/bash

View the postgres user account just to make sure it all looks OK:


$ dscl . -read /Users/postgres
AppleMetaNodeLocation: /Local/Default
GeneratedUID: 5B38F583-CBBF-4082-A32D-C17947394A27
NFSHomeDirectory: /opt/local/var/db/postgresql90
Password: *
PrimaryGroupID: 501
RealName:
 PostgreSQL-90 Server
RecordName: postgres
RecordType: dsRecTypeStandard:Users
UniqueID: 502
UserShell: /bin/bash

Also check the postgres group:


$ dscl . -read /Groups/postgres
AppleMetaNodeLocation: /Local/Default
GeneratedUID: 715FEB22-D0F1-443F-BC93-55896210DB44
Password: *
PrimaryGroupID: 501
RealName: postgres
RecordName: postgres
RecordType: dsRecTypeStandard:Groups

Now edit the pg_hba.conf file to add the appropriate permissions:


# "local" is for Unix domain socket connections only
local   all             all                                     trust
# IPv4 local connections:
host    all             all             127.0.0.1/32            trust

This should allow you to connect easily using the psql utility.

I prefer to use the pg_ctl command to start and stop PostgreSQL. This is simply because I learned PostgreSQL on Linux and that's all there was. To prevent having to retype the full command every time I want to start or stop the database, create a start script and a stop script in the postgres user's home directory named pg_start and pg_stop. Below are the contents of the pg_start file. Make sure to create these files as the postgres user in the home directory:


$ sudo su - postgres 
$ vim ./pg_start 
#!/bin/sh 
/opt/local/lib/postgresql90/bin/pg_ctl -D /opt/local/var/db/postgresql90/defaultdb -l /opt/local/var/log/postgresql90/postgres.log start &

Below are the contents of the pg_stop file:


$ vim ./pg_stop 
#!/bin/sh 
/opt/local/lib/postgresql90/bin/pg_ctl -D /opt/local/var/db/postgresql90/defaultdb -l /opt/local/var/log/postgresql90/postgres.log stop

Don't forget to make them executable:


$ chmod +x ./pg_start 
$ chmod +x ./pg_stop

(There is a Mac OS X way of starting PostgreSQL using launchctl but I don't tend to use that because I'm used to the standard pg_ctl command.)

Now use the pg_start script to start up PostgreSQL. Execute it as the postgres user (sudo su - postgres) I tend to cat the log file just to make sure it's running correctly:


$ ./pg_start 
$ server starting

$ cat ../../log/postgresql90/postgres.log 
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started

Looks good so we'll create my user:


$ /opt/local/lib/postgresql90/bin/createuser bsnyder
Shall the new role be a superuser? (y/n) y

Because I made my user a superuser, I can create my own db schema, so log out of the postgres user account and back to my own account first:


$ exit
$ /opt/local/lib/postgresql90/bin/createdb
CREATE DATABASE

The createdb command automatically uses my username as the schema name.

The only thing left to do before starting up the database is edit your ~/.profile or ~/.bash_profile to put the path to the PosgreSQL bin directory into the PATH:


export PATH=/opt/local/lib/postgresql90/bin:$PATH

Now just log into the PostgreSQL server using psql to make sure we're ready to roll:


$ psql
-bash: psql: command not found
bsnyder@skunk [darwin](DARWIN-1527) $ /opt/local/lib/postgresql90/bin/psql 
psql (9.0.4)
Type "help" for help.

bsnyder=# select version();
                                                                 version                                                                  
------------------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 9.0.4 on x86_64-apple-darwin10.8.0, compiled by GCC i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664), 64-bit
(1 row)

bsnyder=#

And we're good to go!

UPDATE: For those who want a nice seamless experience stopping/starting Postgres via a Mac OS X preference pane, check out John Wang's PGPane.

Remove/Reinstall of MacPorts

Bruce Snyder — Fri, 07 Jun 2013 12:12:45 CDT

After trying to upgrade Mercurial to the latest version via MacPorts, I ran into some compilation issues with a couple dependencies that I could not resolve at all. After realizing that there were dependency resolution issues amongst different versions of dependencies for different ports, I decided to remove and reinstall MacPorts from scratch. This provided a clean slate and everything worked without a problem, though I was annoyed that I had to take this drastic action and spend the time to get MacPorts working again.

Bash Completion For Git

One thing that I ran into was another change to the bash completion and prompt for git. As I mentioned in a previous post about this topic, the location of the location for the git-prompt.sh file had changed but this time I discovered that I had to source both the git-completion.bash file and the git-prompt.sh file to get things working. Below is what worked for me.


if [ -f /opt/local/share/git-core/contrib/completion/git-completion.bash ]; then
  . /opt/local/share/git-core/contrib/completion/git-completion.bash
fi

if [ -f /opt/local/share/git-core/contrib/completion/git-prompt.sh ]; then
  . /opt/local/share/git-core/contrib/completion/git-prompt.sh
fi

Bash Version 4.x

As I was reinstalling ports I noticed some logging that flew by in the terminal stating something about a requirement to use Bash 4 for the bash completion functionality to work properly. So I decided to utilize the newer version of Bash installed by MacPorts and here's how:

Add the path to the new bash (/opt/local/bin/bash) to /etc/shells
Use the chsh utility to change the shell to the new bash: chsh -s /opt/local/bin/bash
After closing the terminal and opening a new one, check the bash version using either of these methods:
```
$ echo $BASH_VERSION
4.2.42(2)-release
```

As long as the version displayed is the newer version from MacPorts, you should be ready to go.

Just for my own sake and in case I need it in the future, I'm going to list a number of items that I had to reinstall:


$ sudo port install git-core +bash_completion +doc +svn
$ sudo port install tig 
$ sudo port install mercurial +bash_completion
$ sudo port install gnupg curl wget ncftp tree pstree watch screen proctools mtr cowsay
$ sudo port install postgresql90-server

For more info about installing and configuring PostgreSQL, see my post titled Installing PostgreSQL 9.0 on Mac OS X 10.6.8 via MacPorts

MacPorts Upgrade Changes Bash Completion

Bruce Snyder — Fri, 07 Jun 2013 11:06:00 CDT

After upgrading Macports recently to version xxx, I started seeing the following error in the terminal when the .bash_profile is loaded:

__git_ps1: command not found

Upon investigation, I discovered that the bash completion for git has been split into more than one file. Previously, all that was needed in the .bash_profile was to source the single file for git like so:


if [ -f /opt/local/etc/bash_completion ]; then
  . /opt/local/etc/bash_completion
fi

Since the update to Macports, now I also need to source the second file like so:


if [ -f /opt/local/share/git-core/contrib/completion/git-prompt.sh ]; then
  . /opt/local/share/git-core/contrib/completion/git-prompt.sh
fi

After closing/creating the terminal tab, the error is gone.

Forget Search, Lets Talk Discovery — A Recap of Enterprise Search Summit

Eric Pugh — Thu, 06 Jun 2013 11:01:00 CDT

The middle of May saw me road tripping to not just one, but two Enterprise Search Summits to tell some war stories in doing search at scale. I attended my first ESS back in New York in 2010, and I was curious to see how the conversation had changed in the intervening three years. Would [...]

Kotlin M5.3: IntelliJ IDEA 13, Delegated Properties and more

Andrey Breslav — Thu, 06 Jun 2013 01:48:00 CDT

Kotlin M5.3 brings support for IntelliJ IDEA 13 and some new features for you to check out. We are moving towards powerful runtime support, including reflection and other framework-enabling features. This milestone marks our first steps in that direction.

IntelliJ IDEA 12.1 and 13

First early access versions of IntelliJ IDEA 13 are coming out, and we ship a Kotlin plugin compatible with these versions. Remember it’s an EAP, use it at your own risk. Of course, good old IntelliJ IDEA 12.1 is supported as well.

Note: Some news about Kotlin support in Android Studio are coming soon.

Many Improvements

As usual M5.3 brings many improvements both in the compiler and the IDE. In the compiler we are still mostly concerned with performance, which is gradually improving. The IDE gets new quick fixes and refactorings, some of which are described below. You can now navigate to properties overriding the one you are looking at (see icons in the left gutter). The editor recognizes the syntax of KDoc (thanks to this pull request)… But first, let’s talk about cool new language features, some long awaited, some unexpected (maybe).

New Language Feature: Delegated Properties

We often get feature requests like:

Support lazy properties: the value gets computed only upon first access.
Support observable properties: listeners get notified about changes to this property.
Support storing properties in a map, not in separate field each.
Support <my favorite kind of property semantics>…

One way of addressing these requests would be to say that life is tough and users have to suffer. Another way would be to support different kinds of properties on the language level. We do not like either of these approaches: too many unhappy users on the one hand, too many ad hoc features on the other hand. So, we take a third approach: support a unified mechanism that covers all these requests (and probably more), so that particular kinds of properties can be implemented in libraries, without altering the language.

Meet delegated properties:

class Example {
  var p: String by Delegate()
}

There’s some new syntax: you can say “val <property name>: <Type> by <expression>”. The expression after by is the delegate, because get() and set() methods corresponding to the property will be delegated to it. Property delegates don’t have to implement any interface, but they have to provide methods named get() and set() to be called. For example:

class Delegate() {
  fun get(thisRef: Any?, prop: PropertyMetadata): String {
    return "$thisRef, thank you for delegating '${prop.name}' to me!"
  }

  fun set(thisRef: Any?, prop: PropertyMetadata, value: String) {
    println("$value has been assigned")
  }
}

When we read from p, the get() function from Delegate is called, so that its first parameter is the object we read p from and the second parameter holds a description of p itself (e.g. you can take its name). For example:

val e = Example()
println(e.p)

This prints “Example@33a17727, thank you for delegating ‘p’ to me!” Similarly, when we assign to p, the set() function is called. The first two parameters are the same, and the third hold the value being assigned:

e.p = "NEW"

This prints “NEW has been assigned to ‘p’ in Example@33a17727″.

Probably, you already see how to implement things like lazy or observable with this mechanism. Try it as a metter of exercise, but most of it is already done in the standard library.

The kotlin.properties.Delegates object holds the most useful stuff. Let’s start with lazy:

import kotlin.properties.Delegates

class LazySample {
    val lazy: String by Delegates.lazy {
        println("computed!")
        "Hello"
    }
}

Delegates.lazy() is a function that returns a delegate that implements a lazy property: the first call to get() executes the lambda expression passed to lazy() as an argument and remembers the result, subsequent calls to get() simply return the remembered result. If you want thread safety, use blockingLazy() instead: it guarantees that the values will be computed only in one thread, and that all threads will see the same value.

Now, let’s turn to observable:

class User {
    var name: String by Delegates.observable("<no name>") {
        d, old, new ->
        println("$old -> $new")
    }
}

The observable() function takes two arguments: initial value and a handler for modifications. The handler gets called every time we assign to ‘name’, it has three parameters: a property being assigned to, the old value and the new one. If you want to be able to ‘veto’ the assignment, use vetoable() instead of observable().

Next may be somewhat unexpected: users frequently ask what to do when you have a non-null var, but you don’t have an appropriate value to assign to it in constructor (i.e. it must be assigned later)? You can’t have an uninitialized non-abstract property in Kotlin:

class Foo {
  var bar: Bar // error: must be initialized
}

You could initialize it with null, bit then you’d have to check every time you access it. Now you have a delegate to handle this:

class Foo {
  var bar: Bar by Delegates.notNull()
}

If you read from this property before writing to it, it throws an exception, after the first assignment it works as expected.

The last thing to show is properties stored in a map. This comes up a lot in applications like parsing JSON or doing other “dynamic” stuff:

class User(val map: Map<String, Any?>) {
    val name: String by Delegates.mapVal(map)
    val age: Int     by Delegates.mapVal(map)
}

In this example, the constructor takes a map:

val user = User(mapOf(
    "name" to "John Doe",
    "age"  to 25
))

Delegates take values from this map (by the string keys – names of properties):

println(user.name) // Prints "John Doe"
println(user.age)  // Prints 25

Of course, you can have var’s as well (with mapVar() function), that will modify the map upon assignment (note that you’d need MutableMap instead of read-only Map).

There are other use cases, and probably numerous improvements to these ones. Fantasize, experiment, enjoy!

First Steps in SAM Conversions

We introduced SAM constructors last time. Of course, this is not enough, so we are working on full SAM conversions. The feature is not finished yet, but you can already use it in simple case like this:

SwingUtilities.invokeLater {
  button.setVisible(true)
}

To remind you, SAM conversions is what Java 8 uses for lambdas: when you have an interface with only one (abstract) method, like Comparator or Runnable, you are allowed to pass in a lambda where an instance of this interface is expected (in this example we pass a lambda instead of a Runnable). Kotlin does not have this as a language feature (for it is not needed in a language with proper function types), so it will only work for Java classes.

First Steps in “Callable References”

Another thing we are working on is “Callable References” or “Feature Literals”, i.e. an ability to pass named functions or properties as values. Users often ask “I have a foo() function, how do I pass it as an argument?”. The answer is: “you prefix it with a ‘::’”. For example:

fun isOdd(x: Int) = x % 2 != 0

val numbers = listOf(1, 2, 3)
println(numbers.filter(::isOdd)) // Prints [1, 3]

Here “::isOdd” is a value of function type “(Int) -> Boolean”, and you can pass it as a filtering predicate. Another example:

fun compose<A, B, C>(f: (B) -> C, g: (A) -> B): (A) -> C {
    return {x -> f(g(x))}
}

This function return a composition of two functions passed to it: compose(f, g) = f(g(*)). Now, you can apply it to callable references:

fun length(s: String) = s.size

val oddLength = compose(::isOdd, ::length)
val strings = listOf("a", "ab", "abc")
println(strings.filter(oddLength)) // Prints "[a, abc]"

If you want to use a member of a class, you need to qualify it, and the result will be of type “extension function”, e.g. String::toCharArray gives you an extension function for type String.

Note that this is early work in progress, so many things do not work yet, for example, overload disambiguation, type inference, support for properties etc. Eventually this feature will evolve into full type-safe reflection, but today we are only starting working on it.

Change Signature Refactoring

Adding/removing/reordering function parameters may be tedious when you have a lot of call sites to update. That’s why IDEs prove a “Change Signature” refactoring. Place your cursor on a function or a constructor and hit Ctrl+F6 (Cmd+F6 on Mac), and you’ll get a dialog like this:

Change type, rename, reorder or delete parameters and all your call sites will be updates accordingly.

Quick Fixes for “Type Mismatch” etc

Thanks to contributions by Jack Zhou, Michał Sapalski, Wojciech Łopata and other students of Open Source Mentorship program led by Stanford University, we now get many cool quick fixes. For example, when you get a type mismatch error, hit Alt+Enter and get some proposals to fix your code:

Code Transformations

Another group of useful IDE actions to convert between equivalent forms of code, like:

You can also use Ctrl+Shift+Up/Down to move statements or declarations:

Installation

As usual, the new plugin can be installed from our plugin repository.

Have a nice Kotlin!

Day 20 of 100 Consider Tests First

Alan Shalloway — Wed, 05 Jun 2013 10:40:00 CDT

Continuing with the 100 Things You Must Know to Be Effective In Software Development

Many consider doing Test-Driven Development but think it is too much work or will lead to future challenges. Many folks find that their tests become unsustainable over time. That's a different story, and you are holding off on TDD because of this, check out our Sustainable TDD site.

This blog is about something much simpler. It's about the insight that considering your tests (not even writing them, but just considering what they will be) will improve your code dramatically. The reason for this is that thinking about how you will test your system is a looking at your system from a behavioral perspective - which is a strong design method. Not coincidentally, one of the critical mandates from design patterns - design to interfaces (which is what tests are).

Anyway, here's the first page and a half from Chapter 3: Define Tests Up-Front from our Essential Skills For the Agile Developer: A Guide to Better Programming and Design book. If you want the rest, just click the link above.

Chapter 3

Define Tests Up-Front

With the advent of agile methods, Test-Driven Development (TDD) has been gaining momentum. A mantra of agile is that stories are completed, not merely written, every iteration. This means they have to go through testing to be considered “done, done, done.” Many teams have experienced the productivity gains and value of TDD. Many teams have, unfortunately, shied away from it as well. We believe that the value and reason that TDD works is not fully appreciated. This chapter discusses both why TDD works, and why it isn’t really testing up-front.

A Trim Tab: Testing and Testability

As mentioned in the preface, this book represents the set of trim tabs we, at Net Objectives, considers to be most useful for enhancing the productivity of software developers. We consider the issue of testability (the focus of this chapter) to be, perhaps, the greatest of these. Hence, we could say this chapter is about the trim tabs of trim tabs.

What Is Testing?

Merriam Webster’s dictionary defines test as “the procedure of submitting a statement to such conditions or operations as will lead to its proof or disproof or to its acceptance or rejection.” This is testing as an action. However, a test can also be a noun, something that is “a basis of evaluation.” We’re sure you recollect a time someone put a “test” on your desk and then you had to take it. The test you were given in this case specified what you needed to know in order to get a good grade. The action of taking the test is something different altogether.

In the same way, tests in software are about what the software needs to do in order to be considered successfully implemented. This is why we can write tests before we have code to test. We are specifying what the software needs to do. We would suggest this insight leads to the observation that test-first is really analysis first using tests. In other words, we use the tests to determine the behavior we want of the functionality we are testing. This is a form of analysis.

But it is actually more than that – it is also a type of design using tests to accomplish the design. That is, simultaneously with the analysis, we are figuring out how to implement the interfaces of the functionality. We are splitting the classes up into their methods. We are, in essence, doing design.

Testability and Code Quality

Why is this useful? We suggest it’s because testability is highly correlated to the code qualities we want to manifest, in particular, loose coupling, strong cohesion and no redundancy. We can recollect times that at the start of testing our code we have remarked:
“I can’t test this code, it does too many things that are so intertwined” (weak cohesion)
“I can’t test this code without access to dozens of other things” (excessive coupling)
“I can’t test this code, it’s been copied all over the place and my tests will have to be duplicated over and over again” (redundancy)
“I can’t test this code, there are too many ways for external objects to change its internal state”

We’ve often summed it up by saying – “gee, I wish they had thought of how this code was going to be tested while they were writing it!”

I’m kind of slow sometimes because it took me quite some time to realize (comment by Alan):
I should consider how my code is going to be tested before writing it!

The reason is clear – testability is related to loose coupling, strong cohesion, no redundancy and proper encapsulation. Another way to say this is that the tighter your coupling, the weaker your cohesion, the more your redundancy and the weaker your encapsulation, the harder it will be to test your code. Therefore, making your code easier to test will result in looser coupling, strong cohesion, less redundancy and better encapsulation.

This leads to a new principles:
Considering how to test your code before you write it is a kind of design.

Since testability results in so many good code qualities and since it is done before you write your code, it is a very highly leveraged action. That is, a little work goes a long way – it’s a great trim tab.

Click here for the rest of the article.

If you are looking to improve your code, I really suggest this simple tip. It will make a difference.

Al Shalloway
CEO, Net Objectives

Did you like this? Please tweet by clicking on twitter icon below.

Author:

alshall

Day 19 of 100 Know the Power of Teams

Alan Shalloway — Sun, 02 Jun 2013 09:41:00 CDT

Continuing with the 100 Things You Must Know to Be Effective In Software Development

This blog is also in preparation for a webinar I am doing this Wednesday called: How to Start an Agile Implementation.

I wrote a blog about a year ago called Why Scrum Works and How This Tells Us When It Won’t it was a follow up to a blog now more than 6 years old Challenging why (not if) Scrum works. I have long held that Scrum was an incomplete manifestation of Lean-Thinking and I have to say it is nice to see thought leaders in the Scrum community finally coming around to this thinking as well.

However, Scrum still provides little insights into how to create teams when they are not present and are difficult to form. The notion that the development group is comprised of generalists is simplistic (see Specialization Exists – Honor It). While the espoused Scrum model is clearly the most efficient, it is not always the most cost-effective, or even an achievable approach in many situations.

This blog will focus on why teams are so important. In a follow up blog I’ll talk about different ways of achieving them. Let’s consider the seven wastes of software development:

Partially done work
Task switching
Handoffs
Defects
Delays
Relearning
Extra features

It not a coincidence that cross-functional teams can help eliminate each of these when they focus on doing iterative development. This is best achieved when the teams understand Lean-Flow and that it implies they should work on the fewest number of features they can – this allows for quick completion of the work with few items in play.

Partially done work. With a team focusing on getting work done, this won’t happen much. Start something, finish it. Stop starting and start finishing works for Scrum teams as much as it does for those doing Kanban.

Task Switching. If we focus working on one story, completing it before going on to the next, there will be little task switching. Pull an item, swarm on it, finish it, go to the next one. No task switching. Nice.

Handoffs. Part of the way to complete stories quickly is getting all of the folks you need work on it. This also eliminates handoffs.

Defects. Swarming as described above means coders and testers will be working together (pairing even?). This will lower the number of defects and enable their quick elimination when they do occur.

Delays. Delays are the cause of extra work, but, by avoiding the above, there will be few delays and overall waste is eliminated.

Relearning. This will be reduced by folks working together.

Extra Features. By building stories incrementally and getting feedback on how the system is manifesting value, we can avoid building things we don’t need. This, of course, requires us to build the most important items first.

Teams are also powerful because they provide for synergy, building camaraderie, creative ideas, higher morale and just plain fun.

Teams are good, work towards having them.

And, of course, if you don’t see how to do that, drop me a line – I’ll be happy to talk to you about how to do that.

Al Shalloway
CEO, Net Objectives

Did you like this? Please tweet by clicking on twitter icon below.

Author:

alshall

Day 18 of 100 Learning What You Don't Know You Don't Know

Alan Shalloway — Sun, 02 Jun 2013 07:12:00 CDT

Continuing with the 100 Things You Must Know to Be Effective In Software Development

I'm writing this and the next day's blog, "The Importance of Teams and How to Create Them" (posting later today) in preparation for a webinar I am doing this Wednesday called: How to Start an Agile Implementation. I believe these two concepts are critical for starting Agile initiatives. Too many folks go down a path that literally dooms them to failure, or, at least, makes it much harder to succeed. Here it is.

“It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so.” - Mark Twain

“The problem ain't what people know. It's what people know that ain't so that's the problem.” – Will Rogers

We’ve all heard that there’s what we know, what we don’t know and what we don’t know we don’t know. While it may appear difficult to discover what you don’t know you don’t know, when it comes to a particular field, such as “how do I do software development” all you have to do is ask others who are doing it. One of the reasons Net Objectives is one of the few (only?) companies that teach and coach in: business portfolio management, Scrum, Kanban, Lean, XP, SAFe, and our own Lean-Agile Framework, is that we’ve found you need lessons in the practices from all of these. With the exception of Lean, which is more mindset and a container for practices, all of the others have different levels of value in different places. However, it is worth knowing the value of each since it provides insights into where you are.

By taking advantage of different ways different folks approach a problem, we can learn about things we might not otherwise become familiar with – and might just let you know about something you previously didn’t know and didn’t know you didn’t know it.

Here’s a list of some things that I think everyone should at least know about, that many folks don’t appear to know, and don’t seem to know they don’t know it.

The best place to start is often at the top, not the team.
If executives appear to development to be acting in a less than ideal way, it may be that they are not being presented with the information they need
Hacking in code is not always the fastest way to do it in the short term
Lean is not about going fast, it is about removing delays
Starting an Agile transformation by creating a pilot by picking all of the people required for it without regard if you are removing skills required by the rest of the organization may be a sure way to early Agile success while dooming the transition to failure (see Day 5 How Successful Pilots Can Hurt an Organization)
The right approach is not to take the time to do the correct thing, the right approach is to take the time to not do the wrong thing

There are probably more, but I don’t know them. But at least I know I don’t know them, so I’ll keep thinking about it. Let me know any you find. Remember Nietzsche's observation:

“Many are stubborn in pursuit of the path they have chosen, few in pursuit of the goal.”

Here are two other quotes by Twain and Rogers that can provide insight:

“A man who carries a cat by the tail learns something he can learn in no other way.” Mark Twain

“There are three kinds of men. The ones that learn by readin’. The few who learn by observation. The rest of them have to pee on the electric fence for themselves.” – Will Rogers

Learn from others, but don't discard your own experience. If one doesn’t understand why others are doing what they are doing, they may be seeing things you are not seeing and they may be ignoring things you are counting on. I would suggest you each have something to learn from each other.

As always, if you or your company are having challenges in business, management, team agile or team technical practices, contact me so I can be of value.

Thanks for reading,

Al Shalloway
CEO, Net Objectives

If you liked this blog please tweet about it by clicking the icon below.

Author:

alshall

Day 17 of 100 The Lessons of Design Patterns

Alan Shalloway — Sat, 01 Jun 2013 10:29:00 CDT

Continuing with the 100 Things You Must Know to Be Effective In Software Development

Years ago people knew they didn't know much about design patterns. Now, most folks think they know design patterns, but my experience in talking to them makes me believe otherwise. Sorry if this sounds a bit aloof, arrogant or elitist, but I do believe it is true. Patterns have been called "solutions to recurring problems in a context" by Christopher Alexander in his Timeless Way of Building (one of my all-time favorite books that I highly recommend, whether you want to learn patterns or just learn more about architecture). While this phrase is bandied about quite a lot, what's not as well know is another quote from the book: "at this final stage, the patterns are no longer important: the patterns have taught you to be receptive to what is real." Alexander is referring to the forces in the problem domain - that is, the issues you must attend to to achieve a high quality design.

The Gang of Four book, Design Patterns: Elements of Reusable Object-Oriented Software actually provides these forces, but if one is looking for solutions, they are often overlooked. While getting into a discussion of how patterns teaches us about forces would be beyond the scope of this blog (see references at the end of this blog for that), we can talk about how design patterns provide us with a solid approach to design - one that is often overlooked. Chapter 1 (Introduction) from the book provides an overview of the lessons learned from Design Patterns:

design to interfaces - that is, design to the behavior of the object, not it's implementation.
find what varies and encapsulate it (essentially, put a layer in, either with an Interface, Abstract class, or even an object that figures out the proper delegation
favor delegation over inheritance. In other words, instead of having different ways of doing things by deriving new methods in a class that uses these methods (which leads to a complex inheritance hierarchies), have objects that are hidden behind interfaces and have the using object refer to this interface

Basically, patterns tell us to hide variation in our solutions and do it by encapsulating the implementations that vary behind Interfaces, abstract classes, function pointers, or any way you can think of. You can quickly see most of the patterns are examples of this by looking at our list of Patterns by Encapsulation on our Design Patterns Repository. You'll notice all of the behavioral patterns are about this. To accomplish this requires separating use from construction. This is essentially the practice of encapsulating construction so that objects don't know which particular implementation of a behavior they are using. You either make an object or use an object, but not both. Patterns that do this are the creational patterns.

But what do you use when you already have objects and want to put them into this encapsulating scheme? That's where the structural patterns come in.

I do strongly suggest you delve more in here in some way or another. While knowing a few patterns (e.g., strategy, template method, bridge, ...) is great, understanding what they truly are and how to think in terms of the forces they expose is much more powerful.

Want more? Check out:

Can Patterns Be Harmful? A good introduction to patterns
Design Patterns Explained: A New Perspective on Object-Oriented Design (while admittedly biased, I believe this is still the only book that talks about patterns as forces a la Alexander)
A webinar: Design Patterns Explained
Our leading edge course that will be a design attitude changing experience: Design Patterns Explained course.

As always, please let me know if you'd like to chat about opportunities.

Al Shalloway
CEO, Net Objectives

A note on the 100 in 100 challenge: I will pick up the daily entries this week. To be able to accommodate this with my existing schedule, I will start writing short blogs that point to other materials when available. The intent will shift to awareness instead of trying to explain the concept. This should make it both easier for both author (me) and reader (you) to keep up.

If you liked this blog please tweet about it by clicking the icon below.

Author:

alshall

Slides and Audio from Let’s Test Posted

Johanna Rothman — Fri, 31 May 2013 07:35:00 CDT

I delivered a keynote at Let’s Test in Sweden last week, “Becoming a Kick-Ass Test Manager.”

A lovely gentleman, Aleksis Tulonen, recorded the audio. Thank you, Aleksis! I attempted to marry the audio to the slides. I was not successful. However, the audio is up on slideshare. You can download the slides, page through them and listen to the audio. There are gems, such as, “I’m not done yet. This is my keynote.”

On the slideshare, there is a link to my original keynote, the video of my StarWest 2012 keynote.

Clojure on Cloud Foundry

Matt Stine — Wed, 29 May 2013 22:10:00 CDT

I was inspired by Brian McClain’s post on bringing Haskell to Cloud Foundry using Cloud Foundry v2 buildpacks, so I decided to go on a buildpack journey of my own. Since Clojure is the language I most enjoying “toying around with,” I thought I’d try to deploy a simple Clojure web application using the Heroku Clojure Buildpack.

To reiterate some of the coolness around buildpacks, they are what allows a PaaS like Cloud Foundry or Heroku to support various runtimes without first building that support into the core platform. If your favorite language or framework runtime isn’t available, there’s absolutely nothing stopping you from providing your own buildpack to support it. Stuff can get crazy – McClain has even hinted at bringing FORTRAN support to Cloud Foundry.

I decided for my experiment to build a VERY basic “Hello World” style application using Ring, which is “a Clojure web applications library inspired by Python’s WSGI and Ruby’s Rack.” The easiest way to get started building Ring applications is to utilize the popular Clojure build tool Leiningen.

First I started by creating a new project:

$ lein new hello-cf
$ cd hello-cf

The next task was to add the Ring dependencies to my project.clj file:

(defproject hello-cf "0.1.0-SNAPSHOT"
  :min-lein-version "2.0.0"
  :description "Hello Clojure on Cloud Foundry!"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.4.0"]
                 [ring/ring-core "1.1.8"]
                 [ring/ring-jetty-adapter "1.1.8"]])

Then it was time to create the Ring application itself, by editing src/hello_cf/core.clj:

(ns hello-cf.core
  (:use [ring.adapter.jetty :only [run-jetty]]))

(defn handler [request]
  {:status 200
   :headers {"Content-Type" "text/html"}
   :body "Hello Cloud Foundry from heroku-buildpack-clojure!"})

(defn -main [port]
  (run-jetty handler {:port (Integer. port)}))

Let’s break this down a bit. The handler function will handle any HTTP request that hits our application, and will return an “OK” response containing a pleasant message indicating that we’ve succeeded. That’s really about it. Our application is complete. We can test it out by running the following:

$ lein trampoline run -m hello-cf.core 8080
2013-05-29 22:42:52.576:INFO:oejs.Server:jetty-7.6.1.v20120215
2013-05-29 22:42:52.804:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:8080

Hitting http://localhost:8080 in the browser confirms that we’re well on our way. Now it’s time to trying pushing the application to Cloud Foundry. As Brian stated in his blog, one of the stellar aspects of Cloud Foundry buildpacks is that they are approximately the same as Heroku buildpacks. Practically, this means that one should be able to utilize a Heroku buildpack on Cloud Foundry with minimal or no modifications. Let’s put that theory to the test, shall we? Before we do, let’s create a Procfile quickly to let the buildpack know what we want to run:

web: lein with-profile production trampoline run -m hello-cf.core $PORT

And on with the push:

cf push hello-cf --buildpack=git://github.com/heroku/heroku-buildpack-clojure.git
Using manifest file manifest.yml

Creating hello-cf... OK

1: hello-cf
2: none
Subdomain> hello-cf

1: mstine.cf-app.com
2: none
Domain> mstine.cf-app.com

Binding hello-cf.mstine.cf-app.com to hello-cf... OK
Uploading hello-cf... OK
Starting hello-cf... OK
-----> Downloaded app package (12K)
Initialized empty Git repository in /tmp/buildpacks/heroku-buildpack-clojure.git/.git/
Installing heroku-buildpack-clojure.git.
-----> Installing OpenJDK 1.6...done
-----> Installing Leiningen
       Downloading: leiningen-2.1.2-standalone.jar
       Writing: lein script
-----> Building with Leiningen
       Running: lein with-profile production compile :all
       Retrieving lein-standalone-repl/lein-standalone-repl/0.1.5/lein-standalone-repl-0.1.5.pom from clojars
       Retrieving lein-standalone-repl/lein-standalone-repl/0.1.5/lein-standalone-repl-0.1.5.jar from clojars
       Performing task 'compile' with profile(s): 'production'
       Retrieving org/clojure/clojure/1.4.0/clojure-1.4.0.pom from
       ...
       Compiling hello-cf.core
-----> Uploading staged droplet (66M)
-----> Uploaded droplet
Checking hello-cf...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
  0/1 instances: 1 starting
  0/1 instances: 1 down
  0/1 instances: 1 starting
  0/1 instances: 1 flapping
Application failed to start.

Drat. Let’s take a quick look at the logs to see what may be awry:

Reading logs/stderr.log... OK
/home/vcap/app/.lein/bin/lein: line 42: java: command not found

Ah-hah! Looks like the existing buildpack is making some assumptions about the structure of our application that no longer hold true on Cloud Foundry. So, I followed in Brian’s footsteps and forked away. One small commit looks like it ought to fix the problem. Let’s give it another try:

cf push hello-cf --buildpack=git://github.com/mstine/heroku-buildpack-clojure.git
Using manifest file manifest.yml

Not applying manifest changes without --reset
See `cf diff` for more details.

Uploading hello-cf... OK
Changes:
  buildpack: 'git://github.com/heroku/heroku-buildpack-clojure.git' -> 'git://github.com/mstine/heroku-buildpack-clojure.git'
Updating hello-cf... OK
Stopping hello-cf... OK

Starting hello-cf... OK
-----> Downloaded app package (8.0K)
-----> Downloaded app buildpack cache (17M)
Initialized empty Git repository in /tmp/buildpacks/heroku-buildpack-clojure.git/.git/
Installing heroku-buildpack-clojure.git.
-----> Installing OpenJDK 1.6...done
-----> Using cached Leiningen 2.1.2
       Writing: lein script
-----> Building with Leiningen
       Running: lein with-profile production compile :all
       Performing task 'compile' with profile(s): 'production'
       Compiling hello-cf.core
-----> Uploading staged droplet (66M)
-----> Uploaded droplet
Checking hello-cf...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
Staging in progress...
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  0/1 instances: 1 starting
  1/1 instances: 1 running
OK

BOOM!

I quickly pointed my browser, and as yesterday’s tweet indicates, success:

Yeah…that just happened. cc: @cloudfoundry #clojure #buildpacks #winning pic.twitter.com/9lJHqmWQPw
— Matt Stine (@mstine) May 28, 2013

Score another win for Cloud Foundry’s buildpack support. I’m now toying with the idea of doing something of a world tour of LISP on Cloud Foundry. My next candidate may be Scheme.

Into the Crucible

Matt Stine — Wed, 29 May 2013 09:00:00 CDT

Wow…it seems I only post to this blog toward the end of May. Well, that all changes now. You see, as of June 3, 2013, this blog is going to become one of many aspects of my new “day job.” On Monday, I start my life as a Community Engineer with Cloud Foundry by Pivotal. What’s a Community Engineer? Quite honestly, I’m not completely sure of the answer to that question yet. But given the many conversations I’ve had over the past few weeks, it seemingly fits right in with the bridge-building roles I’ve played many times over the course of my career. In this case, I have one foot squarely planted in the world of Cloud Foundry engineering, and one foot squarely planted out in the world with you guys and gals - the community. My job is to help you understand how we are quite literally seeking to “build a new platform for a new era.”

Of course, this is a journey that for me started a few years ago. In my previous life as a front-line development manager, I helped lead an agile transformation within my team with “ruthless automation” playing a central role in everything that our team did. However, it seemed that the better we “did agile,” the more pain we felt when dealing with those outside of our circle of control. It was only years later, after reading Eliyahu Goldratt’s The Goal and coming to an understanding of his Theory of Constraints, that I realized what was happening. Our constraints had moved “out of the plant,” if you will, and landed in the world of operations. Even without this understanding, I developed a keen interest in this newly emerging topic called “DevOps” and began to explore the ideas emerging around agile operations and infrastructure as code. I started playing with tools like Puppet, Chef, and Vagrant, and taught sessions on all three of them at the Project Automation Experience in 2011.

You can read my last entry and find out that not much later I joined VMware as a Senior Consultant for its Cloud Application Platform. I was hired into that role based on my extensive background in enterprise Java and the Spring ecosystem, but it was nothing short of a staffing accident that I found myself thrust into a role on a virtualization platform provisioning team helping to build out a private self-service cloud! I was steadily getting carried further away from my role as an application architect, steadily becoming assimilated into that mysterious world of web operations that I knew so little about. These experiences, along with my continued reading and thinking about the worlds of DevOps, Lean, and Kanban, have quite literally changed the way I look at the world of software engineering (or as I prefer to think of it now, value delivery through product engineering that just so happens to involve software!). These experiences have formed around me a crucible, melting me that I might be poured into a new professional mold.

So now it’s time to plunge into the world of building the leading open platform as a service, and to help YOU experience the HUGE can of @cloudfoundry awesome that we at Pivotal are about to unleash on the world. Sound good to you? Join us!

Gr8conf EU 2013 at a glance

Andres Almiray — Wed, 29 May 2013 07:23:00 CDT

Last week Copenhagen became the grooviest city in the world for a few days, as Gr8conf EU 2013 had it's fifth edition. Here's a quick report of what transpired during this awesome time.

The trip began with a quick stop at the annual Groovy DevCon. What's Groovy DevCon you ask? Well it's a meeting of people behind the Groovy language. Topics discussed in this meeting usually shape the future of the language and some of the projects related to it. This time Jürgen Höller was in attendance as the next version of the Spring Framework (version 4) will provide a much improved (and integrated) support for Groovy. Later that evening we had the traditional Speaker's dinner, a perfect occasion to catch up with old friends and make new ones.

During the University Day (May 22nd) Cédric Champeau and myself presented a workshop on Groovy AST transformations. Cédric did an amazing job building a comprehensive set of exersises and documentation (all of which available at https://github.com/melix/ast-workshop), full credit to the workshop's success goes to him He has written down his own conference report too. Basically what happened is I rambled for 30 minutes on my own experiences writing AST transformations then we switched gears into hacking solutions for each exercise.

Groovy AST Demystified from Andres Almiray

Later that evening attenddees joined in what has become a Gr8conf staple: Hackergarten on Tour. We had over 50 participants working on 10 tasks. Some of the tasks included:

Improving the GPars codebase for multiverse.
Adding a @Transactional AST transformation to Grails; no more Spring proxies.
a Griffon + JavaFX document viewer for Asciidoc sources, relying on Asciidoctor.
Updates to a handful Grails plugins (JodaMoney for example).
Improved code coverage for Open Dolphin, reaching 100%.
Spock, Ratpack, Geb and more

Steve took the opportunity to interview some of us during the Hackergarten, videos were posted to Gr8conf's Youtube channel. I must confess that so far Hackergarten is my favorite part of the conference as it brings out the best of our Groovy community: friendship, synergy and lots of fun. We stayed there until the doors closed; I'm pretty sure some of use would had continued if we were allowed in the room for more time (sleep is overrated anyway).

The very next morning Søren declared the conference officially open for business by making an introduction and recounting the history behind it. Then came the turn for Venkat to inspire the audience with his keynote "The rise and fall of empires: Lessons for language designers and programmers", and inspire he did! There's one rule when Venkat appears in a conference: you simply don't miss his presentations. Now, the organizers made an excellent job building up the schedule, as it was hard to choose which session to go next as all of them were of high quality presented by talented people, this trend continued for the rest of the conference -- aside, there was a good number of newcomer speakers to the conference this year. Later in the day Venkat came back with a vengeance presenting "Functional Groovy", shedding some light on Groovy's functional capabilities in case you're fed up with procedural shenanigans. The first day was brought to a close with the Meet'n'Greet, that is, a big grill + beer party for attendees and speakers alike. The Gr8conf crew brew their own brand of beers, ales and malts which were delightfully sampled, tasted and drank by all.

Friday began half an hour earlier than yesterday, a fact Russel felt compelled to mention every 5 minutes during his talk "Is Groovy as fast as Java?", because why not, it was early after all Russel presented some shocking results when running Groovy with JDK8; I won't spoil them here, just be sure to check out his report, fork the code and test it for yourself. Next up was "Griffon Update" by yours truly. I asked the audience who was making use of the framework, turns out more than half was working with it already! The talk was a mixture of what has happened since last Gr8conf (remember Griffon 1.0.0 was released live on stage last year, the book was deveiled too). The slide deck has some interesting numbers in terms of growth and usage; turns out Griffon is being put to work in 86 countries around the world. There was a live demo moment too, where the latest feature "action interceptors" was shown in the context of the shiro and scaffolding plugins. Doctorpad (the Asciidoc viewer built during the Hackergarten) made an appereance as well.

Griffon Update Gr8conf 2013 from Andres Almiray

I was planning to get another surprise shown during the talk, sadly fumbled the plans during my stay in Mexico a week prior to Gr8conf, so it'll have to wait for the next opportunity. However I managed to bring some cool give-aways

What is Gr8conf without Mr. Haki? I'm glad Hubert came back again and shared enlightening and very cool tips and tricks that Groovy 2.0 has to offer. Also, he's probably one the fastest type-while-speaking people I've seen; always a great show. The room was fully packed because ...

Lastly on my list of attended talks is Luke Daley and his Ratpack reboot. Luke showed us the latest tricks coming in Ratpack 0.9, including a full tour of the Java APIs. He dared to show Java code at a Groovy conference, yes, he's that guy Ratpack has matured a lot since Luke joined the team a few months back. He painstakenly pays attention to API design, tool support and documentation, which makes it much easier to get started with Ratpack.

A big shoutout to the Gr8conf Crew and sponsors for making Gr8conf EU an incredible experience and unforgettable event. Very much looking forward to the next edition!

Keep on Groovying!

How We Do the Kanban Method

Alan Shalloway — Tue, 28 May 2013 05:32:00 CDT

Sorry to be pushing back the 100 in 100 project but hopefully the readers of that will find this blog just as interesting. If you are not familiar with Kanban, you should get familiar with it. It is one of the most exciting things to have happened in the Agile space. See my article De-Mystifying Kanban for a brief introduction.

I have been involved in an interesting discussion on the Kanban Dev group (a great place to go to engage in discussions about the Kanban method). I wrote up a description of how we (Net Objectives) does Kanban. This is an appropriate paraphrasing of it. Though it might be of interest.

To summarize our approach to the Kanban Method, and trying to be a bit clearer:

Start where you are
Create visibility by making a Kanban board
Establish explicit workflow policies
Determine if the Kanban Method is appropriate (I’ve sometimes veered away from it at this point and done more of a pure lean flow approach where we mapped out an approach that was based on swarming, not WIP limits). This determination is often made based on who should pull the work. Sometimes the folks doing the work now are not organized correctly and the value stream mapping makes this apparent.
If Kanban Method is appropriate, establish WIP limits and a pull mechanism if it is not in place already
As move forward look for opportunities to:
1. Load balance
2. Redo flow to reduce delays in feedback, error detection
3. Have folks work together
Take these opportunities as they seem appropriate and validate improvement in flow/quality/discovery
Repeat steps 4-7

A note about #4. These seem to fall into two camps. One is a significant change in the manner work to be done is initiated by the business stakeholders. Improvements here can have dramatic affects downstream.

One of my favorite quotes is by Eli Goldratt: “Often reducing batch size is all it takes to bring a system back into control.” By focusing on minimal business increments, this is often achievable. The other camp is to improve workflow directly by changing how work is done. This is often a dramatic change and most often involves changing the order of the work or who pulls the work. We then do continuous improvement a la Toyota Kata. I would not suggest what was happening in these cases was the Kanban method. In our practice we don’t really make such a distinction though, whether it is Kanban Method or Lean – that appears to be arbitrary. But I am wanting to honor the distinction being made on this group.

Regardless of what happens in step 4, our focus is on delay as we believe that is the main problem. Limiting WIP reduces delay, and is a great proxy for process improvement. But I believe these four types of delay is what causes virtually all of the waste in software development:

From when information is obtained until it is used (e.g., getting requirements too soon)
From when an error is made until it is detected (e.g., delayed testing, delayed integration)
From when information is needed until it is obtained (e.g., waiting for someone to be available)
Delays in workflow – just stoppage due to an interruption

I would suggest just about every challenge to workflow shows itself up as a delay. This is the striving for Just In Time of Lean. Limiting WIP is a vehicle for that. There are other mechanisms (teams, ATDD, continuous integration, small batches).

I believe understanding delay is useful to explain why Kanban works so well.

To me, the power of the Kanban Method is not just improving flow, it is a great example of Toyota Kata (I strongly suggest reading this wonderful book by Mike Rother if you haven’t).

As always, if you have interest in this and want to see how it might help your organization, please contact me. Always happy to provide a free consult.

Al Shalloway
CEO, Net Objectives

Author:

alshall

Why I Endorse Both SAFe & the Kanban Method

Alan Shalloway — Mon, 27 May 2013 16:19:00 CDT

This was originally titled "How I Can Endorse SAFe & the Kanban Method" due to consternation my endorsing SAFe seems to be causing some in the Kanban community. But then I realized that most folks in the Agile community don't have this concern, so I thought it's current title to be more appropriate. In a nutshell, I endorse both, because they both have value.

I was recently asked how I can support both the Scaled Agile Framework (SAFe) while being a supporter of evolutionary methods (Lean and Kanban Method).

Here is a paraphrasing of the email:

You sound in so far as I read below as someone who promotes the evolutionary approach. However it is a bit confusing to me where you actually stand.

For example: You seem to endorse SAFe as it stands today, SAFe's prescription (no options) of a disruptive method at the team level does not indicate doing something about that understanding (trusting it exists). As an example Operation teams, Enterprise Data Movement teams balk at this. This prescription as you may already know - does not take into account team's capacity for change and undermines their probable understanding of their scope of Continuous improvement.

To me it's not of prime importance what size an evolutionary step (adaptation) should be but how you improve.

The Kanban method calls for Continuous improvement collaboratively - Not a coach or consultant or method or framework or model or Agile office rolling out adaptations to teams that are going through transformation (small or large).

First, SAFe is not as prescriptive as many hold it to be. It is a starting point and can be tailored considerably. SAFe is explicitly based on Lean and uses Kanban at the portfolio level and Scrum at the team level. Scrum's use is merely because there are lots of Scrum teams out there - Kanban at the team level could just as easily be used. Dean had me present an intro to Kanban at the SPC training I did.

But, in my opinion, apparently counter to both Scrum and Kanban Method thought leaders, there is no right way to transition. Some companies can use a jump, others need evolutionary. Scrum has demonstrated it can be quite useful in jumping, but if that’s the only thing you have it can also be quite dangerous. In all cases, one must look where you are, create visibility, create explicit policies, then make your move and evaluate (PDCA).

I endorse SAFe because it creates a vision about the big picture that many in the software industry are looking for. One that Scrum thought leadership has not provided - continuing to endorse Scrum-of-Scrums. I have/had been involved in LSSC, LSS, LKU primarily to create this vision. SAFe at least creates that. Whether it is the right method to use depends upon the organization.

Net Objectives supports SAFe because in some situations, it is the right way to do it. We also have the Lean-Agile-Framework which is evolutionary (our own IP, consistent with SAFe, but more flexible). The point is you need to do what works for you. If SAFe only introduced the key points that were needed for enterprise agility that would be useful. But it provides a way of doing it that can be a starting point for common understanding. It provides methods that have worked at scale. The irony is, that for all the fuss people have complained about SAFe being heavy or just plain old wrong, there is a lot of evidence that it works. I'd like to see folks point out more case studies for companies with IT organizations greater than 1000 people where Scrum or Kanban has worked.

What I think i like most about SAFe, however, is that it is changing. No one in the SAFe organization says anything is out of scope in the discussions as has happened in the Scrum and Kanban Method communities. If it can help, it is worth discussing. If it can't, then it is worth discussing why the person talking is misled.

I try not to hold to a mindset, but continuously look for solid ideas that can help our clients. So far, I've seen SAFe provide value.

Al Shalloway
CEO, Net Objectives

Author:

alshall

Day 16 of 100 Understanding and Achieving Robust Architectures

Alan Shalloway — Sun, 26 May 2013 17:20:00 CDT

May 17, 2013 — Posted by Al Shalloway

Continuing with the 100 Things You Must Know to Be Effective In Software Development

Many wonder how a quality architecture can be built in an large organization attempting to achieve Agile methods. While most acknowledge big design up front (BDUF) has certain disadvantages, the design as you go approach that many espouse in the Agile community seems to have its own pitfalls and risks. At Net Objectives we believe that both extremes – too much architecture and no architecture - are fraught with risk and waste and are to be equally avoided. To understand what is needed and how to achieve architecture in the Agile space, one must step back and look at what architecture needs to provide to us.

Architecture, whether Agile or otherwise, needs to provide the following:

A vision of the context within which we are designing so as to allow different teams to stay aligned while implementing in the best way for their local needs
Enable extensions to the system, in particular, new concepts, technologies and scaling requirements that are discovered/demanded/required must be able to be implemented with low overhead
Enable new ways of implementing existing concepts (e.g., new pricing, new countries) with only the cost of the new functionality. That is, there should not be any impact to existing functions.
Foster testability throughout the system – both at the acceptance level and team functional level. In particular, new requirements should not adversely affect regression testing.
De-couple systems from the application. As systems evolve, the application should not have to change merely because the system architecture does unless the changing system enables new functionality
Facilitate reuse of common functions across teams

Doing the above is not necessarily easy, but it is made much more possible with an understanding of:

how to evolve systems with emergent design
how to test systems at the behavioral and functional level
the different perspectives of conceptual Vs specification; creation from using; and system architecture from application architecture

The skill sets associated with the above are:

Being able to do quickly identify sets of implementations that are conceptually the same and how they relate to each other (this is readily achieved with commonality / variability analysis using the analysis matrix)
Pattern oriented design
Test-Driven Development
Acceptance Test-Driven Development

Agile architectures will then enable us to:

Inject new concepts with little to no rework
Add new implementations of existing concepts with only minimally affecting the existing system
Change the system with little risk of introducing errors and full confidence that if you do you will detect them

We would suggest the needs of Agile and non-Agile architecture are really the same, Agile architecture merely needs to evolve in smaller increments. Architecture shifts from being this static structure that can accommodate everything to one of an architecture that can be easily modified as new information (often in the form of additional requirements) become known.

As you might guess, we have training and coaching on all of the areas mentioned above. We are introducing an online course, instructor facilitated course as well. If you are interested in improving the skill sets of your folks, please send me a note.

Al Shalloway
CEO, Net Objectives

If you liked this blog please tweet about it by clicking the icon below.

Author:

alshall

Devs in the ‘Ditch Slides Posted

Johanna Rothman — Tue, 21 May 2013 00:29:00 CDT

I gave a talk at Devs in the ‘Ditch last week when I was in London. I posted the slides on slideshare: Overcoming Three Pitfalls of Transitioning to Agile.

The very nice people at 7digital made a video and posted it, too. If you can take the time, watch the entire video. Rob Bowyer gave a great talk about kanban and theory of constraints. My part about overcoming these three pitfalls starts at about 42 minutes in.

There are many other pitfalls to transition. This talk had just three of them: the stories are too big, you need experts to do the work, and you implement as layers instead of through the architecture.

I hope you enjoy the presentation and the video.