The Farr Side

Misconceptions about Team Rooms and Open Floor Plans

Posted on August 15, 2011 by efarr

I see this far too often. Well-meaning software organizations embracing agile software development tear down the walls in order to open up the space and allow easier collaboration. This sounds great, and it’s cheap. An easy win, right? Not if it’s done without some care and thought.

Premise 1: Irrelevant conversations are distractions.

Human beings are trained to pick other human beings’ voices out of the background noise and pay attention to them. There is little that is more distracting to concentration than hearing a conversation that has nothing to do with what you are working on.

Premise 2: A team is a group of people working toward a common goal.

If a team is really acting as a team, there is nothing that one subset of the team could be working on that is irrelevant to the rest of the team.

Imagine Team Green working in an open team room, paired at stations with one monitor and dual mice and keyboards…

Let’s say that pair number 1 is having a conversation…

That conversation is heard by all other members of the team. Because Team Green is a team, the fact that the whole team can hear it is a good thing. I cannot count how many times I have seen this…

Pair #1 overhears pair #2 getting stuck on some problem that sounds familiar. Pair #1 stops and gets involved in what pair #2 is doing. The four people discuss the problem and solve it, based largely on some previous experience that someone on pair #1 had. If one pair had not overheard the other pair struggling, they might have wasted a whole day (or at least until the next stand-up).

Now imagine Team Green is really busy and all working and talking…

I first heard this situations referred to as communication saturation by Jeff Sutherland, but I think he got it from Jim Coplien. Here we have everyone able to hear every other conversation. If a stranger were to come upon this team area, it would sound like noise. But to a high-performing agile software team, it is completely natural because every conversation is related to furthering the goal for this iteration.

Now imagine we have a collaborative, high-performing Team Blue…

Team Blue is enjoying the same open space benefit that Team Green is. Now imagine we take our open-spaces-are-good momentum and co-locate Team Green and Team Blue…

Now we have green conversations in the blue team area and green conversations in the blue area. This is not good. We now have noise distracting both teams.

If the blue conversations are as useful to the green team as green conversations, then you are not doing team based development.

There are two possible remedies: create more space between the teams, put up acoustically meaningful walls, or both.

The bottom line: open spaces and shared conversations are good only within a team, not between teams.

Posted in Agile, Management | Tagged noise, pair programming, teams | Leave a comment

DDD Anti-Pattern #1: Not Accounting for Commands and Queries as Separate Concerns

Posted on August 12, 2011 by efarr

As a long time object-oriented programmer, I was immediately drawn to the DDD concept of Persistence Ignorance. My initial idea of DDD was to simply express the domain concepts in a C# object model, then one one side, map those objects to a persistent store with a technology like NHibernate and on the other side, build a UI without any business logic in it.

Figure 1: The Naïve Transition to DDD

This was a big improvement over the traditional techniques on the Microsoft stack. Databases are inadequate for expressing domain models and business logic in UIs or databases makes unit testing nearly impossible.

How the Naïve DDD Approach Fails

What I didn’t see was that this naïve solution traded one set of problems for another. The new solution looks like Figure 2 below.

Figure 2: The Naïve DDD Approach

This looks OK until you try to build anything remotely complex with this model. The first bit of friction we ran into involved populating some of the screens on the UI. Many screens don’t map smoothly to our rich object model. We get by with technologies like LINQ and lazy loading, but it still feels awkward to spin up a rich object model to simply copy properties to a view.

Things really turn south when we start to have multiple interfaces to the application. Imagine we have our UI and add a SOAP-based API to allow other programs to drive the application.

Figure 3: The Naïve DDD Approach with Multiple Applications

Now the CRUD layer really starts causing trouble. First, notice that because our domain model is defined in an assembly (or package or library), it gets deployed to each separate application that needs it. We have a layered architecture, however, all but the database layer get replicated to multiple applications.

CRUD-Based Models and Meaning

Imagine that a user makes a update to the comments on order 0123 through the UI at the same time that an external program calls GetWorkOrder(“0123”), updates the order description, then passes the updated order into SaveWorkOrder. Now both components are instantiating order 0123, copying the updated property values and saving it back to the database.

We lose the intent of each of those two actors. We are left to sort out any conflicts on simply timing and the intelligence of our ORM tool.

You could avoid the replication in Figure 3 by creating a application server, but you would still have the loss of semantically meaningful information that comes with a CRUD-based object model.

Request/Response: Good for Some Things Not for Others

Our Figure 3 architecture is inherently synchronous request/response. Consider the GetWorkOrder Web service method. Our external system calls that method and gets back the specified work order. This is what we want. The calling code cannot continue until the request comes back.

On the other hand, imagine any method that makes any kind of change to the system (like SaveWorkOrder). We would like to be able to queue these changes and allow the system to process them as compute capacity is available. We also don’t want to have to write code to handle the case where the Web service is temporarily not available.

The Old Way Had Some Things Right

Notice that the bad old traditional system does not exhibit some of these problems:

Figure 4: The Old Smart UI Approach

What the traditional system did right was account for updates being different from queries. Reads used SQL queries or stored procedures and views to get exactly what the screen needs—no more, no less. The write side used SQL inserts and updates to make just the changes necessary to accommodate the user’s actions in the UI. You couldn’t test it and you wouldn’t want to be the one to have to maintain it or enhance it, but it performed well and got the job done.

It did, however, have all the same request/response limitations.

Getting it Right with DDD

Fixing this problem meant an overhaul of the architecture, but the result was well worth the effort. In a nutshell, we redesigned the system to follow the Command Query Responsibility Segregation (CQRS) pattern.

Now all changes to the system are expressed as commands. Anytime the system changes, one or more events are published. Everything is commands in and events out. For reads, we added an Open Data Protocol feed that allows any application to query the data.

Figure 5: CQRS Design

Each of the applications and integration points is now sending commands to a central processing module (which may be dispatching to several machines). The commands express the precise intent of the sender of the command. So, UpdateWorkOrder becomes SetWorkOrderComment or SetWorkOrderDescription. Sending the command allows semantic meaning of the action to follow all the way through. We now log both the commands and the resulting events, creating a detailed audit trail of what happened within our system. This also allows us to better avoid merge conflicts because changes are more targeted with the specific commands.

Asynchronous Messaging versus Request-Response

Another huge win for this design improvement is in how it can scale. Building out the “write” side of the system with NServiceBus means that we can ditch all of the home-grown queuing, retry, and pub/sub logic between components. Now, all of our writes are async, forcing us to learn to deal without getting return codes or statuses from our calls. It took some getting used to, but is liberating in the end.

Our load testing shows that the system is now resilient to peak loads. The message queues swell temporarily, but everything continues to function.

Posted in Architecture, Domain Driven Design | Tagged Architecture, CQRS, CRUD, DDD, WCF | 2 Comments

Domain Driven Design Anti-Patterns

Posted on August 11, 2011 by efarr

[Note: I was going to call this post How to Fail with Domain Driven Design, but that just sounded too pessimistic.]

I’ve been developing business applications on the Microsoft platform since the mid 1990s. The years since then have involved a continual process of learning to be more effective in architectural approach. Along the way, there have been key points where I made leaps forward. Two such examples include object-orientation in the 90s and my first Web service in 2000 (anyone remember SOAP Toolkit2?).

Each of these leaps forward also came with some initial stumbles as I grappled with the promising new approach. Pushing the envelope of productivity means trying new things. New things always create risk. Software is hard. The path of progress in software is never a straight line (unless you are creating commodity level software). However, stumbling when taking on new approaches can be minimized if you learn from other peoples mistakes instead if just your own.

Another big leap forward for me started with my reading of Domain Driven Design in 2005. Since that time, I have been putting DDD principles into practice. I have learned a lot-much of it by my own mistakes. What I offer here are some concrete examples of mistakes I made. I’ll show the mistake I made and how I later fixed it so you can compare and contrast the before and after. Sometimes I learn a pattern best by seeing examples of its anti-patterns. I hope these articles will help you create better software and add to the growing awareness of DDD.

The above list will serve as the table of contents for the series. I’ll enable the links as I write the articles. I’ll probably add an article or two to the list as I go.

Posted in Agile, Architecture, Domain Driven Design | Tagged DDD | 2 Comments

Free Your Data, and UIs are Free

Posted on July 30, 2011 by efarr

This video describes how the Massachusetts Department of Transportation began publishing real-time bus arrival data and within two months there were six compelling applications available that harnessed that data.

Joshua Robins of the Massachusetts Department of Transportation

This shows that if you make your data available, there are legions of developers willing to build applications on various platforms. Massachusetts could have spent months building a lame Web site and then making the data available. By making the data available first, they didn’t need any UI at all.

It’s easy to see how this can work with public data. I believe this same principle works inside the enterprise. If we build our line of business applications to publish their data in open formats, anyone can build the applications that display that data to users in whatever formats users need.

HT: Jack van Hoof

Posted in Architecture, General, Mobile | Tagged Data, Mobile | Leave a comment

Why Software will Always be Hard, Part 2

Posted on July 29, 2011 by efarr

My last post on Why Software will Always be Hard got me thinking about the implications for the software team that wants to create value (i.e., be worth more than they cost). If the bar is constantly being raised on what is considered commodity, then the software team creating value, must continue to get better in order to continue to deliver more next year than they did this year.

In the graph above, I’ve depicted the drive toward higher and higher expectations as the blue line. The green line represents the delivery level of the high-performing team. The space between the two lines represents the value created by that team. In order to continue to deliver value, it must continue to innovate to deliver more for less.

While the commodity line drives steadily upward, the innovation line sometimes plateaus and may even drop prior to new increases. Innovation always brings risk and sometimes they will reduce your output before improving it. This is why even the high-performing team must continue to look for and expect improvements in its processed and technologies.

Posted in General | Tagged software | Leave a comment

Why Software will Always be Hard

Posted on July 26, 2011 by efarr

There is a funny thing about the software business… people expect that building great software should be getting easier. On the surface, that seems like a reasonable expectation. Software is a maturing field. The tools and technology are improving rapidly. The money to be made is attracting bright and capable people. So, why does making great software seem just as hard as it ever was?

There are two market forces working together to ensure that making software will be hard for a long time—at least as long as software will be worth building.

Market Forces

The first force in play is the fact that as soon as technology improves to allow us to build today’s software more efficiently and effectively, the market will raise the expectation of what it will pay for.

The second force in play is that the world is flat. This is especially true in the software business. Unlike automobiles and silly bands, software incurs no shipping costs; so the market of suppliers is truly global.

Result

Those two market forces mean that as soon as a certain level of software functionality becomes easy to provide reliably (think on-time with no bugs), there is such a large supply of providers that no one will pay enough for that software to make a company profitable. Any software company (or department) that wishes to make a profit is going to have to provide something more: better integration to more complex systems; unseen functionality levels; innovative new platforms; more immersive experiences, etc.

That pushing of the envelope means that software will be hard as long as there is any money to be made in it. My dad used to say to me “Eric, if it was easy, anyone could do it.” And if anyone could do it, it hardly ever pays well.

Posted in General | Tagged software | Leave a comment

Teamwork Is an Individual Skill

Posted on July 23, 2011 by efarr

I was intrigued by this interview with Christopher Avery about responsibility on agile teams at InfoQ. Intrigued enough to go download and read his book entitled Teamwork Is an Individual Skill: Getting Your Work Done When Sharing Responsibility.

The premise of the book is that, contrary to conventional wisdom, not only is there an “I” in team, but teams are constituted by nothing more than a bunch of “I”s. If none of those individuals takes responsibility for the team’s success, then no one is taking responsibility.

Whether it be the development team at work, the elder council at my church, or even my family, it seems there is virtually no area of life where my personal success is not dependent on the success of a team. Learning to take more responsibility for the success of those teams seemed like a good idea.

I found that the book could have used a good editor to tighten up some of the writing, and the formatting (at least on the Kindle version) was confusing at times; however, there was enough food for thought in there to make it worth the read.

Here are a few highlights to give you some flavor of the book…

Avery coins the term TeamWisdom, which he defines as follows.

TeamWisdom refers to all the individual mental skills and behaviors that lead to highly responsible and productive relationships at work. The idea is based on my definition of “team”: A team is a group of individuals responding successfully to the opportunity presented by shared responsibility. Thus someone with TeamWisdom takes responsibility for ensuring that the group rises to the occasion, and in the process, makes sure his own work gets done and done well.

Avery makes an important distinction between accountability and responsibility

…accountability can be assigned, but responsibility can only be taken.

Accountability and responsibility are not mutually exclusive. In fact, they are extremely complimentary. It is time for each of us in the workplace to take responsibility for relationships as well as accountability for deliverables, and to engage in the conversations that build productive relationships at work.

Avery makes the astute observation that if each team member acts in his own self-interest, then it is important to learn what motivates the other team members and assure that their interests are aligned with those of the team and with your interests. If your interests cannot be aligned, then you should withdraw from that team.

Once you understand your team members interests, it is in your interest to see that your teammates reach their goals.

The great philosopher/inventor, Buckminster Fuller, taught that the best way for one person to win is not by making others lose, but by making others win too. He taught from the 1940′s until his death that the more people a person helps to win, the more people that person can expect will help her win. Fuller’s teaching was in the forefront of a growing body of literature about the power and humanity of “servant leadership.” Being a servant leader means helping one’s followers become successful, instead of expecting followers to serve one’s personal success.

He then gives the following challenge in the Personal Challenge section that is included in with each TeamWisdom principle.

Do your partners and teammates provide you with access to their thoughts because they experience you as a person who helps them achieve their goals? Listen carefully to your associates to learn what is truly important to them. Check in with yourself to determine your level of commitment to them. If this level of commitment is low, ask yourself why. If it is high, ask yourself how you are willing to help. Then offer that help.

Avery talks about collaboration and includes valuable insights like this:

Most people find it much easier to grant a favor than to ask for one. However, people with TeamWisdom know that asking for a favor actually grants the other person a favor. Asking for a favor communicates to the other person that they are important to us, that we depend on them, and that we are even willing to owe them one. People with TeamWisdom understand that the person who asks for the first favor sets the tone for the collaboration.

Here is an example of Avery’s insight on the effect unmotivated team members:

Is the team leader the most powerful member of your team? Is the most inspired member the most powerful? The smartest member? Nope. None of the above. Like it or not, the most powerful member of your team is the one who cares the least about your team’s task. Sorry, but that’s the truth. The least-committed member of your team is the most powerful because his lack of commitment establishes a low baseline to which other
team members may fall. The success—or mediocrity—of your team likely will be determined by him.

Read the book. Take responsibility for the success of your teams, whether you are the leader or simply a contributor.

Posted in Agile, General | Tagged books, managing, teams | 1 Comment

Shifting from Scrum to Kanban

Posted on July 6, 2011 by efarr

My team had been following (more or less) the Scrum process for two years. The Scrum rhythm of biweekly planning, demo, and retrospective with daily stand-ups served us well most of the time. However, we have since shifted to a process that looks more like Kanban.

The Transition

We had the most friction with the Scrum process after initial releases when our focus turned more toward support and training of our customers. The two-week planning cycle does not accommodate this kind of reactive, responsive work. The dissonance between our stated process and the reality of the work day felt like a process breakdown even though what we were doing was clearly the most important thing for out team. It tended to make everyone a little cranky. Since agile is all about inspect and adapt and choosing a path that gets results, it was time for some adapting.

Our Flavor of Kanban

Unlike Scrum, which has identified creators and a set of doctrines that define orthodoxy, Kanban is more loosely defined. Our idea of Kanban starts with Scrum and and makes a few modifications.

We no longer plan a sprint. We simply pull stories from the backlog as we need more work to do. The VersionOne storyboard gives us a visual of how many items are in each status, and we only pull in a new story if the inventory level are low across the board.

Instead of planning for an entire day before a sprint, we now plan stories whenever the backlog of planned stories is low (less than a few days worth). We still demo the product and hold retrospectives every two weeks.

Since we no longer have sprint boundaries, we needed to make some changes to the way we integrate stories so that we always have a potentially shippable product. We now do all story or defect development on an activity branch in the repository and only merge back to the trunk after it has been thoroughly tested. This creates a bit of new overhead, but we’ve found it worth the effort.

Result

Now we are able to be responsive to new items that come in. The development team feels less jerked around when support issues come in and priorities change. We’d all like to get back to pure product development, but better to face reality as it is than to pretend it is something it is not.

Posted in Agile, General | Tagged Agile, Kanban, scrum, VersionOne | Leave a comment

Peopleware: An Aging Classic

Posted on July 3, 2011 by efarr

Some months ago I was telling a friend at work how everyone in the software industry should read Peopleware by Tom DeMarco and Timothy Lister

I have always said that Peopleware should be required reading for anyone who manages software developers. I’ve also said that if you want to know my philosophy as a manager, just read Peopleware.

However, the first edition was written in 1987 and the second edition, which I had read, was written in 1999. I decided to read it again. I was curious to see how it holds up more than ten years later.

On second read, it is clear that Peopleware was written prior to the agile wave in software development. However, Peopleware was such a prescient work that in 2011 I still found it to be insightful and compelling. Much of what they call for has become more commonplace since 1999.

As DeMarko and Lister railed against public address systems in the office, I felt like I had taken a trip back in time. However, the principle that management too often optimizes the wrong factors in the workplace is as relevant as ever. I’ve seen too many situations where more care was put into color schemes and aesthetics than the mechanics of how the software team would work together (acoustics, workstation configuration, etc.).

The insights and anecdotes on management of software people made the second reading worth the investment. The chapters on building a high performing, jelled team were especially valuable.

Bottom line… I still believe that everyone in software should read Peopleware and that managers must read it.

Posted in Agile, General | Tagged Agile, books, management | Leave a comment

Working with Recruiters

Posted on November 4, 2010 by Eric Farr

Where I work, we needed to add some capacity to our product development team. It was a short term need that could have turned into a long-term need if things go well. We were looking for a contract developer for a three to six month contract. Sounds easy enough, but there were some complicating factors:

In order to help with the short term need, we had to get the person in pretty quickly.
We’re didn’t have a lot of time to devote to recruit someone.
Our current team is very effective.

The first two items are typical of any team needing to increase capacity. Item number three is where it gets interesting.

When I say that the current team is effective, I mean that we get a lot done, we enjoy the work, and we enjoy working together. We believe that the product and the codebase is getting better with each sprint.

I attribute this to core values we have as software developers:

agile: We believe in deferring decisions to the last responsible moment to keep our options open and to be able to adapt.

object oriented/domain driven: We start with the domain concepts and the object model and not the database. We’ve leveraged NHibernate on the server and object serialization on the device to eliminate buggy hand-coded SQL.

software craftsmanship: We view software as a craft to be mastered, not by learning a new technology like WCF or WPF, but by developing sound judgment and applying the basics like those expressed in the SOLID principles.

XP/Scrum: We embrace the engineering practices expressed in eXtreme Programming: collective code ownership, continuous integration, pair programming, and automated testing.

The developer we add to the team must already value these things or must adopt these values to work effectively. A developer arguing to implement a stored procedure on the next feature because it would be more efficient (whether right or wrong) would be counterproductive. Someone opposed to pairing on tasks would reduce the team’s energy and be a distraction.

The first problem is that it is difficult to communicate the values that make our team effective to a recruiter that doesn’t have the context.

Recruiters tend to fall back on what they know… more experience and greater sense of leadership is better. This leads to a paradoxical problem: the more experience a developer has and the more leadership he has shown, the more important it is that he (or she) be a fit for the team. Less experienced developers are 1) more moldable and 2) less responsible for their experience.

The Advantage of Inexperience

Less experienced developers tend to be more open to learning new skills and adopting new values. The good ones recognize that they have a lot to learn—both in techniques and what fundamental values lead to better software.

Less experienced developers also tend to have less control over the the technology and methods they use on the job. They often don’t know what to look for in accepting their first job or two and don’t have the influence to change the direction once they get there. So, I hold less experienced developers less responsible for their lack of experience with object oriented, domain driven technologies and insistence on agile methods.

The Burden of Experience

Developers with many years of experience tend to be the leaders within their teams and organizations. They are the ones who drive the technical decisions and shape the values of their organizations. Because of this, we have a higher expectation that these developers are experienced in the techniques we use and hold the values that we hold. If they don’t, it is mostly likely that their convictions are more settled and they are much less likely to change.

Communicating to Recruiters

Evidently, I was ineffective in communicating this idea to the recruiters who were helping us. Recruiters continued to send people to us touting ten years of SQL Server stored procedure experience or any combination of Microsoft technology TLAs. I don’t think they understood what I meant when I would said that a particular person had too much experience (of the wrong kind).

So, I brainstormed with the team and tried to spell it out with a diagram that looked like this:

I thought maybe the picture would do what the thousand words hadn’t. No such luck… I had to explain the chart to guys on the team; so, there was no way this would help the recruiters. Undaunted, I made one more attempt, this time with a simpler drawing:

I thought maybe this “magic quadrant” style graphic would work better, but consensus was that it still didn’t make it simple enough.

So, I gave up on the conceptual approach, and boiled it down to some questions, based on the years of programming experience the person has.

For developers with 1 to 4 years of experience, only one question:

Would you like to work on a agile team?

I went on to explain that we think a less experienced developer will adapt to our style of working if he/she is interested in learning the way we do things. We also think it will be very valuable and marketable experience.

For developers with 5+ years of experience:

What are the SOLID principles and why are they important?

What are three principles of Domain-Driven Design?

What does CQRS stand for?

What is your favorite ORM and why?

What CI tools have you used?

The candidate should be able to answer at least 3 of the 5 questions without hesitation. Someone without the experience we are looking for will probably not recognize the terms and won’t be able to answer them.

The idea was that an experienced developer who does not embrace the agile/DDD approach would not be familiar enough these terms that are peculiar to that world to answer the questions. The answers themselves are not that important (anyone does not use them won’t be able to interpret those answers, anyway), but familiarity with the terms indicates that we should talk to the person.

Because of some short-term issues, we put off the hire. I am hopeful to give this approach a try soon, though.

Posted in General | Leave a comment

Misconceptions about Team Rooms and Open Floor Plans