Master of the Scrum Dungeon

I was talking to a team mate the other day about why I don’t like Scrum Master as a job title rather than a role, and specifically why I think Scrum Master is not a full time role but something a member of the team does alongside their ‘day job’.

I’ve often been Scrum Master on a team but I’ve always been contributing in one of my other skills at the same time. This is a bit like the concept of ‘multi-class’ characters in Dungeons and Dragons (whoops my ‘geek’ is showing again!) i.e. you can be a Fighter-Wizard and take on the characteristics of both.

Most frequently, I’ve been Developer-Scrum Master in a Scrum team. My primary tasks, the ones I spent most time doing, were development activities. However, my Scrum Master duties trump those and I drop everything to remove impediments, escalate issues, facilitate ceremonies, etc. as these impact the whole team not just my tasks.

I feel this works well because, in a healthy self-managing, self-organising, multi-disciplined team a Scrum Master role is NOT a full time job.

The issue with making Scrum Master a ‘job’ (the only thing you do) is when there are lots of problems, impediments, etc. it takes up your time BUT when the team is functioning smoothly you’ve very little to do most of the time (15 minute stand up per day, 15 minutes updating/reviewing burn down…eh, what now?). SO, this leads to one of two behaviours:

  1. Scrum Master disengages and wanders around talking to team members about irrelevancies as they’re bored, thus interrupting team work,
  2. Scrum Master introduces more ceremony, detailed reports, more meetings, more detailed estimates of future stories that may not even happen, allocating tasks to team members rather than allowing self direction, more detailed plans that need rewriting regularly to cope with change. All this ceremony adds overhead and, although the Scrum Master is busy, it sucks in other team members and prevents them doing ‘their day job’. This is waste and inefficiency of the highest order.

The second reaction is the more common, especially when traditional Project Managers become Scrum Masters as they often feel the need to fill the void with all those comforting spreadsheets and documents they used to use.

The full-time Scrum Master subconsciously wants the team to be disfunctional! If everything is going smoothly they can’t justify their existence. They have nothing to escalate, no blockers to remove, everyone is allocating their own tasks, solving problems together, calling their own meetings spontaneously when required and not holding meetings for the sake of them.

Contrast this to the Developer-Scrum Master or Tester-Scrum Master behaviour. They understand the need to drop their tasks for the good of the team to put on the Scrum Master hat (I’ve actually had one of these in a team before!). However, they are focussed on delivery. They want to get the Scrum Master bit out of the way and get on with the task at hand. Yes, they need to escalate an issue to management, then they need to go back to work. Yes, they need to follow that escalation up but the team won’t let them forget as they are presumably blocked waiting on this to be fixed (peer pressure is a great motivator, unless you’ve a sociopath for a SM!). So they tend to do ‘just enough’ Scrum Mastering, as dictated by the team.

So the moral of this story, for me, is (as I’ve actually shouted across a crowded office at recruiter friends of mine!) “SCRUM MASTER IS NOT A JOB!”

Why learn obscure languages like Clojure?

One of the questions I get asked, quite rightly, by the sales and business development people in our company is why bother with learning Clojure, or even Scala, as none of our clients will use it?

So, I could explain how:

  • Functional programming tends to result in a smaller code base that’s easier to test.
  • Programming with immutable data structures makes reasoning about the impact of multiple threads easier.
  • Concurrent programming support like Akka, atoms, software transactional memory and agents mean modern massively concurrent applications are easier to construct.
  • Clojure encourages developing applications by composing small lightweight libraries rather than large heavyweight frameworks which makes swapping out implementations that don’t support future requirements or adding new functionality much easier.
  • Developing with tools like a REPL (Clojure) shortens the feedback loop to seconds making developing quicker and safer.
  • Functions are composable to make for better reuse at different levels.
  • Meta programming using structures like traits or macros makes developing Domain Specific Languages that express the problem space easier and hence focus on the actual business problem.

However, none of this matters to sales, business development people or most of our clients. They will argue getting large numbers of developers with experience in these languages is hard to impossible.

So why bother?

  • Because these languages generally self select good developers.
  • Because developing in these languages means solving algorithmic problems which develops good problem solving skills.
  • Because developers who have more than one programming language tend to learn other more quickly and are more versatile.
  • Because developing in these languages is fun and that promotes better staff retention.
  • Because techniques beyond OO expand a developers armoury to tackle real world problems.
  • Because developers exposed to smaller, functional code bases tend towards writing cleaner, easier to maintain code and this reduces the overall ‘cost of ownership’.

But if you don’t want any of this, sure don’t bother to learn anything. Go back to writing COBOL, or assembler. Nothing wrong with those languages of course but things move on.


[Update Jan 2016]

Since writing this in Sept 2014 I’ve moved on from the organisation concerned and they’ve adopted Scala. However, the driver to do so was purely client demand, they had clients who wanted to use it and therefore saw a demand that needed fulfilling. I suppose I should be glad the clients have foresight but I feel a consultancy should lead the way.

Monthly Hack Day (Part 1)

I have recently employed two senior developers who are involved in developments run as projects within Opencast Software, either on behalf of clients or in partnership.

Up until this point the company has mainly employed staff for consultancy work i.e. traditional PM’s and BA’s. Although we have had the odd developer (in my case very odd!), they have mainly been consulting on how to develop/architecture/design or have been embedded in client teams.

When choosing these new hires the one thing I made a conscious effort to ignore was the acronyms on their CVs!

I looked at projects they’ve worked on, problems they’ve solved (and how), any contributions to Open Source projects, the range of technologies they’ve learned, and particularly, their attitude to working with new technologies and languages they’ve never been exposed to.

One of them was a .NET developer I know by reputation having worked in the same company but has a yearning to learn new languages and techniques, the other a Java developer of 12 years standing who has converted to Rails over the last 2 years and has some experience of Haskell.

Having established the core of my development practice I want them to ‘practice’ their skills and to stretch themselves.

To that end I’ve established a monthly ‘hack day’ during which they can work on whatever pet project they want. This allows the developers room to ‘grow’ and contributes to the company, either directly through a project or indirectly by learning techniques and languages that we might ‘sell’ in the future.

I am also treating this day as a ‘maker day’ i.e. a day when developers concentrate on making things and don’t attend meetings or other distractions.

The first of these days is this coming Friday (23rd May 2014). I’ve asked the developers to pick their own pet project using whatever technology they like. I’m toying with a Raspberry Pi based autonomous robot to fetch the coffee from the Cafe across the yard, possibly using Clojure or Python for the control/AI.

I will blog on the results of this day in part 2…..

Is everything coming up Roses on Quality Street?

During my travels through my own development projects and coaching others on theirs,  I come across that age old tension between delivery and quality. Someone screaming ‘ship it, ship it’ when the development team are yelling ‘it’s not fit for purpose yet’.

This balancing act is frequently encountered during fixed price contracts and sometime Agile developments when the customer doesn’t really understand the concept of minimum feasible subset. In both cases the problem is one of communication. How does the customer satisfy themselves they are getting faster but good enough and how does the team determine and communicate what is ‘good enough’ and what is not?

You can’t discuss what you can’t understand or see, so making the quality criteria visible to the customer enables the conversation of ‘do you really want to cut this corner?’ to occur.

So we need to measure what quality levels are, and determine what they should be. This provides an objective basis for a discussion around the impact of compromise between functionality vs quality. That’s why I endeavour to measure test quality, coding rules compliance, non functional qualities (performance, security, etc.).

However, I always think metrics are inherently dangerous as they can lead to focusing on only what’s measured to the exclusion of other factors. This leads to ‘sub-optimisation’ of the overall system. Sub-optimisation is not just a performance issue, the ‘system’ may be the features delivered and therefore sub-optimisation on quality criteria may lead, counter-intuitively, to the delivery of something that’s not fulfilling the customers needs. So we also need to measure customer satisfaction with the features delivered. This is more subjective but it’s where Agile processes can help with reviews and retrospectives to provide feedback.

So we need to measure quality and delivery not, however, to the exclusion of other aspects of the system. We also need to ensure the measurements cover the perspective of the full audience of the system, including support people, users of varying degrees to technical literacy etc.

This is a bewildering amount of information to gather and process even in the simplest one page dynamic web application. So one technique is to involve the wider audience.

  • For administrative functions, can you limit the scope to a handful of people that you involve in the testing process so they are trained by default and need little documentation?
  • Can your user representative (product owner/ambassador user) draft guides and documents?
  • Can you get technically literate users to write user documentation during Beta testing?
  • Can infrastructure and support people be involved early in the development to write support documents and automate support processes?

I’m not sure anyone gets the balance of quality versus delivery perfectly correct but recognising that measuring quality as well as delivery and getting other perspectives from the wider audience is a positive step in the right direction down ‘quality street’ and might tell you when you can stop walking!

I Roo the day…

I am currently starting a fairly simple Java CRUD web application for work and at the same time I’ve been playing with developing a simple web application in Clojure.

In order to develop the app for work I wanted something I could use to drive out a prototype quickly but then enhance and augment further. So I took a quick look at Spring Roo.

I found I could generate the basics of a CRUD app in hours, including security, etc. Great…or was it?

I found myself unsettled by a number of things in Roo.

  • Automatically generated integration tests work well in Roo but I found it hard to find good examples of simple extensions to the generated integration tests. For example, how to test a constraint like @DecimalMax on a JPA entity field.
  • I always run my integration tests against a different database in the ‘test’ phase of the build than the database configured for the ‘package’ phase. Usually this is to allow tests to run against an in memory database so they can run fast, clean and without a database server running. This is not something Roo helps you with, and although I’ve done this in Spring and could use the same Spring configuration for the Roo project it’s not obvious or easy how to do this. I just expected it to be something the Roo would support out of the box.
  • I am a TDD addict…Roo does not support TDD at all well. It’s possible but not natural.
  • Although Roo generates nice clean classes with little boilerplate code as it’s still Java it felt unnerving like magic…I didn’t trust it. I kept having to look at the generated AspectJ to check.
  • I always set up my continuous integration server (Jenkins) and maven builds to run code coverage and static code analysis tools (I’m using Sonar this time). As the Java code generated by Roo is minimal, with the aspects adding the bulk of the methods you get a lot of static analysis rules failing (due to rules like, “Unused private field”).
  • You also get low branch coverage messages that don’t equate to any code in source files. I suspect that this is referencing code that has been aspected onto the class.
  • It still felt verbose and heavy weight compared with Clojure code…just look at the number of jars a simple Roo project requires!

Maybe I’m being spoiled by Clojure but it all felt a bit clunky and too much wasn’t apparent or transparent enough. Having said that, Spring Roo cuts out a lot of set up time. Especially for getting a simple CRUD type web application up fast. If, like me, you have to live in the Java world due to skill sets in your organisation/client then it’s a good start.

You can always push the generated Aspect code back into your Java source if the ‘stuff happens by magic’ feeling keeps unnerving you. I did get used to this within a few days though.

So, overall I’m going to use Roo for prototyping and simple CRUD type applications but really I feel that Groovy/Grails, Scala or Clojure give you more, more transparently. You just need to ‘invest’ in learning them.

Political ‘Scrum’

In my travel’s around clients I frequently see organisations trying to introduce agile or lean approaches. Sometimes this is driven by a real understanding of the issues and limitations of their current approach, sometimes it’s ‘new shiny toy syndrome’ (new? – this stuff has it’s roots in late 80’s or earlier). Sometimes it’s ‘keeping up with the Jones (or Howe-Jones!)’

Frequently these agile transformations struggle or fail for a variety of reasons. the most common is ‘politics’. Organisations, regardless of size, are full of people, and some people are inherently out for there own interests.

I frequently see Scrum teams behaving cooperatively and transparently but their message to other stakeholders gets filter and distorted by political animals who are living in their own culture of ‘fear and incompetence’.

How do these teams cope with these political influence? Make sure your metrics are in place, burn down charts, definition of done, test coverage, team velocity, etc. then PUBLISH THEM TO EVERYONE!

If the reality of the teams progress and actions is exposed to every stakeholder for them to see for themselves it’s not possible for the organisational politicians to distort the message.

Be open and transparent.

Seeking Clojure

Over the last three months I’ve been trying to learn Clojure on evenings and weekend. I started this roller coaster ride of joy and frustration  on the recommendation of my close friend and Clojurian @otfrom.

The start of this emotional experience is rooted in a conversation with @otfrom over a glass of wine at the Open Data Institute. The glass of wine may have been a major contributor to my decision to take the plunge into Lisp for the JVM. The conversation went something like this:

  • Me: I’m keen to learn functional programming as I think it has advantages over Object Orientation, especially in the brave new world of ‘big data’ and the increasing emphasis on massively concurrent processing. I’ve started reading a book on Scala because as a Java developer I thought it would be a good place to start.
  • @otfrom: Sounds like a good idea. Scala is a good language if you want to dip your toe in the world of functional programming.
  • Me: Yeah, I thought as it’s like Java I could encourage our other Java developers to start learning and using it.
  • @otfrom: OK. Only thing is, do you think that if they’re struggling to learn they might just abandon the functional style and just start writing Java without semi colons?
  • Me: ..Oh sh*t, you’re right. The whole point of this is to learn functional programming not write OO in a different language.

Continue reading