Acceptance Criteria vs. Acceptance Tests: Minimize your documentation

By ChrisEdwards | Published: December 2, 2012

To know that a story is done, we create a set of acceptance tests (story tests) that, when they pass, we know the story is complete and functions as expected.

The process we follow for defining our acceptance tests is comprised of two steps.

  1. We first come up with high level acceptance criteria.
  2. Right before, or during development, we use the criteria to define the actual acceptance tests.

You’ll notice that we make a distinction between Acceptance Criteria and the Acceptance Tests. Here are the definitions as I see them:

  • Acceptance Criteria is the minimal documentation that ensures a sufficient implementation of acceptance tests.
  • Acceptance Tests are the detailed specification of the system’s behavior for all meaningful scenarios, used to assert its correctness.

Acceptance Criteria

  • Ensures the team knows when they are done
  • Ensures the team does not forget important edge cases and considerations
  • Produced through collaboration of the developers, testers and product owners (3 amigos)
  • Created prior to development, during planning phase
  • Expressed at a high level (conceptual, not detailed)
  • Expressed in whatever form works best for the team…keep it minimal
  • Considers edge cases and failure scenarios
  • Keep it concise (minimum documentation needed by team…may be more for one team, less for another)

Acceptance Criteria is the minimal amount of documentation you need to specify the expected behavior of the feature and any edge cases that need to be considered. Agile favors working software over comprehensive documentation. This is an expression of that principle. We don’t flesh out every possible example, but provide “just enough” documentation to ensure the correct tests get written. The amount of documentation necessary may vary from team to team. We use a simple bulleted list to capture edge cases and things we need to consider. If your team is distributed, you may need more documentation. Teams that are co-located with testers and product owners (like us) need less.

Acceptance Tests

  • Defines behavior of system
  • Ensures the feature works as expected
  • Code implemented by developers and testers
  • Test definition can include product owners or customers if you are using a DSL that decouples the definition of a test from its implementation code (like with Gherkin or FitNesse)
  • Test definition can happen before development if using a DSL as mentioned above
  • Test implementation occurs during development (ideally in a test-first manner)
  • Tests are usually implemented in code, but if testing manually (hopefully only rarely), the “implementation” can be a list of steps/expectations

The Acceptance Tests are the fully detailed expression of the tests….their implementation. We use SpecFlow (which is the .NET equivalent of Cucumber). It allows us to specify Given-When-Then scenarios in the Gherkin syntax. We work with our product owners to create the actual test definitions in Gherkin, then developers pair with testers to implement them. When a test is not feasible to automate, we still develop the Gherkin and can simply follow it as a list of steps to run the test manually.

I have found it useful to conceptually separate the documentation of the tests (Acceptance Criteria) from the implementation (the actual Acceptance Tests). It helps me to remember not to over-specify the tests. As long as the documentation is enough to ensure sufficient tests will be implemented, I don’t need to add more detail.

Posted in Acceptance Testing, Agile, Testing | Tagged , , | 1 Response

The Effective Developer – Passion

By ChrisEdwards | Published: September 9, 2012

This post begins a series of topics I will be posting for a book I am writing entitled “The Effective Developer”. The topics come from my Padawan to Jedi presentation I have given at the Austin Code Camp for the last two years. There seemed to be a lot of curiosity and around the topics I presented, so I decided to dump that knowledge down on paper to benefit a wider audience. I hope you all find it useful.

I do ask one thing. If you read this, and have any feedback, please post it in the comments. I welcome any constructive criticism you may have. Is there something else I can add? Should I remove something? Please let me know… Thanks.

Passion

The effective developer is passionate about his work. He loves what he does and is therefore driven to do it well.

If you really want to be great at something, you have to love it. How can you be motivated to excel at something you don’t enjoy? A healthy passion provides a wellspring of motivation. It drives you to do your best, and constantly improve your best. Just think of the advantage this gives you.

You see, we tend to do the things we enjoy. We think about them, read about them, and practice them–because we like them; we call them hobbies. A passionate developer’s hobby is his job. Because he loves it, he is driven to do it well. As you can guess, I am passionate about software development. I love reading a good tech book or blog, writing code or writing this book. These activities sharpen my skills, but they don’t feel like work. I enjoy them and they come naturally to me. I love what I do and I do what I love. This is the biggest secret to my success.

Sidebar: A warning about passion!

Please be aware that passion can lead to an unbalanced life. Its easy to spend too much time on something you love doing. Resist that temptation. Don’t neglect the important areas of your life, like family, friends, church, etc. These are essential for happiness, and they are far more important than work. It’s tempting to believe that happiness can come from work alone. However, that kind of happiness is fleeting; it’s a lie; burnout and sadness soon take its place.

I will never forget what a wise friend once told me. He said, “I work to live, I don’t live to work”. This should be true for all of us.

When I interview developers, one of the most important things I look for is passion. I will hire a passionate developer who is lacking technically. I know their passion more than compensates for their deficiencies. Because passion can have such a profound affect on the ability to learn and grow as a developer, I believe it is one of the strongest assets a developer can have.

Posted in The Effective Developer | Tagged | 1 Response

Want to work for an awesome company? BancVue is hiring!

By ChrisEdwards | Published: July 31, 2012

BancVue Logo

We are hiring here at BancVue. We are looking for experienced C# developers and automated testers.

Are you passionate about software development? Are you a continuous learner? Are you excited about projects that challenge and stretch your skills? Do you want to work at a place that takes Agile seriously? Do you live, eat and breathe TDD, BDD, CI, CD, etc? Then this is the place for you, and we want you here.

We have multiple software engineer positions for both intermediate and senior developers. Some of these positions are for technical team leads. We are also looking for automation engineers to design automated tests and test infrastructure. All positions are technical coding positions.

To get an idea what type of development we are doing, here is a list of some technologies we are using:

  • C#
  • MongoDB
  • SQL Server
  • RabbitMQ
  • Selenium WebDriver (and Grid)
  • SpecFlow
  • TeamCity
  • Octopus Deploy

This is a great place to work, and the culture is the best I have seen anywhere. If you are interested in joining us here at BancVue, or just want to know more about the opportunities, contact me via email at ChrisEdwards357 (at) gmail.com.

You can see a detailed listing of the opening job positions we have available at our Careers Page.

Posted in Uncategorized | Leave a comment

My Thoughts After Switching from iPhone to Android

By ChrisEdwards | Published: January 16, 2012

Its been a month since I made the switch from iPhone to Android and I thought others might benefit from my experience. What do I miss about my iPhone? What do I love about my Android? How do they compare?

First some background
I have been a loyal iPhone user for the last three years (3GS). I also love Apple products and use a MacBook for my personal laptop even though I am primarily a Windows developer. I made the switch to Android because I wanted a bigger screen and wanted to see what an open platform had to offer. I am active in open source software development, so Android seemed to fit my personality. All that being said, I don’t feel that I am biased toward one platform or the other.

The Android phone I switched to is the Samsung Galaxy Nexus running Ice Cream Sandwich (Android 4.0).

What do I miss about my iPhone?
To be honest, if I had written this article only two weeks in to using my new Android, I would have said I wanted to go back to the iPhone. However, at this point (one month in), I can confidently say I am sold on Android. It took me a while to get comfortable (and overcome my co-dependence on iTunes), but I an over the hump and there’s no turning back. However there are a few things I do miss about my iPhone.

Touch free voice dialing
When voice dialing on the iPhone I never needed to look at, or touch the phone screen. Even when I had multiple numbers for the contact I was calling, it simply asked me which one I wanted to call and I could respond with my voice and it would call the correct number. It was all totally hands-free.

The Android is a completely different experience. I have to turn on the screen to find the voice button and push it. Then I can say “call <name>”. Then, I have to look at the screen again to confirm it found the right contact and select the number to call. I have to do this even if there is only one number for that contact. It’s very clumsy, and dangerous when driving. Luckily, I seldom make calls while driving.

Stability
I have rebooted my Android phone more times in the last month than I did my iPhone in the entire three years I owned it. Granted, I got the Galaxy Nexus phone the day it was released, and it is the first phone to run Ice Cream Sandwich (the latest version of android OS–a major version release), so it makes sense that stability would be somewhat of an issue. However, even when iOS 5 was first released (also a major version release), I didn’t experience this level of difficulty. The stability has been getting better lately. Its really not that big of a problem…more of an annoyance, but I am curious to see what the future holds.

App selection
Android’s app selection is quite good. There are just a few apps I had for the iPhone that are missing in the Android market. The iPhone appears to get development priority in the mobile app space. This is odd given that Android has a considerably larger market share. Apparently the cultures are quite different. iPhone users are far more willing to spend money in the app store than Android users. So currently, the iPhone presents a better ROI, so it gets implemented first. Hopefully this will change as the Android platform continues to mature. There has been a dramatic increase in both the quantity and quality of Android apps in the market in the last year–a trend I expect to continue.

What do I love about Android?
Diversity of hardware options
There are only a handful of iPhone models, which limits your options as a consumer. However, this limitation it comes with the added benefit of increased stability. Since there is a very limited set of hardware to support, Apple can guarantee that apps will run smoothly on all the current iPhone models.

Android, on the other hand, is an open platform that runs on a very diverse range of hardware. There are many models made by many manufacturers. This presents a diverse number of options to the consumer, but at the cost of stability. It simply isn’t feasible to test every app against every known hardware configuration.

But because the platform is open, and any manufacturer may enter the market (unlike Apple, which is the sole designer and manufacturer), there are more designers and manufacturers driving innovation in the hardware. As a consumer, this means you have more options to choose from: large phones, small phones, phones with 3D screens, phones with external memory cards, phones with HDMI outputs, etc. etc.

Customizable user experience
I really like the ability to customize and tweak my phone to work exactly how I want it to. For example, I am using a custom keyboard called SlideIT. Rather than typing each letter one at a time, I just slide over the letters to form words. It is much faster than conventional typing. I just showed it to my wife and she loved it and asked me to install it on her phone. “Sorry sweetie, your iPhone doesn’t allow custom keyboards.” I had to tell her.

The customizable nature of Android goes far beyond just keyboards. Its baked in to the OS–part of it’s very philosophy. Almost anything can be customized or tweaked. And if it can’t, you can always open the source code and change it yourself.

Better app integration
Android makes app integration much easier. The iPhone requires the developers of the app to explicitly integrate with one another. For instance, if you want to use files from your DropBox or other third party app on the iPhone, you must find an app where the developer explicitly wrote in support for DropBox. If they didn’t, the app wont integrate. On Android, any app can access your DropBox because it can just look at its files via the file system. The iPhone does not expose its file system to the user in the name of simplicity.

As another example, if you are want to integrate with your favorite Twitter app you can simply install it and any app that supports sharing will be able to share over your Twitter app natively. Integration on Android is an OS-supported concern, its not left up to the app developers to decide what apps to integrate with. Its based on what apps you have installed and what actions they can support. This allows for much greater flexibility and lets apps integrate with one another easily by decoupling them inherently in the OS.

I prefer Java over ObjectiveC
Yes, yes, I realize my aversion to ObjectiveC is somewhat subjective. To me, ObjectiveC feels like an antiquated language. It just doesn’t feel natural. It has a much steeper learning curve for the average developer than Java. I imagine this is a non-issue once you learn the language. But it presents a high cost of entry that many are unwilling to pay. I admit that being a C# developer does give me a certain bias towards Java since they are so similar.

Freedom
Apple is very restrictive in what apps they allow in the app store. It is commonplace for an app to be pulled from the app store if it allows you to do something Apple thinks you shouldn’t be able to do. This type of thing never happens in the Android market.

While I consider Apple’s tight-fisted control over their app store as a disadvantage, I must admit that it does result in higher quality apps on the app store. They have much higher standards for an app to be accepted into the app store, and reject apps that don’t meet them.

In conclusion
I think it all boils down to the type if user you are. If you are a power user who likes to tweak and hack, then android is right for you. If you value stability and simplicity and don’t mind being restricted in what you are allowed to do, then you will love the iPhone. To each his own.

Posted in Android, iPhone, Mobile | Tagged , , | 1 Response

Error loading .NET 4.0 assembly in TeamCity using MSpec

By ChrisEdwards | Published: December 7, 2011

When we migrated our tests from NUnit to MSpec (Machine.Specifications), we encountered the following error:

Could not load file or assembly 'file:///C:\BuildAgent\work\c7e220bd4bfc29dc\bin\dependencies\SomeProject.dll' or one of its dependencies. This assembly is built by a runtime newer than the currently loaded runtime and cannot be loaded.

After some Googling, I realized that the default MSpec runner (MSpec.exe) was not built using .NET 4.0. Because our assembly (SomeProject.dll in the error above) was a .NET 4.0 assembly, we were getting this error.

The fix was to use the mspec-clr4.exe executable as the test runner instead. This exe is shipped with newer versions of MSpec (we're on Machine.Specifications version 0.5). The clr4 version of the runner is built on .NET 4.0 and works perfectly.

To do this in TeamCity, simply add the full path to the exe in the "Path to MSpec.exe" field in TeamCity's MSpec build step. Even though the field label says it is expecting MSpec.exe, if you give it a path to mspec-clr4.exe, it will use it.

Posted in .NET, Continuous Integration, MSpec, TeamCity | Tagged , , , | Leave a comment