Know your users’ brains
It is important to know who your users are when you are in the business of building anything. Building things is expensive. Software or otherwise.
My mom would rather use a mac over a gentoo. A systems admin would rather use redhat over a mac on a production machine.
It is also important to say “NO” to people who are not your users. Almost all software at eclipse.org are frameworks, tools and app servers written primarily for developers.
The way software and API is designed for the developer is different from how it’s developed for the non-developer (most of the times). SWTBot was originally and still is primarily a test-automation tool for use by QAs to automate test cases. The APIs are designed to be extremely intuitive, simple and easy to extend by QAs by merely looking at examples and not having to read API documentation.
Some context is in order if you haven’t had the opportunity to work closely with QAs before. Most QAs I’ve met understand customer requirements, understand the software and what the customers want out of it. I like to treat them as end users of the buggy software I build. They may not understand SWT threading, the eclipse ui and platform, OSGi and how TCP packets travel across the corporate firewall over to a load distributor in front of a cloud hosted at a datacenter on the other side of the continent. Try explaining a “org.eclipse.swt.SWTException: Invalid thread access” to a QA and you’ll get interesting looks.
QAs are good at finding bugs in whatever software they are testing. Testing all of the software is time consuming, not to mention testing backward compatibility with older software. Nobody likes clicking around the same navigation path in the UI all the time.
QAs like exploring interesting ways to use the software and ensure that it works consistently as software evolves over time. The traditional favorites have been the ‘heavy weight’ automation tools like HP’s QTP, IBM’s Rational Robot, Borland’s SilkTest and the open source ‘light weight’ tools like selenium, sahi, and a lot others.
Each of these tools has its advantages and disadvantages. One of the most powerful features of each of these toolchains is the ability to author scripts in a dynamic language. The commercial vendors use vbscript or some other variant of the language, and provide some forms of integration with defect tracking tools. The open source tools are primarily light weight, use the languages like javascript, ruby, python and don’t integrate much with anything but your IDE and things like ant or rake.
The end goal is to make SWTBot an extremely good, open source, light weight tool for QAs to automate tests, and be just about useful for developers to write simple test cases with and not something in between that neither of them like.
As part of evolving SWTBot APIs we need to ensure that we stay true to our goal. This sometimes means saying NO to feature requests and even patches from developers if it hurts the non-developers.
+1
Sriram Narayanan
11 Apr 10 at 12:06 pm
[...] This post was mentioned on Twitter by EclipsePlanet and Ketan Padegaonkar, David Green. David Green said: @ketanpkr good post on saying no and staying true to project's goals http://ketan.padegaonkar.name/2010/04/11/know-your-users-brains.html [...]
Tweets that mention Know your users’ brains « Ketan's Musings -- Topsy.com
12 Apr 10 at 3:21 am
+1 You cant satisfy all everytime.
Just another perspective from my side.
Can a software made for both ? For example Selenium, Its excellent for QA as they can really use IDE to record what exactly they want and useful for developer who knows XPath and all to see finer details with Java API etc..?
Sometimes you are saying more “No ” then “yes ” And users of your software are not the you thought. Its really good situation sometimes.
If you recollect delhi and washing machine incident, Suddenly there was huge washing machine sales in Delhi, And market analyst found that they not really used for washing clothe but used for making Lassi..!!
Jigar Shah
12 Apr 10 at 11:57 am
Nice post, Ketan! +1 for catering to testers.
Narayan
12 Apr 10 at 3:35 pm
What about using SWTBot for TDD? I don’t see any conflicts here, but it’s clearly a case where the user is a developer and it would be interesting to know where your “for QA” goal collides with my “for TDD” goal.
In which points does an API “for developers” differ from one for “for QAs” / where do you draw the line here? I mean developers don’t want unintuitive, complex and hard to use API.
Ralf Ebert
12 Apr 10 at 6:04 pm
For being plain wrong it works really great
I write user stories first (“click this, observe that”). I then convert the prose into an executable specification, by writing a SWTBot test. Then I write the code required to fulfill this test.
Of course this is an integration test and while implementing the code to make the test green, I write lots of unit tests for the things under the hood. But I want to have a test that proves that at the end of the day, the complete system works as the user expects. And I want to have this test before I start the work on the system itself.
What’s “plain wrong” about this?
And, separate from that, what’s wrong about using SWTBot to test a view class in separation in a unit test, just as a library to send events to the view?
Ralf Ebert
12 Apr 10 at 11:27 pm
@Ralf,
Using SWTBot to do any TDD is just plain wrong.
If at all developers wish to practice TDD, unit tests and mock tests before writing any production code. This is the only way I know of doing TDD. UI level tests are the last thing a developer should be using for writing tests because they only provide feedback at the UI level when there is something else underneath the hood that has gone wrong.
UI level tests are integration tests at the very least. Regression tests come in later that are more end-to-end and try out every crazy possible thing with the software.
SWTBot is a UI and regression testing framework. Having written it, I myself do not use it or recommend it for unit testing.
I think this calls for another blog post.
Ketan
12 Apr 10 at 8:33 pm
@Ralf,
So “plain wrong” is probably an oversimplification for most developers who forget the unit tests part in the middle.
UI/integration tests are generally are bad at pinpointing reasons for failures apart from telling you that a button did not appear or something did not have the right label on it.
Having UI/integration tests is infinitely more valuable than having no tests at all.
TDD for the most part (for me personally) is about unit tests, short quick feedback cycles that let me know that the change I made does not break existing functionality.
Integration/UI tests don’t always fit that bill of quick feedback cycles and for the most part developers have a tendency to take shortcuts when tests take too long to run.
Ketan
29 Apr 10 at 9:34 pm