SEETest 2008 was a 2-day conference, the first testing conference focused on the South Eastern Europe testing community.

It was held in lovely Sofia, the capital of Bulgaria, on the 2nd and 3rd of July, in the luxurious Kempinski hotel (the Ciragan Palace Kempinski in Istanbul is the one that John F. Kennedy Jr. stayed at on his honeymoon, before taking a dive into the sea with his plane, so if you aren’t familiar with the Kempinski’s, you get the picture).

The first day consisted of 6 tutorials (3 morning and 3 afternoon), and the second day consisting of 21 (!) presentations and 2 keynotes speeches, one from BJ Rollins from Microsoft, and the other from Ina Schieferdecker from Fraunhofer FOKUS.

It was, as in SIGiST Israel 2008, a hard decision as to which tutorials to go to.

In the morning, the choice was between Vipul Kocher, co-founder of Puretesting and President of the Indian Testing Board, who spoke alking about ”Q-Patterns”; Patricia McQuaid, President of the ASTQB, and lecturer at California Polytechnic State University, talking about “Test Design: Specifying Test Cases”, and Yaron Tsubery and Effi Bruker, from Comverse, Israel, talking about “Performance & Load”.

The afternoon was a bit easier, since I had already heard Alon Linetzky, from Sela and Best-Testing, Israel, giving a tutorial on “TPI”, and I had already heard Rex Black, of RBCS-US, and President of the ISTQB, talking about “Identify and mitigate risks through testing”. That left me with Geoff Thompson from Experimentus, who spoke about “The Essentials of Test Management”. I will admit to having been in the business a bit too long to get the most out of Geoff’s talk, but I did learn something new, and I also gave Geoff a bit of a hard time with my questions (which he did thank me for, as most of the attendees didn’t really talk much)!

More posts about SEETest 2008 coming soon …

If you enjoyed this post, please consider subscribing to my RSS Feed, to keep up with all future updates

Bookmark

We had a “conference budget” which allowed a small percentage of employees to attend international conferences every year. I was a nice boss, and always tried to send my staff to the international ones, like STAREast and STARWest, EuroSTAR, and the like, while I went to the local ones.

One day I woke up!

Last June, I attended the Better Software Conference in Las Vegas, and I kicked myself all the way home. What an idiot I was. Attending good international conferences was something I should always have allowed myself to do, at least sometimes. But then, there was that dratted conference budget …

So, one of the nice things about being independent, is that I only have to ask 2 people permission to go to International conferences:

1. My husband, David, who always says ”of course, and while you’re about it, why don’t you add a few days and do some sightseeing and shopping” (David is THE world’s best shopper, bar none. I had to drag him away from the shops in Venice, so that we could go and see some historical sights !!)
2. My daughter’s nanny …

Which is why it may look to you like I’m spending all my time going to conferences.

I’m not, it’s just that 2 conferences in a row, one week (SEETest 2008 in Sofia, Bulgaria) after the other (SIGiST Israel 2008 in Tel Aviv), both with their international speakers, was too much not to take advantage of. [It also didn't hurt that the SIGiST conference was free, since I was a speaker, and my plane ticket and hotel to Sofia were being paid for, since I was also attending an ISTQB workshop ].

So, if you have the chance, take it, and attend as many GOOD International Conferences as you can.

Coming next, some posts about SEETest 2008.

If you enjoyed this post, please consider subscribing to my RSS Feed, to keep up with all future updates

Bookmark

Sorry about the length of this post – even by MY standards it’s a bit long. But if you’re into Quality in particular, and Requirements Engineering in particular, I think you’ll find it worthwhile.

So, What was the Exercise?

The goal was to find as many ambiguities as possible in the exercise below (the “Postal Regulations” text is courtesy of Gary Mogyorodi), and to re-write it, fixing the ambiguities:

You are a computer science student, and twice a week you work in the local Post Office. Two weeks ago the post ffice received new regulations, and all the employees in your branch start implementing the regulations. The manager of your branch quickly realized that the employees were making mistakes with the postal charges. Since you are the “technologist” of the branch, the manager asked you to explain to all the employees what the new regulations mean.

The Postal Regulation (Draft)

The following postal regulation determines postal surcharges. This regulation applies only to parcels, not other types of postal items, and only to parcels sent to South America. For parcels which people mail to South America between December 1 and December 24 of each year, the teller will apply these surcharges in addition to the standard postage

Country and Weight Surcharge

Argentina                    All weights                                35 NIS
Brazil                           Weight > 33 pounds                 70 NIS
Brazil                           Weight < 33 pounds                 60 NIS

The Results

There were a number of ambiguities that almost everyone wrote, and there were a few that only some people thought of. Below is the complete list, with the unusual answers in bold. If the person who gave the bold answers wrote his/her name, they are mentioned by name . So, if you wrote one of those answers but forgot to tell me who you are, and you want your name added, send me a comment, and I will add you.

See how many of them you managed to find

And if you want to add more, that’s great – send me a comment, and I’ll add it to the list.

1. Who pays for the surcharge – the sender or receiver?
2. What is the surcharge for exactly 33 pounds?
3. Does “between December 1 and December 24″ include those dates or not?
4. What is the “standard” postage?
5. What about the other countries in South America, is there a surcharge for them?
6. Does South America include Central America?
7. What about the rest of the year, is there any surcharge?
8. What happens if you mail something from home, i.e. there is no teller?
9. The regulations don’t say that they apply to parcels sent from Israel to SA. Maybe they are from a different country?
10. “For parcels which people mail”, what about companies? Do they have a surcharge? (from Shaul Weinstein of Intel, Janna Fridman of Intel, and Michael Stahl of Intel and Joshua Weiss of Intel Valor
11. What about postal items that are not parcels?
12. What is considered a parcel?
13. What about the size or volume of parcels (is sized used to determine if something is a parcel)?
14. What about special delivery items, do they have a surcharge?
15. What about insurance, packaging, etc. is that included in the surcharge? Do they have a surcharge of their own? (this was from Efi, who wrote “what about insurance, packaging, etc.”. I added 2 possibilities of what he could have meant. Are there others?)
16. Do the prices include VAT?
17. “Each year” – who will remember it after the 1st year? (Leonid Z of Marvell)
18. It’s confusing that Brazil is written twice in the list (Leonid Z of Marvell)
19. Is there a difference between “send” and “mail”? If yes, explain, if no, use just one term (Gadi Shiloah of BMC, Michael Stahl and Joshua Weiss of Intel)
20. Why is the surcharge required? (Gadi Shiloah of BMC) I liked this , because Gadi used the “validation tool” that I explained in the first half of the presentation!
21. Why does Argentina have a lower surcharge, and why is their a difference in weights for Brazil but not for Argentina? (Erez521 Erez Landwer of Interwise, outsourced from Sela) I liked this too :-) , because here also, Erez used the “validation tool” that I explained in the first half of the presentation!
22. “The teller will apply …”, the use of the word “will” is ambiguous (Debi adds e.g. is this automatic, or does the teller have to add this by hand). I don’t know who wrote this, but it is really good. Another example of someone using something mentioned earlier in the presentation, but in this case, part of the ambiguity review explanation.
23. It is not clear if the surcharge is per pound or for the whole pacakge. (Janna Fridman of Intel)
24. The regulations are marked “Draft”, so it’s not clear whether they are final and when the regulations start being valid (Janna Fridman of Intel, Barak Benjo of NDS, and one other person who didn’t leave their name)
25. The regulations use ”pounds” which is not the measure used in Israel. There should be a conversion table (Janna Fridman of Intel)
26. There is no maximum size or weight mentioned
27. When does the surcharge start, from the moment the parcel is handed in? And what about companies that pay at the end of the month by credit? (Michael Stahl of Intel and Joshua Weiss of Intel Valor)
28. “The teller” – does the surcharge apply if someone else, e.g. the branch manager, handles the parcel? (Michael Stahl of Intel and Joshua Weiss of Intel Valor)
29. “The teller will apply the surcharges” – is this a separate payment, or part of the same payment? (Michael Stahl of Intel and Joshua Weiss of Intel Valor, and one other person who didn’t leave their name)

The Winners

Firstly, you are all winners, since you all now have more tools to use when writing or reviewing requirements. The companies you work for are also winners, since they will get a return on their investment in sending you to this conference – you will find more bugs in the early stages of your projects, and will save them a lot of money.

But there are 3 people who had the most interesting list of unusual ambiguities.

They are …..  Michael Stahl and Janna Fridman, from Intel, and Joshua Weiss from Valor!

And there is an honourable mention to 2 people, who used lessons from the presentation in the exercise itself: Gaid Shiloah of BMC, and Erez521 of ??  (Erez – if you send me your name and company, I will update the post). Erez Landwer of Interwise, outsourced from Sela.

Well done, everyone, and thank you for taking part. I hope you found it useful.

And of course, many many thanks to all of you for voting my presentation as one of the top 3 of the day

If you enjoyed this post, please consider subscribing to my RSS Feed, to keep up with all future updates

Bookmark

Danko used techniques from the world of Agile to explain how testing and testers are involved in Scrum, and his use of the audience was highly effective. Among other things, he taught us how to do “planning poker” - a really cool method of estimating, that gets all the stakeholders in the project actually talking to each other (who’ld have thought it was possible!!) .

Danko explained the dynamics of the Sprint Life Cycle, and showed us a few of the other tools and techniques used in Sprint, like “Planning - just before committing”, Visibility Charts – Task Charts and Burn Down charts.

There’s no way I can explain the dynamics of this session in a few blog lines. Suffice to say that it was highly entertaining as well as very educational.

And if you are asking what the roles and responsibilities of QA are in the world of Scrum?

Now wouldn’t it be nice if we could all say that we work in a fully integrated Scrum organization, where “testers and coders are fully Integrated“??

If you enjoyed this post, please consider subscribing to my RSS Feed, to keep up with all future updates

Bookmark

Well, unfortunately, some of the keynotes should really take a lesson from the presentations. True, in many cases, keynotes are really good, especially if they are technical keynotes from domain specialists.

However, my gripe is when the keynotes are given by the sponsors. If they do a good job, fine, as that’s a win-win situation.

But guys, really …. we KNOW that you are plugging your product, even if you say you aren’t. At least give the audience the benefit of the doubt that we are intelligent beings: what you have to say has to be INTERESTING!! Don’t send us to sleep …

At SIGiST 2008 I did enjoy Vipul (but then he wasn’t a sponsor), and I also enjoyed and learnt something from the keynote on “Microsoft’s ALM solution – Implementing Testing  & QA Measurements“, given by Shai Raiten and Hila Lahav Rice.

With regards to the other 2 keynotes, ‘Nuff said 

If you enjoyed this post, please consider subscribing to my RSS Feed, to keep up with all future updates

Bookmark

Requirements Review – it sounds dreadfully boring. Months ago, when I was asked to speak, I thought “what can I do to make it a bit more interesting?”. I came up with the idea of involving the audience, making it interactive. How so interactive? Well, lots of questions.

And then, an exercise at the end, for which the answers can be found – guess where? On this blog !!!! During the 1st week of July, the best answers will be up on the blog, with the names of their authors. (I must admit that the idea of putting the answers on the blog was something I thought of the day before the presentation, but I think it was a cool way to get people to come and see what I’m writing about, and hopefully turn it into a great forum for discussions ).

I also wanted to have the audience actually come away with something that they could use straight away.

So I started with a story about one of my favourite bugs – actually it was 5 bugs that I found by just clicking on a button that wasn’t supposed to be there. The button said “clear database”, and believe me, it did a lot more than clear!
What were the bugs I found?

• “clear database” button was not supposed to be there (not in the requirements)
• there were no mechanisms to check whether the button was clicked on purpose or by mistake
• the “cancel” button, that was supposed to stop the “clear” didn’t work
• the whole database was erased, both data and tables
• there was no backup of the database ….

The system analyst had to sit for 3 days and recreate the database by hand.  OUCH! But at least it was the day after the demo to the CEO, and not the day before.

Why am I bothering you with this story? Well, I should have felt great, I just found this really serious bug on the test system, before someone completely wiped out a live system with 10 million records on it.

But I didn’t.

I was frustrated that I hadn’t been working on the project from day 1, that I hadn’t found this bug in the requirements documents, like the fact that there was no requirement to backup the database once a day.

I didn’t want the audience (both at the presentation, and those of you who are reading this) to feel the same sense of frustration, so my presentation was actually an interactive lesson about 2 tools that can help find bugs in requirements … pretty easily.

The first tool is used for validation of requirements. What is validation? Validation answers the question “Are we building the right thing?”. In this case, should we give our customer (the baby) a bottle or a new nappy/diaper? By validating his requirements, we know that he wants a bottle, so that is what we will give him.

The second tool is used for verification of requirements. What is verification? Verification answers the question “Are we building the thing right?” In this case, which bottle are we going to build for our customer, the baby? Using verification, we can clarify whether we are building the thing (the bottle) right?

The Validation Tool: “Problem Space” vs. “Solution Space”

Normally, one would expect customer requirements to be written by customers. But in the world of hi-tech, they very often are written by “techies” – system architects, project managers and the like. These “pseudo-customers” already know how they can build a system that answers the customer’s needs, and so they can all too easily write the requirements with a particular solution in mind. This solution may not be the best answer to the real need – the real requirement. It should be no surprise, then, if the final product does not satisfy the customer.

How should you, as a test engineer or test manager, go about finding these types of problems?

1. You need to look for solutions masquerading as requirements.
2. Then you need to find out what the REAL requirement is.

How do you do that? You use a technique called “the 5 why’s” (a technique originally developed by Sakichi Toyoda, and often used for “root cause analysis” of defects; but just as useful for finding the “root requirement”).

The Verification Tool: “Ambiguity Review”

Many test professionals are aware of the list of words and phrases that make requirements ambiguous. Things like “user-friendly”, “etc.”, “and/or”, “efficient”, “state-of-the-art”. In fact there are any number of ambiguity lists.

The idea behind the ambiguity review is to find:

1. All words and expressions that are ambiguous
2. Anything else that can be interpreted in more than one way, for example: “If this happens, then do that“. If there is no requirement regarding what to do if “this” does NOT happen, then we are leaving it up to the developer to decide what to develop (not a very desirable situation!!!).
3. The thing we really don’t like doing : GO AND BUG SOMEONE with Domain knowledge, who can give you an answer as to what the ambiguous or unclear requirement is REALLY meant to say.
4. Go back and fix the requirements, getting rid of the ambiguities

Why bother with all this work?

Because you probably don’t want to end up with this ….

If you enjoyed this post, please consider subscribing to my RSS Feed, to keep up with all future updates

Bookmark

Mieke Gevers, from aQis in Belgium, is our last speaker.

Mieke lives and breathes Testing, and in fact she told me that she is “passionate about testing – it is like blood running into her veins”. In addition to her 20 years’ experience as a software developer, analyst, project manager and ultimately QA Manager for several companies in Europe, including Borland and Segue, she is a regular conference speaker throughout Europe, the United States, Asia and Australia ( I really do envy her!). Mieke is also EuroSTAR’s country coordinator for Belgium, and was a team member of the EuroSTAR 2007 committee.

Mieke has a style of presenting that includes walking around the room (which is small enough to not make it weird), and asking the audience questions. Interestingly enough, Danny (Danko) Kovatch (look out for my blog about him soon) uses the same technique. Personally, I find it very effective. Just haven’t really had the guts to do it myself yet – I’m still connected by umbilical cord to the laptop …

Meike’s presentation, or maybe I should call it a discussion, is about Advanced Testing, but in order to talk about that, she has to take us back in time, through older methods, mentioning the V-model, the O-model (looks to me like iterative) and the O-model with a Time Axis (looks to me like Agile).

Mieke also takes us back to quickly mention Testing Techniques, past (like unit testing, static analysis, functional testing) and present (like Heuristic testing, exploratory testing, agile testing, scrum testing, RBT – requirements based testing).

And she does the same thing with requirements definition (past: word, excel, printed, home-made tools; present: stuff from the past, plus UML, Storyboarding, and Automatic Test-Case generation based on requirements); and testing tools.

Organizational structure gets the same treatment, where Mieke shows an organization from the past, and from the present – component based.

Then Mieke starts giving us food for thought. She mentioned three methods by which one might achieve Advanced Testing:

1. Component based development (CBD) and component based testing (CBT). Coming from an organization that did just that, I’m not convinced, but I must admit that I didn’t think of it at the time…
2. Models and Model-based Testing
3. Techniques, like Exporatory Testing combined with tools
4. Testing on the Move … a question with no answer, or that anyone can answer

What do you think? Do any of you come from a CBD/CBT organization, but feel like testing is still in dark ages? I’d be interested to hear, and we could get a great discussion going on this.

Then Mieke goes off in the direction of Advanced Testing for Managers, which has to save time, reduce risk, reduce costs, and save money! Her focus is mainly on Tools – which of the following will be part of Advanced Testing for Managers? Who knows, it could include any or all of the following, and more:

• automatically define tests?
• uniform test tools? uniform management of test data? Opensource?
• totally integrated test facilities, parsing commands and sharing data?
• generating data together with risk and impact analysis?
• surely supporting existing and new test strategies
• learning modes and maintenance strategies, accelerating testing and assuring high test ROI?
• patterns for testing, and not just design?

In the end advanced testing is going to change the process, and lead to a new project management approach, organization, test techniques, infrastructure and architecture tools, and test approach.

The only thing that might not change is the importance of aligning testing with the customer’s use.

Do you have a vision of what Advanced Testing might be? We heard Mieke, and we heard Vipul at the keynote. Do they have the same vision, or is it different? What do you think about where the Testing Profession is going to be in 10 or 20 years? Should this be the topic of a Working Group?

Use this forum, or any other, and maybe we can make a difference to the profession

Bookmark

Accompanied by some wonderful photographs from his ATP (acceptance test plan) travels around the world, to India, Taiwan and Italy, Avi explained why getting formal acceptance from the customer is probably the most important milestone in the life cycle of a software project – duuuhhh, it’s the stage after which you get paid !!

Avi started with an example of how an inexperienced test engineer managed to insult a Chinese customer, so that Avi had to fly out to China, just after returning from California…

According to Avi, there are 3 types of customer, and any of them can turn up in any part of the world:

1. The customer who understands Quality and QA, and who is very organized, and has his own SQA (Software Quality Assurance) staff or test team. When you have a customer like this, very often all you need to do is to accompany them, and answer questions – the actual work of the acceptance test is done by the customer’s quality staff. It’s generally a win/win situation. You are there to handle small issues, if they arise. You are there to channel any problems they find directly back to your company, so that they can start fixing them. And you are there so that your CEO knows about any issues before your customer’s CEO does… so that he can be ready with answers if necessary.
2. The second type of customer doesn’t understand anything about QA. You need to take charge, and run the tests yourself. Make sure that your customer cannot easy get hold of your mouse – don’t let him win the battle - or he may do something to crash the system! On the other hand, you do need to exercise the system as your customer would do, and attempt to find the types of problems that your customer would find. Just make sure that YOU find them.
3. The final type of customer is a novice. You have to use the opportunity of running the ATP to train him in what the system does. Just be careful not to assume that he’s an idiot just because he doesn’t understand the system. Don’t con him in any way.

Avi explained that in the Far East (especially), the customer can appear as a Fire-breathing Dragon. However, if you take a real interest in his country and culture, he doesn’t have to be.

In any case, this is NOT a task for amateurs: one needs specific skills and experience in various domains, including a deep knowledge of the software life cycle and quality, a customer-oriented approach, a broad cultural horizon for offshore missions, and high flexibility toward the changing conditions during the process.

This activity is commonly (and somewhat wrongly) known as “performing ATP”. But, more than a technical challenge, performing successful ATP is a complicated art.

Bookmark

2nd Session: Katy Podolner-Rotman, Aternity, on “Surviving “crunch time”: Dealing with constant change, high workload and communication issues“.

Katy started off by giving us her list of “unwritten rules” about how to deal with these issues at crunch time:

DO:

• Plan for tired people
• Make time for rest/fun
• Appreciate people’s work and effort
• Improve “while running”
• Communicate – within the team and between teams
• Laugh

DONTs:

• Over schedule
• Multitask
• Overwork yourself and others
• Let small issues develop into huge ones

I like these rules.

Then Katy entertained us with 5 real-life “fictionalized” case studies (none of which anyone in the room could possibly have actually experience!), in which these rules were broken. She told us what the result was in each case, and asked us what could have done better.

Here’s one of Katy’s examples:

Story: “… but everything was going so well …”

• high profile project
• a new, anxious-to-succeed, project manager
• in order to meet an impossible deadline, he makes everyone work impossible hours, from day 1
• project ended, almost successfully, and almost on schedule

BUT …        80% of the project’s development and test team quit afterwards

PROBLEMS:    Burnout : BAD, which leads to Turnover: BAD

How to avoid burnout:

• eliminate multitasking, and work at a sustainable pace
• keep away from ongoing “crisis mode” (usually a result of bad management)
• downtime after a “crunch” is a must
• at times, a “lessons learned” session after a “crunch” might prevent the next one
• don’t forbid cynicism – laughing takes away some of the stress – and use the level of the team’s cynicism as a “burnout barometer”

You get the idea. There was a lot of discussion about this and the other examples, throughout the presentation.

Next presentation, next post – see you there

Bookmark

Yaron started off the morning with a presentation that can be summed up in three words: “Communication, Communication, Communication“.

Yaron started off with 4 messages that he then elaborated upon during the presentation:

After that, he moved on to discuss the need for expectation management between the test manager and all his/her colleagues, and that the test manager has to be proactive in explaining what the test team does to each stakeholder, and getting their feedback.

We were told to view each colleague as a customer, not a new concept, but one that never really took hold in the hi-tech field.

Yaron went on to discuss how easy it is to break trust, as opposed to the effort it takes to build trust.

Transactional Analysis

Next we learnt a bit about Transactional Analysis, a psychoanalytic theory of psychology developed by psychiatrist Eric Berne. Berne postulated three ego states, the Parent, Adult and Child states. Berne considered how individuals interact with one another, and how the ego states affected each set of transactions. This model helps understanding how people function.

Yaron explained Berne’s Interpersonal Conversation Model, where the

Parent (“exteropsychic”) is a state in which people behave, feel and think in response to an unconscious mimicking of how their parents, or other parental figures, acted.

Adult (“neopsychic”) is a state in which people behave, feel and think in response to what is going on in the “here and now,” using all of their resources as an adult human being with many years of life experience to guide them. While a person is in the Adult ego state, he/she is directed towards an objective appraisal of reality.

Child (“archeopsychic”) is a state in which people revert to behaving, feeling and thinking close to how they did in childhood.

There are 2 types of conversational transaction:

• Reciprocal or Complimentary transactions:  A simple, reciprocal transaction occurs when both partners are addressing the ego-state that the other is in
• Crossed transactions:  Communication failures are typically caused by a “crossed transaction”, where partners address ego-states other than that which their partner is in

At the end, Yaron gave some tips for prevention, correction, maintenance and some advice for the road, and he closed with his 5th message,

“Remember: It is not enough being a professional to earn your colleagues’ trust, you need also to communicate it in order to earn it!”

More on track day I after I get some sleep …

Bookmark