Thursday, June 19, 2008

test

A DAY IN THE LIFE OF A TESTER…
ABSTRACT:
In this paper I would like to share the experiences of my colleagues and mine, who
are a part of the product-testing group while interacting with other involved parties
especially development teams in Baan Information Systems. The paper focuses on
how to overcome the embarrassing situations between developers and testers with
Ease and elan. It stresses the need for mutual respect of skills between development
teams and test teams which are equally important and unique to achieve company’s
mission. Some of the common mistakes that are done by development teams
towards test teams vice versa and the possible solutions are also explained. It states
unique skills that a Testing profession calls for which are very much different from
development skills.
If you belong to a company where you also have a separate independent test team
to test the products, the following situations may not be new to you.
What role should a Tester play? Optimistic or pessimistic.
While I was involved in the technical installation and translation verification testing
for translated products of Baan, interaction with release managers, Baan translation
group and configuration & assembly department is very frequent. These tests are
performed at the end of the system life cycle before shipping the product to the
customers. Release managers responsibility is to get the product out of doorsteps as
soon as possible with good enough quality to get bucks for us. Test report from
Product testing group helps them in taking better decision of giving GO/ No GO to
the release. Baan translation group, who is responsible for translating our products
into different languages, playing a role of software engineers in our standard
development group. Configuration and development group is responsible for
processing the application dumps and putting different pieces of the software
(installation scripts, tool components, and different products…) which is received
from the different product development units on the physical medium. While
participating in the discussions with them I was always confused whether to play a
role of optimistic or pessimistic. Everybody else except product testing group wants
to get the product out of doors as soon as possible. If we don’t release the products
we will not get money, releasing with less quality also is not good in the long run
(support department lines would soon get busy). I had come to the decision of
playing an optimistic role in front Of release managers, talking like “ I wish I would
give GO to this product”. As a tester I get paid for proving that “product does not
meet specified requirements”. I was always pessimistic at the back of my mind
thinking, “iam going to find critical bugs which has been escaped through the eyes of
development groups”. This is the only way I can show my credibility and usefulness
to our internal customers. Generally product-testing group is perceived as
gatekeepers who does not allow product get out of the doorsteps. It is very
important that testing group should get the message out to other departments that
our objective is not to stop the release but get the good enough quality product out
so that we can avoid customers receiving the product which is not fit for use. As I
mentioned above the above tests are performed at the end of the system life cycle,
fixing the bugs by changing the code could be very costly affair because it consumes
a lot if time. Testers useful ness can be shown here by suggesting work around for
the show stoppers so that product can be released by mentioning these workarounds
in the release notes.
Are we really cracking the product?
While I was involved in doing the technical testing for Baan products, one of the
developers sarcastically commented about me “here comes the man who is going to
crack our product”. As a tester I never cracked any software product, because by the
time it reaches us, it already had cracks, which were made by the developer. Iam
here just to find those cracks. One of the developers told me that though he can do
everything perfectly he does not do because he wants testers also let live. If
developer attempts to do everything correct, may be we will take more time to find
the remark, but if he does his work to keep us busy may be he will not have a job to
do in future. It should be made very clear that the quality can not be built in by
testing the product. It is the developer’s responsibility to build the quality into the
product and tester can assess the quality of the product looking into different aspects
of products. By involving testers from the early phases of the development life cycle
tester also can become the co-builder of the product.
Breadth and Ignorance:
It is very common that a single tester works with development teams of bigger size
(more than 6). Developer is generally responsible for his own piece of the product,
where as a tester has to work across the product. In other words tester is
responsible for major portion of the product or for the complete product. It is not
very unlikely that developer very easily write off tester saying “ he is dumb, he does
not understand what DLL, function, display driver, binary means…”. As a tester we
should always feel comfortable being a generalist rather than going for eye to eye
detail in other words breadth is important than depth. So sometimes we better
ignore the technical details. It is always an advantage to understand what developer
is doing to make the product, to appreciate his piece of craft but not necessary.
Constructive Confrontation:
While talking to one of my friend who is a developer he told me “ designing of an
application is challenging work and testing is a boring job, which is nothing more
than data entry operator’s job “. As a tester shot back saying that “ pointing the
failures and omission in the design is equally challenging and interesting as making
the design.” I fully believe that testing is a challenging job which requires
comprehensive knowledge of the product, domain knowledge, comfortable with
conflict, grasping things quickly, skeptical about specifications, negotiating skills,
getting into the role of the customer while testing and having the courage to say NO
when things are wrong.
Mixture of different personalities:
While resolving an issue with the configuration engineer, it had become almost
impossible for me to convince him that it is an issue which the customer may not be
able to understand and it has be mentioned more detailed in the release notes.
Rather continuing an argument with him, which iam very sure will end up
In vain. I had approached international technical consultant who has the history of
working with customers and better understanding of the knowledge levels of the
customer about the technical details of Baan product. It took him no time to
convince the configuration engineer that it is an issue that need to mentioned in
detail. Here my interest is not to point out who has won the situation. It is very
important that test team should be a mixed team which should have individuals who
worked in customer support, technical support and customer community. Keep it in
mind at least to give an opportunity to testers that they would talk to customer
community or at least to the individuals who has got targeted user industry
experience.
Testing by developers:
I had come across a situation where enough testing manpower is not available to
execute the test project. So developers were allowed to execute the test scripts that
were prepared by tester. Developer’s enthusiasm levels were very less while logging
the defects compared to executing the test scripts. So obviously defect numbers and
defect reports were very poor. So when ever you are planning to involve developers
make sure that you will explain the importance of logging defects, detailed bug
reports and how to make them interesting and reproducible. Do not for get to tell
them that the visible indicators
Of the test execution phase are remarks and found and bug reports.
Ego Clashes:
While working with one of the developer, I had tough time interacting with him. He
gets demotivated for the every defect I log. He thinks the product that he is making
is a part of him, and showing faults in it is a disgrace to him. I explained him my job
as a “Professional bad message delivery man” and it is my duty to find defects and
that is what iam paid for. Definitely everybody wants to have as many friends as
possible in their personal life, so is the tester. If software developers become my
friends only if I neglect the mistakes in the products that they make, Do I really need
that friendship? Company pays me to find the defects in the software, not to make
friends. There are a lot of tactics involved to do our job effectively particularly if the
development teams are less mature. It is easier said than done to have a good
Rapport with development teams while doing your job effectively.
Till now as you may have observed I was more or less talking about the
shortcomings of developers while interacting with test teams. It definitely does not
mean that we are perfect in everything. Let me share some of my experiences where
testers behaved in a way that they should not have.
Dismissal of developer’s job as scrap:
I had seen some of the young talented testers dismissing developers job as
Scrap after finding few defects. We should understand that the errors in the program might
be because of the lack of clarity in requirements, erroneous designs, lack of training and
time pressure rather than developers intellectual ability. I had seen testers insisting upon
every defect that is logged by them to be solved. We should also evaluate the impact and
severity the of defect that it can have on the other programs, next phase of test execution
and ultimately on the customer before bluntly asking the developer to solve every remark
that is logged. Friction / end less arguments between developers and testers should be
minimized to spend the precious time in doing bug fixing and hunting for gold mine
(defects).
Early Involvement of Testers:
Early involvement of testers in the software development life cycle to detect the defects early
so that cost of the fixing them comes down is the known philosophy. Allowing junior testers
(generally fresh from campus with out domain knowledge) to participate in the design
reviews does more of harm than good. During the review senior software developers and
consultants spend most of their time educating the Tester about system design, but the
purpose was different. He is there to find the faults and omissions in the design. So don’t
bluntly argue to involve tester from the early phase, let the testers gain the knowledge about
the product and slowly get involved in the early phases of the development life cycle.
Importance of Bug Reports:
One of my friend who is a tester comes to me and says “ Defect logging is such a boring job,
I do not find any interest in doing them.” It is very important that bug report needs to be
complete and correct to reduce the BUG AGE considerably. What I mean by bug age is “ The
time difference between the time at which defect has been found and the time at which it is
resolved”. Bug reports are the most visible indicators of tester’s deliverables. Cost of the bug
solving would be very high incase of terse bug reports stating “ it does not work the way it is
supposed to work” and “ The functionality is good”. It is also very important that we should
do root cause analysis (Etiology) to pinpoint what exactly caused the failure. This kind of
analysis would reduce the costs of fixing the bug and increases your value in the eyes of
developer. Defects, some times, which are very difficult to reproduce and can be
reproducible after doing the same set of actions more than 10 times or so. If it is getting
impossible for you to pinpoint the exact conditions, that is making the system to behave in
the way that it behaved previously, most of the testers hesitate to log them. If you don’t log
it, creator of the bug will never get a chance to look at it. Because the developer know the
internals of the system very well, they will help you in finding out the cause. So we better
log everything which is suspicious.
Conclusion:
Testers and developers definitely require unique skills, which are very complimentary in
nature. Both the test teams and development teams should understand this to be more
productive and to reach our common goal of producing good enough quality products. By
demonstrating values between the test teams and development teams to each other, we
gain mutual respect for each other. A master of one should be appreciated rather than being
criticized for jack of all and vice versa. As we are seeing now testing is being recognized as a
profession by it self… I believe it continues to go stronger and finally reaches a stage where
it is inseparable from the development and indispensable.

Life of a Tester

A DAY IN THE LIFE OF A TESTER…
ABSTRACT:
In this paper I would like to share the experiences of my colleagues and mine, who
are a part of the product-testing group while interacting with other involved parties
especially development teams in Baan Information Systems. The paper focuses on
how to overcome the embarrassing situations between developers and testers with
Ease and elan. It stresses the need for mutual respect of skills between development
teams and test teams which are equally important and unique to achieve company’s
mission. Some of the common mistakes that are done by development teams
towards test teams vice versa and the possible solutions are also explained. It states
unique skills that a Testing profession calls for which are very much different from
development skills.
If you belong to a company where you also have a separate independent test team
to test the products, the following situations may not be new to you.
What role should a Tester play? Optimistic or pessimistic.
While I was involved in the technical installation and translation verification testing
for translated products of Baan, interaction with release managers, Baan translation
group and configuration & assembly department is very frequent. These tests are
performed at the end of the system life cycle before shipping the product to the
customers. Release managers responsibility is to get the product out of doorsteps as
soon as possible with good enough quality to get bucks for us. Test report from
Product testing group helps them in taking better decision of giving GO/ No GO to
the release. Baan translation group, who is responsible for translating our products
into different languages, playing a role of software engineers in our standard
development group. Configuration and development group is responsible for
processing the application dumps and putting different pieces of the software
(installation scripts, tool components, and different products…) which is received
from the different product development units on the physical medium. While
participating in the discussions with them I was always confused whether to play a
role of optimistic or pessimistic. Everybody else except product testing group wants
to get the product out of doors as soon as possible. If we don’t release the products
we will not get money, releasing with less quality also is not good in the long run
(support department lines would soon get busy). I had come to the decision of
playing an optimistic role in front Of release managers, talking like “ I wish I would
give GO to this product”. As a tester I get paid for proving that “product does not
meet specified requirements”. I was always pessimistic at the back of my mind
thinking, “iam going to find critical bugs which has been escaped through the eyes of
development groups”. This is the only way I can show my credibility and usefulness
to our internal customers. Generally product-testing group is perceived as
gatekeepers who does not allow product get out of the doorsteps. It is very
important that testing group should get the message out to other departments that
our objective is not to stop the release but get the good enough quality product out
so that we can avoid customers receiving the product which is not fit for use. As I
mentioned above the above tests are performed at the end of the system life cycle,
fixing the bugs by changing the code could be very costly affair because it consumes
a lot if time. Testers useful ness can be shown here by suggesting work around for
the show stoppers so that product can be released by mentioning these workarounds
in the release notes.
Are we really cracking the product?
While I was involved in doing the technical testing for Baan products, one of the
developers sarcastically commented about me “here comes the man who is going to
crack our product”. As a tester I never cracked any software product, because by the
time it reaches us, it already had cracks, which were made by the developer. Iam
here just to find those cracks. One of the developers told me that though he can do
everything perfectly he does not do because he wants testers also let live. If
developer attempts to do everything correct, may be we will take more time to find
the remark, but if he does his work to keep us busy may be he will not have a job to
do in future. It should be made very clear that the quality can not be built in by
testing the product. It is the developer’s responsibility to build the quality into the
product and tester can assess the quality of the product looking into different aspects
of products. By involving testers from the early phases of the development life cycle
tester also can become the co-builder of the product.
Breadth and Ignorance:
It is very common that a single tester works with development teams of bigger size
(more than 6). Developer is generally responsible for his own piece of the product,
where as a tester has to work across the product. In other words tester is
responsible for major portion of the product or for the complete product. It is not
very unlikely that developer very easily write off tester saying “ he is dumb, he does
not understand what DLL, function, display driver, binary means…”. As a tester we
should always feel comfortable being a generalist rather than going for eye to eye
detail in other words breadth is important than depth. So sometimes we better
ignore the technical details. It is always an advantage to understand what developer
is doing to make the product, to appreciate his piece of craft but not necessary.
Constructive Confrontation:
While talking to one of my friend who is a developer he told me “ designing of an
application is challenging work and testing is a boring job, which is nothing more
than data entry operator’s job “. As a tester shot back saying that “ pointing the
failures and omission in the design is equally challenging and interesting as making
the design.” I fully believe that testing is a challenging job which requires
comprehensive knowledge of the product, domain knowledge, comfortable with
conflict, grasping things quickly, skeptical about specifications, negotiating skills,
getting into the role of the customer while testing and having the courage to say NO
when things are wrong.
Mixture of different personalities:
While resolving an issue with the configuration engineer, it had become almost
impossible for me to convince him that it is an issue which the customer may not be
able to understand and it has be mentioned more detailed in the release notes.
Rather continuing an argument with him, which iam very sure will end up
In vain. I had approached international technical consultant who has the history of
working with customers and better understanding of the knowledge levels of the
customer about the technical details of Baan product. It took him no time to
convince the configuration engineer that it is an issue that need to mentioned in
detail. Here my interest is not to point out who has won the situation. It is very
important that test team should be a mixed team which should have individuals who
worked in customer support, technical support and customer community. Keep it in
mind at least to give an opportunity to testers that they would talk to customer
community or at least to the individuals who has got targeted user industry
experience.
Testing by developers:
I had come across a situation where enough testing manpower is not available to
execute the test project. So developers were allowed to execute the test scripts that
were prepared by tester. Developer’s enthusiasm levels were very less while logging
the defects compared to executing the test scripts. So obviously defect numbers and
defect reports were very poor. So when ever you are planning to involve developers
make sure that you will explain the importance of logging defects, detailed bug
reports and how to make them interesting and reproducible. Do not for get to tell
them that the visible indicators
Of the test execution phase are remarks and found and bug reports.
Ego Clashes:
While working with one of the developer, I had tough time interacting with him. He
gets demotivated for the every defect I log. He thinks the product that he is making
is a part of him, and showing faults in it is a disgrace to him. I explained him my job
as a “Professional bad message delivery man” and it is my duty to find defects and
that is what iam paid for. Definitely everybody wants to have as many friends as
possible in their personal life, so is the tester. If software developers become my
friends only if I neglect the mistakes in the products that they make, Do I really need
that friendship? Company pays me to find the defects in the software, not to make
friends. There are a lot of tactics involved to do our job effectively particularly if the
development teams are less mature. It is easier said than done to have a good
Rapport with development teams while doing your job effectively.
Till now as you may have observed I was more or less talking about the
shortcomings of developers while interacting with test teams. It definitely does not
mean that we are perfect in everything. Let me share some of my experiences where
testers behaved in a way that they should not have.
Dismissal of developer’s job as scrap:
I had seen some of the young talented testers dismissing developers job as
Scrap after finding few defects. We should understand that the errors in the program might
be because of the lack of clarity in requirements, erroneous designs, lack of training and
time pressure rather than developers intellectual ability. I had seen testers insisting upon
every defect that is logged by them to be solved. We should also evaluate the impact and
severity the of defect that it can have on the other programs, next phase of test execution
and ultimately on the customer before bluntly asking the developer to solve every remark
that is logged. Friction / end less arguments between developers and testers should be
minimized to spend the precious time in doing bug fixing and hunting for gold mine
(defects).
Early Involvement of Testers:
Early involvement of testers in the software development life cycle to detect the defects early
so that cost of the fixing them comes down is the known philosophy. Allowing junior testers
(generally fresh from campus with out domain knowledge) to participate in the design
reviews does more of harm than good. During the review senior software developers and
consultants spend most of their time educating the Tester about system design, but the
purpose was different. He is there to find the faults and omissions in the design. So don’t
bluntly argue to involve tester from the early phase, let the testers gain the knowledge about
the product and slowly get involved in the early phases of the development life cycle.
Importance of Bug Reports:
One of my friend who is a tester comes to me and says “ Defect logging is such a boring job,
I do not find any interest in doing them.” It is very important that bug report needs to be
complete and correct to reduce the BUG AGE considerably. What I mean by bug age is “ The
time difference between the time at which defect has been found and the time at which it is
resolved”. Bug reports are the most visible indicators of tester’s deliverables. Cost of the bug
solving would be very high incase of terse bug reports stating “ it does not work the way it is
supposed to work” and “ The functionality is good”. It is also very important that we should
do root cause analysis (Etiology) to pinpoint what exactly caused the failure. This kind of
analysis would reduce the costs of fixing the bug and increases your value in the eyes of
developer. Defects, some times, which are very difficult to reproduce and can be
reproducible after doing the same set of actions more than 10 times or so. If it is getting
impossible for you to pinpoint the exact conditions, that is making the system to behave in
the way that it behaved previously, most of the testers hesitate to log them. If you don’t log
it, creator of the bug will never get a chance to look at it. Because the developer know the
internals of the system very well, they will help you in finding out the cause. So we better
log everything which is suspicious.
Conclusion:
Testers and developers definitely require unique skills, which are very complimentary in
nature. Both the test teams and development teams should understand this to be more
productive and to reach our common goal of producing good enough quality products. By
demonstrating values between the test teams and development teams to each other, we
gain mutual respect for each other. A master of one should be appreciated rather than being
criticized for jack of all and vice versa. As we are seeing now testing is being recognized as a
profession by it self… I believe it continues to go stronger and finally reaches a stage where
it is inseparable from the development and indispensable.

DON

Srinivas.Ch - DON1