Rays Development Blog - Expectations

Judge, Jury and Executioner - The life of a QA analyst

Ray Cassick — Fri, 05 Nov 2010 16:48:11 GMT

So, I have experienced what I feel is a failure of a project that I was recently a part of in my personal life and have been thinking about it a lot lately. Partly because as an systems architect it is my job to always be trying to understand where I can improve myself and ensure that I do not repeat mistakes, but also just because, well darn it, I hate failing.

Who the heck doesn’t hate failing?

Really, I am not counting this as a ‘failure’ per-se because I did bring it up as an issue at the onset of the project and even noted my personal objections to it in the review notes that were taken in the meetings I had. I am noting it as a time of shame in that I allowed my PERSONAL level of professional conduct to be driven by an outside group instead of recusing myself and just walking away. In short, I let go of my principals and am now paying for it.

Not a mistake I will be making again.

How did I come up with the title of this entry? What does a QA analyst have to do with the legal system? Just so you know I am a huge fan of the TV series Law an Order. Not so much the recent off shoots, but the old shows with Jerry Orbach (Lennie Briscoe), Sam Waterston (Jack McCoy), and one of my personal favorites, Chris Noth (Mike Logan), but I digress…

I have always been fascinated by the law. I almost decided to become a layer ate one point but decided that I was not hard enough (or perhaps too hard) to take the role. I looked at it for a while and decided that there were potentially too many gray areas to have to deal with ethically, so I took the IT route instead. Hehehehehe, yeah, who knew?

So, the relation here is this.

In the legal system you have several areas of a legal issue, each on represented by a specific area of expertise looking at the case in a different way. The accused is, by matter of the same legal system that is currently citing them as a ‘bad guy’ provided a way to prove their innocence before a panel of impartial people, and is offered representation to help them. There are people on both sides that defend their position, present their case and in the end the judge and jury make a decision based upon a preponderance of the evidence if the accused is guilty or innocent, and what the method/mode of punishment should be. Remember, the legal system is represented by the scales held by lady justice’s left hand with a sword in her right, and her eyes covered with a blind fold giving the indication that she is unable to be influenced by any outside party, and are driven only by the written matter of law currently established.

In the project system you have several areas of a project issue, each on represented by a specific area of expertise looking at the problem in a different way. The project is what it is, being defined by the specifications that were approved by all the parties involved upon its initiation. There are people on both sides that defend their position, present their case, and in the end someone makes a decision if the delivered system met the requirements or not, and how to correct what needs to be corrected moving forward.

In a business environment, the business owner comes to IT with a need. They understand (probably very well) what needs to be accomplished and can usually state those goals very well in what are referred to as High Level Requirements. These requirements are used to establish a baseline timeframe and budget that is then referenced by the business plan to check validity to the established mission and cash flow for the year to determine if it can be(or even should be) perused. Once they get the green light it moves on.

In the IT environment an architect is assigned the project, provided the business requirements, a basic timeline and a budget framework and told to go off and design, then come back with more specifics to move forward. Once they do the design and pass it back to the company for final approval (timeline and budget) the project then gets assigned to developers to complete according to the specification.

The developers do the work based upon the design of the architect; perform some base level tests to make sure that what they release meets the stated objectives, and then release a build for testing.

Here is where the problem ALWAYS happens.

The business will sometimes NOT want to include a QA test resource.

WHY? I am not sure. Usually the business says that they are too busy to be bothered with anything. They are, after all, the ones making the money for the company, why should they want to do anything else? But I have heard more than a few times that THEY want to be the test people on the project because THEY know the DATA better than ANYONE and can be the best judge of the system processing quality than a QA persona can ever be.

It was HERE where I was bitten.

I fought hard and lost my battle. I was made to allow this abomination into my project. I was provided with the business requirements, I created the low level design, handed that off to developers that created their individual designs and had them reviewed by other developers, then implemented them long with a series of basic test cases that they deemed were required, and then handed the ‘completed’ project over to the business for THEM to test. The business ran their TESTS (I have yet to see an established – IE: written – test plan or results document) and signed off on the completed work. The total time for QA testing ended up being about 4-6 hours.

My right eyebrow rose a bit but it was apparently not for me to say anything and the project went into production where it was run for the first time and the resulting data set was sent off to the next step of the process (not something that I have any control over at all there), and within hours THEY saw issues in the data that they were presented with as a result of this projects processing and kicked it back to us. The business took a look at the data (that they already saw by the way, remember, they ‘QA Tested’ this system just hours before and had ‘signed off’ –approval via email- on its viability and correctness).

The reaction was shocking to say the least. The business came back and questioned the systems correctness. I was shocked, but not at all that surprised, but still a bit ticked off. I am not a person that enjoys assigning blame, but when I am asked to explicitly locate a problem, that job gets done for me. I find the error and the fault is assigned by the simple act of doing that. Who did that work gets the ‘blame’. In my opinion though the blame should be shared by the developer and the person that did the review of the code, and ALSO the QA Analyst that either missed a test case or did not execute one correctly. In this case we had NO QA Analyst, or in reality, I was getting asked BY THE QA Analyst (the business unit in this case) what the problem was. Again, I was a little miffed, but took it. The problem ended up being something that I knew was going to be a potential issue, and that we had even discussed in meetings as part of the implementation and design. A direction was decided upon between me and the PM that the business (err… QA) would manually process through this data list and perform some further cleanup that would take a significant effort in dollars, time and specialized software to accomplish in an automated manner, and that we would look at other more automated solutions in the next round prior to this process needing to be used again next year. Being the diligent architect that I am kept this all documented in the projects documentation, partially because I am just a thorough person, but also as a way to provide some CYA to both myself and the next unlucky architect that got any revisions the next time this project needed to have changes made to it.

The manual processing was done, requiring the business to manually look through every record and try to remediate possible duplicates. I figured this would FORCE them to look at each and every record and if there were any OTHER errors they would see them. They were after all ‘the best people to judge the correctness of the data’ hence the reason that they mandated themselves as the QA team in the first place. I again, shook my head, scratched a bit, and let it go. They completed their manual processing, removed about 1000 or so records that they felt were dupes and handed the file back to me to get converted over and sent back to the vendor for processing. That being done, the project was run, my involvement was closed out, and I was assigned on to other work.

Ding dong, the alarm bell rings again as a new problem is found, and then another.

Once again I am asked to look at the data. Amazingly enough, I am asked by the same team that certified this exact same data, and even had to read through it all manually record by record in their last cleanup effort, to find the ‘problem’. I found the problem, a common mistake in this type of processing (the order that records are placed in when a lookup is performed) that was not caught by the developer, the reviewer of their code, nor the QA team that certified the data TWICE now before it was allowed out the door.

So, what’s the result here? I am going to spend my weekend looking over the data between what we HAD then and what we HAVE now as the result of a change made to address the issue and try to determine what to do next.

Being a process oriented guy and always one to try to learn from my mistakes I have taken a hard look at this and made a determination that I was right at the start and I am not going to ever accept a project that does not have QA resources assigned. Could I be potentially signing my own walking papers? Perhaps, but at this point it is a case based upon principles and not just me being a whiney architect not willing to take blame. In fact all I have been asking all along is that someone who is impartial to the business process, design of the solution, and the development of the solution look at the data going in, the processing, and the data coming out, and TELL ME if there are problems.

I welcome being told there is a problem so it can be addressed BEFORE we ship. That’s the idea of testing, to catch problems before they make it to production. I just fail to see how people cannot understand that. Just as Lady Justice stands outside of every courthouse to ensure fair and impartial judgment on the application of the rules of law, so should QA be allowed to stand and judge the usability of a system before it is relied upon to perform its tasks.

Now I ask you, how many people ASK to be judged like this?

Am I wrong?

Irony!

Ray Cassick — Fri, 15 Oct 2010 02:07:56 GMT

I just found this so funny I had to share it. Although things like this have happened to me in code before so I probably should not be casting stones, I do still admit that I find it funny and point an occasional finger and giggling allows me to vent sometimes :)

I received an email the other day from a company (nameless - unless you happen to recognise the email :) )that provides a daily supply of white-papers and other technically oriented and marketing type documents for us geeky folk to read in our volumes of spare time.

I just found it sooo funny when I got to the bottom of the page and read the last item on the list...

(website link and company name clipped to protect the ~~innocent~~goofy)

I'm really sorry, but I have to admit that I spent a good 10 minutes laughing at this and finding great joy for some reason.

So, let this be a lesson to everyone out there, and me too. People WILL laugh at you for making a goofy mistake. How do you want your marketing efforts to be remembered?

Hmmmm...

Maybe they did this on purpose?

Nah... I doubt it...

Note: Yes, the actual link to the document DID work, and it was actually pretty good.

Pick a design and stick to it (please).

Ray Cassick — Mon, 27 Sep 2010 01:55:02 GMT

Well it has happened again?

Found yet another inconsistency on the Microsoft Office suite that for some reason annoys me to no end now that I found it. The issue lies around the ability to select colors.

Within outlook you can create message categories and assign them to messages in the mail list. This is pretty useful when you need to quickly identify groups of messages visually. I actually make use of this myself to help me separate personal emails from work related emails, and then a bit more granular to help rate them by importance via color (IE: light are less priority, darker are higher priority). It seems to work pretty good so far and helps keep me organized. However, lately something has just been itching at me about the way it works and looks, and suddenly last night it hit me. The color picker dropdown box that the Outlook colorization category uses is not a standard color selection dropdown. Here is what it looks like:

Problem #1 - It's just wrong! Windows has a standard color picker dialog box, why not use it? Someone had to:

Take the time to design the control interface.
Take the time to create a drop down control JUST for this use and include it in the code.
Test it.
Release it

Problem #2 - It's been done wrong! The UI of this control is awful and it is non-standard.

It mixes the look and feel of a combo box and a menu control. Notice that the control uses a dropdown arrow to indicate the available action but then the option area is not a list style control, but instead is a menu style area, and even contains the light shading on the left where the menu would normally place icons that show this options equivalent location on a tool bar.
The available colors are all mixed up. They are not in any specific order (IE: Light to dark, grouped by tone value, etc?) Very bad design indeed.

It's clear that someone at Microsoft does know how to design a color selection control, because they offer one. I see it used all the time, even in the same application (Outlook) when I want to change the color of my text. It looks like this in case you have forgotten:

See? Now THAT'S how it is supposed to look. Gives you access to the standard colors most often used, allows you to select form a nice wide range of other colors, arranged in (some) hue order to create a theme style, and then gives you the option to jump over to a more advanced selection dialog (using the More Colors?) option, where you can mix your own colors using not just the RGB or HSL scale but also brightness.

THAT is how it's done.

Now I am not going to argue about the effectiveness of the color selection dropdown I am showing above, I am simply showing that it exists, is considered a standard within the Microsoft community, both by internal developers and external customers, and seems to be effective enough for general use. I would not settle for this style in a high end art-centric application, but for what it needs to do as part of the Microsoft Office package I think this design is clear, concise and effective.

I do have to say however that even this design seems to have been perverted a bit. Taking a look into the same type of area but within the Microsoft Publisher application reveals this color option:

Bluch! Boring!

BUT at least it gives you the ability to jump to the more advanced option just like the others do using the ‘More Colors…’ option, and there you get the same dialog as shown above. In fact, because this is a professional (semi at least) desk top publishing application, this dialog box offers you the added option of using the Pantone color scale as well as the other RGB and HSL styles (who ever though that selecting a color would be so darn complicated?)

Since I was poking around I decided to crank open my copy of Microsoft Expression Blend 3 and see what that did for me. I figured as I went upwards in application complexity and relative use in the graphics industry I figured this one would be a bit more advanced (IE: Complicated) and I was presently surprised at the available features AND the obvious usability.

There is surprisingly very little labeling within the control itself, but I think most you can get the idea about what you can do with it pretty quickly. As I have often sighted to various groups in the past that have had to listen to me drone through usability discussions, quite often it is simple UI and the ability to play that leads towards an efficient design. You do not always have to GIVE the answers explicitly as long as you allow people the latitude to try and undo, leading to the adage that experience often leads to the best learning

So, I just HAD to do one more thing and revisit my old buddy PhotoShop.

Wow, it is actually not too bad. Funny, but it was kind of better than I had expected it to be. I do however notice that they may need a usability person to review this because the various types of color selections should really (IMHO at least) have a box around them showing that they are grouped together.

So, what have we learned here?

Yeah, I know, beyond the fact that I can get overly picky?

I think that the message is clear. Consistency is key.

As a developer, there is not really a need to go off reinventing the wheel. What would have been wrong with a dropdown solution similar to this?

At very least I think this leverages the idea I am trying to get across. And, it would give me more darn color options instead of just the static 25 that I am apparently stuck with for my categories.

Oh, and DO NOT GET ME STARTED on the fact that this listing of categories is the SAME listing that I have available between my CALENDAR and my CONTACTS!

UGH!

That is the subject for another blog post!

Problems with Avast Support

Ray Cassick — Sun, 04 Jul 2010 20:20:36 GMT

Honestly I would never have thought that I would be here using my technical blog to be venting about a vendor, but I really feel the time has come. This effort has gone far too long to go 'un-published' any longer.

The offender is Avast Antivirus. http://www.avast.com/index

I have been using their free home version for years and have been promoting it all over the place to every one. It works flawlessly and has caught so many things that other AV vendors have missed that I had decided to NEVER use anything but Avast on any system that I set up for any person. Business of course have to pay for the product because that the rules at Avast. I am FINE with that. in fact I think it is GREAT that they have thought enough to understand that home users are in need of a great AV app (the Avast provides) and that business are in a better position to pay for the support and all that. I have NO problem with them charge business and not home users at all.

So... the McAfee install that came as part of my Motorola Q phone had run out and I figured that I would go to my friends at Avast and purchase the mobile version of the app and install it on my phone and get rid of the junk that came on it for free. On 11\19\2009 I ordered and received my copy of Avast 4 PDA version and installed it. Email confirmation assured me that I would get my license code via email latter, and I did. The instructions told me how to enter the Avast license code. The PROBLEM was that the UI on the phone display did not include the 'About' button that they referenced so I could enter any code. Hmmmm

I opened a support case on 11\28\2009 explaining the issue I was having, and was responded to by Tomas the very next day that my phones 320x200 screen resolution was not yet supported and they would try to implement this feature in a future release. I asked him when they anticipate having this since I love using Avast and wanted to keep it going. I was told that they should have support for that resolution by sometime in January 2010. I felt OK with that date and told them that I would wait for the next release and figured that I would work out the license cost issues latter with them since I was going to be a few months off between having the license and being able to actually use it. I figured they would be good for it and we would work it out.

So, January comes and goes... Other things in my life had happened... I decided to follow up on April 4th 2010. Noticed that I could no longer find any listing of my original support case (AFO-665966) so I opened a new one (RSB-517949) asking for a status on my old case and if no help was going to be forthcoming then I would be expecting a refund of my $20. on 4\9\2010 I got an email back from Petr telling me that I can download Avast 5, get my license file resume to me if I forgot it, yadda, yadda, yadda.... I replied to him, and explained that I thought he was a bit confused, I was referring to Windows Mobile, etc... provided with again with the original support case # and asked for followup. As of 6\6\2010 I have yet to hear back from Petr at all and that case is still open.

In the mean time I posted about this on the public Avast message area, thinking that maybe if I started asking for any other people who maybe had the same issue I would get a response form at least the public. So far, no response from anyone (public or Avast) there. OK, what ever...

So, I decide enough is enough, I contact Avast sales and am at this point simply now asking for my $20 back. Shown below is the opening interaction I posted to get this started:

"I would like to request a refund of my purchase of Avast 4 PDA edition. REF#305093845. I have been trying to get support now since my purchase on 11/19/2009 under 2 ticket numbers (AFO-665966 and RSB-517949) and have gotten no where. I hate to move away from using Avast on my mobile phone (I use it at home on all my PCs there and love it and just recently just purchased a package of license for there) but I am not getting any help with the version I am running on my mobile phone at all. If someone can help me get a version that works on my phone I will keep running it (I love Avast) but I do not want to go longer without a functioning AV on the device and I am not getting any responses or help."

Thusly opens another ticket (XAP-831717) with a person named Adam. He asks me for details again and promises to 'look into it' for me. I give him the same technical details i Had given originally:

Motorola Q 9h global running Windows Mobile 6.1 - 2.4 Inch display 320x240 - 120 MB RAM - Avast 4.1.19 PDA

That post was on 6\23\2010. Here we are on 7\5\2010, no response. No refund, and no functional AV on my phone.

I don't get it.

When did support stop being important to people that pay?

I have worked in support most of my technical life. I worked as a phone jockey for Sony desktop and laptop support (I was agent C02E way back in the day if you remember me :) ) and was held to EXTREMELY high standards (phone monitoring, customer surveys, technical information and documentation audits, etc...). I Left there to do engineering level support and technical training for a local company called Voice Technologies Group (VTG) that built interface hardware to allow systems like IVRs, Voice Mail, and latter on Unified messaging systems interface between servers and PBX systems. They were bought by Dialogic so I ended up working for them, they got bought by Intel, and then Intel sold them off to a company called Icon Networks from Europe that ended up putting the name back to Dialogic again. Every step of the way I was involved in the servicing processes in some way, weather through support (phone or on-site) acting as a technical evangelist at all the various trade shows, writing and presenting training classes, or even stepping in and helping customers design solutions, and then test them using our hardware\software. At every step of the way I was held to the highest standards of customer interaction. If I just let a support case lie dead for a week my butt would be in a sling and I would be getting emails from customers, CEOs, you name it.

Now, I will admit that that perhaps the support expectations for a $20 bit of software is slightly lower than a hardware card or IP gateway that runs between $2000 and $10,000, but the idea is the same. You have a paying customer that plopped down some cold hard cash for and has some expectations. with respect to how they are treated and how they are interacted with. I am at this point not even expecting to get it working. I KNOW that they are simply NOT going to address my issue only because of the fact that my phone (as I have been told so many times now) is outdated and has too small a screen. The software is built for an actual 'smart phone' with a larger screen and that is that.

Simply have the personal where-with-all to just email me back and TELL me that fact and then GIVE ME A REFUND.

At this point I am not sure if I would ever buy Avast again even if I did upgrade my phone to something newer. Why? Because the level of support I have received until this point has been abysmal. Why would I expect that to change?

Avast support: Suggestion - Give me a version of SW that works or give me back my $20. I am not going away and I can have a tendency to become very persistent. I have tons of experience knowing what gets attention from the customer side of a support case.