Rays Development Blog - Roles

Judge, Jury and Executioner - The life of a QA analyst

Ray Cassick — Fri, 05 Nov 2010 16:48:11 GMT

So, I have experienced what I feel is a failure of a project that I was recently a part of in my personal life and have been thinking about it a lot lately. Partly because as an systems architect it is my job to always be trying to understand where I can improve myself and ensure that I do not repeat mistakes, but also just because, well darn it, I hate failing.

Who the heck doesn’t hate failing?

Really, I am not counting this as a ‘failure’ per-se because I did bring it up as an issue at the onset of the project and even noted my personal objections to it in the review notes that were taken in the meetings I had. I am noting it as a time of shame in that I allowed my PERSONAL level of professional conduct to be driven by an outside group instead of recusing myself and just walking away. In short, I let go of my principals and am now paying for it.

Not a mistake I will be making again.

How did I come up with the title of this entry? What does a QA analyst have to do with the legal system? Just so you know I am a huge fan of the TV series Law an Order. Not so much the recent off shoots, but the old shows with Jerry Orbach (Lennie Briscoe), Sam Waterston (Jack McCoy), and one of my personal favorites, Chris Noth (Mike Logan), but I digress…

I have always been fascinated by the law. I almost decided to become a layer ate one point but decided that I was not hard enough (or perhaps too hard) to take the role. I looked at it for a while and decided that there were potentially too many gray areas to have to deal with ethically, so I took the IT route instead. Hehehehehe, yeah, who knew?

So, the relation here is this.

In the legal system you have several areas of a legal issue, each on represented by a specific area of expertise looking at the case in a different way. The accused is, by matter of the same legal system that is currently citing them as a ‘bad guy’ provided a way to prove their innocence before a panel of impartial people, and is offered representation to help them. There are people on both sides that defend their position, present their case and in the end the judge and jury make a decision based upon a preponderance of the evidence if the accused is guilty or innocent, and what the method/mode of punishment should be. Remember, the legal system is represented by the scales held by lady justice’s left hand with a sword in her right, and her eyes covered with a blind fold giving the indication that she is unable to be influenced by any outside party, and are driven only by the written matter of law currently established.

In the project system you have several areas of a project issue, each on represented by a specific area of expertise looking at the problem in a different way. The project is what it is, being defined by the specifications that were approved by all the parties involved upon its initiation. There are people on both sides that defend their position, present their case, and in the end someone makes a decision if the delivered system met the requirements or not, and how to correct what needs to be corrected moving forward.

In a business environment, the business owner comes to IT with a need. They understand (probably very well) what needs to be accomplished and can usually state those goals very well in what are referred to as High Level Requirements. These requirements are used to establish a baseline timeframe and budget that is then referenced by the business plan to check validity to the established mission and cash flow for the year to determine if it can be(or even should be) perused. Once they get the green light it moves on.

In the IT environment an architect is assigned the project, provided the business requirements, a basic timeline and a budget framework and told to go off and design, then come back with more specifics to move forward. Once they do the design and pass it back to the company for final approval (timeline and budget) the project then gets assigned to developers to complete according to the specification.

The developers do the work based upon the design of the architect; perform some base level tests to make sure that what they release meets the stated objectives, and then release a build for testing.

Here is where the problem ALWAYS happens.

The business will sometimes NOT want to include a QA test resource.

WHY? I am not sure. Usually the business says that they are too busy to be bothered with anything. They are, after all, the ones making the money for the company, why should they want to do anything else? But I have heard more than a few times that THEY want to be the test people on the project because THEY know the DATA better than ANYONE and can be the best judge of the system processing quality than a QA persona can ever be.

It was HERE where I was bitten.

I fought hard and lost my battle. I was made to allow this abomination into my project. I was provided with the business requirements, I created the low level design, handed that off to developers that created their individual designs and had them reviewed by other developers, then implemented them long with a series of basic test cases that they deemed were required, and then handed the ‘completed’ project over to the business for THEM to test. The business ran their TESTS (I have yet to see an established – IE: written – test plan or results document) and signed off on the completed work. The total time for QA testing ended up being about 4-6 hours.

My right eyebrow rose a bit but it was apparently not for me to say anything and the project went into production where it was run for the first time and the resulting data set was sent off to the next step of the process (not something that I have any control over at all there), and within hours THEY saw issues in the data that they were presented with as a result of this projects processing and kicked it back to us. The business took a look at the data (that they already saw by the way, remember, they ‘QA Tested’ this system just hours before and had ‘signed off’ –approval via email- on its viability and correctness).

The reaction was shocking to say the least. The business came back and questioned the systems correctness. I was shocked, but not at all that surprised, but still a bit ticked off. I am not a person that enjoys assigning blame, but when I am asked to explicitly locate a problem, that job gets done for me. I find the error and the fault is assigned by the simple act of doing that. Who did that work gets the ‘blame’. In my opinion though the blame should be shared by the developer and the person that did the review of the code, and ALSO the QA Analyst that either missed a test case or did not execute one correctly. In this case we had NO QA Analyst, or in reality, I was getting asked BY THE QA Analyst (the business unit in this case) what the problem was. Again, I was a little miffed, but took it. The problem ended up being something that I knew was going to be a potential issue, and that we had even discussed in meetings as part of the implementation and design. A direction was decided upon between me and the PM that the business (err… QA) would manually process through this data list and perform some further cleanup that would take a significant effort in dollars, time and specialized software to accomplish in an automated manner, and that we would look at other more automated solutions in the next round prior to this process needing to be used again next year. Being the diligent architect that I am kept this all documented in the projects documentation, partially because I am just a thorough person, but also as a way to provide some CYA to both myself and the next unlucky architect that got any revisions the next time this project needed to have changes made to it.

The manual processing was done, requiring the business to manually look through every record and try to remediate possible duplicates. I figured this would FORCE them to look at each and every record and if there were any OTHER errors they would see them. They were after all ‘the best people to judge the correctness of the data’ hence the reason that they mandated themselves as the QA team in the first place. I again, shook my head, scratched a bit, and let it go. They completed their manual processing, removed about 1000 or so records that they felt were dupes and handed the file back to me to get converted over and sent back to the vendor for processing. That being done, the project was run, my involvement was closed out, and I was assigned on to other work.

Ding dong, the alarm bell rings again as a new problem is found, and then another.

Once again I am asked to look at the data. Amazingly enough, I am asked by the same team that certified this exact same data, and even had to read through it all manually record by record in their last cleanup effort, to find the ‘problem’. I found the problem, a common mistake in this type of processing (the order that records are placed in when a lookup is performed) that was not caught by the developer, the reviewer of their code, nor the QA team that certified the data TWICE now before it was allowed out the door.

So, what’s the result here? I am going to spend my weekend looking over the data between what we HAD then and what we HAVE now as the result of a change made to address the issue and try to determine what to do next.

Being a process oriented guy and always one to try to learn from my mistakes I have taken a hard look at this and made a determination that I was right at the start and I am not going to ever accept a project that does not have QA resources assigned. Could I be potentially signing my own walking papers? Perhaps, but at this point it is a case based upon principles and not just me being a whiney architect not willing to take blame. In fact all I have been asking all along is that someone who is impartial to the business process, design of the solution, and the development of the solution look at the data going in, the processing, and the data coming out, and TELL ME if there are problems.

I welcome being told there is a problem so it can be addressed BEFORE we ship. That’s the idea of testing, to catch problems before they make it to production. I just fail to see how people cannot understand that. Just as Lady Justice stands outside of every courthouse to ensure fair and impartial judgment on the application of the rules of law, so should QA be allowed to stand and judge the usability of a system before it is relied upon to perform its tasks.

Now I ask you, how many people ASK to be judged like this?

Am I wrong?

Problems with Avast Support

Ray Cassick — Sun, 04 Jul 2010 20:20:36 GMT

Honestly I would never have thought that I would be here using my technical blog to be venting about a vendor, but I really feel the time has come. This effort has gone far too long to go 'un-published' any longer.

The offender is Avast Antivirus. http://www.avast.com/index

I have been using their free home version for years and have been promoting it all over the place to every one. It works flawlessly and has caught so many things that other AV vendors have missed that I had decided to NEVER use anything but Avast on any system that I set up for any person. Business of course have to pay for the product because that the rules at Avast. I am FINE with that. in fact I think it is GREAT that they have thought enough to understand that home users are in need of a great AV app (the Avast provides) and that business are in a better position to pay for the support and all that. I have NO problem with them charge business and not home users at all.

So... the McAfee install that came as part of my Motorola Q phone had run out and I figured that I would go to my friends at Avast and purchase the mobile version of the app and install it on my phone and get rid of the junk that came on it for free. On 11\19\2009 I ordered and received my copy of Avast 4 PDA version and installed it. Email confirmation assured me that I would get my license code via email latter, and I did. The instructions told me how to enter the Avast license code. The PROBLEM was that the UI on the phone display did not include the 'About' button that they referenced so I could enter any code. Hmmmm

I opened a support case on 11\28\2009 explaining the issue I was having, and was responded to by Tomas the very next day that my phones 320x200 screen resolution was not yet supported and they would try to implement this feature in a future release. I asked him when they anticipate having this since I love using Avast and wanted to keep it going. I was told that they should have support for that resolution by sometime in January 2010. I felt OK with that date and told them that I would wait for the next release and figured that I would work out the license cost issues latter with them since I was going to be a few months off between having the license and being able to actually use it. I figured they would be good for it and we would work it out.

So, January comes and goes... Other things in my life had happened... I decided to follow up on April 4th 2010. Noticed that I could no longer find any listing of my original support case (AFO-665966) so I opened a new one (RSB-517949) asking for a status on my old case and if no help was going to be forthcoming then I would be expecting a refund of my $20. on 4\9\2010 I got an email back from Petr telling me that I can download Avast 5, get my license file resume to me if I forgot it, yadda, yadda, yadda.... I replied to him, and explained that I thought he was a bit confused, I was referring to Windows Mobile, etc... provided with again with the original support case # and asked for followup. As of 6\6\2010 I have yet to hear back from Petr at all and that case is still open.

In the mean time I posted about this on the public Avast message area, thinking that maybe if I started asking for any other people who maybe had the same issue I would get a response form at least the public. So far, no response from anyone (public or Avast) there. OK, what ever...

So, I decide enough is enough, I contact Avast sales and am at this point simply now asking for my $20 back. Shown below is the opening interaction I posted to get this started:

"I would like to request a refund of my purchase of Avast 4 PDA edition. REF#305093845. I have been trying to get support now since my purchase on 11/19/2009 under 2 ticket numbers (AFO-665966 and RSB-517949) and have gotten no where. I hate to move away from using Avast on my mobile phone (I use it at home on all my PCs there and love it and just recently just purchased a package of license for there) but I am not getting any help with the version I am running on my mobile phone at all. If someone can help me get a version that works on my phone I will keep running it (I love Avast) but I do not want to go longer without a functioning AV on the device and I am not getting any responses or help."

Thusly opens another ticket (XAP-831717) with a person named Adam. He asks me for details again and promises to 'look into it' for me. I give him the same technical details i Had given originally:

Motorola Q 9h global running Windows Mobile 6.1 - 2.4 Inch display 320x240 - 120 MB RAM - Avast 4.1.19 PDA

That post was on 6\23\2010. Here we are on 7\5\2010, no response. No refund, and no functional AV on my phone.

I don't get it.

When did support stop being important to people that pay?

I have worked in support most of my technical life. I worked as a phone jockey for Sony desktop and laptop support (I was agent C02E way back in the day if you remember me :) ) and was held to EXTREMELY high standards (phone monitoring, customer surveys, technical information and documentation audits, etc...). I Left there to do engineering level support and technical training for a local company called Voice Technologies Group (VTG) that built interface hardware to allow systems like IVRs, Voice Mail, and latter on Unified messaging systems interface between servers and PBX systems. They were bought by Dialogic so I ended up working for them, they got bought by Intel, and then Intel sold them off to a company called Icon Networks from Europe that ended up putting the name back to Dialogic again. Every step of the way I was involved in the servicing processes in some way, weather through support (phone or on-site) acting as a technical evangelist at all the various trade shows, writing and presenting training classes, or even stepping in and helping customers design solutions, and then test them using our hardware\software. At every step of the way I was held to the highest standards of customer interaction. If I just let a support case lie dead for a week my butt would be in a sling and I would be getting emails from customers, CEOs, you name it.

Now, I will admit that that perhaps the support expectations for a $20 bit of software is slightly lower than a hardware card or IP gateway that runs between $2000 and $10,000, but the idea is the same. You have a paying customer that plopped down some cold hard cash for and has some expectations. with respect to how they are treated and how they are interacted with. I am at this point not even expecting to get it working. I KNOW that they are simply NOT going to address my issue only because of the fact that my phone (as I have been told so many times now) is outdated and has too small a screen. The software is built for an actual 'smart phone' with a larger screen and that is that.

Simply have the personal where-with-all to just email me back and TELL me that fact and then GIVE ME A REFUND.

At this point I am not sure if I would ever buy Avast again even if I did upgrade my phone to something newer. Why? Because the level of support I have received until this point has been abysmal. Why would I expect that to change?

Avast support: Suggestion - Give me a version of SW that works or give me back my $20. I am not going away and I can have a tendency to become very persistent. I have tons of experience knowing what gets attention from the customer side of a support case.

Architects, Developers, and Programmers - Oh my.

Ray Cassick — Thu, 30 Apr 2009 17:05:59 GMT

I catch myself correcting people all the time when these terms are used because not many seem to use them correctly, at least correctly by my judgment. I am going to speak my mind here, and put out into the public eye, what I think the difference is between each of these roles.

I say roles because they are not people. They can be people, they most certainly are almost always jobs within a company, but at the lowest level they are roles. Each of these is a pattern that a person has to fit into to serve that particular purpose. Multiple people can fill each of them at once, just like one person can fill several, but at any one given time a person fits into either one of them. Because of a persons experience and knowledge levels, as well as their underlying personality, they may be qualified to fill one of these roles or they may not. They may be good at one or not.

I am going to start by putting out a very crude diagram to show my personal view (perhaps rash generalization based upon my experience) of how these roles fit within a development hierarchy.

The first thing you will notice is that architects are on top and programmers are on the bottom with developers nicely placed in between. This is not out of any disrespect for either developers or programmers, but we must be honest with ourselves, there is a certain level of expectation between these roles that places them within a very specific hierarchy. Like it or not, professionally speaking, one step up and better than the lower. I use ‘better’ as a relative term here to mean more experienced, more accepting of responsibility, and shouldering more expectations. I know that sometimes programmers can feel the entire weight of the project on their shoulders, but in reality, if they are then someone above them in the hierarchy is not performing in their role properly.

So, how do I place these roles within this hierarchy? What criteria do I use? How do I measure the expectations?

Architect

This person(s) is responsible for the technical footprint of the solution. When it comes down to understanding how all the various piece-parts talk to each other, this person knows. When it comes down to understanding the difference between a clustered and a load-balanced set of servers, this person knows. When it comes down to understanding why clustering is better than load-balancing within the context of the enterprises architecture, this person knows. When it comes to understanding how a specific messaging architecture fits in the system, this person gets it. When it comes to understanding why it may be better to use a server with multiple physical CPUs vs. one with multiple processing cores, this is the guy to ask.

Can they do the work of everyone below them? About 80-90% of it, yes. Should they be responsible for doing low-level work within their project? I don’t think so. Why? Because for a really technical person that has to work at the implementation level it is very difficult to shift gears to a high level technical view and stay objective, to not select one method over another strictly on the merits of its contribution to the overall business need instead of what may be simpler, or cooler, to implement. If an architect is going to be required to actually do work on a project at a lower level then I don’t think it should be on their project. If they are going to switch gears then I think it should be a clean switch.

Architects have to not only be able to work at this high level, but they need to be happy working there. I have seen many cases where developers have been promoted to architect simply on merits such as length of service or their great ability to lead a team of developers and programmers, but be miserable wrecks when they reach the level of an architect because they miss the thrill of the compile. They need to be able to feel personal fulfillment by the act of a project coming together more than the rush of seeing a passing unit test. They need to be well with the fact that they made a good decision on what message transport they selected rather than feeling the high of spending all day working with WSDL and message versioning. They need to feel comfortable sitting in an ivory tower once in a while, even if those bellow them feel a bit off because of the view..

Developer

Developers are the top of the ‘do-er’ list. These people do the work. The build the systems designed by the architects and understand the low-level implementation details of HOW to build the stuff that was designed. You want to know the various methods available on an object? This is who you ask. You want to know how large an XML message is as it goes across the wire between servers? This person can answer that. Do you want to know how two objects connect and what the ripple effect of a change is going to be? Ask these folks.

Developers know it all within their areas of expertise. And to be honest, developers need to maintain a specific area of expertise because software development changes so darn fast that you cannot possibly know it all to a high degree of efficiency and knowledge. You can be very knowledgeable in a ton of areas, but when it comes down to knowing how the bits move in a specific way you need to really have a core set of technologies that you are great in. These folks need to understand how tools like UML help them and how they can hinder. They need to know the difference between book theory and implementation reality. They need to know that ‘pattern’ is not a magic word unless it can really solve your problem, and that OOP is not a mandatory way of life, but you better think at least a little before you decide that it isn’t. This role also understands why you should need a note form your mother to use a global variable in development, but also understand that doing so does not make you an evil Satan worshiper. Developers understand the reason that code comments are useful and that not every line needs to be commented.

Some people can feel confused and worried living here because they think that they need to know it all at a very low level. I think these people are best to live at one level lower, as a programmer until they get a level head enough to move a level higher as an architect, and they may actually end up being very good architects given enough experience.

Programmer

Beginner, Script-kitty, copy-paste-developer, these are the first words that come to mind when I think of this moniker. Don’t get me wrong, being a programmer is part of the natural progression of becoming a developer, and then an architect. Most of us learned to crawl before we could walk, and learning to write software is no different. Programmers understand the syntax, but probably not the reason behind using different patterns. They understand the idea behind separation of concerns and multi-tier development, but are probably not completely clear on the subtle nuances that can make it work well or bring a system down around their knees. They can debug most of the code they write, but get itchy when they have to read others code, or work on code that was written years ago but someone else. They also may not view the process of design, review, and code as having much worth and feel more comfortable by just sitting down with their beverage of choice and writing code to hit a mark. These folks maybe great at writing glue, the code that binds the ‘stuff’ of a project together, but they have not yet had enough experience to be responsible for all the low level details of an objects overall implementation. They are, the good ones, hungry for knowledge and want to learn as much as they can, but focus until they get closer to being a developer they are in an endless search for the silver bullet, the best way, the one true method that allows them to work efficiently and write the next killer bit of code. These guys comment their code because they are told that comments are good but for the most part it is feast or famine. They either comment everything or nothing.

So there we go. If I make it sound like one role is better than the other as in architects are just better people than programmers are, then please accept my apologies as that was not my intention. I think every one of thee roles is very important for a well balanced development team. Like I have always said, the world needs both planners and doers if it wants to get anything done. If there was no one to put their head down and code then it does not matter how good the design is, nothing gets done. So, if you are a programmer that is learning and growing, and understands their role and plays well there, then I say congratulations to you for being a necessary cog in the system. If you are an architect and feel that I am giving programmers or developers too much credit for their jobs then shame on you and get out of the industry because your attitude is getting in the way. Everyone has to start somewhere, it's a natural progression that everyone should go through.