Part 2 Getting Ready - 2.1 Knowing the Needs and Wants

2.1 Knowing the Needs and Wants

7. The Neighborhood Restaurant

I am sure you remember your neighborhood restaurant.  It is that tiny but lovely restaurant, which could barely accommodate not more than ten customers at a time.  It is that homely place, where you experienced the warm reception of the owner of the restaurant. She and her family members were the most active participants in that business. She was not at the cash counter all the time. She was a passionate cook.  She knew you and your family members by name. She knew what you wanted. She was flexible, fast and caring. She did more that the so-called order taking. She took orders, made suggestions and recommendations, connected with your family very well through her conversation and delivered more than what you wanted. Every time you visited that place, it was a delightful experience.  Sometimes, your parents took you there to celebrate your birthday!

Do you remember the neighborhood restaurant? I will take you there once more at the end of this chapter.

In our industry, requirement management continues to be one of the challenging areas. I have discussed this in some of my articles and blog posts. In this chapter, I have shared some more thoughts and ideas on this subject.

*-*-*-*-*

8. Can They Be Different?

Can they be different? I mean, a team or a person who identifies, gathers, analyzes, and manages requirements and a team or a person who delivers the outcome or product to customers – can they be different?

The answer is ‘May be. It depends.’ Let me change this question. Should they be different?

They should not be different.  That is what many of us would command. Why not? If you want to manage customer expectations effectively, they should not be different.  You cannot have business analysts to do requirements gathering and let developers deliver.  Why don’t you involve business analysts taste what is delivered and make them deliver?

If they are trained to do business analysis, I don’t think it is going to be an uphill task for them to deliver.

Remember the neighborhood restaurant!

In small projects, it can happen. What about large projects? Is that what comes to your mind?  Let me assure you, it is possible to deliver that experience in large projects too. We can do that by visualizing large projects as a collection of well-orchestrated small teams.

This is where the waterfall approach falls through cracks.  You need to embrace the paradigm shift. Alternatively, apply some Band-Aids.  Try to be iterative or agile.  Let your business analysts face your customers. Let them gather and refine requirements. Let them demonstrate and deliver the software your team has created.

This approach will open up new possibilities. 

By the way, what kind of role does your sales team play here?

*-*-*-*-*

9. Requirements Are Owned by Our Customer!

This happened when I was assessing a project that had more than fifteen team members. It was a long running maintenance project. I asked the project manager three fundamental questions. 

‘How do you manage requirements? When is the last time you reviewed requirements? What have been your findings?’

To my surprise, the response was fast.

“Requirements are owned by our customer. We start with design and move on to coding and testing. We do this iteratively and build this application.”

Obviously, that did not answer my question. I had to put forward some more questions, and listen curiously and concluded that nobody was managing requirements in that project and the review mechanism was very weak and insufficient.  My fellow assessors agreed.

When you hear someone say, “Requirements are owned by our customer or provided by our customer”, smile and think.

The team that develops and delivers software need to understand requirements. In addition, they need to review and manage requirements.  Never compromise.  If your customer’s business analysts are managing requirements, ensure that you have one or more business analysts who co-manage or collaborate with customer’s business analysts.

This approach will make you track changes, prioritize, negotiate and deliver the most important things first.

That your customer owns requirements does not mean that you can disown requirements! You go forward and own it or at least co-own it. That is it.  There are no other options!

Who owns requirements in product development organizations?  Product Manager. That is an easy answer. However, thinking deep, you would admit that the entire team that owns product development owns requirements as well. They need to be inquisitive, collaborative, involved, challenging and passionate in understanding what needs to be delivered.  Sometimes they learn from leaders who create a new market thru their innovative products and services.  Uber is a great example.

*-*-*-*-*

10. A Physician Who Can Hear You Well

This is about Dr. John, a physician who can hear you well.  One day you are unwell and you visit him. He wants to know what is bothering you. You explain him your state of health. He nods his head couple of times, does not ask you enough questions or engages in a conversation and gives you a shot, prescribes some medicine and moves on to the next patient.

Will you visit Dr. John again? Certainly not! 

Most Physicians actively engage with their patients. They ask questions.  They listen.  They rephrase the answers or ask more questions. They do this with curiosity and attention. Next, they examine their patient further before they write a prescription.

Even in this electronic age, you cannot raise a ticket in a tracking system about your symptoms and get a prescription. Of course, I am not talking about the usual flu, headache and such health issues. 

Think! Sometimes we do react as software engineers. We look at a call ticket assigned to us. We fix it and move on.   The next day, it boomerangs.  We do some rework. This does not lead to first-time-right fixes.  Let us spend fifty percent of our time in listening to our customers and understanding their needs. Designing, coding and testing come next. How else can we deliver the right thing?

*-*-*-*-*

11. Specifications:  In Which Format?

Years ago, the day before we started a new project, one of my team members got curious. He asked, “Are we going to receive requirement specifications from customer?  How will the specifications look like?  Let us make sure that there are no communication gaps.”

I responded, “In a document with text, and UML notations in our standard format. We will get wireframes too.”

We had a short pause.  I continued, “We will have a look at existing data and reports.  We will do our best to reduce communication gaps.”  He nodded silently.

I could read his mind.  We had serious gaps in understanding requirements in our recent project and the result was a huge schedule over run.  We wanted to do it better this time. 

Specifications come in what form?  Obviously in documents with UML diagrams, wireframes, etc.  When it comes to requirements, we cannot afford to say, “Let us not worry about form.  Substance is critical.”    

“The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems.  No other part of the work so cripples the resulting system if done wrong.  No other part is more difficult to go rectify later”, says Fredrick P. Brooks, Jr. in his book ‘The Mythical Man-Month (Addison-Wesley, 1995).’  He adds, “I would go a step further and assert that it is really impossible for clients, even those working with software engineers, to specify completely, precisely, and correctly the exact requirements of a modern software product before having built and tried some versions of the product they are specifying.”

Can Rapid Prototyping help?  Yes. It can help to a certain extent.  In fact, Rapid Prototyping can be done at the time of Requirement Gathering and Elicitation.   Rapid Prototyping tools are available in the market these days.

When we combine the power of such tools with the way we create software, using methodologies based on iterative and incremental delivery, we get an opportunity to deliver working software early and frequently.  Delivering working software early at regular intervals is a way to make your specifications powerful.  This is when you reduce the gap between what is specified, what is delivered and what is needed.

When you have dated specifications you need to get them validated. By this, I mean written text, UML notations or wireframes created more than a month or two ago.   When you do rapid prototyping (on a case-to-case basis) and focus on short-iterations to deliver working software you get an opportunity to demonstrate working software and get feedback.  You get to improve what you are going to deliver.

Above all, when you get specifications in the form of written text and written text only, beware!  Text-only requirements are raw.   They need to undergo analysis and refinement.  Analysis needs to result in a more-than-text-only form, which includes analysis models – such as use case diagrams, sequence diagrams etc.  You need to resolve conflicting requirements.  You need to negotiate, prioritize and validate requirements.

To know more, read my blog post titled ‘Requirement Engineering = Elicitation + Analysis and Negotiation + Specification + Validation’.

*-*-*-*-*

12. The Specification Myth

When I started my career in IT industry, one of my friends said, “You know, my cousin works for a large organization where developers are provided with detailed specifications. All they have to do is to produce code.”

I was thrilled to hear that. At that time, I was working on a medium size project. It was more or less like our neighborhood restaurant.  Some days we used to wonder how great it would be to get detailed specifications from someone so that we, the developers focus on coding and create bug-free code!

We waited for that moment. It never happened.  I have never heard it happening elsewhere!

After years of experience, I accepted the reality that it is a myth! In typical enterprise projects or software product organizations, detailed specifications do not come to developers on a sliver plate!

Detailed specifications are created in other ecosystems such as R&D establishments, government agencies, and large businesses with several compliance needs.  Even in those organizations, specifications get outdated unless there is a concerted effort in keeping them up-to-date.  In all other places, you cannot afford to wait for detailed specifications.  Also, you cannot afford to rely on detailed specifications.

Let us embrace the reality!  Reality can be messy at time. This applies to software development too.

Accept high-level requirements, create a prioritized list, organize white board sessions to refine requirements, and embrace iterative and incremental development.

*-*-*-*-*

13. Ten Key Things

Here are ten key things worth remembering.

1. Quality is the top priority and it is possible to deliver high-quality software.  In order to make this happen, business analysts need to focus on delivering high-quality requirements.  Obviously, both the form and the substance of specifications are equally important.
2. Understand the problem before specifying requirements.  When business analysts understand the problem accurately, it is highly probable that they are going to create specifications that are very close to what the user wants.
3. Use appropriate tools, models and practices.  It is necessary to understand the organizational culture of customers and be aware of what tools, models and practices have worked for them.   Tools, models and practices that provided results in collocated teams may not work well in case of distributed teams. 
4. Awareness, skills, techniques, and discipline are more important that tools.  The outcome of any practice is not determined just by the tools that you use.  For example, a plumber who is in-charge of your facility requires awareness of your facility.  He needs to be skilled.  He has to follow certain discipline and understand relevant techniques.  Without these qualities, if he claims his strength because of a powerful tool set he has, he is a dangerous plumber.  This applies to team members who participate in requirement engineering activities too.
5.  Establish appropriate measurement criteria.  How do you measure the progress of requirement engineering activities?  It is necessary to have appropriate measurement criteria.  Document exchange or data collection is not requirement engineering.
6. Establish an appropriate verification and validation criteria.  How do you verify requirements? How do you validate them? It is essential to have appropriate criteria for V&V activities.  Doing it right is more important that making it faster.
7. Good team and good management are vital to success.   Technology comes next.   Start-ups succeed when they have a good team that captures product requirements and a good management that fosters product development.  Technology is essential. More than technology, a good team and good management are important.
8. Understanding of customer’s priorities is paramount.  This is what helps the team understand priority requirements at early stages and find ways to deliver working software early.   The more the customers see, the more they will need.   Understanding customer’s priority is a way to deliver valuable features early.  Delivering early is the way to understand what else the customer wants.
9. Use the most efficient communication tools.   Information exchange over documents can lead to misinterpretation or inadequate grasp.  White board sessions, face-to-face meetings or video conferencing sessions are the effective ways of information exchange.
10. Ask questions.  While this is applicable for all areas of software engineering areas, it is very appropriate for requirement engineering.  Questions based on various aspects such as business need, functionality, scalability, performance, usability, test data, data volume, geography of users, compliance, etc. help business analyst understand requirements in detail.  Asking context-free questions are equally important to make a positive impact.  We will discuss more about context-free questions in this chapter.

*-*-*-*-*

14. The Weak Links

Most of us believe that Software Requirement Specification documents handed over to project teams enable them in designing, coding, testing and delivering the product.   We trust, in every project we get into, that the business analysts and user representatives collaborated well in order to create such documents.   We must be positive and think optimistically. We must trust.  We must believe.  However, our positive thinking, trust and belief can take us only so far unless we do our homework right.

Sometime ago, I was reading a paper titled, ‘What’s Wrong with Requirements Specification? An Analysis of the Fundamental Failings of Conventional Thinking about Software Requirements, and Some Suggestions for Getting it Right’ written by Tom Gilb. (Link:  http://www.gilb.com/tiki-download_file.php?fileId=443)   Tom is the author of nine books, and hundreds of papers on topics related to Software Engineering.  He has been a keynote speaker at dozens of international conferences.  He has delivered guest lecturers in universities all over the world.

In this paper, Tom has outlined ten key principles for creating successful requirements specification documents.  He has provided very good examples to illustrate these principles.   I found it very interesting as it helped me realize several weak links in the way we gather and specify requirements.  Here are my seven takeaways from Tom’s paper.

1. Check if requirements specification helps you understand the top-level project objectives.   Make sure that these objectives are not verbose and qualitative.  They need to be detailed enough but fit into a single page. Tom says that a good first draft of the top ten critical objectives can be made in a day’s work assuming that the key management personnel are available for discussions. The top-level project objectives remain weak links if you are not able to translate these top level objectives into success criteria that can help you measure the success of your project.
2. Understand the value delivered to stakeholders by means of satisfying the specified requirements.   More than the defined functionality or user stories, the value delivered is what counts.   Traditionally, we are not used to considering value when we think about requirements specifications.   The ability to prioritize requirements based on value delivery is critical.  Product Owner (Scrum), Business Analysts, Marketing Managers, and Product Managers need to focus on this function.  When you implement a requirement (or satisfy a requirement) validate if you are delivering value to stakeholders.
3. Quantify requirements.   Qualitative descriptions or textual representation of requirements does not help.   Quantification enables measurement.  Measurement is necessary if you want to improve.
4. Specify the value you want to deliver to stakeholders clearly. Do not mix this up with how you want to achieve the value (the 'design'). When you are able to specify your needs clearly, the solutions to your requirements will naturally follow.  Do not let the solutions change your real needs.
5. Instead of focusing on each functionality, focus on system quality.   Great products go to market because of this focus.
6. Make the specifications rich with several elements such as historical data, test data, scenarios, etc.
7. Inspections can help you improve the quality of requirements. So, consider Specification Quality Control (SQC). ( Link:  http://www.gilb.com/tiki-download_file.php?fileId=64  )  

Finally, requirements do change or evolve.  If you spend lot of efforts in hardening all requirements in advance, it is probably a waste of energy.  Embrace change!

When you read, Tom’s paper you will realize the weak links in the current state of creating Requirements Specifications.  Also, you will get an opportunity to learn from his experience and thoughts.

Can we ignore the suggestions presented in this chapter just because we write user stories? No. What is important is important irrespective of the way we attempt to deliver! There is no short cut!

*-*-*-*-*

15. Context-Free Questions

‘Exploring Requirements: Quality before Design’ by Donald C. Gause and Gerald M. Weinberg is a must-read for students as well as professionals.   In this book, the authors emphasize that project teams need to ask context-free questions in order to understand high-level details that are necessary to create the right product in the right way.  According to them, there are two primary categories of context-free questions:  a) context-free process questions and b) context-free product questions. Also, they discuss about the third broad category of questions called meta-questions or questions about questions. Context-free questions are applicable to any product design.     They consume less effort but provide an opportunity to obtain valuable information.

Context-free questions can be used in Software Engineering irrespective of the lifecycle methodology used.  Also, IT consulting teams can use context-free questions during portfolio analysis or assessment phase in order to better understand the customer requirements.

Context-free process questions relate to the process to be followed and the answers to context-free questions relate to the project or the lifecycle activities.  Context-free product questions relate to the design of the product and the answers to context-free product questions provide details on attributes, considerations or trade-offs related to design of the product.

Let me share few examples.  Most of these examples are based on my experience.

Examples of Context-Free Process Questions:

·         What are the success criteria for this project? Or how do we measure the success of this project?

·         What are the driving factors for executing this project with a collocated team (or distributed team) or a consulting partner?

·         How much time do we have for this project?

·         What is your trade-off between schedule and quality?

·         Where else can the solution to this design problem be obtained?

·         Can we copy something that already exists? Can we use open source?

·         Who are the competitors for this product?

Examples of Context-Free Product Questions:

·         What problems does this product solve?

·         What problems could this product create?

·         What are the operational environments for this product?

·         What kind of precision is required or desired in the product?

·         What are the interfacing standards to be followed?

Examples of Meta-questions:

·         Am I asking you too many questions?

·         Do my questions seem relevant?

·         Who is the right person to answer these questions?

·         Do you think our team needs to contact multiple members in your organization to get answers to these questions?

Context-free questions help us identify hidden assumptions. They help us identify the right approach to design products.  Also they help us design the right product.

*-*-*-*-*

16. Neighborhood Restaurant Revisited

What happens in small projects with teams of five to seven engineers?  Like what happens in our neighborhood restaurant, in small projects team members play multiple roles.   This is what happens in start-ups.  You need to be more than a coder to succeed in such environments! You need to learn business analysis. You need to master configuration management. You need to focus on quality and test your code.  You need to be fearless and have the ability to wear multiple hats.

Even in large enterprises, you will come across projects that resemble our neighborhood restaurant.  Software engineering ecosystems that mimic our neighborhood restaurant are exceptional and they provide you a unique experience.  You get the opportunity to interact with business users and develop requirements. Also, you do design, coding and testing.   Besides, you experience true DevOps where Agile development teams and operations teams come together to put together a tool chain with start-of-the-art tools.  You move the software system into production and resolve production issues with unparalleled efficiency and predictability.  This is where engineers groom to become FSEs or Full Stack Engineers and deliver value.

Those who carry this experience do well in larger teams too.  They are open and ready to perform overlapping roles. They are ready to exchange hats. They deliver.

Next time, when you get an opportunity to work in a small project, grab it.  That is a great step to enrich your experience and scale up.

When you happen to work in medium or large projects comprised of several team members, remember the neighborhood restaurant.  Staying close to your customers, understanding their needs, focusing on the deliverable, learning from customer feedback at regular intervals cannot be forgotten in such projects.  Agility is critical.

*-*-*-*-*

Part 1 Introduction                              Table of Contents                 2.2  Discerning the Magnitude