- Any Questions?
- 080-4710-6006
- [email protected]
By Vamsi Krishna Vutukuru – Head of Product and GTM, AWS Computer Vision, Amazon Web Services
Today, with the help of evolving technology and market, the role of a product manager has become more diverse with the integration of AI. So what is different in building AI products? It’s mostly the same. To do good in AI product management, you need to be a good product manager, in general. You still need to do all the things that a typical product manager does in traditional product management- which is product requirements, voice of customers, problem solving, product positioning, UX design, go to market strategy, etc. This does not go away and continues to be important. But on top of this, there are some key differences that exist between traditional product management and AI product management.
The first difference is you as a product manager will need to work with more stakeholders if you deal with AI products. Let us take an example. Take a look at stakeholders you work with as a regular product manager. These are engineering, marketing, customer success, sales, business development, customers and so on. This gives you an overview of how complex the product management role can be. Let us focus more on product management and engineering because these two are the teams which work together to actually build the product.
In a typical product, product management along with UX are used to define requirements. This can be product requirement documents, user stories, or wireframes. These requirements are provided to engineering and you iterate with them over a period of time to actually build and ship the product. This is how the traditional product life cycle works.
What’s different if you are building AI products is that there are two new teams who enter the mix. The first team is data sourcing and annotation. Let us say that you expect this team to detect cats and build a classifier for all animals in the cat family. There are 100-200 animals in the cat family. You have to get imagers for each and every cat type like tiger, lion, panther, Bobcat and so on and then source or annotate them.
Another example is credit card fraud detection, it will be in terms of getting data across hundreds of transactions which happened previously.
So there is an entire role in this team which is just focussed on how you source data, and this data can be in thousands or millions of imagers.
The second team is the science team, which essentially takes this data and builds the models. This is a continuous process. You build the models, you see if the models are working fine, you may ask for more data and so on.
Finally, the engineering team essentially takes in science models as the input and builds the overall product. So the complexity of building the product goes up significantly. This means that if you as a product manager are spending time right now with your engineering team, to actually work with them to ship a product out. Your coordination goes up and this is very critical to do this right. It also means that you have a longer lead time for product iterations. If you get feedback and you want to change something in the product, especially if it impacts the AI models, then again you have to get involved with the data process. In small organizations, you might not have separate teams for these purposes. But the roles do exist.
Fundamentally, product managers deal with more ambiguity in AI product management as compared to other products. There are three aspects that need to be taken into consideration.
The first aspect is that you need to take a step back and understand whether AI is actually needed to solve the business problem or not. Take an example of a web survey form where you are onboarding someone into health insurance. Typical question for a product manager would be- how do you optimize this form so that you can reduce the number of questions you are asking and get more people to answer. Let us take two questions from this form- (i) What is your gender? (ii) Are you pregnant? If you want to optimize this web form, one way is to use AI. After reading multiple customer responses, the AI might say the answer to question (i) is male. So, it makes sense to not ask the next question (ii). But you don’t need AI to tell you this. So the point is, don’t use AI, when simple business rules can solve the problem.
The second aspect is, if you have decided to use AI but don’t know how to go about the process. We know that all AI models want high accuracy. What does it mean? Let’s take an example of credit card fraud detection which is a very common use of AI. So if you want to improve fraud detection, what is more important? Is it important to deny a fraudulent transaction or is it important not to deny a legitimate transaction. These two are opposing forces. This is actually a difficult, real problem and is what the science team will ask you. Because based on the response, they will tweak your model accordingly.
The third aspect is cost. Suppose you say that, you don’t want to deny a legitimate transaction, but even there, we have a measure of accuracy. So the next question is, how do you measure accuracy and what level of accuracy is okay from a business perspective. Note that increasing accuracy has a cost. It has a cost in the amount of time in which you build the model, the cost in terms of money, or time taken to respond to the customer. It is important for product managers to understand if they would be more okay if they missed one fraud in 100000 transactions or missed one fraud in 10000000 transactions. In terms of cost, the first option should be preferred because the second one comes at a very high cost of time and money.
These are the questions that you have to answer for science because scientists, just like engineers, tend to think very precisely and unless you provide clarity, they can take a lot of time solving the wrong problem.
AI bias is an anomaly in the output of machine learning algorithms. These could be due to prejudiced assumptions made during the algorithm development process or prejudices in the training data.
Let us take a look at a few examples to know what it is and why it is important.
Say, in AI you have a training phase and your objective is to create a face recognition product. Let’s take a training set, in which you only have dark skin tone faces. If you put this model in production, the moment a light skin tone face shows up, your model will get confused. It may still output the correct result, but then, as more and more light skin faces show up, the model starts misbehaving and starts giving errors. This is a very classic problem in face recognition which is difficulty in ensuring that your data set is balanced across demographics, ethnicities, skin color, age groups. So as AI product managers, it is crucial to start solving the business problem from backwards.
Another funny, yet common example is where people want to detect skin cancer. In one particular situation, the images that were given as a set of the problem had rulers, to measure the size of the tumor. But if an image without a ruler was given, AI could not detect it. This happened because the model was always correlating the ruler with the skin cancer image and when the ruler did not show up in the image, it did not detect. This is another example which shows how imbalance in the data set can show wrong outcomes.
Machine learning is probabilistic. Feedback and customer data are critical to improve model performance. You may build something assuming certain data in mind, but in the real world you may see that the data is actually different. So how do you embed feedback loops? One way is, you actually get this feedback by embedding a feedback loop in the core customer workflow. For example, platforms like TripAdvisor utilize user feedback through features like the ‘like’ button on reviews, creating a continuous loop of user-generated data that contributes to model enhancement.
Another way to do this is, try to get customers to opt in to share data, because getting data is critical. But at the same time it is important to protect privacy. Product managers should be very aware and conform with the data privacy regulations.
While the majority of the fundamentals of product management are applicable in AI product management, there are still quite a few differences between the two. These differences exist mainly in the domain of number of stakeholders, higher ambiguity, AI bias and feedback loop. Strategically navigating these differences ensures that product managers can thrive in the field of AI product management.
AI product management involves solving customer problems using data enabled by artificial intelligence and machine learning.
In traditional product management, product behavior is usually binary and predetermined. AI product management deals with probabilistic outcomes.
Challenges associated with AI product management include working with more stakeholders, ambiguity of outcomes, difficulty in explaining the rationale behind the outcomes, addressing fairness and bias concerns, adapting to new infrastructure and tools, and selecting the right problems to solve with AI.
It is highly unlikely that AI will fully replace product management. AI is used to assist product managers and enhance aspects of product management, like data analytics and customer support. But it cannot completely replace a product manager.
About the Author:
Vamsi Krishna Vutukuru – Head of Product and GTM, AWS Computer Vision, Amazon Web Services
