4 Ways How AI Product Management is Different from Conventional PM

There are 4 key differences between AI and traditional product management.

1. Increased Stakeholder Involvement

In traditional product management, a product manager typically coordinates with a range of stakeholders such as engineering, customers, customer success, marketing, and sales. This multi-faceted interaction forms the foundation of product development. However, when managing AI products, the number of stakeholders increases, adding layers of complexity to the process.

Core Collaboration: Product Management and Engineering

At the core of product development are the product management and engineering teams. These two teams work closely to transform product requirements—whether through documents, user stories, wireframes, or UX mocks—into a functional product. This iterative process involves continuous feedback and refinement to ensure the product meets its objectives.

New Stakeholders in AI Product Development

AI product management introduces two additional critical teams:

(i) Data Sourcing and Annotation Team:

Role: This team is crucial for collecting and labeling the training data needed for AI models. For instance, creating a classifier to detect cats and dogs requires gathering and annotating thousands of images. The complexity grows if the classifier needs to recognize various animals, necessitating a vast and diverse dataset.

Challenges: Ensuring data accuracy and relevance is vital. This involves meticulous annotation and handling large volumes of data, whether it’s images for computer vision models or transaction data for fraud detection systems.

(ii) Data Science Team:

Role: This team utilizes the sourced and annotated data to build and refine AI models. This iterative process requires continuous model evaluation and adjustments based on performance metrics.

Challenges: Effective model training demands not only data but also significant computational resources and expertise in selecting suitable algorithms and tuning hyperparameters.

Evolving Role of the Engineering Team

The engineering team, while continuing to build and deploy the product, now also integrates AI models developed by the data science team. This integration adds another layer of complexity to the development process.

Increased Coordination and Longer Lead Times

With these additional teams, the coordination efforts of a product manager increase significantly. More time is spent aligning these diverse groups to ensure seamless product development. This extended coordination results in longer lead times for product iterations. Feedback requiring changes to AI models sets off a series of steps: gathering new data, updating models, and then integrating these models into the product, which extends the development cycle.

In smaller companies, these roles might be combined within the same team, but the complexity remains, necessitating careful management and coordination.

2. Dealing with Ambiguity

In AI product management, ambiguity is a more significant challenge compared to traditional product management. This inherent uncertainty requires careful consideration and a strategic approach to decision-making. Here’s a closer look at how ambiguity plays a crucial role in AI product management.

Assessing the Need for AI

The first step in AI product management is determining whether AI is necessary to solve a particular business problem. Often, simple business rules or common sense can address the issue without the need for complex AI solutions. For instance, optimizing a web form by not asking irrelevant questions based on previous answers can be done without AI. Understanding when to leverage AI and when to rely on straightforward solutions is crucial to avoid unnecessary complexity.

Science Problems and Accuracy

Once the decision to use AI is made, addressing the science problems becomes essential. AI models strive for high accuracy, but achieving this involves balancing conflicting priorities. For example, in credit card fraud detection, it is vital to decide whether it is more important to deny fraudulent transactions or to avoid denying legitimate ones. These opposing goals require careful thought because the choice influences how the AI model is trained and tuned.

Defining Acceptable Accuracy Levels

Determining acceptable accuracy levels involves making trade-offs between cost, development time, and business impact. High accuracy comes at a cost, whether in terms of computational resources, time spent developing the model, or response time to customers. Product managers must decide what level of accuracy is acceptable. For instance, is it acceptable to miss one fraudulent transaction in 100,000 or in 10 million? Each level of accuracy has different implications for the model’s complexity and development cost.

Applying Business Judgment

AI product management requires strong business judgment to understand these ambiguities. Balancing the need for high accuracy with the practical constraints of time and resources is a constant challenge. Sometimes, it may be better to deploy a less accurate model quickly and improve it over time, rather than striving for near-perfect accuracy from the start. This iterative approach allows for continuous improvement and adaptation based on real-world feedback.

3. Watching Out for Bias

Bias refers to anomalies in the output of machine learning algorithms caused by underlying assumptions or prejudices in the training data. Understanding and mitigating bias is critical for effective AI product management. Let’s delve deeper into this important aspect.

Defining Bias in AI

Bias in AI refers to discrepancies in the model’s behavior that can lead to unfair or inaccurate outcomes. These biases may arise from the training data or the algorithmic design process. It’s essential to recognize and address biases to ensure that AI systems produce equitable and reliable results.

Examples of Bias in AI

To illustrate the impact of bias, consider the following examples:

(i) Facial Recognition Models:

Suppose you’re developing a facial recognition model, but the training data primarily consists of faces with dark skin tones. When deployed in the real world, the model may struggle to accurately recognize faces with lighter skin tones, leading to errors and misidentifications. This highlights the importance of ensuring a balanced dataset representing diverse demographics to avoid bias.

(ii) Skin Cancer Detection:

In another example, biased training data can lead to incorrect outcomes. If the training images for a skin cancer detection model predominantly feature rulers alongside cancerous lesions, the model may inadvertently learn to associate the presence of rulers with cancer, rather than focusing on the actual characteristics of the lesions. As a result, when presented with images without rulers in real-world scenarios, the model may fail to detect skin cancer accurately.

Mitigating Bias in AI Products

To mitigate bias in AI products, it’s essential to start by understanding the business problem and examining the data collection process critically. Balancing the dataset across various demographics, ethnicities, skin colors, and age groups helps prevent unintentional biases from creeping into the model. Additionally, rigorous testing and validation procedures can identify and rectify biases before deploying AI systems in production environments.

4. Feedback Loop is More Critical

Let’s explore why feedback loops are even more critical in AI product development and how to effectively incorporate them into the process.

Understanding the Significance of Feedback Loops

As a product manager, you’re likely familiar with the importance of feedback loops in iteratively building great products. However, in AI product management, where machine learning models are probabilistic by nature, feedback loops become even more vital. These systems rely on continuous feedback to improve the quality of their outputs over time.

Embedding Feedback Loops in Customer Workflow

One effective way to gather feedback is by embedding feedback loops directly into the core customer workflow. For example, consider the reviews and ratings page on platforms like TripAdvisor. By incorporating a simple “like” button, users can provide real-time feedback on reviews they find helpful or informative. This feedback is invaluable for refining the underlying algorithms and enhancing the overall user experience seamlessly.

Encouraging Data Sharing with Privacy Protection

Another strategy involves encouraging customers to opt-in to share data voluntarily. While data collection is crucial for refining AI models, it’s essential to prioritize user privacy and adhere to data protection regulations. For instance, many mobile apps prompt users to share usage patterns to improve the app’s functionality. However, it’s crucial to transparently communicate the purpose of data collection and ensure compliance with ethical and legal standards.

Balancing Effectiveness with Ethical Considerations

While feedback loops are instrumental in enhancing AI products, it’s essential to implement them responsibly and ethically. Striking the right balance between improving product effectiveness and safeguarding user privacy is paramount. As product managers, it’s our responsibility to ensure that feedback loops are implemented in a manner that respects user rights and aligns with ethical standards.

AI product management is like exploring new territory in managing products. It brings exciting chances to innovate and change things up. Product managers need to understand the special challenges that come with managing AI products. With this knowledge, they can confidently tackle these challenges and look ahead to what’s coming. Embracing AI in product management means embracing the chance to create products that make a big impact and do good for society, all while being fair and ethical.