How AI Is Pushing Product Managers Beyond Their Role

The journey to product management

I've been a product manager for over 15 years now. Earlier in my career, I did everything else that I could, including design, sales, consulting, marketing, and even coding back in the PHP days — yuck.

I got into product management because I really enjoyed bouncing between various topics and interfacing with users, developers, executives, etc. I also had a pretty good eye for design and a good sense of product taste. After a lot of research, I found that there was a role that suited that really well — Product Management. So, I applied to my first PM role and that’s how I got here.

I’ve worked in all sorts of verticals: B2B, B2B2C, retail, and consumer software. I started teaching product management at General Assembly over a decade ago, and after that, I launched my own courses online independently with Udemy and, later, LinkedIn Learning. Today, I have over 1.6 million students combined across both platforms. And I’ve spoken at a lot of conferences about product and project management, including working with Google teaching a product course in Kyiv, Ukraine.

Currently, I’m working with Probably.dev, a deterministic data agent app funded by a16z. We’re a small team, so I’m not just a PM, but also an engineer, sales rep, designer, etc.

Our app is a native desktop app that does deterministic, PhD-level data science with supporting visualizations to directly answer any given natural language query without sensitive data leaving your machine. You can think of it as a "data copilot." We work directly on top of data warehouse schemas (e.g. BigQuery, Snowflake, Clickhouse, etc), as well as with local files like CSV, JSON, and Parquet. All of the answers our agent provides are traceable and reproducible by a human, so typical LLM hallucinations aren't an issue. It's not just another SQL generator agent, and we can handle billions of rows/columns and complex schemas with tens of thousands of tables.

How AI helps teams get more done per person

AI has had a big impact on my work. I used to be "just a PM." That sounds silly because PMs do so much, but now I'm able to contribute to anything else now, like engineering and sales. And that helps expedite the team. With AI transforming project management, the traditional role boundaries are becoming more fluid.

Time-intensive tasks like research and analysis are much faster, and the time savings allow for much more face time and comms with stakeholders. It also allows for faster decision-making.

Much of the job of a PM is to ensure everyone else can get their jobs done effectively, so with me being able to do even more myself, we're able to get much more done per person. This human-AI hybrid approach is becoming essential for modern delivery teams.

Why a deep understanding of AI lets product managers do almost anything

The more I get into it, the more I realize how high the learning curve for effective and efficient AI/LLM use actually is, as well as the sub-skill of getting it to fit the workflows that people around you or on your teams prefer. This further reinforces my belief that every person at a company must be intimately familiar with not only LLMs in general and the models available, but also with the multitude of ecosystem tools around them like MCPs, modalities like CLI, GUI, and most importantly, the transformer architecture. For those looking to deepen their understanding, exploring AI project management books can provide valuable insights into implementing these technologies effectively.

Working well with AI in the vast sea of AI slop these days, whether that be content or tools, is a skill that you have to hone, and that can only be done with experience. You've got to be able to understand how LLMs work and what they are or are not capable of doing well to get the best out of them and perform optimally.

Many “groundbreaking" SaaS products and tools out there seem to be good solutions — making PRDs for example — but I've found that 99% of everything you need to do can be done yourself cheaper, faster, more customized, and vastly better. That is, as long as you have an understanding of the above concepts and the skills to use LLMs effectively.

It's like those infomercials where there's a person doing something simple, then they show how "difficult" it is, and they try to sell you an entire device to solve that tiny use case? Like instead of cutting a banana up into pieces with a knife like a normal person, they want to sell you a banana shaped slicer that cuts all the pieces in one motion...

The knife is an LLM and the banana slicer is a junk niche app. Learn to use the damn knife. Buy a good quality one, use it, learn it, and stop shopping for stupid stuff because tech Twitter tells you that some random SaaS app just "made designers obsolete" or "killed coding."

There’s basically nothing you can’t do with an LLM in a CLI, web browser, and every so often an open-ended task-agnostic automation tool like n8n.

Save your money and learn how to build things that reflect the wisdom of your learned experiences instead of accepting the output of an overnight popup SaaS tool.

The only time this wouldn’t apply is for single-purpose tools that focus specifically on solving problems for which the transformer LLM architecture in vanilla frontier models (i.e. not fine-tuned) is inherently flawed. For example, large-scale data science or mathematics.

How to get started learning with AI as a product manager

The best part about LLMs is that you can ask them how to learn about them. If you're a visual learner like me, YouTube is the icing on the cake, there's a ton of good resources on AI and LLMs there.

If I were starting over, I would approach learning like this: Tell an LLM you want to learn about everything in the AI/LLM space adapted to your level of technical understanding. You can even instruct it to ask you progressively more technical questions in order to understand your current level of proficiency for anything in the AI/LLM space, starting from the fundamentals. And when you reach a point where you're out of your depth, you can have it recommend topics to learn, from the basics of the transformer architecture all the way to how an app like ChatGPT works.

The idea is to get a list of topics to learn, and then search each one of those on YouTube to find a video to understand each topic.

Here's an example prompt:

Act as my AI/LLM “starting point” assessor.

Ask me up to 12 questions (mix of easy + technical) to gauge:

• my general technical comfort (coding, data, APIs)
• my understanding of how LLMs work (model modalities like CLI, GUIs, as well as tokens, context, hallucinations, etc)
• my familiarity with modern LLM app patterns (RAG/embeddings, agents/tool-use)
• my ability to judge quality (evals, verification)
• my awareness of risks (privacy, prompt injection, etc)
• my awareness of AI limitations (good vs bad use cases for LLMs , accuracy, uncontrollable nature of LLM outputs, etc)

After I answer, output ONLY:

• 5–10 bullets of the highest-leverage YouTube topics I should learn next, ordered by priority. Each bullet must be a YouTube search phrase (not a sentence) + a 5–10 word “why it matters”.

Start with your questions now.

What AI skills product managers should learn first

In my opinion, understanding where and how you can use LLMs is important. By that, I mean understanding that they can be in an app vs in your computer terminal vs in code operating in an automation via MCPs or APIs.

Secondarily, you must understand that they are:

1. Nondeterministic, uncontrollable and prone to mistakes
2. Never to be relied upon for accurate information, especially with critical decisions
3. Able to be automated creatively to reduce the effects of the prior two points, depending on your use case

The classic example I give is to never use an LLM to write an essay for you entirely, but it's totally fine to ask it to list some ideas of directions to go if you're stuck on what to write next. In other words, LLMs are great supplements to the human mind, but it's a dark road to go down when you try to have them do all of the thinking in place of the human mind.

The last thing I'll mention here is that, while you're learning the above concepts, you need to be trying those things yourself. If you're learning about how prompt engineering works, for example, you should try the same question to an LLM phrased two different ways to see the difference in the output. Never learn without practicing the learning at the same time.

Why you shouldn’t automate high-value customer interactions with AI

Once you learn what you need to know, you can do just about anything. But that doesn't mean you should.

For an early-stage company with a premium product, customer interaction and sales "seem" to be most ripe for AI and automation. But in practice, I’ve found that a human touch is absolutely necessary. Personally, I find the interaction with users or potential customers to be a major value add on both sides, and I don’t even want to try to automate that.

As a PM, you’d be doing yourself a disservice to distance yourself from talking to users to any degree at all. The more you personally chat with them with your own voice, the more information you gain, and the more trust you engender.

Why it's hard — but possible — for product managers to contribute to code

So, I'm cautious about what I use AI for, but I use it a lot.

For example, right before a release on prod, there's always a bug-fix frenzy. Typically at a larger company, the engineers and QA team would be doing these types of final touches. Of course, the senior product person would be in the loop, but they’d also be planning out next steps and strategies for monitoring that release, collecting user feedback, prepping several cycles ahead, or collecting metrics once the feature or product is launched.

Since I'm currently at a smaller company, a game-changer for me has been the ability to help crush bugs along with the engineers, and I've only been able to accomplish that via the usage of AI/LLMs.

I've always been involved in QA on products and features in my career, but rarely have I been able to get in the trenches and fix a bug that involves code changes through the full stack.

Veteran tech folks will understand that engineers are very particular about their codebases and coding standards. So, even if you technically know how to fix a particular bug, you also need to have the context of the coding standards and the personalities on the engineering team, as well as understand the engineers who have worked on each portion of the code. So, fixing some bugs is several degrees away from being easy even if you know how to, and most engineers would scoff at your pull request for the fix.

Thanks to AI, though, I’ve been able to not only contribute to the code by pushing straight to main, but also remedy the secondary cultural factors involved.

How to build an AI agent that can safely contribute to your codebase

My initial approach was to establish a coding standards Markdown document by having an agent swarm in Claude Code explore the entire repository for the previous six months and absorb the style of coding, the repo Claude.md file, and most importantly, all comments on GitHub diffs, conflict resolutions, commits, and edits to PRs made.

From there, I had a pretty solid coding standards document which I augmented by turning it into a set of skills and commands in Claude Code that would take any code that I generated for a bug fix and rectify it against the coding standards the team had already implemented in the codebase. Then, I created another sub-agent that would use hooks with GitHub that I could also manually trigger to look at the coding I had done as well as the comments or fixes, and then assimilate all of that into an addition to the coding standards document.

Eventually, after only a few PRs, the code that I would generate for a given fix could very easily be translated into the type of code we like to see in the codebase via a simple Markdown document that was essentially a continuously improving and self-learning coding style guide triggered through multiple cron jobs and automations with Claude Code.

The agent is so good now that I'm able to merge straight to main.

How small product teams can increase agility

That's a good example of how I'm going beyond the traditional PM role. But going beyond traditional roles requires good processes.

We have a well-groomed and tagged backlog in Linear, and we do mandatory meetings on Monday, Wednesday, and Friday mornings. On days without a sync meeting, we keep up to date with a slack standup check-in. All other prioritization and discussion happens in real time via slack.

This may not be the best solution for larger companies, but you'd be surprised how well it works for this use case and how agile it keeps us. I used to use Google Drive, spreadsheets, documents, slides, and a million other apps. Now, as a small team, we can get by with GSuite, Slack, Linear, and GitHub.

Lightweight PM methods are important in this constantly-evolving AI/LLM space. Something could happen any day of the week with user feedback or releases from potential competitors that we'll have to keep on top of and prioritize around rapidly.

How AI is forcing product teams to rethink rituals

When it comes to rituals, the biggest rethink is that it truly is a circular, never-ending ritual now, rather than something with stagnating documents you’re continually having to create and reference.

Scope starts as a tight Linear ticket with variable degrees of detail, then I use AI to stress-test it for ambiguity, edge cases, and sequencing options, and I tighten until it’s shippable in the smallest slice.

For us on a small remote team, alignment is mostly async — a short Slack thread that ends in a clear decision and next actions, with lightweight standups handled by Geekbot. However, there’s still immense value in three or four team meetings per week to riff together and build morale.

Validation is where I’ve gotten more stringent, not looser. LLMs typically generate convincing garbage if you don’t know how to use them correctly, so every feature or app change gets a small evaluation harness — realistic inputs, expected outputs, adversarial cases — that AI can help draft but I will curate and test.

Execution is then just Linear + Claude Code + self-hosted Posthog analytics. Ship, observe, adjust, and keep the loop fast because the tech market is changing at a pace that seems like it’s hourly these days.

Why a minimalist AI-first tool stack is best

So here's my basic PM stack:

• GSuite
• Slack
• The Geekbot Slack bot (for standups)
• Linear
• VSCode
• CLI-based LLM (Claude Code in our case)

Why Linear? It is bare essentials. It's powerful enough to meet nearly any demands a project team needs, but simple enough to be approachable. It's the anti-JIRA. And most importantly, it's got easy integration into GitHub and Slack, which is where engineers work to begin with. It's simple, lightweight, and adds minimal additional friction for engineers to keep up with tasks or update them. It's also extremely extensible so you can do some cool stuff like have your LLM in your CLI automatically try to fix a bug when it's reported in a draft PR — before a human even begins to look at it.

Nothing else is needed. Any kind of work needed outside of this can be done with open-source tools in the terminal.

Cole's Favorite AI Tool for Personal Productivity

On a personal, non-team level, I'm absolutely obsessed with Raycast for productivity. It's the ultimate quick access, fully customizable, ultra-lightweight app that replaces the terrible "Spotlight" app for Mac or the "Windows" key in Windows. You'll decrease your usage of your mouse by 75% for most things. You can integrate nearly any app with their existing integrations or create your own custom ones easily — just ask Claude Code to make an extension for you!

If I want to get the text of an image I see onto my clipboard, I hit the raycast shortcut buttons, type "OCR", and hit enter. That triggers the cleanshotX OCR screenshot tool and I've got the text of any image in less than five seconds. Similarly, I can hit the shortcut, type "Ask spotify", and then type a natural language sentence like "give me some good coding music that is high energy," which will interpret my sentence through an LLM, search music tags and properties in the Spotify API, then start a playlist of the kind of music I just requested instantly in Spotify.

The use cases are endless. Just download it, activate AI through either your own API keys or their subscription, and go through the top extensions on their website to find stuff you think would be useful for you. Total game-changer.

Why product management and engineering is beginning to overlap

Soon, most feature engineers will be forced to start to think and make decisions like product managers, and product managers will need to learn enough about machine learning, neural networks, LLMs, and various architectures to get them as close as they can be to being an engineer using LLMs without actually being one. This transformation reflects bold approaches to AI integration that are reshaping traditional roles.

Obviously, there are many caveats and areas where this won’t apply, but in general, the Venn diagram of all previous roles in a company, from sales to marketing to product to engineering, will inevitably close in on one another, and very little in terms of skills will be exclusive to one particular role.

How to avoid the inevitable existential crisis as AI reshapes product work

Here's my advice: Try to avoid the inevitable existential crisis you’ll be tempted to have as this era of technology continues to evolve.

Instead, focus on the skills your mind has that you’ve developed through experience. LLMs can generate content, but they really suck at having good taste.

Answers are a dime a dozen now, so asking unique, experience-specific questions to AI is a skill now. Moreover, discovering the right question to ask in your own mind is the true intellectual work.

Follow along

You can follow Cole on X and his personal website, as he continues to challenge what's possible for PMs. And check out Probably.dev!

More expert interviews to come on The Digital Project Manager!