Career FAQs: Generative AI
This post is part of a series of posts answering the questions that people on LinkedIn most commonly ask about my career. The other posts in the series are on working at Google, product management, and accessibility. Each post covers what it’s like working in that area and tips on how to get into it yourself.
What’s it like working in generative AI?
Everything that makes working in generative AI feel exciting for me could equally feel maddening for someone else.
- The energy around generative AI is a mix of optimism, urgency, and hype
Projects have high visibility to leadership - What the tech can do keeps changing, which means projects keep needing to pivot
- Best practices and processes are nascent and evolving, so teams do a lot of improvising
- Figuring out whether a feature is likely to be launchable takes a mix of intuition, experimentation, creativity, risk tolerance, and good judgment about how people in the world will perceive the feature
It takes a broader, softer skill set and more improvisational mindset than for a regular engineering project. If building a regular feature is like building a house out of bricks, building a gen AI feature is like training a squishy, slippery alien critter who is enthusiastically helpful and also a little tipsy. If this kind of work doesn’t sound fun to you, it’s totally fine to go do something else.
On my first gen AI project, the team had already gotten the attention of executives with a cool prototype. When I joined, I evaluated the feature by running experiments, adversarial testing, UXR, competitive analysis, and risk assessment. What I determined was that we should not ship the feature. (The squishy, tipsy critter wasn’t up for the job.) I had to find a gentle way to break the news to the team and the execs, and strategize a new path.
On my second gen AI project, almost the same thing happened. This time, however, I was able to catch the issue much earlier, pivot the project, and launch a related gen AI feature within six months.
One reason this happens is because generative AI has made it very easy to build a good demo. It's easy to make something that works 30% of the time or 80% of the time. But in order to ship, depending on the application, it might need to work 99% of the time. Or 99.9%. If you're doing gen AI work right, issues of inclusion, representation, and safety come up all the time. Most of work is in the edge cases.
Part of what determines whether a feature is high enough quality to ship is whether it meets what people will expect from it. When there's a gap, one approach is to improve quality of the model. Another approach is to design and constrain the surrounding product experience to guide the user's expectations. As a product manager, I have more control over the latter. I get to know the squishy alien’s strengths and weaknesses, and then create the context where it can succeed in being helpful to people. That's basically what working generative AI as a product manager is like in 2024.
How can I get a job in generative AI?
Get to know the squishy alien critter. Experiment and see how it performs. Before I started working in generative AI, I started playing with Midjourney in my personal time. I got the idea to make a visual journal and started trying to generate one image a day that represented a feeling or moment from my day. It wasn’t a particularly ambitious or technical project, but it helped build my intuition for the strengths and weaknesses of these models and how prompting works. Then when I was interviewing for my first gen AI role, I had relevant experience to talk about. I felt a bit silly talking about my little visual journal, but it worked. So try a little project and see where it leads you.
Consider documenting your tests in a table: what you tried, what the response was, a little analysis of whether the response was good or bad, and what a better response could have been. Documenting experiments makes it much easier to see patterns over time. (As you may know, I’m a big proponent of running lists.)
If you don’t have a technical background, first get familiar with the technical vocabulary: prompt, training, inference, LoRA, fine-tuning, etc. Then work on discerning the difference between hype and reality around generative AI. When you see something that works well once, think about whether it’s likely to be high quality, fast, and cost-effective to run it for thousands or millions of people.
If you have a technical background, focus on learning about the social implications. How would an incorrect response make someone feel? What if that person was someone very different than you? How might society change if people used this feature very often?
Companies are rapidly expanding their generative AI teams, so a candidate with good intuition and judgment about gen AI should stand a good chance.
Best of luck!
