Everything I learned from getting roasted by our users

You’re reading Release Notes, a weekly newsletter covering product lessons from scaling beehiiv and building AI products in the creator economy. (Read the full archive here.)

If you're new here, I'm Jake, a product expert in the creator economy, currently Head of Product at beehiiv.

There’s nothing more humbling as a PM/founder than getting roasted by your customers online. And last month, we got roasted A LOT.

Let me give some context:

Over the last 12 months we’ve invested in expanding the beehiiv product suite beyond newsletters to include an advanced Website Builder, Advanced Automations, Digital Products, Podcast hosting and most recently Community.

And while that has allowed us to scale revenue from $20M ARR to $32M ARR in 12 months, maintaining an incredible product experience across such a wide surface area (whilst also expanding that surface area) has its challenges.

Namely, where do you invest your product and engineering time, and how do you balance your focus across new vs old features?

And as any product person who has attempted an all-in-one strategy will tell you, the biggest mistake you can make is thinking your core features are ‘good enough’ and focus elsewhere. Cue some roasting…

This week:

[sponsored]

You don't send a colleague a three-word brief. You explain the context, the constraints, what you've already tried. But typing all that into ChatGPT takes forever — so you don't.

Wispr Flow lets you speak your prompts instead. Talk through your thinking naturally and get clean, paste-ready text. No filler words. No cleanup. Just detailed prompts that actually get you useful answers on the first try.

Millions of users worldwide. Works system-wide on Mac, Windows, and iPhone.

Try Wispr Flow free

Let me set the scene, it’s the start of July, we’re one week out from our Summer Release Event (our biggest feature release of the year), the entire team has just spent 5 months building the SRE features,  the roadmap is locked in for Q3, and we feel like we know where we need to invest over the next 3-6 months.

Then Tyler (beehiiv CEO) drops this on X/Linkedin:

Now, we are constantly speaking to customers, reviewing support trends, looking at user behavior, so we felt confident we knew what to expect from the feedback.

However the framing of the question was important: “What is the worst part of the product?”

The feedback was overwhelmingly clear (and did not match what we thought it was going to be):

Seeing the Website Builder and Ad network, two of the most important pieces of the beehiiv ecosystem, being highlighted so prominently as the worst part of the product set off some serious alarm bells.

To be clear, all the specific feedback had been heard before for both features. However, both products had arguably fallen into the ‘good enough’ trap (getting regular small improvements but never claiming the P0 spot for the business).

The alarm bells forced us to rethink whether our prioritization for Q3/Q4 was correct. (Spoiler, it wasn’t - roadmap pivot incoming).

Fast-forward a week after Tyler’s post and we go live with SRE.

Thousands of people tune in live to watch the stream on Youtube and be the first to hear what was coming out. 

During the stream the live chat was pumping non-stop and for every announcement we got an immediate (and often extreme) reaction from the audience. If you’d built the wrong thing, they’d tell you.

Fortunately we’re very intentional around what features make it into the SRE. So it was packed with wins for our core email-first customers, with a huge update to the email editor and improvements to subscriptions, paywalls, MCP and more. 

There was still plenty of constructive feedback and gentle roasting, but overall it was positive.

What came next however, was a career-first for me: having industry experts immediately jump on a podcast to discuss and dissect the announcement. 

Having 3 people you deeply respect independently analyze every decision you’ve made over the past 6 months, and then sharing it with the world, definitely takes some getting used to!

So far I’ve seen five reaction podcasts like this and while we typically come out with a very positive review, they usually include some pretty damning critiques around specific parts of the product and provide some valid challenges to our strategy.

In short, they are a pretty humbling watch if you’re the PM who was involved in the product.

Like all feedback, however, these reaction videos are incredibly useful and importantly act as a stark reminder of how far we have left to go.

We pride ourselves in building for our users and have no problem admitting when we’re wrong. Based on all the feedback, it was clear we were about to spend the next quarter improving the wrong things.

Importantly, this wasn’t a case of us questioning our decision making and roadmap just because of some customers asked for things (that is a bad precedent to set), this is because Tyler’s Linkedin/X post highlighted that some of our assumptions that underpinned the roadmap were wrong.

Within 2 days of the linkedin post going out, we’d identified all the highest-leverage parts of  the website builder and the ad network that needed to be addressed, rebuilt the Q3 roadmap, and shuffled the engineers to bring it to life.

The most important change was shifting how we handled design resources:

 In the past, our two amazing product designers (Sisi & Priyanka) would always be working across multiple products at once, constantly context switching between un-related features. 

The website builder is an incredibly complex product to build, and switching in and out to design specific web features / UX wasn’t going to work. This needed far more attention to ensure that everything we change in the website builder had continuity and could genuinely be best-in-class.

So in addition to shifting engineers around to tackle web builder initiatives and bumping lots of other projects to Q4, we also allocated one of our designers 100% to website builder for Q3 to make sure we 10X this experience in the next 90 days.

Changes like this have plenty of side-effects - in this case, the main risk is the inevitable design bottleneck for other parts of the business that this will create. However, this is a necessary evil to allow us to solve big problems like this.

My last thought is highlighted by this post from Jason Lemkin (Investor & Founder of SaaStr). It illustrates one of the biggest problems that product teams are facing right now.

In today’s world of AI, companies are going through the biggest workflow re-orientation since the introduction of computers.

Every single tool is being re-evaluated internally, and products are rapidly becoming either redundant (replaced by AI workflow) or not fit for purpose (eg not having an MCP).

If getting roasted is a proxy for anything, it’s that your customers actually care about what you build.

And in a world where the biggest risk for a product is becoming irrelevant to their users, a gentle roast every now and then is something you should look forward to.

Before you go, I have one favor to ask:

Despite a small hiatus (sorry about that) we are incredibly close to hitting 2k subscribers for this newsletter.

If you know anyone that you think would find this useful or interesting, please share this link with them and help me hit this milestone before the end of August!

https://the-release-notes.beehiiv.com/p/getting-roasted-by-users

Thank you!
Jake

PS. If you’re considering launching your own newsletter, you can get a free month & 20% off by clicking here.