“Don’t make a move until you know how you’ll measure the impact.”

Sergey Kyune built a product to help him learn how to sing, and now it brings in $10k/mo.

The key to his success? Measuring everything. He doesn’t do a thing until he has his hypothesis and test plan in place.

I caught up with him to understand his testing process and how he grew an atypical software product. 👇

Measure everything

James: What’s different about how you do business?

Sergey: Two things:

1. The industry I’m building in.

2. My geekiness about measuring everything. Features, design, marketing experiments — everything.

James: What's the benefit of measuring everything?

Sergey: As an entrepreneur, you only have so much time and a myriad of things to work on. And there’s so much ambiguity. You must focus on things that impact your business or you are out of the game.

If you do not measure the impact, you do not know what works and what doesn’t. If you do not know what works, you do not learn, and you cannot find things to focus on that will eventually lead you to success.

James: What's an example?

Sergey: It helped me find my customers’ “aha” moment — the interactive piano layout that we use in our product that visualizes pitch-matching.

I learned that once the user is introduced to the layout and its mechanics, they understand the value of our product. And that's the moment when the paywall should appear: no earlier, no later.

James: That's valuable. Any downsides to measuring?

Sergey: I actually have a lot of issues when working with contractors. Most of them are fast with suggesting changes, but when I ask them how they’ll know if their change works, they start referring to their experience and industry standards instead of coming up with a test plan.

How to run an experiment effectively

James: Yeah, it’s a rare thing. What’s your process?

Sergey: There are entire books written on the topic of how to be data-driven and how to run experiments. Below is my personal simplified interpretation:

1. Define the problem you want to solve and why it's important right now.

2. Explain how the work you are planning to do contributes to solving that problem.

3. Make a hypothesis statement: Why do you think the change is going to work? What data supports it?

4. Go deeper into your hypothesis. It should answer these questions: Once the change is implemented, how do you measure it? What metrics matter? And very importantly, what is the exact change in the metrics that will mean success? You have to decide that upfront.

5. Rank the idea based on the source of the data: a hunch (worst), expert opinion (weak), relevant case study (okay-ish), relevant industry research (good), your own customer data (best!).

6. Choose your setup. Best-case scenario, you can set up a split test. In this case, ensure you have enough traffic to get statistically significant results within a reasonable timeframe. The fallbacks are a time series post-analysis (testing one time period against another) or qualitative research methods like user surveys and interviews.

7. Implement the change.

8. Verify that your metrics are showing that your change is working as expected. The setup should be done in such a way that metrics can only be influenced by your actions. You don't want other factors affecting the experiment.

If you did all this properly, analyzing the results should be fairly easy.

James: How do you remove all the variables?

Sergey: In reality, it’s almost impossible to exclude all the variables, but that’s what we have to aim for. At a minimum, make sure you are only changing one thing at a time; not multiple things.

And make that thing as small as possible. Smaller changes mean less work.

James: Do you log all of your tests?

Sergey: In a team setup, I would, but since I’m running solo, I keep a lot of these steps in my head to speed up the process. I only note the most important learnings.

James: Is there anything you shouldn’t measure?

Sergey: I can think of 3 main cases:

1. Measurements are too expensive. For example, they're slowing down customer experience or exposing the fact that you are running a split test to the end user.

2. Ethical and privacy-related concerns. I try not to collect PII unless I absolutely have to. Some people also think running AB tests on pricing is unethical, in my opinion it depends on the case. And you have to think about the use case. For example, if you're B2B, you don't want to put one employee at a disadvantage with a variant that makes them slower at their work.

3. Regulations. Governments also might have an opinion!

Subscribe for more how-tos, roundtables, and interviews with people in the thick of it.

Burnout and scary decisions

James: You built Singing Carrots for yourself.

Sergey: Yep, for the last two years of my employment, I was studying vocals and building various useful tools to help me with my studies.

James: So you were side-hustling. Why did you take the leap to full-time?

Sergey: Burnout. I started feeling that every new technology I had to learn for my job was just a copy of something I'd already seen before. The names of terms in various frameworks and programming languages were different, but the essence and underlying principles were the same.

I began getting bored and simultaneously irritated that every 2-3 years of my career, I had to learn another fancy storage solution people invented to attempt to replace the relational database, or a new approach to using a client-side purposed programming language for the server.

I started looking for an escape from this for my career and attempted to switch to product management. I quickly learned that I'm very bad at corporate politics.

I ended up in deep burnout and decided to quit.

James: Was it making enough to support you?

Sergey: It wasn’t even close to what I needed to sustain my life, but I had some savings.

James: Brave decision.

Sergey: It was freaking scary. But after I slipped into burnout, this project was the only thing that still held my interest. So I decided to quit and educate myself on how to turn it into something that could feed me.

James: And does it feed you now?

Sergey: Three years later, I’m at the point where it feeds my family. My MRR is €7,300, but my one-time sales and ad revenue bring it to about €10k/mo.

This project probably saved my mental health and opened a new way of approaching life for me.

James: So you lived off your savings for three years?

Sergey: No, I actually took a detour. I used my savings to sustain myself for the first year, but then my wife got pregnant, so I got a job for extra security.

I even found a person to sell Singing Carrots to. The plan was to sell it and go back to normal employment long term, but the sale fell through. Six months after my son was born, I quit my job (again).

Realities of building in an unusual space

James: What has been your biggest challenge building in an unusual space?

Sergey: The biggest challenge has to do with the fact that the product is educational. Even though I’m selling software, the problem I’m solving for the customers is not repetitive. They come to improve their singing to a certain level, and once they reach that level, they naturally churn.

Because of that, I can’t really think in terms of MRR and churn. I view it more like one-time payments that are paid over the duration of a course.

James: So how do you increase lTV?

Sergey: I could offer more advanced training, but as an Indie, I’m forced to work on a niche. Otherwise, I won't be able to compete with bigger players. So I stick to beginners, and upsells are limited.

LTV limits the marketing channels I can rely on. So I have to be creative with the ones that are left.

James: Anything else?

Sergey: I cannot directly use customer feedback as a way to improve the product because my customers are learners, and they don’t know the best way to learn something they have yet to learn.

Therefore, I have to filter customer feedback more stringently and involve the expertise of singing teachers more often than I’d like. This slows down the process.

Growth via free courses and tools

James: Alright, let’s hear it. How did you grow Singing Carrots?

Sergey: In the early days, I tried to post links to my product in every corner of the internet that mentioned the word “singing.” The first users came from Reddit and Quora.

Reaching out directly to music blog owners, teachers with websites, and relevant Facebook group admins brought some users too.

And those early efforts allowed me to build some backlinks and lay a foundation for my domain authority.

James: What else?

Sergey: Youtube. I decided that my strategy would be to teach theory for free and sell tools that assist with practice. So, I started by recording video lessons, vocal exercises, and warmups, and publishing them on YouTube.

Then, I added some articles to our blog.

Over time, as the size of our educational library grew, I converted it into a free course for beginners. That has been a huge driver of growth.

Additionally, I created simplified versions of the tools we sell and made them available for free on the website. Like this simple pitch monitor, for example.

James: Free tools are powerful.

Sergey: People love free tools. They share links on the internet, which strengthens your domain positions.

James: Any growth tactics that you refuse to do?

Sergey: When people hear about my product, they often tell me to grow it on Instagram or TikTok. I ignore these platforms.

James: Why?

Sergey: They are way too time-consuming for an indie. The content you publish there drowns insanely quickly, and I don’t like doing work that doesn’t scale.

YouTube is different, though. Videos that you upload there get views years after you publish them.

Growing via data

James: You mentioned that you have to be creative with your channels. How?

Sergey: I'll give you examples.

I wanted to retarget people who hit my paywall, but most of the people who hit my paywall will never buy. So I started thinking about how I could narrow down the audience to make my ads more cost effective. I started digging into my MixPanel analytics to find what other events were connected with a likelihood of purchase.

I ended up placing a single survey question on the paywall to see how the answers would affect the purchase rate, then I used that information to set up my Google Ads audience.

James: Your dedication to data is paying dividends.

Sergey: Or when I wanted to reach out to singing teachers to promote the product. The response rate was really low to direct emails. So instead, I ran an “industry research” survey where I invited teachers to participate in a survey or an interview as experts.

Then I introduced them to my product.

Data-driven pricing tips

James: What else can you share from your experiments?

Sergey: I’ve been testing pricing a lot. Here are a few things that I found counterintuitive:

• Offering a weekly membership plan works much better than any sort of free trial.

• Price sensitivity turns out to be a very relative thing. I used to have only one type of membership, and when I was experimenting with raising the price for it, barely anybody bought a membership at prices higher than $15/month. Now, I have 4 tiers at $10/month, $20/month, $25/month, and $35/month, and even the most expensive one generates some sales. The price progression ladder and the surrounding price context create a better value-for-money perception.

• Running promotional campaigns offering discounted prices for events like Black Friday generates sales, but the kind of customers it brings are the worst: complaints, refund requests, etc. I’m not sure it’s worth dealing with.

• People don’t read descriptions; they make intuitive moves.

Caution here. All of this only worked for Singing Carrots. I have no idea if these learnings are scalable.

James: Gotta love that data-driven approach.

Sergey: It’s so important. If I could leave indie hackers with anything, it’s this: Plan how you will measure the impact before you make a move.

James: Where can people find you?

Sergey: You can find me on LinkedIn. If you like singing, check out Singing Carrots

Subscribe to the Boot's Trap series