Hey, so do y'all write tests?

I 100% disagree with the premise that you need tests.

Tests are about understanding your costs. If it costs you less to fail than it does to write a test, then you don't need a test. If it costs more to write a test than it does to test it manually, then you don't need a test.

There is, however, a point at which the cost of testing manually, or failing in production or staging environments likely outweighs the cost of writing automated tests. There is also a point at which the cost of on-boarding new team members and contributing to the codebase will be greater than the cost of writing tests.

Anyone who says that a lack of tests is technical debt, does not understand economics. Debt is incurred when you take a short term loss for a long term gain and the gain is LESS than the loss. Investment occurs when you take that same loss and the gain is GREATER than the loss. Sometimes not writing tests is technical debt, other times it's technical investment. It's all about understanding your costs. I really don't think that's understood enough.

It's your product and it's up to you to understand your cost allocation and your gains and make informed technical decisions.

Testing is like brushing your teeth. It's not fun but you do it for health reasons.

I like to create tests because I find it to be a good way to ensure that most of the heavy lifting functions are as pure as possible. You sometimes find that some functions are just a pain in the ass to test, and this is usually a result of poor architecture. You can then go and refactor until testing is a delight. Existing tests also mean that you can refactor extensively and be sure that your application still works.

If you're looking for minimum viable testing, what I'd do is identify the core concept of your app, identify the main functions, and just test them.

I found when I have to move very fast, test can slow me down too much. So for Prototypes I don't write many tests or even none at all. For MVPs I definitely write tests especially around more complex logic, mostly unit tests though. Everything that's not tested will get a test as soon as it appears to be part of a bug.
Downside of my approach: Sometimes I end up in a sub-optimal architecture that does not let me test code easily and is also not replaceable without friction. I see that writing more tests would help me with that. I guess, it's always a balance between speed at the beginning and the slow down later through technical debt.

I only unit test for things I find complex and have high potential for errors - the last thing I wrote tests for was a lexer and parser for customised compression of json in swift and in python 🐍 Things like regex usually get a bunch of tests as well.

Yes, there is a formula for automated testing.

Automated tests are an investment. That means that you put effort in, and you expect to save time or have a higher quality product.

So you need to calculate your Return On Investment for your automated tests. How much does it cost to make them? How much does it cost to maintain them? (A lot of people forget about the 2nd part). How much does it cost to run them? (this one might be neglectable)

Then you need to calculate your benefit: How much time does it save you from manual testing? How much time does it save you on debugging.

So my advice is to start out with no tests. And then along the way, see if you could save time on certain aspects by writing those tests.

Some people think they have to test everything, but this is not efficient. You need to test those things that have the highest chance of breaking.

And of course, it also depends on how stable your product needs to be.

Depends what you want to do with your project. Validating and going fast on the market does not go along with having the healthiest codebase. Thus, if you are building a product that you want to make money of fast, you shouldn't do testing. You need every minute.

All of the successful projects I've been working on have had low coverage as the people started catching up on testing only after some ground was secured.

It's a funny coincidence with my tweet from two days ago that opened some controversy. I am a software engineer of 14y experience (mostly working for clients and startups) and I am all into testing as a practice. But for small side projects with the intention to grow to Saas, it's an overhead.

I run tests with my own tool: https://reflow.io (Low-Code Automated UI Tests).

About a year ago (6 months into development) the complexity of the tool was high enough that it could both break in subtle ways, and it could test itself sufficiently well to catch those subtle bugs. I worked out a set of deterministic UI flows that cover almost all application functionality, recorded them into the tool, and trigger them to replay each release. Now I only manually test the new features.

For some critical areas I have blocking visual assertions that validate an area of the page looks exactly (+-1%) the same as the previous release. For other critical areas (e.g. Billing) I write unit tests.

The right time to implement tests is when the development productivity is impacted by lack of stability in the application: if you've started to test manually, then you should probably started considering automating a subset of those test actions and embedding them into your release processes.

This is just an ROI calculation: how long does it take to return your investment into building automated tests. Relevant XKCD: https://xkcd.com/1205/.

Is there a formula to get the optimal benefit with the least amount of work

Very Biased Opinion: try a low-code tool: they've been designed to solve this problem, giving you testing workflows that require very little effort to create/maintain/execute/validate.

Solo dev here. It's very painful to write tests while working on your own. Thinking of making it a habit for a healthier future :)

automated tests are a good thing, because once you have them written, they're written. All you need to then do is run them. let's say you had 100 test scenarios developed over the course of time in developing an app. And you didn't use automated testing tools. You would have executed each of your tests, probably recorded the outcome in a spreadsheet, and once they all passed, it was good to deploy. Now lets say a few months later, you need to implement a new feature of your app. If you don't have automated test tools, you would have to go back and re-execute every one of your previous tests to make sure everything is still working. Otherwise if you don't, you wouldn't catch an issue that was caused by one of your new changes. If you had an automated testing tool that took care of running all those tests, you would know that one or more of them failed, and you wouldn't spend hours re-executing all your previous tests for regression testing.

I just don't see the advantage to not using automated testing tools. They will save a lot of time and you will catch more bugs when the tests fail

I think it will depend on how comfortable you feel with your product. Does it crash frequently or does it have a lot of defects on certain parts? If yes, I think you can make it better by adding tests. Otherwise, I will spend time on other important tasks.

When you write tests you become a better developer.

Sometimes principles like using TDD could really help you write code faster.

I think testing is really important but it needs to be correlated with your business,
how many developers do you have?
How hard is it for you to write a test VS the time it takes you manually to check every time
what is more urgent currently?
and so on...

When I worked in companies of 100+ people usually the startup was in a place to write tests and we had time and it was okay.

Now I'm working in a startup and we write tests only on critical places that can affect us seriously, and focus on delivering a better product and features to our clients.

On my side Project(sekker.io), We are just 2 developers and we didn't write even one test.

Absolutely. The cost of finding and resolving things that break can be high. In many cases I'm already writing tests to see if the code behaves as I expect it to because manually triggering the scenario and verifying the data would be tedious. There are other things I don't test until there's a problem. Basically, I just take a quick assessment of what can go wrong, the impacts, and how difficult would it be to add a test and ease my mind. In most cases, it's rather simple. When it's not, it's usually a sign that I've over complicated the software. Proper separation of concerns help a great deal and tests can help identify those areas. I've found this workflow helps me grow organically. I'm at $0 MRR.

You are rolling an engineering debt. At some point, it may bite you, and explosive growth, for example.

I usually do quite some unit tests. Depends on the application - not so much on the UX side, not for basic CRUD stuff, but the core business logic definitely.

The main reason is that it does increase my productivity. Instead of spending time on triggering the right state in the application to test the code I'm writing, a test lets me execute only that code path and nothing else.

I also know that in 6 months, I will have lost some context of what I was doing. Then I'll find having a look at the unit tests very helpful to see exactly what something is used for, how it is invoked and what result is expected, and what kind of edge cases there is. Especially when refactoring I'll find this useful.

In addition, I find that doing unit tests helps me in creating a much better low-level architecture, in terms of dependencies, separation of concern, and so on.

So funnily enough, using tests to find bugs is probably the last thing I use tests for.

I do have tests. Mostly unit tests.

But that’s also because my project will double as a portfolio piece. Also, I find that writing tests as you go is a lot, lot, lot easier than adding them later. Same with linters and formatters. It was a hassle setting them up, but I’m so glad I have them now. Helps me write better, cleaner code that I am confident in and enjoy writing.

Is it necessary? Not at this stage. Am I still glad? Yes.

Tests do actually help reason about and catch mistakes in your code, especially when you have multiple test cases. I plan on adding a couple e2e smoke tests and use automated tests for accessibility and google lighthouse.

I will clarify, I actually wish I had a couple tests.

Namely for the auth flow and the minimal-path-to-awesome. But in my opinion, if somebody finds a bug after that point in using your application they'll usually tell you it's broken, so tests aren't necessary (at small scale of course).

Here's the thing. No matter what you think, you probably do write tests as you develop things. Unless you're a real crazy person, you will probably do things like copy the function to your console and try it with different inputs, or click the button and see it does the right thing, or validate your commit some other way. So the question, is whether you are mature enough to actually save these tests in a structured way so you can rerun them down the road.

I say "mature" because I view writing tests as an act of choosing long term over short term, and that is inherently a mature thing to do.

Tests are also IMHO the best kind of documentation. You can read them as examples of how different parts of the code work, and if they fail it means the code is wrong, not you. It's good for your sanity.

Side note: Most of my code is not tested properly, and I still suffer for this.