AWS VPC charges
We decided to go with RDS and Lambda and SQS as the main building blocks of our app.
Before long we realized that if you want to have an RDS cluster, it better live inside a VPC, otherwise it's on the internet, which is a huge no-no. But then in order to access that RDS, each lambda also had to be in the same VPC.
Our lambdas use other AWS services (secret manager, s3, sqs, etc.), so we need interfaces from the VPC to those services. But interfaces are expensive. We are still in the early stages of development and a huge proportion of the total cost comes from those VPC interfaces just being up; regardless if we are using them and how much traffic they receive.
So here's my question: How do you deal with that cost?
I had several ideas:
- The cost doesn't scale with the users, so it will reduce in proportion as we gain momentum, so just wait it out?
- We could have gone with Dynamo, which doesn't have to be in a VPC, but we feel we need the SQL
- A proxy lambda that is inside the VPC and all other lambdas use it to access the RDS. Not sure how the DB interaction will be happening from within the code. I don't want to reimplement the sql library I'm using.
That's already too long of a post for the issue at hand. So thanks for reading. :)
I have had AWS Elastic Beanstalk for some time, it was very expensive (mainly because you need a load balancer for SSL). I am in the transition to Digital Ocean with Laravel Forge (1/6 of the costs). Works for me. When in the future it does not fit, I will then take actions.
RDS is not directly live on the internet, AWS is blocking all traffic with the firewall. I had quite some difficulties opening the firewall for my EC2 container..
Hope this helps :)
Thanks for the reply. DO is very easy and cheap. That was our other option. But AWS has scalable user management, queue services, databases, monitoring that come out of the box (after you pay, but you don't have to maintain any of that). I think that got us on the AWS train.
Yeah can understand. I'm building with laravel and use Laravel Forge to basically do all of this - for a one man team :)
Don't have to do much with servers and such, can just commit in git and it will automatically update the server
I don’t use a load balancer: I get SSL from API gateway.
I agree Database on the internet is a huge no no but I do it.
I add basic auth and use ufw to whitelist the relevant IP ranges.
I guess it depends on how sensitive the data you're storing is, but if bootstrapping, in my case, it was fine.
I think "security" is one of those problems we tend to worry about way before it's an actual problem. The other I see people worry too much about is "Scaling".
I guess you're right if we can restrict the IP ranges. And tbh we don't store anything sensitive. We're architecting it so that we delegate everything sensitive to someone else (thus also kind of vendor locking ourselves, but it's not a perfect world, is it).
But we're running lambdas, which don't have fixed IP addresses :/ I think we might end up using dynamo for some of the lambdas so that we keep them outside the VPC. Or a postgres RDS proxy when that's available
It makes sense.
Don't overlook basic auth with a strong password over TLS or SSL. it's simple to set and secure enough in most cases.
Have you looked into connecting to your RDS instance through a lambda using an RDS proxy? This will require you also setup API Gateway for access externally to your lambda. It will be more of a serverless approach and the cost should decrease dramatically while your user base is still relatively low.
This will probably be the way forward. We just need to wait for the proxy to become available for postgres too.
Although I'm not sure if the proxy will help with moving our lambdas outside the VPC (and get rid of the VPC endpoints/interfaces).
I think you will still need to integrate with API Gateway to expose your lambdas and then get rid of the VPC.
We use AWS and other cloud instance as well, which VPCs won't help. So we use our own product OmniEdge to replace VPC for the hybrid cloud. And OmniEdge is 1ms better then VPC even between two AWS EC2s.
See the benchmark:
Can you elaborate on what you mean by expensive? There are a number of ways to approach what you are trying to do. I would recommend staying on AWS, as migrating into AWS will be harder after you've been running for a while.
Regarding your access issues, I would run a micro instance as a bastion host and SSH tunnel through it for your remote RDS access. Very secure.