When I first used Apache Spark about 3 years ago I was blown away how fast and easy to learn it was. So I have been using it wherever I could ever since and gained quite a bit of experience with it.
Since day one I have also been looking for a visual front end for it to build ETL processes and data pipelines with. There are some products out there but most of them are expensive and bloated with features I will never use.
So a few months ago I decided to create my own visual data pipeline builder backed by Spark. It makes it a breeze to quickly inspect large amounts of data in Amazon S3.
It's still heavily under development but is fully usable. The key points are:
can be used to build pipelines step-by-step or just inspect data
I also managed to grab the perfect domain: https://www.datapipelines.com
With the MVP done comes the hard part. Marketing, business development, etc...
I'm very technically minded so currently looking for someone with business experience to team up with.
I'm also looking for a company who would be willing to give the product a go. I will provide it for free and ask only for feedback in return. If you are interested let me know.
great product.
Thanks