We consolidated Lighthouse onto AWS, cut cloud costs by 22%, and sped up pipeline by 30%
Growth has a way of exposing infrastructure. Lighthouse’s AI-powered data pipeline was straining a fragmented GCP/Azure setup that made costs hard to control and scaling harder still.
We consolidated everything onto AWS: their crawler, their production app, and their AI workloads, and created one platform, with full cost visibility and near-zero downtime during the move.
This made the platform cleaner, cheaper to run, and easier to scale.
Project basics
- Industry Venture Capital / AI SaaS
- Challenge Migration from GCP/Azure to AWS, re-architecting a cost-sensitive AI data pipeline, infrastructure consolidation
Client
Lighthouse is an all-in-one sourcing and CRM platform for venture capital firms and investors.
It brings dealflow sourcing and relationship management together in one workspace, letting investors discover, track, and manage companies, people, and deals all in a single place.
Under the hood, it’s powered by a large-scale data pipeline that crawls, classifies, and enriches startup data to surface high-potential companies and automate due diligence, so investors can go from sourcing to decision without switching tools.
We'd happily recommend allOps to anyone facing a non-trivial cloud migration, especially teams with heavy data or AI workloads where the migration isn't just moving boxes but rethinking how things run.
A genuinely great team to work with.
Challenge
Lighthouse is an AI-powered platform serving VC firms and investors. Its differentiator (and heaviest workload) is a large-scale data pipeline that scrapes, classifies, and enriches startup data using AI models, feeding everything from pitch deck analysis to investment memos and signal feeds.
As the platform grew, the existing GCP/Azure infrastructure created real friction:
Fragmented multi-cloud setup
Workloads split across providers meant scattered operational visibility and no single source of truth for costs.
Cost-sensitive data pipeline
The team had built an intricate crawler setup specifically engineered to keep costs down. Migrating it without breaking those optimizations was the hardest part of the entire project, and a lift-and-shift would have destroyed the economics.
Scaling pressure
The pipeline, the user-facing app, and the AI workloads all needed room to grow with better cost control. They needed a single, consolidated platform.
Our Solution
We carried out a complete, controlled migration from GCP/Azure to AWS with the data pipeline as the centerpiece:
Re-platformed the backend onto Amazon ECS Fargate
Existing cloud functions and service workloads became containerized services behind an Application Load Balancer, which made the infrastructure scalable, observable, and consistent.
Re-architected the data pipeline.
The old crawler relied on an intricate web of cost optimizations that worked until you touch it. Instead of replicating those workarounds on AWS, we redesigned the pipeline around them becoming unnecessary.
We containerized workloads on ECS Fargate, with Amazon SQS decoupling document processing so heavy enrichment jobs run asynchronously without blocking the rest of the system.
Migrated the database to Amazon RDS PostgreSQL
Because this is the production database behind the investor workspace, the cutover was staged first, validated for data integrity, then executed as part of a migration. Lighthouse now gets managed backups, patching, and scaling without the operational overhead of self-managed Postgres.
Moved frontend and storage to S3 + CloudFront
We served the static frontend and selected storage paths through secure, optimized content delivery. DNS moved to Route 53.
Hardened operations across the board
Container images in Amazon ECR, runtime secrets in AWS Secrets Manager and SSM Parameter Store, logs and metrics in CloudWatch, and private EC2 instances (managed via AWS Systems Manager) for internal workflow and CRM workloads.
Everything as code
The entire infrastructure was provisioned with Terraform across separate staging and production environments, with CI/CD through GitHub Actions using AWS OIDC and tag-based deployments.
This cut deployment time from ~20 minutes to under 5 minutes.
The migration of the main application happened with less than 5 minutes of downtime.
Results
Lighthouse moved from a fragmented multi-cloud setup to a single, consolidated AWS platform, and the numbers followed.
- ~22% reduction in monthly cloud spend
- ~30% faster data pipeline processing
- Deployment time cut from ~20 minutes to under 5
- One platform, full operational and cost visibility
- Reproducible infrastructure (Terraform + separate environments)
- Data pipeline that's more efficient and easier to maintain
About allOps Solutions
allOps is an AWS Advanced Tier Services Partner founded by two AWS Heroes – a distinction held by around 255 people worldwide – and the only AWS Authorized Training Partner in Bosnia & Herzegovina.
allOps designs, builds and operates secure, scalable AWS for companies from startups to global enterprises, turning AWS funding into outcomes customers can see.
Migrate to AWS with us, save on cloud costs, and improve your workflows.