Scaling Analytics: Vanguard's Journey with Amazon Redshift

Vanguard, one of the world’s leading investment companies, faced significant challenges as its Financial Advisor Services (FAS) division scaled. The rapid growth in data analytics requirements led to performance bottlenecks and resource contention, impacting everything from daily operational reporting to strategic business analysis. The initial architecture, a single Amazon Redshift provisioned cluster, began experiencing severe performance issues that threatened business operations.

To address these challenges, Vanguard FAS implemented a multi-warehouse architecture leveraging Amazon Redshift's advanced data sharing capabilities. This architecture included a central hub (Amazon Redshift Provisioned Cluster) optimized for consistent workloads and multiple Amazon Redshift Serverless instances serving as specialized consumer endpoints. This setup allowed for workload isolation and independent scaling, enhancing both performance and operational efficiency.

The new architecture delivered significant improvements. Nightly ETL cycles now consistently completed before the 9 AM SLA, ensuring fresh data was available for morning business activities. Analysts experienced higher productivity with faster query response times and the ability to run complex analytical workloads without impacting other users or ETL processes. This change led to higher analyst satisfaction and productivity across the team.

The architecture introduced a dedicated “Data Lab” environment where analysts could experiment with data using `CREATE TABLE AS SELECT` (CTAS) commands. Each workload type could now scale independently based on demand, enabling more sophisticated analytical approaches. This flexibility allowed Vanguard to explore new data products and leverage the system more effectively.

As Vanguard’s data environment matured, they recognized the opportunity to evolve their architecture further. The success of the multi-warehouse architecture enabled broader adoption across the organization, leading to ongoing modernization efforts. Vanguard is now exploring the evolution toward a data mesh architecture to match their organizational growth and the expanding portfolio of data products.