Brighthouse Financial ran ~100 daily batch jobs across datasets with tables nearing 500 million rows, and volume growth was making runs increasingly unstable.
Critical query runtimes dropped from nearly 30 minutes to seconds, recurring failures were eliminated, and batch operations became reliably scalable.
What moved forward
Critical query runtimes dropped from nearly 30 minutes to seconds, recurring failures were eliminated, and batch operations became reliably scalable.
Situation
Brighthouse Financial ran ~100 daily batch jobs across datasets with tables nearing 500 million rows, and volume growth was making runs increasingly unstable.
What we did
We joined midstream in a production support and development role with one priority: stabilize daily operations first. We investigated failures, performed deep root-cause analysis, fixed recurring issues permanently, optimized database and SSIS bottlenecks, and documented system dependencies so the team could move out of firefighting mode.
What changed
Critical query runtimes dropped from nearly 30 minutes to seconds, recurring failures were eliminated, and batch operations became reliably scalable.
Overview
Brighthouse Financial is a leading U.S.-based life insurance and annuity provider managing more than $240 billion in assets. As data volumes increased across daily operational workloads, they needed stronger batch reliability to protect reporting timelines and internal confidence in system outputs.
Critical query runtimes dropped from nearly 30 minutes to seconds, recurring failures were eliminated, and batch operations became reliably scalable.
Brighthouse Financial ran ~100 daily batch jobs across datasets with tables nearing 500 million rows, and volume growth was making runs increasingly unstable.
This engagement sat in a scale & stabilize context and was delivered through our outcome partnership support model.
Brighthouse Financial ran ~100 daily batch jobs across datasets with tables nearing 500 million rows, and volume growth was making runs increasingly unstable.
The work had to strengthen the weak loops in the operating system before more complexity piled on top of them.
Brighthouse Financial is a leading U.S.-based life insurance and annuity provider managing more than $240 billion in assets. As data volumes increased across daily operational workloads, they needed stronger batch reliability to protect reporting timelines and internal confidence in system outputs.
Core constraints we had to address
We joined midstream in a production support and development role with one priority: stabilize daily operations first. We investigated failures, performed deep root-cause analysis, fixed recurring issues permanently, optimized database and SSIS bottlenecks, and documented system dependencies so the team could move out of firefighting mode.
The sequence focused on the parts of the workflow where reliability, usability, and system clarity were under the most pressure.
Core technologies and system elements
First move
Prioritized every live batch incident, diagnosed root causes quickly, and coordinated fixes across upstream/downstream owners to protect daily run continuity.
Critical query and batch-processing bottlenecks were reduced from nearly half an hour to near-real-time response windows.
Pressure addressed
Date-based queries on very large tables could take nearly 30 minutes due to missing indexes
Change unlocked
30 minutes to seconds
Critical query and batch-processing bottlenecks were reduced from nearly half an hour to near-real-time response windows.
Next move
Redesigned fragile process paths after deep analysis, including replacing memory-heavy SSIS handling of multi-gigabyte files with a lightweight line-ending detection strategy.
Repeated batch-run failures were permanently resolved, increasing day-to-day pipeline success rates.
Pressure addressed
Recurring SSIS batch failures increased as upstream files grew in size and memory pressure
Change unlocked
Recurring failures removed
Repeated batch-run failures were permanently resolved, increasing day-to-day pipeline success rates.
Next move
Identified missing indexes on high-volume tables, optimized query paths, and reduced key query times from ~30 minutes to seconds.
Fewer incidents and faster recovery reduced manual intervention and eased pressure on internal teams.
Pressure addressed
Frequent manual interventions stretched limited internal resources and increased SLA risk
Change unlocked
Lower support overhead
Fewer incidents and faster recovery reduced manual intervention and eased pressure on internal teams.
Final move
Documented batch flows and dependencies with clear visual references so onboarding and troubleshooting no longer depended on tribal knowledge.
IT management and business stakeholders regained confidence in data timeliness and platform stability, and Brighthouse renewed the engagement.
Pressure addressed
Business users uploaded reports without timely data availability, reducing trust in the system
Change unlocked
Confidence restored
IT management and business stakeholders regained confidence in data timeliness and platform stability, and Brighthouse renewed the engagement.
Critical query runtimes dropped from nearly 30 minutes to seconds, recurring failures were eliminated, and batch operations became reliably scalable.
The clearest gains showed up in reliability, workflow stability, and day-to-day operating confidence.
Performance
Critical query and batch-processing bottlenecks were reduced from nearly half an hour to near-real-time response windows.
Reliability
Repeated batch-run failures were permanently resolved, increasing day-to-day pipeline success rates.
Operations
Fewer incidents and faster recovery reduced manual intervention and eased pressure on internal teams.
Trust
IT management and business stakeholders regained confidence in data timeliness and platform stability, and Brighthouse renewed the engagement.
Additional case studies with comparable pressure patterns, operating context, or delivery stage.
Systems fragile
Barclays Bank US
The redesigned website gave Barclays Bank US a stronger digital experience with better accessibility, clearer product communication, and a more flexible content...
Systems fragile
Admin Partners
Admin Partners moved toward a more connected operating model with stronger workflow reliability, less duplicate handling, and clearer visibility across core business...
It's hard to trust the data
Premier Financial Alliance
PFA moved to a stronger incentive management system with secure team communication and more flexible reporting, making day-to-day operations easier to run with...
The clearest shifts in the system, workflow, and operating confidence after the work moved.
Performance
30 minutes to seconds
Critical query and batch-processing bottlenecks were reduced from nearly half an hour to near-real-time response windows.
Reliability
Recurring failures removed
Repeated batch-run failures were permanently resolved, increasing day-to-day pipeline success rates.
Operations
Lower support overhead
Fewer incidents and faster recovery reduced manual intervention and eased pressure on internal teams.
Trust
Confidence restored
IT management and business stakeholders regained confidence in data timeliness and platform stability, and Brighthouse renewed the engagement.
Common questions from teams facing similar pressure, implementation choices, and operating constraints.
Brighthouse Financial ran ~100 daily batch jobs across datasets with tables nearing 500 million rows, and volume growth was making runs increasingly unstable. One early constraint was: Date-based queries on very large tables could take nearly 30 minutes due to missing indexes.
Critical query runtimes dropped from nearly 30 minutes to seconds, recurring failures were eliminated, and batch operations became reliably scalable. A concrete shift was: Critical query and batch-processing bottlenecks were reduced from nearly half an hour to near-real-time response windows.
The work centered on SQL Server, SSIS, and Database Indexing. The choices mattered because they supported the real operating constraint instead of adding unnecessary complexity.
By sequencing around the highest-leverage constraint, using technology choices that fit the operating context, and staying focused on what actually needs to move for the team.
The first move is usually a focused systems read: identify where reliability is breaking down, then address the highest-risk loop first.
The approach is phased and practical. This engagement started with "Production-First Stabilization" and expanded from there once core stability improved.
Clear ownership, reliable operating loops, and stronger coordination patterns become critical so growth does not erode delivery confidence.
Prioritized every live batch incident, diagnosed root causes quickly, and coordinated fixes across upstream/downstream owners to protect daily run continuity. That mattered because it directly addressed constraints such as: Recurring SSIS batch failures increased as upstream files grew in size and memory pressure.
This case is most relevant for teams facing systems fragile during the scale & stabilize stage. A strong next step in similar situations is to Run the Scale Readiness Check.
Use this case study as a starting point, then branch into the closest next move, supporting guidance, or comparable work.
Browse similar situations
Browse the full library by pressure, stage, and industry context.
Run the Scale Readiness Check
Surface decision drag and reliability gaps as team and system complexity increase.
Read: Should you modernize or rebuild?
Use a risk-based decision standard when current systems are under pressure.
Read: Did growth break your operating rhythm?
Restore execution rhythm as team size, delivery load, and systems complexity keep increasing.
Explore Outcome Partnership
See how this support path is structured when similar constraints are in play.
Related case: Barclays Bank US
The redesigned website gave Barclays Bank US a stronger digital experience with better accessibility, clearer product communication, and a more flexible content...
If your team is dealing with systems fragile during the scale & stabilize stage, we can help diagnose the constraint, sequence the next move, and stay close through execution.
