Secure Data Aggregation and Statistical Processing for Multi-Agency Research

Introduction

Multi-agency research requiring sensitive state data presents an immediate tension between research utility and information security. For the LA FIRST initiative housed at ULL’s Blanco Public Policy Center, the Office of Technology Services (OTS) resolved that tension by building a governed, PII-free data aggregation environment spanning six state agencies—enabling longitudinal policy research without exposing personally identifiable information. The result is a replicable model for secure inter-agency data collaboration in Louisiana state government.

Background

The LA FIRST research initiative required access to sensitive administrative records held by six separate state agencies: the Department of Corrections, Office of Juvenile Justice, Department of Education, Department of Children and Family Services, Louisiana Department of Health, and Louisiana Workforce Commission. Each agency maintained data in its own format and systems, with no existing mechanism for consolidated, research-ready output.

The core challenge was not simply aggregating data—it was doing so in a way that fully excluded Personally Identifiable Information while preserving enough record linkage to support meaningful longitudinal analysis. This required coordinating data governance agreements, background screening, secure infrastructure, and statistical processing across multiple institutions simultaneously. No existing shared infrastructure was in place to support this scope of work.

Choices

Allowing LA FIRST researchers direct access to agency source systems was not an option. The security and compliance exposure of granting external researchers access to live agency data was prohibitive, and no existing Data Sharing Agreement framework covered that scenario.

A centralized data warehouse owned by a single agency was evaluated but presented governance complications: each agency retained legal responsibility for its own data, and no agency was positioned to act as a universal custodian. Instead, OTS established agency-specific directories within its own secure network, with OTS acting as the neutral infrastructure host rather than a data owner.

Processing was initially performed in R, but as dataset sizes grew into the millions of records, R’s performance limitations required a transition to Python. This was not a design failure but an anticipated scaling decision—the architecture accommodated the tool change without structural rework. Stata export was retained as a researcher-facing format given its prevalence in policy research workflows.

Solution

OTS’s Information Security team coordinated background checks for all LA FIRST technical staff and required that all work be conducted on OTS-imaged and controlled laptops within OTS’s secure network. Data Sharing Agreements were executed with all six participating agencies to formalize access terms and compliance obligations.

Within the secure environment, agency-specific directories were established to receive incoming data. OTS created corresponding SharePoint repositories where LA FIRST researchers could upload aggregated datasets for agency review and PII validation. For agencies without SharePoint access, files were compressed into encrypted archives and transmitted via secure email. No dataset was cleared for research use until all relevant agencies provided final approval.

Statistical processing used Python and R to clean and standardize heterogeneous input files from each agency. A unique LA FIRST ID was assigned to each individual to enable record linkage across datasets while eliminating direct identifiers. Following linkage, all remaining personal identifiers were removed and analysis-ready datasets were generated. Each dataset underwent quality assurance checks for accuracy and completeness.

OTS configured a SQL Server environment—using SSMS, Visual Studio, and SSIS—for long-term storage and management of the aggregated datasets. Final datasets were exported in Stata format to align with researcher workflows.

Results and Impact

OTS’s secure data aggregation environment for the LA FIRST initiative cut nondisclosure review times by 30%, giving researchers at ULL’s Blanco Public Policy Center access to longitudinal, de-identified records from six state agencies—without exposure to personally identifiable information or direct access to source systems. The governance framework, built on Data Sharing Agreements and InfoSec background checks, created a replicable compliance model for future multi-agency projects, with OTS serving as a neutral infrastructure host to resolve agency custody complications. Early legislative outputs have already influenced policy decisions, and the initiative positions Louisiana as a leader in evidence-based, inter-agency data governance.