Balancing Innovation and Compliance: How New Regulations Will Shape Data Strategies

8 min readAug 1, 2024

As we go about our daily lives, we generate near-unfathomable amounts of data. From online purchases and fitness tracking to social media posts and using maps on our phones, our digital footprints are expanding rapidly. This data holds immense potential for tailored preventative healthcare, enhanced quality of life, and innovations in public health.

The scale of this data generation is staggering. By 2025, the average number of data interactions per connected person per day is expected to reach almost 5,000, up from 298 in 2010[1]. The global datasphere is projected to grow to 175,000,000,000,000 gigabytes, (175 zettabytes) by 2025[2]. To put this in perspective, this is about three and a half billion times larger than the largest storage capacity of the latest iPhone. However, a recent survey from Seagate reveals that only 32% of data available to enterprises is put to work, leaving 68% unleveraged[3].

In regulated sectors like healthcare, the challenges are even more pronounced. While healthcare generates an estimated 30% of the world’s data volume — equivalent to about 52,500,000,000,000 gigabytes (52.5 zettabytes) by 2025[4] — a large portion remains unutilized[5]. This stands in stark contrast to less regulated sectors like manufacturing, which is expected to produce 44,000,000,000,000 gigabytes (44 zettabytes) of data annually by 2025[6].

We find ourselves at a unique inflection point, where unbridled opportunity meets new headwinds of regulatory complexity. Emerging regulations worldwide are giving consumers more rights over their data, including privacy, security, and control over usage and sharing. When dealing with healthcare data, especially in the United States, navigating these regulations becomes particularly challenging, but also represents an unprecedented opportunity. As technology advances and data analysis techniques improve, there’s enormous potential to explore the value hidden within regulated data, which remains a largely untapped resource.

Unleveraged data is often attributed to regulatory constraints, but it’s not the whole story. While regulations are routinely cited as a blocker, other factors contribute to this underutilization. These include the cost of data storage and movement, as well as the complexity of implementing advanced data analysis techniques. The key lies in bridging the gap between the vast amounts of data we produce and its effective utilization. By addressing the reasons behind data underutilization and finding creative solutions that maintain compliance, we can support the full potential of our data resources. The challenge now is to stay compliant while getting creative in our approaches to data analysis and utilization.

To better understand the complex web of regulations governing sensitive data, let’s follow an example, using the journey of Sarah, a 35-year-old marketing professional living in California who frequently travels for work. By tracing how Sarah’s personal information moves through various systems and companies, we can illustrate how different privacy laws and regulations apply at each step. This example will help clarify which regulations come into play, when and how they affect Sarah’s data, and the responsibilities of companies handling her information.

At the federal level, Sarah’s data is protected by:

The Health Insurance Portability and Accountability Act (HIPAA): Safeguards Sarah’s medical information when she visits her doctor or fills a prescription.
The Gramm-Leach-Bliley Act (GLBA): Protects Sarah’s financial data when she uses her bank accounts or credit cards.
The Fair Credit Reporting Act (FCRA): Governs how Sarah’s credit information is collected, used, and shared.

As a California resident, Sarah benefits from state-specific protections:

The California Consumer Privacy Act (CCPA) and its update, the California Privacy Rights Act (CPRA): These give Sarah more control over her personal information, including the right to know what data companies collect about her and the ability to request its deletion.

When Sarah travels to New York for work, her data falls under the protection of:

The New York SHIELD Act: Requires companies to implement safeguards for her private information and notify her of breaches.

During her business trip to Europe, Sarah’s data is subject to:

The General Data Protection Regulation (GDPR): This comprehensive law gives Sarah extensive rights over her personal data, including the right to access, rectify, and erase her information.

As Sarah uses various online services and apps, her data may also be governed by:

The Children’s Online Privacy Protection Act (COPPA): If Sarah has children who use online services, this law protects their privacy.
The newly enacted American Data Privacy and Protection Act (ADPPA): This federal law aims to provide a comprehensive framework for data protection across the United States.

Recent developments that affect Sarah’s data privacy include:

The Washington My Health My Data Act (MHMDA): While Sarah doesn’t live in Washington, this law could impact how health-related apps and websites handle her data if she uses services based in that state.
Updates to HIPAA: Recent changes have expanded patients’ rights to access their health information and strengthened protections for electronic health records.

This consumer-centric view illustrates the complexity of data protection regulations. As Sarah goes about her daily life — shopping online, using social media, traveling for work, and managing her health — her data intersects with numerous regulatory frameworks. Each of these laws aims to protect different aspects of Sarah’s privacy, but they also create a complex landscape for businesses to navigate.

The challenge for organizations is to ensure compliance with all applicable regulations while still leveraging data to provide Sarah with the personalized, efficient services she expects. This balancing act requires a sophisticated approach to data management, one that puts the consumer at the center while also meeting the diverse requirements of an ever-evolving regulatory landscape.

Common Pitfalls and Misconceptions

“The regulatory landscape for data privacy is evolving rapidly, presenting both challenges and opportunities for businesses. Many organizations fall into the trap of relying solely on basic de-identification techniques like tokenization, but true compliance goes far beyond that. It’s about understanding the entire lifecycle of data — from collection to deletion — and implementing robust systems for chain of custody, access auditing, and continuous risk assessment.” — Brian Mullin, CEO at Karlsgate.

Navigating the intricacies of such a rapidly evolving regulatory landscape is no easy task. Regulation has long been considered a “dirty word” for companies. Complying with regulations can be resource-intensive for a company. Pair this with often minimally precise definitions of how to confirm compliance with regulated data, companies are faced with a complex decision matrix of the risk/reward/consequence of to do or not to do.

One frequent mistake is the misinterpretation of regulatory scope. For instance, some organizations assume that if they’re compliant with one regulation (e.g., HIPAA), they’re automatically compliant with others (e.g., GDPR). This misconception can lead to significant gaps in data protection strategies.

Another common error is inadequate risk assessment. Many organizations conduct superficial assessments that fail to identify all potential vulnerabilities in their data handling processes. This oversight can result in unexpected compliance issues and data breaches.

Insufficient data governance practices also pose a significant risk. Without clear policies and procedures for data collection, storage, and usage, organizations may inadvertently violate regulatory requirements or mishandle sensitive information.

There are also widespread misconceptions about de-identification and data sharing. For example:

Myth: Simple anonymization techniques (like removing names and addresses) are sufficient to protect privacy.
Reality: Advanced re-identification techniques can often piece together supposedly anonymized data, necessitating more sophisticated approaches to truly safeguard sensitive information.
Myth: Once data is de-identified, it can be freely shared without regulatory concerns.
Reality: Many regulations have specific requirements for de-identified data, and re-identification risk must be continually assessed.

Understanding these pitfalls and misconceptions is the first step toward developing a robust, privacy-preserving data strategy.

Best Practices for Compliance

To effectively manage regulated data, organizations can benefit from implementing a comprehensive framework for assessing and mitigating data privacy risks. This framework should include:

Data Management:

Regular data audits and classification: Systematically catalog all data assets, classifying them based on sensitivity and applicable regulations.
Data minimization: Collect and retain only the data necessary for specific, defined purposes.

Security Measures:

Implementing robust access controls: Utilize the principle of least privilege, ensuring that individuals only have access to the data necessary for their roles.
Continuous monitoring and updating of security measures: Regularly assess and upgrade security protocols to address emerging threats and vulnerabilities.

Organizational Culture:

Employee training and awareness programs: Foster a culture of data privacy awareness throughout the organization.
Privacy by design: Incorporate privacy considerations into every stage of product and process development.

It’s crucial to emphasize that compliance is not a one-time effort but an ongoing process. This is where solutions like those offered by Integral play a vital role, particularly in the pre-purchase evaluation and continuous monitoring of datasets.

To address these challenges and implement best practices, many organizations are turning to specialized solutions like those offered by Integral. Integral enables companies to thoroughly assess privacy-sensitive data before acquisition and maintain compliance throughout its lifecycle. This approach is critical in today’s data-driven environment, where understanding the full implications of a dataset is essential for both compliance and strategic value.

This method addresses several key challenges:

Pre-purchase Evaluation: Integral helps organizations thoroughly assess datasets before acquisition. This includes understanding the embedded risks and opportunities for remediation before purchase.
Attribute Intent and Necessity: By analyzing the purpose and relevance of each data attribute, Integral ensures that organizations only acquire and retain data that is truly necessary for their business objectives. This aligns with data minimization principles and reduces compliance risks.
Business Use Case Alignment: Integral’s solutions help organizations clearly define and document the specific business use cases for each dataset. This ensures that data usage remains within the bounds of its intended purpose, a key requirement in many privacy regulations.
Continuous Monitoring: As datasets evolve and business needs change, Integral provides ongoing monitoring to detect any shifts that might impact compliance or data utility. This includes alerting organizations to new sensitive data elements, changes in data quality, or alterations that might affect the original use case.
Change Management: When changes are made to a dataset, Integral helps organizations reassess the impact on privacy, utility, and compliance. This includes evaluating whether new attributes introduce additional risks or if removed attributes affect the dataset’s intended functionality.
Efficiency and Agility: By streamlining the process of evaluating and monitoring regulated data, Integral helps organizations remain responsive to market demands while maintaining compliance.

By leveraging these capabilities, organizations can transform their approach to regulated data from a reactive compliance exercise into a proactive strategy for responsible data management. This not only ensures regulatory compliance but also maximizes the strategic value of data assets while minimizing risk.

Future Outlook

As regulations continue to evolve and data volumes grow, several key trends are likely to shape the future of regulated data management:

Increased regulatory scrutiny: As data breaches and privacy concerns continue to make headlines, we can expect more stringent regulations and enforcement.
Global standards: There may be movement towards more unified, global standards for data protection, potentially simplifying compliance for multinational organizations.
AI and machine learning regulations: As these technologies become more prevalent in data analysis, new regulations may emerge to govern their use with sensitive data.
Enhanced individual rights: Future regulations may grant individuals even greater control over their personal data, including more comprehensive “right to be forgotten” provisions.
Emphasis on algorithmic fairness: Regulations may increasingly focus on ensuring that data-driven algorithms do not perpetuate biases or discrimination.

Collaboration between industry stakeholders and regulators will be crucial in shaping these future regulations. Organizations that proactively engage in these discussions and adapt their strategies accordingly will be better positioned to thrive in this evolving landscape.

Conclusion

Navigating the complex world of regulated data requires a careful balance between innovation and compliance. By understanding the regulatory landscape, avoiding common pitfalls, and implementing best practices, organizations can maximize the potential of their data assets while maintaining the trust of their stakeholders.

Viewing compliance as a foundation for data-driven innovation, rather than a burden, paves the way for organizational success. By striking the right balance between innovation and compliance, organizations can not only navigate the complexities of new regulations but also leverage them as a catalyst for responsible data-driven growth and innovation.

About Integral

Integral enables companies to safely leverage regulated data at unprecedented speeds by automating the data de-identification and compliance certification process, allowing our customers to stay agile and iteratively drive outcomes. www.useintegral.com

[1] https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf

[2] https://www.forbes.com/sites/tomcoughlin/2018/11/27/175-zettabytes-by-2025/

[3] https://www.seagate.com/files/www-content/our-story/rethink-data/files/Rethink_Data_Report_2020.pdf

[4] https://www.rbccm.com/en/gib/healthcare/episode/the_healthcare_data_explosion#content-panel

[5] https://www.wipo.int/edocs/pubdocs/en/wipo_pub_gii_2019-chapter8.pdf

[6] https://www.statista.com/statistics/871513/worldwide-data-created/