The Objectives

Our client has embarked on a mission to develop Synopsis Clinical, an enterprise-level analytics dashboard product suite. This initiative aimed to integrate data from diverse sources such as CTMS, OSA, EBS, and Veeva Vault. It encompassed automating manual processes, centralizing study data, and constructing a consolidated KPI dashboard for Clinical leadership. This transformative project had the primary business objective of empowering more than 13,000 users with data-driven decision-making capabilities, thus revolutionizing data analysis within the Clinical domain at our client’s esteemed organization.

Challenge

• Data Integration Complexity: Clinical research customers struggle to compile and integrate data from various sources, including CTMS, OSA, Oracle (EBS), Veeva Vault, and more, hampering effective data management.
• Manual Spreadsheet Usage: Some coordinators still rely on manual spreadsheets for critical business decisions, leading to significant inefficiencies.
• Key Challenges in Data Consolidation and Monitoring:
• Site Development Oversight: Difficulty monitoring and managing different stages of clinical trial site development.
• Research Site Selection Challenges: Significant hurdles in selecting suitable research sites for projects.
• Study Data Centralization Issues: Difficulty consolidating and centralizing study-related information from diverse sources.
• Patient Recruitment and Retention Struggles: Formidable challenges in ensuring successful recruitment and patient retention in clinical trials.
• Preserving Study Data Quality: Facing difficulties in maintaining the quality and integrity of clinical study data.
• Inefficiencies in Payment Tracking: Dealing with an inefficient and error-prone process for managing and tracking payments to investigators.
• Study Progress and Budget Management Complexities: Complexity in tracking study progress and efficiently managing budgets from inception to date.

Solution 

Data Integration and Processing:

• Integrated Data Sources: Successfully integrated data from diverse sources, including Clinical Trial Management System (CTMS), Oracle Site Activate (OSA), Oracle (EBS), and Veeva Vault.
• Centralized KPI Dashboard: Implemented a unified dashboard for Clinical Leadership and Project Managers, consolidating clinical study KPIs.
• Building Delta Lake: Utilized Databricks Delta Lake and partitioning to seamlessly integrate 19+ source system raw data into the cloud.
• Streamlined Data Ingestion: Azure Data Factory (ADF) and Databricks Notebooks were deployed for data ingestion and scheduling, ensuring a smooth data flow.
• Streamlined Site Life Cycle Tracking: Successfully developed the Study Start Up Module (SSU) for monitoring and managing site status.
• Patient Enrollment Monitoring: Achieved integration of CTMS data to track patient statistics.
• Enhanced Data Security: Robust role-based security measures were implemented at workspace and cluster levels.

Architectural Framework:

• Seamless Data Integration: Successfully ingested raw data from all source systems into Azure Data Lake Storage (ADLS).
• Structured Data Model: Formalized data within a Common Data Model, enabling standardized data handling.
• Optimized Data Flow: Data from the Common Data Model flows seamlessly into a Presentation Layer, facilitating Spotfire visualizations.

Portability and Reusability:

• Open Data Formats: Strategically architected the solution to embrace open data format standards, ensuring portability across diverse environments.
• Presentation Layer Migration: Successfully migrated presentation tables to Databricks for optimized Spotfire Dashboards.

Capacity and Performance:

• Capacity Expansion: Addressed capacity needs without additional infrastructure, following the successful 2020 re-architecture.
• Performance SLAs: Engineered the application to meet performance Service Level Agreements (SLAs), ensuring swift login, dashboard load times, and seamless navigation.

High Availability and Scalability:

• 99.999% Uptime: Engineered the application for a 99.999% uptime, ensuring continuous availability of Spotfire, Azure Data Lake, and network connections.
• Scalability Prepared: Anticipated scalability to accommodate the expanding user base, ensuring smooth operation.

Impact

• 30% Increase in User Efficiency: Empowering Over 13,000 Users with Enhanced Data Accessibility.
• 99.999% Uptime Ensured: Exceptional Uptime for Seamless Operations.
• 50% Faster Insights: Accelerating Decision-Making with Real-Time Data.
• Enhanced Data Accuracy for Informed Decision-Making: Streamlining data integration and management for Reliable Information.
• Faster Patient Enrollment: Reducing Enrollment Time for Swift Patient Engagement.
• Substantial Improvement in Patient Retention: Leveraging Data Insights for Increased Retention Rates.