Ethics and compliance (E&C) play a vital role in strengthening business culture. Derived from a company’s core values, ethics determine decisions, choices, actions, and behaviors, and can be thought of as codes of conduct. Alongside ethics sits compliance – conforming to additional external standards, rules, or laws. For example, a financial organization provides baseline employee training that underpins its company codes of conduct, whereas industry-specific regulations may require additional training, e.g., cybersecurity training.
Most enterprises provide annual online E&C programs, using various available platforms. The customer at the center of this case study inspires principled performance in organizations by providing end-to-end ethics and compliance management programs. However, the question remained: how can the value of these programs be measured?
The customer was looking for an innovative solution to consolidate data from disparate sources and then display company insights and industry/competitor benchmarks based on measuring ongoing program effectiveness.
Providing 500+ courses in 70+ languages, the company inspires 30+ million learners each year – that generates a lot of data. It provided different dashboards to its clients representing critical metrics on the effectiveness of its E&C programs. The dashboards were developed using Angular, Node.js, and third-party open-source libraries, with MongoDB as a data warehouse.
The entire architecture depended on many separate systems, with incoming data being collated from a variety of sources to display different results/reports:
|Dashboard / Report||Data Sources|
|Culture Pulse Survey and EOCS (End of Course Survey)||
|Internal Usage POC||
One of the key challenges was the ability to segregate client data efficiently and effectively, and reduce manual data processing. In addition, due to the number of languages served, the company wanted to effortlessly manage region-level data being generated using a generalized structure. The core challenges included:
SourceFuse was selected to partner with the company following a successful RFP response, based on its cloud migration experience and technical expertise.
Taking a discovery-first approach, SourceFuse carried out a full assessment of the customer’s current ecosystem and took the time to identify and understand the desired business objectives. It then developed the business case for providing a very well-structured Data Lake on AWS which could create an infrastructure capable of handling data from different regions. All incoming data is now consolidated into a single data lake providing the appropriate level of advanced security, scalability, and availability.
Additionally, the proposed AWS infrastructure provided a reliable structure to manage the various types of data ingestion and transformation. In this way, the datasets created would be represented visually via a dashboard in a flawless form, which will also incorporate end-user access authentication and authorization.
Sourcefuse delivered a modernized automated data migration and extracted data processing residing in a well-architectured cloud Data Lake. This handles complex data formats while controlling the flow of data in a very segmented form, helping to rapidly process data to its endpoint. The speedy data mart, used for processing the results, increases the data refresh rate from a daily to an hourly basis, which enables real-time data visualization on the dashboard. A secured data pipeline enables unified data sets, provides controlled access to only authorized end-users, and enables cross-region data sharing for comparative data visualization.