Implementing machine-learning to deliver business intelligence and insights, improve efficiencies, and empower data-driven decisions, with Amazon QuickSight
Any enterprise serious about streamlining business workflows and improving efficiencies, would have made the move away from manual spreadsheets towards dynamic workspace solutions long ago. Even though basic spreadsheets still have relevance for certain tasks, there are associated risks, for example, calculation errors or version control, with several examples hitting the headlines. Using data management solutions, for example, Smartsheet, enables an organization’s data to be managed and accessed in a single online environment. However, the key to gaining competitive advantage is the ability to leverage automation to unlock intelligent and actionable insights from the data. Analysing data and reporting results, whether presenting internally or to clients, needs to be quick and intuitive, and the fewer manual processes involved the better. And yet, even with a dynamic workspace solution it can prove a challenge.
The more comprehensive a data management tool, the more complicated and time-consuming it can be to identify the information, stored in multiple worksheets, that comprehensively answers a business query. Collating data into a visually impactful dashboard is great for reviewing everyday business queries, but even creating dashboards requires a certain level of aptitude. When it comes to ad hoc business queries, many hours may be spent reaching a conclusive answer.
And this was exactly what we were finding, here at SourceFuse. The company had been using Smartsheet for several years, but experienced several challenges when responding to ad hoc queries, including:
- Manual efforts: with no intuitive way to work on multiple sheets, end-users had to manually search for the information by creating a calculated column. Unfortunately, not all end-users possessed the skills required to write these calculations, and in this case, they would have to tediously scroll through the sheet.
- Skill dependency: in order to use Smartsheet as effectively as possible, end-users would need to rely on an expert with the right technical knowledge, in order to generate insights with complex formulas.
- Performance: the need for creating a lot of calculations was slowing down the time for retrieving a result. In addition, time-to-result was impacted when multiple users were trying to retrieve some information at same time.
- User interface: if multiple users logged in simultaneously, it would slow down the processing of information, and cause further delays when responding to clients. The UI included filters, which helped narrow down the search, but users would still need to know in which files to locate the relevant information.
Time for Change: The SourceFuse Approach
Building on our ethos of digital transformation, deploying artificial intelligence and machine learning would overcome these challenges. However, the key would be a seamless integration of Smartsheet, without disrupting day-to-day processes and workflows. Building an automated data pipeline would mean the source data, in Smartsheet, is regularly updated so that it’s readily available for consumption.
As an AWS Advanced Consulting Partner, we were provided the unique opportunity to preview a new tool, called Amazon QuickSight Q. Quicksight Q leverages AWS machine learning natural language capabilities, enabling businesses to ask queries about their data in plain English, and get highly accurate answers in seconds. This game-changing solution uses machine learning to understand the relationship between data from multiple sources, without the need for complicated dashboard development. A user simply poses a question, and Amazon QuickSight Q will return the answer in an appropriate format, for example, tabular, chart or graphical.
Since Smartsheet was heavily embedded in day-to-day business processes and used by multiple departments, rather than the upheaval and disruption of replacing it entirely, we were able to use it as the ‘data source’. Building data pipeline automation then fed data into the new architecture, which incorporated AWS QuickSight Q:
Additional AWS Services Incorporated Include:
- AWS Glue – for simple, scalable, and serverless data integration and extraction
- AWS S3 – for data storage and retrieval
- AWS CloudWatch – to maintain logs for glue job
- AWS CloudTrail – for tracking user activity in AWS QuickSight Q
- AWS Identity and Access Management (IAM) – for managing users access and policy
- AWS Key Management Service (KMS) – for controlling the keys used to encrypt or digitally sign data
Amazon QuickSight Q: The Results
Being part of AWS’s incubation program, we were able to collaborate with the AWS solution build team, and provide relevant feedback. Examples of business queries that are now answered by AWS Quicksight Q in seconds include:
- What is the monthly revenue by [project] in 2020?
- What is the average salary for an iOS developer?
- What projects are due for renewal in September 2021?
- Which employees have a leaving date in September 2021?
- What is the billable utilization by [Resource] in January 2021?
- What is the project name and end date for [client]?
Since implementing AWS QuickSight Q, the company is now experiencing many workflow benefits, such as:
- Improved productivity: no need to manually manipulate data and track information across various sheets.
- Increased efficiency: end-users are no longer dependent on an in-house expert to develop reports or create dashboards as and when new data fields are added. AWS Quicksight Q handles a comprehensive menu of aggregations, such as sum, count, min, max, average, so users no longer have to create separate calculated fields.
- Empowered users: the intuitive nature of AWS QuickSight Q means no training is required in order for business users to find answers to relevant business questions.
- Better performance: multiple users can now log into the AWS Quicksight Q application without there being any impact on speed or performance.
Readily accessible and basic spreadsheets are still relevant for ad hoc analysis, but require a lot of manual and skilled manipulation to output complex reports. The ability to perform predictive, descriptive, or prescriptive data analytics, and identify relationships between data is lacking.
AWS QuickSight Q comes with inbuilt industry-based data knowledge so it understands your business language and terminology from day one. The machine learning capabilities automatically improves and hones its understanding of the data relationships, providing increasingly accurate answers with relevant visualizations.
And the quality of responses gets better and better over time. The more questions that are asked, the more the users can see the most popular queries, and feed those results back into the dashboard. And if you make typos or can’t remember exact phrasing, AWS QuickSight Q provides suggestions for phrases, business terms, and performs a spell check, making your search super-efficient and easy.
With this modernized infrastructure, the knock-on effect has also provided:
- 100% automation via AI functionality, eliminating repetitive manual processes
- Enhanced security, through authorized access via AWS IAM and data encryption via AWS KMS
- Reduced costs, by deploying a ‘pay for what you use’ cloud application.