Tuesday, April 16, 2019

Augmented Analytics: What It Is and Why You Need It

You hear it everywhere that business teams are hungry for analytics. They crave accurate predictions and forecasts to allow them to make better business decisions, faster. The problem is data has become so complex that most of the time they have to wait for it to be analyzed and presented. They waste time waiting and when the report finally arrives it doesn’t provide what they wanted.

The whole exercise of doing business analytics today is a time-consuming, costly, and frustrating experience for everyone involved. At the same time, getting actionable insights into the hands of everyone is increasingly important to improving business operations and future competitiveness. Ugh. It’s time for something to change in a big way.

This is why you're seeing a new wave of disruption in data analytics tools with the concept of augmented analytics gaining momentum.

“Augmented analytics will democratize insights from analytics including AI to all business roles. It will make data science and ML/AI model building accessible to new citizen data science roles (business analysts, developers, and others) while making expert data scientists more productive and collaborative, freeing them for high-value tasks.” - Gartner

What Is Augmented Analytics?

Augmented analytics gives anyone from technical users to data scientists, to business users the ability to access the intelligence they need to make decisions. It combines AI and Natural Language Processing (NLP) to expose previously unseen insights on any data.

Under the hood, an augmented analytics platform uses artificial intelligence and machine learning to automate data preparation, insight discovery, and sharing. They may still include traditional BI functions like dashboards and reports but augment them with natural language BI for a search-like experience. These interfaces allow business users to interact with their data and find insights on their own. Imagine what your data science teams could get up to if they didn’t have to answer every single business user’s question.

For your data scientists, augmented analytics provides an approach that automates insights using machine learning. It’s unlikely full automation is possible but a number of the more time-consuming steps to discover and prep data can be automated. It’s anticipated that automating these steps alone could shave weeks off analytics projects. Additionally, because these platforms can connect directly to any data, anywhere, the chance for adding unintended bias is reduced. This is important when creating training datasets for machine learning because removing bias will make models more accurate.

Why You Need It

Augmented analytics improves multiple areas within your current analytics workflows. Some of these include:

Augmented interfaces: you can expect augmented analytics tools to be easier to use. Providing natural language processing as well as conversational interfaces are naturally more intuitive and user-friendly for business people.

Automate data discovery: because augmented platforms are connected directly to data sources, data engineers can instantly discover data, relationships, and correlations. This enables them to dynamically blend data across sources. The result is the creation of the widest possible datasets within a few minutes.

More agile business intelligence: augmented analytics will enable data analytics to be delivered faster across the enterprise. Analytics will require fewer resources but increase data access by allowing business users a true self-service experience.

Benefits for Every Business

Augmented analytics solutions hold the promise to transform everything about how we do analytics today. It gives businesses of any size the ability to use data to their advantage by:

  • Better utilizing analytics resources by eliminating time-consuming data prep tasks
  • Enabling integration to any data not matter its structure or size
  • Streamline data pipelines even when using complex data
  • Democratizing data through more natural and conversational interfaces 
  • Broadening analytics adoption by embedding analytics in applications people are already using, making it easier to access actionable intelligence
  • Exposing more relevant insights 

The net result of implementing an augmented analytics solution is it exponentially increases in your organizational data literacy. You’ve broken down the old barriers so data and analytics become the backbone of every decision at every level of your organization.

With that, you leap forward in the analytics maturity model and never need to look back.  We’re at the beginning of the augmented analytics journey and it promises to be an exciting ride. Buckle up!

Tuesday, February 5, 2019

How to Tell a Compelling Story with Your Data Using Knowi

Great Storytelling Starts with Using the Right Data Visualization



Knowi Data Visualizations

Choosing the right visualization to tell your story is important, as the wrong visualization can cause confusion and distract your audience.  We walk you through dozens of visualization types and how to best use them to get your point across. 


We all have our goto charts for visualizing data but if you need to present data to others, you have to think from the viewer perspective.  Picking the wrong data visualization can easily confuse the viewer and lead to mistaken data interpretation. Before you create a chart, first understand the reasons for the chart. Start by asking a couple of questions, What are you trying to convey? What conclusions do you want the viewer to see using their data? This will help you narrow down the type of data visualization to use.

COMPARING VALUES

There are two main categories for comparing values: 1) you want to compare one or many items to show low and high values; 2) and you want to show trends over a time period. For example, if you have specific time intervals you want to highlight, such as hours, months, quarters, Line, Column, or Area charts are good options. If linear progression is not available in the dataset, Column and Bar charts are commonly used for simple comparisons among items. If allowing for viewers to search data is required, Data Grids (tables) or Pivot Tables are typically used.


Knowi comparison charts











CONVEYING RELATIONSHIPS

Relationship charts are suited to showing how one variable relates to one or more other variables. You could use relationship charts to show correlation or connections between variables in a data set. For example, Scatter Plot, Bubble, Heatmaps and Dual Axis Column charts are good for showing how something positively effects or negatively effects another variable over time. Cord Diagrams are a really interesting way to show connections between the data.

Knowi relationship charts














SHOWING DATA COMPOSITION

Data composition charts show how individual parts make up the whole of something and change over time. These charts are used to show how something is divided up. The only rule to remember is the sum of the parts must total 100%. If you have relatively few data points, Pie, Doughnut and Stacked Column charts are good options. If you have a large number of data points, try using a Treemap to keep the visualization compact but still readable. Sankey is a good option to show flows as well as proportion such as user behavior flow through app pages. Waterfall charts are used to understand the influence of several positive and negative factors on the initial value. They are typically used for financial reporting such as visualizing financial statements.

Knowi composition charts













SHOWING DATA DISTRIBUTION


Visualizing the distribution of data in a given interval, geographic area, etc. is a great solution for quickly spotting outliers, clustering trends, and relationships within a data set. Bubble, Heatmaps, Box Plot visualizations display frequency, how data spread out over an interval, or how data is grouped. Maps also show frequency and data spread but add geographic context so are often used for population-based visualizations. Word Count diagrams are used to show text-based distributions. For example, Word Count can quickly visualize top revenue generating customers.

Knowi data distribution charts



















KEY PERFORMANCE MEASUREMENT

If you are building executive or business performance visualizations, you typically want to include Key Performance Indicators (KPI) measurement. These visualizations are great for displaying a single value/measure within a quantitative context such as comparing to the previous period or to a target value. It allows viewers to see, at a glance, totals such as sales, revenue to target, number of active users, etc.

This is probably the easiest data visualization type to build with the only consideration being the period you want to track. You just need to remember that less is more with KPI visualizations. KPIs can be very impactful on a dashboard but the more of the same type of visualization you add the more you risk diluting the key metrics. Reserve these visualizations for the important metrics you want to immediately draw the viewers
attention towards.

Knowi KPI charts








IOT ANALYTICS

In IoT use cases, typically the end users are non-technical so selecting the right visualization is critical to making complex IoT data easily consumable. Simplifying the IoT data story can be accomplished using a variety of visualizations including Timelines, Heatmaps, Geo-cluster, and Image overlays. For example, a Timeline diagram can instantly show if all the lights in a building were off overnight. A geo-cluster diagram or image overlay can show where sensors are deployed and their health with drill-down capabilities to see specific device details. Bubble charts and heatmaps are a quick way to highlight anomalies for further introspection.

Knowi IoT Analytics Charts


















Hope you found this useful.  If you'd like to get a PDF version, click here.

Saturday, February 2, 2019

How Three 2019 Analytics Trends Change the Status Quo


Originally posted on medium.com

Demands of the Digital Enterprise Force Rapid Innovation Towards Augmented Analytics

Before we dive in, let me overlay this discussion with the main business trend that is behind some of these technology trends. Digital Transformation.
“Digital transformation marks a radical rethinking of how an organization uses technology, people and processes to radically change business performance”, says George Westerman, a principal research scientist with the MIT Sloan Initiative on the Digital Economy.


According to a recent CIO Gartner survey, Digital initiatives and revenue growth are top priorities for enterprises in 2019. Digital initiatives combine disruptive technologies, like Big Data and AI, with existing infrastructure to innovate business processes and deliver new products and services that improve customer experience and therefore drive growth.




In 2019, digital initiative investments cover a wide range of technology with analytics and BI solutions leading the list. According to the survey, investment in analytics and BI tools is expected to increase by 45% to support Digital Transformation efforts. Technology leaders see analytics as a core component to successfully delivering transformative solutions to the business. With that, let’s breakdown three of the top analytics trends for 2019 and how they support Digital Transformation.

2019 Analytics Trends to Watch

According to Forrester, insight-driven organizations grow 30% faster than their less informed peers.

1. Analytics Reaches the Other 78%

In the past, self-service analytics promised to enable business teams with analytics, but reality shows that traditional BI, even with self-service, achieve adoption rates of less than 25%. There are a variety of reasons for this anemic performance, but in simple terms, it is because they require business users to understand too much about how the data is constructed and dedicate precious time to learning a new application.

As a result, most business users rely on others to answer even basic questions which could mean they wait days, if not weeks, for the result. If only 25% of your organization has access to analytics, it is virtually impossible to become a data-driven organization.
To combat this problem, in 2019, more analytics platforms will integrate Natural Language into their platforms. Natural Language Processing (NLP) promises to turn self-service analytics on its head by focusing on delivering solutions for non-technical business users, i.e., the other 78%.

What Is Natural Language Processing?
Natural Language Processing (NLP) is a form of deep learning AI that creates the ability for a computer to understand human language. In the past, we had to learn computer languages like C++, Javascript, etc. for a computer program to do what we wanted it to do. The shoe is finally on the other foot! With NLP, computers learn our language, English, French, Chinese, etc. and interpret key elements of a sentence to understand the intent. Sentiment analysis is a good example of how NLP is used in analytics today to understand the intent behind a statement. More recently, NLP got a voice. Call her Siri or Alexa but behind the scenes is highly complex NLP that is learning every time you talk to her.

Natural Language BI allows business users to ask questions, in human language, and get answers, usually in the form of a dynamically generated chart. They can ask follow-on questions to explore deeper and run what-if scenarios without needing to understand how the data is constructed. More importantly, the interactions with data analytics can be done within applications your business users are already familiar with using, like Slack, using NLP APIs. Done right, Natural Language BI has the promise to close the data gap for business users and create the data agility needed to uncover insights that can transform business.



2. Self-Service BI Rebrands to Augmented Analytics

Adding NLP to BI platforms is just the beginning changes we’ll see in 2019 geared towards non-technical business users and in support of Digital Transformation efforts.

Let’s say you run a SaaS company. Like all SaaS companies, an important metric you pay close attention to is Churn Rate. With natural language queries, you could ask a question like “what was my churn rate for the last three quarters.” It is obvious that you’re trying to understand if actions you’ve taken have caused any changes (good or bad) to your churn rate. However, the question you want to ask is “what should I change to reduce my churn rate?”

The answer to this question is at the junction of AI and NLP. NLP enables the question to be asked, in plain English, and AI answers the question. This is also where AI and BI converge to create something new, Augmented Analytics.
In 2019, analytics platforms will move in earnest to leverage AI/machine learning under the hood to prepare data for analysis, analyze it, and interpret results into predictive insights that can be used to take action.



3. Analytics Platforms get Smarter about Data Discovery

Data is only getting more complex and dispersed. Enterprises are awash with data, but IT continues to struggle with finding ways to collect it from diverse sources and make available in time for the business to get value out of it. The current status quo is to involve multiple teams to move, aggregate, transform and prep the raw data before using it for machine learning or analytics. This means you are applying machine learning on already biased data that was created based on predetermined questions.
Modern BI platforms with augmented data discovery capabilities will show two times the adoption growth rate of all other modern BI platforms by the year 2020, Gartner predicts
2019 marks an inflection point for analytics, as machine learning/AI becomes more mainstream the traditional extracting, transform, and load data into analytics platforms make even less sense. Data and business pushed by new digital initiative are just moving too fast to wait weeks for data pipelines to be built. We’ve entered a time where data agility is critical to success meaning the old way of custom coding data pipelines or building rigid schemas for a centralized data warehouses/data lakes needs a rethink.
In 2019, more analytics solution providers will expand their data management capabilities. New approaches, like smart data discovery, which applies machine learning/AI to raw data to discover relevant data relationships and correlations automatically will become more prevalent. The intent behind using AI here is to augment human component in a way that reduces unintended bias created by manipulating data when prepping it for analytics. By using smart data discovery on raw data, it becomes possible for the corresponding insights to be surfaced without any preconception of the answer or perhaps even the question.

Inhibitors

Implementing technology is only part of the battle there are other moving parts involved. Below are some potential inhibitors that need to be addressed alongside technology implementation if business leaders hope to advance their Digital Transformation efforts forwards in 2019.

AI Trust

AI is a black box and just because it has the “AI” stamp of approval doesn’t mean the result can be trusted. Business executives will ask how you came to a specific conclusion. Saying “the AI told me” probably isn’t the going to be enough to convince anyone to take action. Being able to explain how you came a particular recommendation is going to be required before anyone will trust the AI behind it.

Data Ethics

Data ethics is a top of mind topic for many with data privacy and data use laws like GDPR going into effect in 2018. Additionally, the use of personal data to feed the “fake news” engine may result in similar legislation being applied more globally. As a result, the topic of data ethics is no longer restricted to data collection but also applies to how data and resulting insights are used. With more data put into the hands of more people additional training on the do’s and don’ts of data use become essential to avoid unwanted appearances in news cycles.

Data Literacy

We’ve talked a lot about how technology is helping reach the other 78%, but there is still a human factor to the equation that can make or break success. Developing a data-driven culture within your organization means not only providing the tools but developing the processes and providing necessary data literacy training. Data literacy training ensures everyone knows how to perform experiments and create compelling stories using data to put forward a persuasive argument to take action.

Conclusion

This year will be interesting on many fronts but mostly to how analytics solution providers, new and old, start to leverage AI for good, the level of push back within leading organizations, and the corresponding fallout for product roadmaps. I think the technology is moving faster than business right now and the increase in anxiety AI creates for analytics leaders will force a slowdown in its adoption. Some of the predictions about the level of impact of AI in the market sound overly aggressive, at least to me. However, the movement is unmistakable. If you are in a position to influence your organization’s analytics strategy, it will make sense to prepare for the world of augmented analytics. Begin to assess the impacts implementing augmented analytics may have to data architectures and near term technology decisions. At the same time, the long pole for Digital Transformation is more likely to be process and people than technology. History has proven that humans are good at stifling innovation so the sooner you being to evangelize augmented analytics, to inspire imaginations versus instill fear, the more likely your Digital Transformation efforts will succeed enterprise-wide and actually transform how your business operates.

Tuesday, December 4, 2018

Knowi Product Update - Q3 2018




We are super excited to announce that we've added Natural Language BI to the Knowi platform.  With this addition, business users gain the ability to type, in English, a query and watch as we automagically interpret the question and present the results.  It is specifically designed to provide self-service analytics capabilities to non-technical end users.  They can ask questions and get answers without an in-depth understanding of the data or how it is constructed.

There is a lot more to tell you about so keep reading!  If you're more of a visual person, you can watch the replay of our product update webinar here.

Use Your Words - Natural Language Query

Natural Language BI is a powerful way to enable self-service analytics to non-technical users by enabling them to ask questions in plain English and get to insights quickly without needing any in-depth knowledge of the data or how it was constructed.

Natural Language BI is available for all widgets within a dashboard. At the moment, it can be accessed using the Natural Language/Self Serve Analytics icon on a widget or by going into Analyze mode. 

Start typing your question in the search bar.  We use machine learning to automatically suggest the next part of the question.  When you're happy with the question, hit enter, and the results will be displayed.  Easy as pie.

Image

Users can select the chart type they want to use and save the results to their dashboard to create their custom dashboards.



Trigger Alerts Gets a New Modern Look and Feel

If you're not familiar with Trigger Alert or Trigger Notifications, they are a powerful way to trigger an action based on criteria you specify on any dataset.  Trigger Alerts allows you to monitor datasets for conditions and send alerts, invoke an API, or send a notification to a Slack Channel.  This capability allows you to get your data working for you.   Instead of business users having to go to a dashboard to gain insights, the insights are delivered to them so they can take the appropriate action. 

To create a trigger alert, go to Alerts from Setting Menu.


Image



Enterprise Security Enhancements

Knowi now supports the setting up of a connection with an LDAP server which will then allow users to login into Knowi using their LDAP credentials. If you require this functionality, please contact support.

Support for sharing the administration rights on all Dashboards, Widgets, Reports, and Alerts has now been implemented. This will allow a user to grant admin access to assets they created for the purpose of on-going administration including edits, setting Global Filters, and changing widget level Cloud9QL. 

Image

For secure embedding use cases, you can now specify a time to expire for a given share URL so the link cannot be reused beyond the expiry date.



Other Cool Stuff
Localization
The display of time series chart legends and filters can be localized for a user by selecting the Locale option under User Settings. Useful for embedded dashboard and widget modes, this will then render any time series chart selected in the locale specified. Current locales supported are English (en), French (fr) and German (de).

Run-Time Date Query Parameters
Runtime parameters can be applied to Direct queries to pass dynamic parameters into a query.  We've added the ability to pass date parameters based on input from the UI or user configuration. 

Knowi Warehouse Retention
With ElasticStore usage, the overwrite strategy provides you powerful control over how data is managed.  We've added the ability within the overwrite strategy to specify how far back you want to retain your data.  

For example, Date-1m will remove all existing rows in ElasticStore which have Date field's value less than 1 month before the MIN value of the incoming data's Date field.  You can use data retention in conjunction with other overwrite strategies.

Cloud9QL Enhancements
Knowi's Cloud9QL library has been enhanced to now include the following date delta functions :
  • MINUTES_DELTA: number of minutes between two dates
  • HOURS_DELTA: number of hours between two dates
  • DAYS_DELTA: number of days between two dates
  • MONTHS_DELTA: number of months between two dates
  • REGEX_REPLACE: regex function that can act on a field

Tuesday, July 24, 2018

Knowi Product Update - Q2 2018



Join us on Wednesday, August 8, 2018, at 10:30AM Pacific for a live demo of the new features and enhancements we've added to the Knowi platform. If you can't make the time, register anyway and we will send you the recording of the session.  In the meantime, please keep reading to see a summary of the product updates for this past quarter.

Anomaly Detection

Time-series anomaly detection is used to identify unusual patterns that do not conform to expected behavior otherwise know as outliers. There are a number of business applications for this type of machine learning. For example, IoT use cases where an unusual traffic pattern that might indicate an issue with a traffic light and business applications like detecting strange network behavior that could indicate a hack attempt.

We provide a number of anomaly forecasting algorithms within the workspace so you can determine the best one for your specific use case. To use, simply select Anomaly Detection as your machine learning option and create a new workspace. Follow the configuration steps and test out different algorithms for accuracy. The precision of the model increases over time as more data is made available.



The anomaly detection visualization itself consists of a configurable blue band range of expected values (acceptable threshold limit) along with the actual metric data points. Any values outside of the blue band range are considered anomalies and will appear in red.





Sparkly New Data Visualizations

Data Grid

You may think data grids are boring but then you haven't tried our new data grid visualization. You can do a ton with this new grid type, including formatting, conditional highlighting, sorting, grouping, and search along with the ability to download formatted grid information in Excel format.

The other pretty cool thing you can do is add charts to cells. The embedded chart options are sparkline, area, bar, spline, spline area, and pie. Not so boring anymore, right?

Knowi Data Grid with Sparkle line


Image Overlay Heatmap

You can now upload an image and overlay x, y coordinate values. Example use cases are tracking how a stadium is filling up as people are scanned at the entry gates or showing the location of IoT sensors deployed on each floor of a building.

Knowi image overlay data visualization




It's Your Dashboard - Customize It!

We enhanced the level of customization of the look and feel of your dashboards by adding several new configurable options.





Getting Clicky With It

Tightly integrating Knowi visualizations within your applications is an important step to give users a seamless experience. To further that integration, we've added an OnClick Event Handler which is available for various visualizations. You can use it to customize what action to take when a data point is clicked on the visualization.






Other Cool Stuff
Connected Charts
Added a drill down feature to allow the current dashboard to be filtered with settings based on the clicked points.

Hidden Filters
Admin or dashboard owners can now hide certain filters from end users.

REST-API Enhancements
Added epoch secs support for REST API datasource url parameters, which allows you to pass in date in epoch time formats, dynamically.

Added a way to handle the return of multiple JSON objects within the same file

Cloud9QL Enhancements
We've added an ARRAY function that creates an array with a specified grouping to return an array for the field.  We've also added  UPPER and LOWER functions to change the case of string fields.