Build end-to-end solutions with Fabric Real-Time Intelligence

What is Real-Time Intelligence?

Real-Time Intelligence is a solution that allows the extraction and visualisation of data as it happens in real-time, turning it into a live analytics framework. Furthermore, this solution enables the integration of scenarios such as events, data streams and logs in parallel, allowing them to converge in Real-Time Hub. Additionally, data from other sources can be connected and immediate analysis and reactions based on triggers can be generated.

Ingestion, transformation, storage, analysis, visualisation, tracking and artificial intelligence working in the same real-time environment is what Real-time data analytics enables. This means that data remains protected, governed and integrated into the business, aligned in the same data integration platform.

real-time-intelligence-fabric

End-to-End solution on Real-Time data analytics

To develop a real-time big data analytics project, every step of the project will be carried out in order to achieve the expected result. The stages will include the following:

  • Create a flow of events
  • Create a destination to send the transformed events
  • Transform the events
  • Create queries in the KQL database.

1. Get data in Real-Time hub

Working with data collection is the first step to start with.

data real time hub

This workshop is based on sample streaming data called Bicycles.

select-data-source-fabric

This sample is a bicycle data set with a preset outline column such as street, neighbourhood, number of bikes, location and more. This sample data allows the real-time number of bike events to be simulated and analysed with various destinations, such as the KQL database.

sample data solution with microsoft fabric

add destination kql database

Once published, select Add Destination to work with the KQL Database. To continue with the configuration, the KQL database data must be configured.

configuring kql database microsoft fabric

kql database microsoft fabric

2. Event processing

When the process is concluded, the data will be ready to be added to the transformations.

From the Eventstream canvas, transformations can also be performed to prepare the data according to the requirements.

transformation microsoft fabric

As an example, grouping on the bike data will be the transformation to be worked on.

data grouping transformation microsoft fabric

data grouping microsoft fabric

After creating the Group by transformation event, in this case, the eventstream will need to be connected to the Group by event.

connect eventstream to group microsoft fabric

3. Data stream with KQL Queries

Kusto Query Language (KQL) is designed as a read-only query to process data and return results. It is requested in plain text, using a dataflow model that is easy to read, create and automate. All queries are executed always in the context of a concrete table or database. As a minimum, a query contains a reference to the original data and one or more query operators applied in sequence, which is indicated visually by the use of a pipe character.

This first query uses the take operator to return a sample number of records, and is useful to get a first look at the data structure and possible values.

kql query fabric

Return to the data tree to select the next query, which uses summarize operator to count the number of records ingested in 15-minute intervals.

kql database summarize operator

Another way to get results from queries is to visualise the data by using the render operator.

kql data visualisation fabric

Power BI reports are multi-perspective insights into a semantic model, with visuals that display findings and insights from that model. KQL output will be created to build a new Power BI report, working with summarize operator.

summarize-operator-fabric

Select Build Power BI report. The Power BI report editor opens with the query result available as a data source named Kusto Query Result.

explore data solution microsoft fabric

Once Power BI displays the table that has been created from the data developed in KQL using the summarise operator, visualisations can be created.

power bi report kql database

In this way, the visualisations that will be required on the data that is displayed can be developed from within Power BI.

power bi report fabric

Industry

Related CONTENT by Industry
Services & Solutions
Related CONTENT by SERVICE
Technology
Related CONTENT by TECHNOLOGY

More Content