Lab 4 - AP Performance Advisories
Task Goal
In this laboratory session, you'll explore the AP Peformance Advisory feature and how to get the most out of the insights generated by the AI/ML algorithm, identifying radios persistently exhibiting poor client experience, helping you understanding the root cause and providing guidance on how to resolve such issues.
You'll learn how this feature works and how you can use it to identify specific APs or areas where you can focus your efforts in optimizing the client experience.
This lab task will guide you through an AP Peformance Advisory detection issue workflow:
- get an overview of the client experience issues detected on the network, organized by root cause
- for each group of problematic radios, understand the common root cause
- get further details when focusing on individual radios
Benefits
Quality of Experience (QoE) has become an important performance metric in wireless networks due to the usage of wireless connectivity in supporting critical business applications such as VoIP calling, video collaboration applications such as WebEx, and HD media streaming.
The goal of AP Performance Advisories feature is to develop a system that can identify wireless Access Points (APs) that persistently offer a poor client QoE.
In addition to detecting problematic APs, insights are generated consisting of a set of radios and network features that allow a network engineer to diagnose an underlying issue that can be remedied.
The AP Performance Advisories insights is a long-term analytics feature, as it makes use of up to 4 weeks of data to generate the insights (the insights are refreshed every week).
Like the other Cisco AI Analytics features explored so far, no configuration is needed to get these insights, except having the Wireless LAN Controllers configured to send telemetry data to the Cisco Catalyst Center appliance, and having the Cisco AI Analytics cloud service activated (See Lab 8 - Service Operations for more details).
As the long-term nature of this analytics feature is not compatible with the Cisco Live Walk-in Lab timelines, here we provide you with a pre-configured lab where you can directly explore the results.
Algorithm
-
Focus on top radios to constrain analysis to only the most important radios.
This not only limits generated insights to the customer's most important radios, but further improves the detection of underperforming radios by removing any inherent bias in radios that the customer has less concern over the QoE. -
Identify underperforming radios with poor QoE metric.
Analyze KPIs such as RSSI, SNR etc, compared to the global baseline and reference radios which have QoE metrics comparable to global baseline.
The global baseline is derived from the global distribution of QoE metric of all customers' radios on a given frequency band (2.4 GHz, 5 GHz, 6 GHz).
- Apply unsupervised learning to group radios with similar issues by applying multi-dimensional clustering.
-
Use supervised learning like decisions tree to identify explanatory features like interference, power, CPU usage etc which best distinguish between underperforming radios from reference radios.
-
Leverage intelectual capital developed by SME to derive the root cause analysis (RCA) depending on differentiating KPIs identified in previous step.
-
Provide detailed visualisation and table views for customers
Usecase workflow
The overall design goal of the workflow is to progressively provide increasing levels of details to the generated insights:
- Start with very high-level overview of the overall network with all discovered insights aggregated by common root cause and frequency band, represented as cards
- When a user clicks on a card, they are taken to a summary page with an intermediate level of detail, including charts of the most important explanatory KPIs aggregated over all radios sharing a similar root cause, contextual information such as locations, and finally a table of all the affected radios in the group that can be searched, sorted, and filtered.
- The radio name in the table links to the final detailed page, where specific anomalous KPIs particular to the individual radio are presented, along with suggested actions to remediate the problem.
Landing page
Navigate to the AP Performance Advisories landing page:
Menu > Assurance > Trends and Insights
The initial landing page of shows all discovered radios with suspected client experience issues, grouped as cards by common root cause, for each frequency band. The cards include a summary of the root cause, the numbers of radios in the group and an estimate of the impacted endpoints.
Additionally, the names of the top 3 impacted radios are indicated, acting as a link to jump straight to the details page for those APs.
Summary page
Let's Click on High co-channel interference on 2.4 GHz to investigate further.
After the user clicks on a card from the landing page they are shown a summary page that provides an aggregated view of all radios with a common root cause.
The summary page has a Hero bar containing an overview of the root cause, number of radios and buildings impacted, link detaled page for top radio impacted.
The analysis charts show the distribution of the explanatory KPIs that supports the root cause analysis, aggregated over the entire basis period (e.g., 4-weeks) and for all radios in the group.
A complementary distribution of the same KPIs collected over a set of reference radios that do not have recorded client experience issues, are used as a baseline to enable comparisons within the customer's network.
Those distributions are built using all the datapoints collected for all radios on each group over the entire period, where:
- the horizontal (x) axis shows the ranges of values for the KPIs
- the vertical (y) axis shows how often the KPI values in each range/bucket were observed
Only the KPIs that are pertinent to the root cause are shown.
The root cause is derived using the output of the AI/ML algorithm and it takes into account the combination of KPIs, as well as the individual of impact of each KPI (expressed by the AI/ML algorithm in form of a weight).
The selected example shows how the distributions of Co-channel Interference and Channel Utilization on the problematic radios in this group are significantly higher than those of the reference radios.
This is a common scenario observed in the field, and it can be resolved or mitigated by tuning the Radio Resource Management (RRM) RF Profile settings for the affected APs and their neighbors, for instance:
- Tuning the Transmit Power Control (TPC) settings to lower the radio transmission power
- Disabling some of the lower datarates, to reduce the airtime
These suggestions are explained directly on the user interface, along with a detailed explanation about the meaning and relevance of each KPI in the context of the selected insight.
Lastly, the radio table lists all radios in this group with a common root cause.
By default, the radios are ordered by their Impact rating, which is a combination of the number of impacted clients and the relative magnitude of statistical deviation of all the client experience KPIs.
Each column of the table can be used to sort, filter and search.
The column Insight Details provides a link to the detail page for a particular radio.
Details Page
Click on an AP to navigate to details page, CW9166I-LDN1-05 in this example.
The last part of the user workflow is referred to as the Details page as it contains significant additional information for the user to develop a deeper understanding of the problem and actions to resolve it.
As with the other page, a Hero bar provides an overview of information including information on the AP-model, location, a link to AP360 in Assurance and impacted clients.
Below the Hero bar you can find the charts of the underlying client experience KPIs that were used to determine that this radio had an identified client QoE problem.
Client expericence KPIs
The client experience KPIs include RSSI, SNR, link-speed, packet retries and packet failures.
In the details page you'll see by default the client experience KPIs that were tagged with a statistically significant deviation from the baseline for the selected radio.
You can add more KPIs, by clicking on the + button, to get more contextual information:
In this example, you can add the Speed KPI and observe the distributions for this KPI.
In this case you can see that the lower data-rates are used more often on the selected radio as compared to the reference radios; having lower data rates has a direct impact on the client experience.
Radio Root Cause
Below the Client Experience charts are the root cause analysis charts.
These are similar to the those shown in the previous Summary Page, but instead of being an aggregation over all radios with the similar root cause, the data represents the single radio.
All KPIs that were statistically anomalous are shown by default, which may be a superset of the KPIs shown in the Summary Page.
Indeed, the summary page only shows up to 3 KPIs, as those are used to group radios based on similar root cause; however, the radios in the group may have additional KPIs as part of the individual radio root cause analysis.
There are many additional KPIs that can be selected to add contextual information, such as CPU and memory usage, AP model, primary image versions, channel change counts etc.
You can use the + button here as well to add more KPIs.
Key takeaways
AP Performance Advisories automates the processing of large amounts of data about the radio performance, allowing to identify the most critical radios on the network, based on utilization, that are persistently delivering poor client experience.
This feature provides a way to identify radios and areas on the network requiring optimization, helping the network administrator to always ensure optimal client experience.
This concludes the exploration of the AP Performance Advisories feature.
You can use the link below to proceed with the exploration of other use cases.