Which of the following would be a reason why an organization would use web usage mining?

14.1 Application Contexts for Data Profiling

A retrospective look at the wave of consolidations among vendors in the data quality and data management industry shows one specific similarity across the board – the transition to building an end-to-end data management suite is incomplete without the incorporation of a data profiling product. The reason for this is that many good data management practices are based on a clear understanding of “content,” ranging from specific data values, the characteristics of the data elements holding those values, relationships between data elements across records in one table, or associations across multiple tables.

It is worth reviewing some basic application contexts in which profiling plays a part, and that will ultimately help to demonstrate how a collection of relatively straightforward analytic techniques can be combined to shed light on the fundamental perspective of information utility for multiple purposes. We will then provide greater detail in subsequent sections for each of these applications.

3.1 Introduction to anomaly detection

Data points that are inconsistent with the major data distribution are called anomalies [2]. Originally, the problem of anomaly analysis has been tackled by the statistics community and fundamental results have been published in the literature in the field [60,31,53].

According to Barnett and Lewis' definition [60], an anomaly is “an observation [or subset of observations], which appears to be inconsistent with the remainder of that set of data”. Hawkins [31] defined an anomaly as “an observation which deviates so much from other observations as to arouse suspicions that it was generated by a different mechanism”. Anomalies are also referred to as rare events, abnormalities, deviants, or outliers.

More recently, anomaly detection has received significant attention from many other research communities such as machine learning and data mining, networking, health, security, fraud detection, etc., due to the insights that rare events can provide for the phenomenon under study. Thus, outlier detection techniques can be used to monitor a wireless sensor network to identify faulty sensors or interesting behavior patterns. The availability of data used for the anomaly detection task depends on the properties of the data set. In static datasets, anomaly detection can be conducted over the whole data set in which all observations are available.

In continuous data stream scenarios, the observations may not be available at any moment and arrive sequentially. The observations in data streams can be seen only once and anomalies should be detected in real time. In environments where the distribution changes over time [nonstationary], traditional detection methods cannot be applied, and the models change. Therefore, adaptive models need to be considered to deal with dynamically changing characteristics to detect anomalies in the evolving time series.

In recent years, several algorithms have been proposed to detect anomalies in data sets or data streams. Some of them consider scenarios where a sequence with one or more characteristics unfolds over time. While others focus on more complex scenarios in which streaming elements with one or more characteristics have causal / noncausal relationships with each other.

Extensive research on the detection of outliers in static data related to various applications scenarios can be found in [4,8,13,14,30].

20.3.1 Data Profiling

Data profiling incorporates a collection of analysis and assessment algorithms that provide empirical insight about potential data issues, and has become a ubiquitous set of tools employed for data quality processes supporting numerous information management programs, including assessment, validation, metadata management, data integration processing, migrations, and modernization projects. Chapter 14 discussed analyses and algorithms profiling tools employ and how those analyses provide value in a number of application contexts. Profiling plays a part in some relatively straightforward analytic techniques that, when combined, shed light on the fundamental perspective of information utility for multiple purposes, such as:

Data reverse engineering: Data reverse engineering is used to review the structure of a data set for which there is few or no existing metadata or for which the existing metadata are suspect for the purpose of discovering and documenting the actual current state of its metadata. Data profiling is used to grow a knowledge base associated with data element structure and use. Column values are analyzed to determine if there are commonly used value domains, to reveal whether those domains map to known conceptual value domains, to review the size and types of each data element, to identify any embedded pattern structures associated with any data element, and to identify keys and how those keys are used to refer to other data entities. The metadata discovered as a result of this reverse engineering process can be used facilitate dependent development activities such as business process renovation, enterprise data architecture, or data migrations.

Anomaly analysis: There is often little visibility into data peculiarities in relation to existing data dependencies, especially when data sets are reused. Profiling is used to establish baseline measures of data set quality, even distinct from specific downstream application uses. Anomaly analysis is a process for empirically analyzing the values in a data set to look for unexpected behavior to provide that initial baseline review and is used to reveal potentially flawed data values, data elements, or records. Discovered flaws are typically documented and can be brought to the attention of the business clients to determine whether each flaw has any critical business impact.

Data quality rule discovery: The need to observe dependencies within a data set manifests itself through the emergence [either by design or organically through use] of data quality rules. In many situations, though, there is no documentation of the rules for a number of reasons. Data profiling can be used to examine a data set to identify and extract embedded business rules, whether they are intentional but undocumented, or purely unintentional. These rules can be combined with predefined data quality expectations, as described in chapter 9, and used as the targets for data quality auditing and monitoring.

Metadata compliance and data model integrity: The results of profiling can also be used as a way of determining the degree to which the data actually observes any already existent metadata, ranging from data element specifications, validity rules associated with table consistency [such as uniqueness of a primary key]. The results can also be used to demonstrate that referential integrity constraints are enforced properly in the data set.

Review chapter 14 to see how data profiling tools are engineered and how they support aspects of the data quality program. When evaluating data profiling products, it is valuable to first assess the business needs for a data profiling tools [as a by-product of the data requirements analysis process and determination of remediation as described in chapters 9 and 12]. In general, when evaluating data profiling tools, consider these capabilities discussed in chapter 14:

Column profiling

Cross-column [dependency]

Cross-table [redundancy]

Structure analysis

Business rules discovery

Business rules management

Metadata management

Historical tracking

Proactive auditing

Business rule importing

Business rule exporting

Metadata importing

Metadata exporting

5.4 Employing Data Quality and Data Integration Tools

Data quality and data integration tools have evolved from simple standardization and pattern matching into suites of tools for complex automation of data analysis, standardization, matching, and aggregation. For example, data profiling has matured from a simplistic distribution analysis into a suite of complex automated analysis techniques that can be used to identify, isolate, monitor, audit, and help address anomalies that degrade the value of an enterprise information asset. Early uses of data profiling for anomaly analysis have been superseded by more complex uses that are integrated into proactive information quality processes. When coupled with other data quality technologies, these processes provide a wide range of functional capabilities. In fact, there is a growing trend to employ data profiling for identification of master data objects in their various instantiations across the enterprise.

A core expectation of the MDM program is the ability to consolidate multiple data sets representing a master data object [such as “customer”] and to resolve variant representations into a conceptual “best representation” whose presentation is promoted as representing a master version for all participating applications. This capability relies on consulting metadata and data standards that have been discovered through the data profiling and discovery process to parse, standardize, match, and resolve the surviving data values from identified replicated records. More relevant is that the tools and techniques used to identify duplicate data and to identify data anomalies are exactly the same ones used to facilitate an effective strategy for resolving those anomalies within an MDM framework. The fact that these capabilities are available from traditional data cleansing vendors is indicated by the numerous consolidations, acquisitions, and partnerships between data integration vendors and data quality tools vendors, but this underscores the conventional wisdom that data quality tools are required for a successful MDM implementation.

Most important is the ability to transparently aggregate data in preparation for presenting a uniquely identifiable representation via a central authority and to provide access for applications to interact with the central authority. In the absence of a standardized integration strategy [and its accompanying tools], the attempt to transition to an MDM environment would be stymied by the need to modernize all existing production applications. Data integration products have evolved to the point where they can adapt to practically any data representation framework and can provide the means for transforming existing data into a form that can be materialized, presented, and manipulated via a master data system.

Advanced

The adversary understands security and knows how to defeat or get around most traditional security measures. Not to depress anyone but if an organization has security devices that have been purchased more than 3 years ago and have a standard/default configuration, in most cases it will be ineffective against the APT. The reason is that these devices were not built or configured to deal with this level of a threat. They were meant to deal with traditional worms and viruses that had distinct signatures that could be tracked and detected on a network. Now the good news is that the standard configuration for older security devices was set to look for the standard threat. With proper tuning many of these devices, in coordination with other devices, can be used as part of the solution. The second important point is that security devices that are focused on data, data flow, and anomaly analysis are much more effective at dealing with the APT than anything that is based on signatures or specific instances of an attack.

It is important when we talk about the advanced nature of the attack, we still have to remember that the attacker has to follow the general principles of finding a weakness and exploiting them. Yes, the APT is good but it is not super human or something that no matter what you do, it will still break in and go undetected. The mistake that is often made is many organizations give the APT more credit than it deserves and says no matter what you do it is unstoppable and undetectable. That statement is just not true. An attack has to take advantage of a weakness in a system and make modifications to a system once it is compromised. Based on those two factors means that the APT can be prevented and detected, if we know what vulnerabilities to close down and where to look. The correct phrase to use when talking about the APT is that it is unstoppable and undetectable using traditional security and normal remediation methods. However, if we change how we approach security, in much the same way that the APT changed how systems were compromised, organizations do have a chance of properly dealing with the threat. It is also important to remember that based on the persistent nature, an organization might not be able to prevent all attacks, some will break in and the reason for detection is so important.

Now the main problem with the APT is that it takes advantage of vulnerabilities that are needed in order for the organization to function. This makes it very difficult to track and close down the attack vector. With traditional attacks, the vulnerability that was exploited was an extraneous service or an unpatched system which meant if an organization focused in on the correct area they could remediate the threat and cause minimal impact to the enterprise. Today, the APT is taking advantage of email attachments and employees within the organization. An organization can only remediate this 100% by shutting down email and firing all employees. While they might be tempting, that is not practical. Therefore, organizations have to be very creative in how they deal with the threat.

What is the Problem That is Being Solved?

Growing up one of the jokes that people would make is that every Miss America candidate when asked what her dream goals are would state “to solve world hunger” or “bring peace to the world.” These are high-level goals but in order to solve them you have to identify what the problem is that is trying to be solved. Every great journey begins with the first step. An organization cannot identify the first step, if they do not know what the problem is they are trying to address. We have worked with many clients and when asked what they are trying to accomplish they would state “to be 100% secure” or “to never be compromised by the APT.” While these are notable goals, they are as nebulous as solving world hunger. The million dollar question is where do you start? What are the problems and associated project plans that need to be developed to ultimately maintain forward progress toward the goal? Goals are hard to achieve by themselves but problems can be solved with the right focus.

Every organization should always perform an assessment within their environment to identify the main problems that need to be solved in order to do the right thing in terms of defending against the APT. The good news is the high-risk problems for many organizations typically are similar. In shifting from doing good to doing the right things, the following are typical problems that organizations are ignoring but need to solve to make forward progress in dealing with the APT:

The ability to detect compromised systems—One of the number one problems today is that organizations have most of their focus and attention on inbound prevention. Trying to stop attacks is important and should continue. Even though the APT is very stealthy, if an organization can prevent 20% initial attacks, that is still better than nothing and is 20% less attacks that have to be detected after the fact. However, the APT is persistent and will continue to target an organization until they accomplish their goal. Bottom line is regardless of what number of attacks can be prevented with inbound traffic, it is not going to be 100%. Organizations must be examining outbound traffic looking for signs of a compromised system. One of the million dollar questions is if an organization had a compromised system, would they be able to detect it and how long would it take. In some cases organization can take 6–8 months in order to detect a compromised system. Think about how much information is leaving the organization every day. Even if it was detected within 3 months instead of 6 months, that would still be a lot less damage and exposure to the organization. Any skill including security takes time. Ideally you want to be able to detect attacks as soon as they occur and contain/control immediately or within a short period of time like 12 h; however the most basic question is regardless of time, would your organization have any chance at all of being able to detect a compromised system? The immediate response is of course, but step back for a second. If a system was currently compromised on your network today, leaking information how would you know? It is critical in combatting the APT that organizations solve the problem of being able to detect compromised systems. Once an organization has the capability to do so, they need to continually decrease the amount of time it would take to detect the attack. Every minute a compromised system goes undetected is an increased damage and exposure to the organization.

Being able to identify anomalies from known baselines—anomaly detection is critical to dealing with and detecting the APT, but the fundamental question is how can something be an anomaly if you do not know what is normal. An organization must track usage, network patterns, connectivity, and bandwidth to understand and build a profile of what is normal and expected to be seen in an organization. Depending on the type of attack it might be very visible or it might be subtle, but if someone compromises a system they are going to act differently than a normal user. If their behavior is exactly the same, then they are not an attacker or your normal users are attackers. Organizations need to figure out which activities to build baselines against, but the ones that work well against the APT are: [1] length of the connection; [2] amount of outbound data; [3] external IPs being connected to. In almost all cases that we have seen the APT creates an obvious anomaly across all three areas that is quite different than normal traffic. The powerful component of anomaly analysis is the correlation across multiple sources. While one item might give a little insight into something being an anomaly, the real value is when the results are compared across 3–5 different variables. When this is done it becomes quite obvious that something is an anomaly. Creating baselines are quite easy if you have the traffic, sniffer output, or logs, but the base analysis has to be done for it to be of value. The big problem for many organizations is that they have the data but it has never been normalized. It is like there is an oil deposit under someone’s house that is worth millions but because they never checked, they never realized and therefore was not able to take advantage of the value of it.

Properly segmented networks—Flat networks are easy for users to be able to access data but it is also extremely easy for attackers to also be able to access the information. What is ironic is that properly segmented networks can also be configured to be very easy for users to be able to access data but also very difficult for attackers to be able to cause harm. The question for your organization is whether you want option [1] easy for the user and easy for the attacker or option [2] easy for the user and hard for the attacker. Life is full of adventures so when it comes to protecting information, let us stay away from excitement and pick option 2. The big problem with many organizations is that they cannot properly control who can access what systems and once a client system is compromised, it is very easy for the attacker to have full access to any information they want. Gateways and control points have been used for many years in the physical security realm to protect valuable possessions and the same concepts can be used in with electronic information to protect critical information.

Better correlation to prevent/detect attacks—The APT is like a puzzle, in order to solve it you need to have all of the pieces. With a large puzzle if you only have one piece you really have no idea what the picture of the puzzle is. The more pieces you have the clearer the picture. A puzzle piece is an entry in a log file on a single device. If you just look at the logs or information on a single computer you only have a small number of pieces, the more devices you gather and correlate information against, the clearer the picture and the more useful the data. With the APT correlation is king. In many cases better detection can lead to improved prevention. Once an organization is able to detect an attack and see what they missed, that information can be used to build better defensive measures in the future. Proper detection should lead to improved metrics and better information for enhanced prevention of future attacks that are similar. Now the critical piece is to correlate the information and look for general patterns that can be blocked, not specific signatures. Since the attacker is always changing, signatures will provide minimal protection in dealing with the APT. While looking at specific log entries is good for detailed analysis, most of the energy and effort in dealing with the APT should be focused on high-level correlation, tied with anomaly analysis.

Proper incident response to prevent reinfection—When an organization finds out that they have been compromised for 6–8 months and it went undetected, the response is not usually a calm and peaceful response, it is usually people freaking out. The typical response is to make the problem go away as fast as possible. Most organizations forget about the six-step process for handling an incident, skip steps and all focus is on recovery. Get the systems back up and running as quickly as possible. Since people are under stress they often forget the obvious. If the attacker compromised a system once, there is very good chance they will compromise it a second time. While catching an attacker is important, reinfection is deadly. If the first time the attacker broke in you eventually caught them, they are not happy. They will break back in but they will work even harder to be stealthy and not get caught. If it took an organization 6–8 months to detect the attacker the first time and now they are really trying to be stealthy, how long do you think it will take the organization to catch them the second time. When it comes to compromise, do it right the first time, there are no second chances. Many organizations that have not been compromised take a logical look at the problem and say the longer a system is down the more money it is going to cost the organization. Therefore the quicker we can recovery the better off the organization is. The problem is it is better to be down once, for a longer period of time and fix the problem, than recover quickly but become reinfected and have additional data loss. However, it is important to point out that there are some systems in which availability is critical. In these cases systems might need to be brought up before they are fully remediated, but this should be a business decision and all systems should be carefully monitored and controlled in these circumstances. The important lesson with an incident, but especially with a devastating attack like the APT, is do it right the first time and fix the problem. There are no second chances.

Security is very challenging and it is always important to make sure the problem you are fixing is the highest priority problem.

13.1.3 Mining Other Kinds of Data

In addition to sequences and graphs, there are many other kinds of semi-structured or unstructured data, such as spatiotemporal, multimedia, and hypertext data, which have interesting applications. Such data carry various kinds of semantics, are either stored in or dynamically streamed through a system, and call for specialized data mining methodologies. Thus, mining multiple kinds of data, including spatial data, spatiotemporal data, cyber-physical system data, multimedia data, text data, web data, and data streams, are increasingly important tasks in data mining. In this subsection, we overview the methodologies for mining these kinds of data.

5.1.3 Time-series statistical techniques

Time-series based statistical techniques use previously observed values to forecast new values. Sperotto et al. [2008] used time-series analysis anomaly characterization in flow traffic. Nguyen et al. [2008] apply Holt–Winters forecasting method to detect an anomaly in the flow traffic. They use four metrics to construct a flow: total bytes, total packets, the number of flows with similar volume to the same destination socket and the number of flows that has a similar volume and same source and destination address, but to different ports. These four metrics are used to detect three types of anomalies: flooding, TCP SYN, and port scan. The Holt–Winters method keeps track of the normal metrics values and raises an anomaly flag if any value goes out of range. The technique is limited to only three anomalies and can be bypassed if the attacker keeps the flow metrics values within range.

A high-speed flow level intrusion detection system [HiFIND] is presented in Li et al. [2010]. The use of flow information for high speed and DoS resilient intrusion detection was initially proposed in Li et al. [2005] and Gao et al. [2006]. HiFIND uses a small set of packet header fields including Source/Destination IP and Source/Destination ports. It focuses on three types of attacks: SYN Flooding, Horizontal Scan and Vertical Scan. The authors use Holt–Winters double exponential smoothing and EWMA with season indices method for change detection in network traffic. HiFIND is applied in three phases and false-positives are reduced by separating the intrusion and network anomalies caused by misconfiguration. The performance evolution of HiFIND is carried out using both simulation and on-site deployment. A custom dataset of one-day traffic traces consisted of 900M flow records is used. The authors compared the HiFIND with other statistical detection techniques for flow-based detection, and results show that HiFIND has similar accuracy but is memory efficient in worst case scenarios. HiFIND is one of the few models to implement the intrusion detection systems security [Sadre et al., 2012]. The use of Holt–Winters double exponential smoothing and EWMA with season indices can have the drawback of statistical seasoning effect. The authors only used a 4-feature NetFlow record without including the protocol field. Therefore the system may not be able to detect the attacks sent on UDP packets. Another limitation of the HiFIND system is the inability to detect small and slow-ramped attacks.

7 Big data analytics in the industry

Throughout our survey, we came across several companies that offer network solutions based on big data analytics. These companies and solutions are highlighted in Table 3. It should be noted that these solutions are enabled by research conducted in their corresponding areas. We have added academic research papers related to each solution in Table 3.

Table 3. Big data analytics-powered industrial solutions.

Due to the proprietary nature of industrial products, the exact algorithms or methods behind these products is not available in the open literature. Therefore, academic papers with related concept[s] are cited. NetReflex IP and NetReflex MPLS utilizes big data analytics [27,73,108] to provide services like anomaly analysis and traffic analysis. Nokia provided several solutions targeting the wireless field. For example, Traffica introduces itself as a real-time traffic monitoring tool that analyzes user behavior to gain network insights, similar approaches were presented in academia by the authors of [69,109]. The Wireless Network Guardian detects user anomalies in mobile networks where a comparable topic was discussed in [110]. Preventive Complaint Analysis makes use of big data analytics to detect behavioral anomalies in mobile network elements where the authors in [111] provided a similar approach. Predictive Care utilizes big data analytics to identify anomalies in network elements before affecting the user, a comparable academic approach is presented in [110,112]. HP presented Vertica, a solution that exploits CDRs for network planning, optimization, and fault prediction purposes.

The authors in [64,113] researched akin approaches. Amdoc's Deep Network

Analyzer provides predictive maintenance and proactive network deployment for cellular networks. The authors in [114] presented a similar approach. Log analytics can be used for a variety of purposes. Aprevi's ARLAS solution provided real-time collection and storage of network logs. Related academic research was presented by the authors in [115–117].

Examining the above solutions, one can note that the majority of the solutions are in the wireless field. This, in fact, coincides with the orientation of the academically-researched topics. Sampling through the offered solutions, we noticed the increased interest in anomaly prediction and network node deployment. Thus, offering the customer a service that is as close to optimal as possible, while minimizing network expansion expenditures.

7 RQ2: Data mining and data quality management in large-scale CPSs

This section is to answer the second SLR review question [RQ2] listed in Table 2 based on the results of the SLR.

Data quality assessment in large-scale CPS applications using traditional methods is no longer efficient because of the heterogeneous large volume of data that these systems typically exchange [57]. Thus, such systems, usually rely on numerous sensor nodes that stream large volume of data in real-time which requires a high-performance, scalable and flexible tools to effectively provide insight real-time data processing and analysing mechanisms [44,59,80,88]. Based on the results of the SLR data extraction process illustrated in Table A.10, many statistical, technical and machine-learning models were proposed, tested and evaluated mostly for identifying data quality issues, decreasing their occurrence probability and overcoming their impact on the system. Most of these proposed solutions, methods, or models were developed to enhance the reliability of a particular system by improving its data quality based on prior knowledge extracted from the data itself, a process known as Data Mining. Considering the SLR empirical studies only, it is possible to categorise all the adopted data quality assessment/management methods, techniques or solutions into three primary groups:

•

Data mining.

Technical solutions/models.

Mathematical models.

Fig. 7 shows the usage ratio of the methods of each of the above groups, indicating that data mining methods are the most widely used compared to other technical or mathematical techniques.

Data mining is the process of auto-discovering knowledge, patterns or models from large volumes of data using advance data analysis methods [89]. Data mining techniques are essential for data analysis in large-scale CPS applications which relay on sensor node networks that, typically, stream a continuous flow of spatiotemporal4 data at a relatively high-speed and dynamicity [90]. Focusing on the SLR primary studies that adopted data mining techniques for tackling data quality challenges in large-scale CPS applications reveals that these methods are mainly divided into statistical and machine-learning based methods. Furthermore, it reveals that most popular data mining techniques used for data mining in large-scale CPS applications are anomaly analysis, predictive analysis and clustering analysis, as shown in Fig. 8. Moreover, these three leading data mining techniques were applied to address various data quality issues associated with the main data quality dimensions, as shown in Fig. 9.

Fig. 10 shows a holistic diagram of the main data quality management/assessment techniques and data quality dimensions based on the SLR results.