Measure of Dispersion

Range, Quantile, Quartile and Percentile:

  • Range and Quartile are measure of dispersion or spread of Numerical data in a set.
  • Useful in findings outliers in data set.
  • Range:
    • Range of a data set is calculated by taking the difference of smallest and largest numbers.
    • First sort the set and then take the difference of first and last elements.
    • Example:
      • 1. Data : 20, 30, 12, 8, 50, 70, 100, 2, 40, 90
        • Sort the data : 
          • 2, 8, 12, 20, 30, 40, 50, 70, 90, 100
        • Take difference to calculate Range = 100-2 = 98
      • 2. Data : 20, 30, -12, 8, 50, 70, 100, -2, 40, 90
        • Sort the data : 
          • -12, -2,8, 20, 30, 40, 50, 70, 90, 100
        • Take difference to calculate Range = 100-(-12) = 112
  • Quantile:
      • Quantiles are data points, at which split the data distribution into equal size consecutive sets. 

 



                                    Here, Q1, Q2 and Q3 are three quantiles.

  • Quantiles are point taken at regular intervals of a data distribution, dividing it into essential equal size consecutive sets. [Important point to note down, I have mentioned essential, means not exactly equal sized.]

2-Quantiles:

    • Data set divided into two half parts are called 2-Quantile. 
    • Here, Number of quantile = 1 (1 data point) 


4-Quantiles (Quartile):

    • Data set divided into four essential equal parts are called 4-Quantile. 
    • Here, Number of quantiles = 3 (3 data point)

            



    • Each parts represent one quarter of data distribution. All parts are commonly known as Quartile.

Quartile:

    • Divide the data distribution into four equal parts, means, each part represents a quarter of data distribution. So, it is called Quartile.

Quartile = 4-Quantiles = 3 Quantiles point


100-Quantiles (Percentile):

    • Data set divided into hundred essential equal parts are called 100-Quantile. It is also called Percentile. 

Percentile = 100-Quartiles = 99 Quantile points.

So, in Quartile :

    • Q1 =  25th Percentile
    • Q2 =  50th Percentile = Median
    • Q3 = 75th Percentile
                  


Comments

Post a Comment

Popular posts from this blog

Understanding of Attribute types

Image
  Know your Data What is data It is set of records, having one or more attributes or features. It is also called collection of data objects. Example of Data Name Income Profession Mother Tongue Native Place Ram 70000 Doctor Bengali Village Shyam 50000 Carpenter Hindi Small Town Mohan 60000 Engineer Hindi Suburban   Kabir 90000 Doctor   Bengali Metropolitan   Fig1. As shown in Fig1, There are four rows and five columns. Record: Each row is called record. There are four records in the example dataset. Features or Attribute: Each column is called feature or attribute of a record in the dataset. There are five attributes for each record in example dataset. Understanding of Data Knowing of data is very important in data science, whether you are doing Data Mining or applying Machine Learning Techniques on data. To understand the data you have to understand each attribute of a record. The following properties of numbers are typically used to describe attributes. Distinctne...
Read more

Density Based clustering evaluation with Modified Silhouette Score

Image
  Modified Silhouette Score for HDBSCAN: Silhouette Score works well with convex clusters. However, when dealing with density-based clustering algorithm such as HDBSCAN, the traditional Sihouette Score doesn't perform well due to the irregular cluster shapes and the presence of nose points. To address this, modified version of Silhouette Score can be applied to HDBSCAN cluster, where the calculation is based on core points and the distance metrics used reflects the density-based clustering. Core Points: HDBSCAN categorizes points as core (high-density points), border (points near the cluster boundary), or noise . The modified Silhouette Score focuses on core points, as they represent the true structure of the cluster. Density-based Distance Metric : Traditional distance metrics (like Euclidean distance) assume spherical clusters. For HDBSCAN, a density-aware distance metric is more appropriate, such as the distance between core points based on density connectivity or mutual re...
Read more

Cluster Evaluation: Silhouette Score

Image
  Technical Concept: The Silhouette Score evaluates the cluster quality based on the notion of cohesion and separation both. That means it evaluate the quality of clustering by measuring how well data points fit within their assigned clusters and how good separation is among clusters.   It provides insight into the separation and compactness of clusters, combining information about intra-cluster similarity and inter-cluster dissimilarity. Formula For a given data point i i : a ( i ) a(i)  ->   the average distance between point i i  and all other points in the same cluster. b ( i ) b(i)    -> Calculate the data point's average distance to all the objects in any cluster (not containing i data point) . Find the minimum such value with respect to all clusters.  The Silhouette Score s ( i ) s(i)  is computed as: s ( i ) = b ( i ) − a ( i ) max ⁡ ( a ( i ) , b ( i ) ) s(i) = \frac{b(i) - a(i)}{\max(a(i), b(i))} ​ Silhouette coef...
Read more