clustering provides means of what

finding structure in collection of unlabeled data

reasons for clustering

determine intrinsic groupings

1. classification

2. simplification

3. to create populations of types for downstream analysis

two kinds of metrics to calculate similarity

give example of each

statistical

(Pearson correlation)

geometric

(Euclidean distance)

Pearson Correlation

tells you what two things about correlation?

tells you degree and direction of correlation

measure of the correlation (linear dependence) between two variables X and Y, giving a value between +1 and −1 inclusive.

used as a measure of the strength of linear dependence between two variables.

distance metrics

if paying attention to most deviant conditions, use what method to measure distance

Chebyche

k-means clustering

method of cluster analysis in which n objects are grouped into k clusters in which each component belongs to the cluster with the nearest mean

k-means clustering

advantages and disadvantages

advantages:

simplicity

speed

can use on large datasets

disadvantages:

gives you different results with each run

in k-means clustering, how is each cluster primarily defined?

via its centroid

centroid

mean over all cluster members for each dimension

steps to k-means clustering

1. pick K random points and define as cluster centroids

2. add points to cluter with cloest centroid (assoc every oberservation with the nearest mean)

3. recalculate the means

4. repeat until you have reached max iteration OR until centroids don't move anymore when you recalculate them

why are outliers a problem in k-means clustering?

Every case is forced to join a cluster no matter how atypical or remote it might be, with the result that the classification can be substantially distorted.

hierarchical clustering

find successive clusters using previously established clusters

two main classes of clustering:

agglomerative

divisive

greedy algorithm

example

k-means clustering

perform a single procedure over and over until it can't be done any more and see what kind of results it will produce.

the procedure tries to maximize the return based on examining local conditions, with the hope that the outcome will lead to a desired outcome for the global problem.

In some cases such a strategy is guaranteed to offer optimal solutions, and in some other cases it may provide a compromise that produces acceptable approximations.

how to measure distance between two groups in agglomerative hierarchical clustering

3 ways

average linkage

single linkage

complete linkage

single linkage

way to calculate distance btw two groups in agglomerative hierarchical clustering

minimum distance btw any two points in each cluster

complete linkage

way to calculate distance btw two groups in agglomerative hierarchical clustering

maximum distance btw any two points in each cluster

how do you denote clusters in a dendrogram?

horizontal cuts

Silhouette

what s(i) values of 1,-1, & zero tells you

method of interpretation and validation of clusters of data

provides a succinct graphical representation of how well each object lies within its cluster.

value of 1means datum in approp cluster

value of 0 means btw clusters

-1 means datum should be in neighboring cluster

what can computing the average silhouette distance of a cluster and of the entire dataset tell you?

average s(i) of a cluster: how tightly grouped all the data in the cluster are

average s(i) of entire data set: measure of how appropriately the data has been clustered

silhouette plots and averages are a powerful tool for determining what?

the natural number of clusters within a dataset.

PAM

partitioning around mediods

more robust k-means

computes medoids instead of centroids as cluster centers

unlike k-means, it uses data to define cluster # (often silhouette)

model based clustering

two characteristics

strategy for determination of # of clusters and cluster membership

two characteristics:

1.fits gaussian distributions to data

2. uses BIC to pick # of distributions

Bayesian information criterion

BIC

helps determine the best clustering method along with the # of clusters