Download Introduction to Data Mining: Techniques and Applications using Pandas and more Cheat Sheet Data Mining in PDF only on Docsity!
Intro to Data Mining
Data mining is the process of searching and analyzing a large batch of raw data in order to identify patterns and extract useful information.
KDD Process (Knowledge Discovery from Data)
- Data Cleaning
- Data Integrating
- Data Reduction
- Data Transformation
- Data Mining
- Pattern Evaluation
- Knowledge Representation
Hands in to Pandas (a python library used for data preprocessing)
#import pandas in Jupyter Notebook import pandas as pd
read data from .csv file
df= pd.read_csv("xyz.csv") df
check row-column amount from dataframe
df.shape
check first row/first 3 rows of the dataframe
df.head() df.head( 3 )
check last row from dataframe
df.tail()
check all the values of one specific column
df["humidity"]
Missing data handling techniques: โ Ignore the tuple โ Fill missing values manually โ Use global constant for missing values (NaN) โ Central tendency (Mean, Median, Mode) โ Data mining algorithm (most probable value)
check whether the dataframe has null values or not
df.isnull()
check total null values of columns
df.isnull.sum()
drop rows with null values
df.dropna()
fill null attributes with 0
df.fillna( 0 )
see mean values of of a column
df["visibilityMiles"].mean()
fill specific column values with specific number
df.fillna({"VisibilityMiles": 9 })
forward filling the missing values
df.fillna(method="ffill")
Further read: pandas official doc for file preprocessing
