Data Analytics with Statistical Software Notes
COM3MN210 - Data Analytics with Statistical Software study notes and resources.
Chapter-Wise Module Notes & PDF Downloads
Short Notes
All Modules IncludedComplete syllabus revision notes covering all modules in one single PDF
Module 1
Theoretical Foundations & Principles
Module 2
Operational Frameworks & Methodologies
Module 3
Regulatory Compliance & Quantitative Analysis
Module 4
Strategic Applications & Contemporary Trends
Read Module Notes Online
Module I: An Introduction to SPSS
In an era dictated by empirical inquiry and data-driven governance, statistical software packages constitute the core intellectual infrastructure of modern business research. IBM SPSS Statistics (originally Statistical Package for the Social Sciences) represents one of the most widely adopted, comprehensive, and user-friendly statistical computing environments in corporate enterprise and academic research. This module introduces the fundamental architecture of SPSS, examining its historical origins, corporate applications, operational merits, inherent constraints, comparative positioning against competing analytical suites (Excel, R, Python, SAS, STATA), step-by-step software installation protocols, and the essential mechanics of creating, structuring, and manipulating data files (.sav).
1.1 Meaning, Evolution, and Nature of SPSS
Historical Background and Nomenclature
SPSS was originally developed in 1968 by Norman H. Nie, C. Hadlai (Tex) Hull, and Dale H. Bent at Stanford University. The software was initially conceived to automate complex computational statistical procedures for social scientists who lacked formal computer programming credentials. The original acronym stood for Statistical Package for the Social Sciences.
Over five decades of continuous technological iteration, the platform expanded far beyond traditional sociology and political science into corporate commerce, banking, healthcare epidemiology, marketing analytics, and government administration. In 2009, IBM Corporation acquired SPSS Inc. for approximately $1.2 billion, formally rebranding the suite as IBM SPSS Statistics. Today, SPSS operates as an enterprise-grade analytical engine combining a graphical point-and-click interface with a robust, reproducible command syntax language (SPSS Syntax).
Core Architecture and File Types
SPSS operates through three primary, interconnected file environments, each serving a distinct operational function in the analytical workflow:
1. Data Editor (.sav): The primary spreadsheet interface containing raw data cases and variable definitions. Displays two views: Data View (cases in rows, variables in columns) and Variable View (metadata attributes for each variable).
2. Output Viewer (.spv): A dedicated, tree-structured viewer displaying all statistical computations, summary tables, test statistics, p-values, and graphics generated by executed procedures. Allows export to Word, PDF, Excel, or HTML.
3. Syntax Editor (.sps): A code editor window where analysts write, edit, and execute native SPSS command scripts. Ensures absolute research reproducibility, batch execution, and automation of repetitive daily analytical pipelines.
1.2 Applications and Practical Uses of SPSS in Business
In contemporary corporate environments, IBM SPSS serves as a vital decision-support engine across diverse business functions:
| Business Domain | Specific Analytical Applications in SPSS | Strategic Management Value |
|---|---|---|
| Market Research & Consumer Insights | Survey analysis, Likert-scale questionnaire processing, Brand perception mapping, Conjoint analysis for new product design, Factor analysis for psychographic segmentation. | Identifies target consumer clusters, optimizes product pricing thresholds, and quantifies brand equity drivers. |
| Human Resource Management (HRM) | Employee satisfaction and pulse survey analytics, Attrition/Turnover hazard modeling (survival analysis), Performance appraisal metric validation, Training effectiveness evaluation. | Reduces costly employee turnover, identifies workplace burnout determinants, and benchmarks organizational climate. |
| Banking, Credit & Financial Services | Credit scoring models using binary logistic regression, Customer churn forecasting, Fraudulent transaction classification, Portfolio risk assessment. | Minimizes non-performing assets (NPAs), establishes automated loan approval scorecards, and prevents revenue loss. |
| Retail & Supply Chain Management | Sales trend forecasting using time-series decomposition, Inventory re-order point variance analysis, Cross-tabulation of purchase baskets across geographic regions. | Enhances inventory turnover ratios, eliminates stockout penalties, and tailors localized promotional strategies. |
| Healthcare & Pharmaceutical Administration | Clinical trial efficacy testing (paired and independent sample t-tests), Survival analysis (Kaplan-Meier), Hospital patient readmission predictive modeling. | Ensures regulatory compliance, validates medical drug performance, and enhances hospital operational efficiency. |
1.3 Key Features, Merits, and Limitations of SPSS
Salient Features of the Platform
- Dual-Interface Flexibility: Seamlessly alternates between an intuitive, menu-driven Graphical User Interface (GUI) for business novices and a rigorous Command Syntax language for advanced statistical programmers.
- Comprehensive Statistical Arsenal: Native implementation of univariate, bivariate, and multivariate procedures—ranging from basic descriptive statistics and cross-tabulations to multi-way ANOVA, linear and logistic regression, exploratory factor analysis, cluster analysis, and non-parametric tests.
- Robust Data Transformation Engine: Powerful internal commands including
COMPUTE VARIABLE,RECODE INTO SAME/DIFFERENT VARIABLES,COUNT VALUES WITHIN CASES, and automatic data restructuring wizards (folding rows to columns or vice versa). - Extensible Integration: Native bridges allowing analysts to embed and run Python and R scripts directly inside the SPSS workflow, combining SPSS's data management strengths with cutting-edge open-source machine learning libraries.
- Publication-Quality Reporting: Advanced Chart Builder and Pivot Table editors enabling automated styling, conditional formatting, dynamic dimension pivoting, and high-resolution chart rendering.
- Zero Coding Requirement: Analysts can perform complex multivariate statistical tests via dropdown dialog boxes without writing lines of code.
- Integrated Metadata Management: Stores extensive variable labels, value codes, missing data declarations, and measurement levels directly within the file (.sav).
- Specialized Survey Tools: Unmatched capabilities for handling complex multi-stage survey sample designs, stratification weights, and multiple-response sets.
- High Numerical Reliability: Proven, mathematically verified computational algorithms rigorously vetted by academic and corporate audit standards over 50+ years.
- High Commercial Licensing Costs: Enterprise subscription seats and academic site licenses represent a substantial financial burden compared to free tools (R, Python).
- Big Data Inefficiency: Struggling performance when handling streaming datasets, unstructured text, or tables containing tens of millions of rows (RAM bound).
- Proprietary File Formats: Output files (.spv) require SPSS or a proprietary viewer utility, hindering effortless cross-platform sharing.
- Lag in Cutting-Edge AI/Deep Learning: While basic predictive modeling is robust, advanced neural network architectures, transformers, and computer vision models are absent.
1.4 Comparative Analysis: SPSS vs. Other Statistical Tools
Selecting the appropriate statistical platform requires weighing organizational resources, computational scale, statistical complexity, and team technical competencies:
| Parameter | IBM SPSS | Microsoft Excel | R Programming | Python (SciPy/Stats) | SAS |
|---|---|---|---|---|---|
| Primary Interface | GUI (Menus) + Command Syntax | Spreadsheet Grid + Formulas | Command-Line Scripting / RStudio | Code Notebooks (Jupyter/VS Code) | Syntax Scripting + Enterprise Guide |
| Learning Curve | Low (Very gentle for non-programmers) | Extremely Low (Ubiquitous business tool) | Steep (Requires programming literacy) | Moderate to Steep (General purpose language) | Steep (Proprietary syntax structure) |
| Licensing & Cost | Commercial / Proprietary (High Cost) | Commercial (Bundled in Microsoft 365) | Free / Open-Source (GNU GPL) | Free / Open-Source (PSF License) | Commercial / Proprietary (Highest Cost) |
| Data Scale Capacity | Medium to Large (RAM constrained) | Limited (1,048,576 rows max per sheet) | Large (Optimized memory packages) | Massive (Integrates with Spark, SQL, Big Data) | Massive (Industry gold standard for big data) |
| Statistical Sophistication | Comprehensive for standard research | Rudimentary without specialized add-ins | Exhaustive (Latest academic packages) | Exhaustive (Unmatched in ML / AI algorithms) | Highly robust, especially in biostatistics |
| Target User Group | Social scientists, Marketers, Business Analysts | Accountants, General Business Executives | Academic Statisticians, Data Scientists | Machine Learning Engineers, Data Analysts | Pharma, Large Banks, Enterprise Risk Teams |
1.5 Step-by-Step Installation & Setup Guide for SPSS
Minimum System Requirements
Before proceeding with the installation of IBM SPSS Statistics (Version 26, 27, 28, or 29), verify that the host workstation satisfies the following minimum hardware and operating system benchmarks:
- Operating System: Microsoft Windows 10/11 (64-bit editions) or macOS 11.0 (Big Sur) or higher.
- Processor: 1.6 GHz or higher multi-core 64-bit Intel/AMD processor (Apple Silicon M1/M2 supported via native/Rosetta installers).
- Memory (RAM): Minimum 4 GB RAM (8 GB or 16 GB strongly recommended for processing large multi-case survey datasets).
- Hard Disk Storage: Minimum 4 GB of free contiguous disk space for application files, plus temporary scratch space.
- Display Resolution: Minimum 1024 x 768 display resolution (1920 x 1080 recommended for comfortable dual-view navigation).
Installation Workflow (Windows Environment)
1.6 Creating, Structuring, and Editing an SPSS Data File (.sav)
The Data Editor Architecture: Data View vs. Variable View
Upon opening SPSS, the primary workspace is the Data Editor window. At the bottom-left corner of this window are two clickable tabs that form the bedrock of all data operations:
- Data View: Resembles a traditional spreadsheet where columns represent Variables (metrics, questions, features) and rows represent individual Cases (respondents, companies, transactions, subjects). Every intersection cell holds a single data observation.
- Variable View: The metadata control room of SPSS. Here, each row defines a single variable, and each of the 11 columns governs a specific architectural attribute of that variable.
- Name: The unique variable identifier used in syntax and internal indexing. Rules: Must start with a letter; cannot contain spaces or special punctuation (except underscore
_); cannot exceed 64 bytes; cannot duplicate reserved keywords (ALL,AND,BY,EQ,GE,GT,LE,LT,NE,NOT,OR,TO,WITH). Example:emp_salary,cust_age,satisfaction_q1. - Type: The mathematical/computational data structure. Common types include Numeric (standard numbers), String (text characters), Date (calendar dates in specific formats like dd-mmm-yyyy), and Dollar / Custom Currency.
- Width: The maximum number of characters or numerical digits allocated for the data value.
- Decimals: The number of digits displayed to the right of the decimal point for numeric variables. Does not truncate underlying computational precision.
- Label: A comprehensive, human-readable description of the variable (up to 256 characters). This label automatically appears on all output tables, charts, and summary reports in place of the terse variable name. Example: "Monthly Net Disposable Household Income (in INR)".
- Values (Value Labels): Crucial for categorical and ordinal variables. Allows analysts to assign textual definitions to discrete numeric codes (e.g., Value:
1= "Male", Value:2= "Female"; or1= "Strongly Disagree",5= "Strongly Agree"). This preserves compact numerical storage while delivering readable output. - Missing: Identifies values that should be excluded from statistical calculations. Analysts can specify Discrete Missing Values (e.g.,
999for "Refused to Answer",99for "Not Applicable") or a numeric range. Unrecorded blanks are automatically treated as System-Missing, represented by a period (.). - Columns: The visual display width of the column inside the Data View grid (measured in character spaces).
- Align: Visual text alignment in Data View cells (Left, Right, Center). Standard practice: Right-align numeric variables, Left-align string variables.
- Measure (Measurement Level): Governs the mathematical properties of the variable and dictates which statistical procedures SPSS will permit. Three distinct levels exist:
- Nominal: Qualitative categories without intrinsic ranking (e.g., Gender, Religion, Department, Marital Status).
- Ordinal: Qualitative categories with an explicit, meaningful rank order, but where intervals between ranks are unequal or non-quantifiable (e.g., Likert scales: Low/Medium/High, Education Level: High School/Bachelor/Master/PhD).
- Scale (Continuous/Metric): Quantitative interval or ratio data with meaningful numerical distances and potential absolute zeros (e.g., Age in years, Annual Income, Product Price, Weight in kg).
- Role: Designates the operational function of the variable in automated modeling dialogs (e.g., Input for independent predictor variables, Target for dependent response variables, Both, or None).
Step-by-Step Data Entry Protocols
Constructing an empirical data file from scratch in SPSS follows a disciplined three-phase protocol:
Essential Data Editing & Manipulation Utilities
- Inserting and Deleting: To add a new case, right-click any row number in Data View and select
Insert Cases. To add a new variable, right-click any column header and selectInsert Variable. Deleting is accomplished by highlighting rows/columns and pressingDelete. - Sort Cases: Reorders observations based on specified keys via
Data -> Sort Cases. Can sort by ascending or descending order across single or multiple hierarchical variables (e.g., sorting primary by Department, then secondary by Salary). - Split File: Divides the dataset into subgroups for comparative analysis via
Data -> Split File. Selecting "Compare groups" produces combined output tables partitioned by the grouping variable (e.g., generating separate descriptive statistics for male and female respondents). - Select Cases: Filters the active dataset to execute procedures on a specific sub-population via
Data -> Select Cases. Using the conditional logic dialog (If condition is satisfied), an analyst can filter cases whereage >= 25 AND monthly_exp > 30000. Unselected cases are either temporarily filtered or permanently deleted. - Compute Variable: Calculates new variables based on mathematical transformations of existing variables via
Transform -> Compute Variable(e.g.,annual_income = monthly_salary * 12 + annual_bonus). - Recode into Different Variables: Re-bins continuous metrics into discrete ordinal categories via
Transform -> Recode into Different Variables(e.g., converting continuousageinto age brackets: 1 = "Under 25", 2 = "25-40", 3 = "Above 40"). Best Practice: Always recode into different variables to preserve raw original data.
Module II: Inferential Statistics
While descriptive statistics serve to organize, summarize, and tabulate the empirical properties of an observed sample, Inferential Statistics enables business researchers to make probabilistic deductions about an entire unobserved target population based on sample data. In corporate research, inferential analytics bridges the gap between limited empirical evidence and executive decision-making under uncertainty. This module examines the foundational architecture of hypothesis formulation, the formal mechanics of statistical significance testing in SPSS, the application and interpretation of the Non-Parametric Chi-Square (χ²) Test of Independence, and Bivariate Correlation Analysis spanning Pearson's parametric and Spearman's non-parametric formulations.
2.1 Hypothesis Formulation & Testing Architecture in SPSS
The Logic of Statistical Inference
In corporate commerce, surveying an entire consumer census is economically prohibitive. Instead, analysts draw a representative random sample of size n from a broader target population of size N. Sample statistics (such as the sample mean ¯x or sample proportion p) serve as point estimates of underlying population parameters (such as the true population mean μ or proportion P).
Because samples inevitably exhibit sampling error—natural random variation from one sample draw to another—inferential statistical testing provides a rigorous probabilistic framework to determine whether an observed business outcome reflects a genuine real-world effect or merely random sampling fluctuations.
Formulating the Null and Alternative Hypotheses
Every statistical investigation requires the formal specification of two mutually exclusive and exhaustive propositions:
The Null Hypothesis (H&sub0;): Represents the benchmark proposition of "no effect", "no difference", or "no association". It presumes that any observed difference in sample data is purely the artifact of chance and sampling error. In statistical modeling, H&sub0; is assumed true until empirical evidence demonstrates otherwise beyond a reasonable doubt.
Examples:
• H&sub0;: μ&sub1; = μ&sub2; (No difference in mean sales between Region A and Region B).
• H&sub0;: ρ = 0 (No linear correlation between advertising budget and revenue).
The Alternative Hypothesis (H&sub1; or H&suba;): The research hypothesis that the investigator seeks to validate. It asserts the presence of a real relationship, treatment effect, or parameter divergence in the population.
Directional (One-Tailed) vs. Non-Directional (Two-Tailed):
• Two-Tailed (Non-Directional): H&sub1;: μ&sub1; ≠ μ&sub2; (Tests for any significant difference in either direction).
• One-Tailed (Directional): H&sub1;: μ&sub1; > μ&sub2; (Predicts specifically that Region A will outperform Region B). Standard corporate research defaults to two-tailed tests to remain objective.
The Decision Framework: Type I & Type II Errors and Significance Levels
Because statistical inference relies on probability rather than absolute certainty, decision-makers are exposed to two fundamental categories of decision error:
| Reality in Population → Statistical Decision ↓ |
Null Hypothesis (H&sub0;) is TRUE (No real business effect exists) |
Null Hypothesis (H&sub0;) is FALSE (A real business effect exists) |
|---|---|---|
| Reject H&sub0; (Claim an effect exists) |
Type I Error (α - Alpha) "False Positive" Rejecting a true null hypothesis. Example: Concluding a new ad campaign drives sales when it actually has no effect. Probability = Significance Level (α = 0.05). |
Correct Decision (1 - β) Statistical Power Correctly detecting and validating a real market effect. Standard target power is 0.80 (80%). |
| Fail to Reject H&sub0; (Claim no effect detected) |
Correct Decision (1 - α) Confidence Level Correctly maintaining the status quo when no real difference exists (typically 0.95 or 95%). |
Type II Error (β - Beta) "False Negative" Failing to reject a false null hypothesis. Example: Abandoning a viable product launch because the sample was too small to detect the true positive effect. |
The p-value Decision Rule in SPSS
In IBM SPSS, the analyst does not manually calculate critical rejection regions from statistical tables. Instead, SPSS outputs the exact p-value, labeled in output tables as "Sig. (2-tailed)" or "Asymptotic Significance".
• If p-value ≤ α (e.g., Sig. ≤ 0.05): Reject the Null Hypothesis (H&sub0;). The result is Statistically Significant. There is sufficient empirical evidence to support the research hypothesis H&sub1;.
• If p-value > α (e.g., Sig. > 0.05): Fail to Reject the Null Hypothesis (H&sub0;). The result is Not Statistically Significant. The observed difference can be reasonably attributed to random sampling variation.
2.2 The Chi-Square (χ²) Test of Independence in SPSS
Theoretical Framework & Business Scope
The Pearson Chi-Square (χ²) Test of Independence is a non-parametric statistical procedure used to evaluate whether a significant relationship exists between two categorical (nominal or ordinal) variables. In business research, it answers questions such as:
- Is customer brand preference (Brand A vs. Brand B vs. Brand C) dependent on gender (Male vs. Female)?
- Is customer loan default status (Default vs. Non-Default) associated with occupational category (Salaried vs. Self-Employed vs. Professional)?
- Does consumer subscription churn depend on preferred payment method (Credit Card vs. UPI vs. Net Banking)?
Mathematical Foundation & Calculation of Expected Frequencies
The test compares the Observed Frequencies (O)—the actual case counts tallying in each cell of a cross-tabulation grid—against the Expected Frequencies (E)—the theoretical counts that would occur if the two variables were completely independent.
E = (Row Total × Column Total) / Grand Total (N)
Pearson Chi-Square Test Statistic:
χ² = ∑ [ (O - E)² / E ]
Degrees of Freedom (df):
df = (r - 1) × (c - 1) (where r = number of rows, c = number of columns)
Underlying Assumptions and Special Adjustments
- Categorical Data: Both variables must be nominal or ordinal. Continuous variables must be recoded into discrete bins before running Chi-Square.
- Mutual Exclusivity: Each subject or observation belongs to one and only one cell.
- Cochran's Minimum Cell Frequency Rule: For the test to be mathematically valid, no cell should have an expected frequency less than 1, and no more than 20% of the cells should have expected frequencies less than 5.
- Continuity Correction (Yates' Correction): For 2 × 2 contingency tables, SPSS reports Continuity Correction, which subtracts 0.5 from the absolute difference |O - E| to adjust for the continuous approximation of a discrete distribution.
- Fisher's Exact Test: Automatically computed by SPSS for 2 × 2 tables when expected cell counts violate the sample size assumption (E < 5).
Quantifying Effect Size: Phi and Cramér's V
A statistically significant Chi-Square test indicates that an association exists, but does not communicate the practical magnitude or strength of the relationship. SPSS computes normalized effect size metrics:
| Metric | Table Geometry | Interpretation Scale |
|---|---|---|
| Phi (φ) Coefficient | Strictly 2 × 2 contingency tables | Ranges from 0 to 1.0. (0.10 = Small effect, 0.30 = Moderate effect, 0.50+ = Large effect). |
| Cramér's V (V) | Tables larger than 2 × 2 (e.g., 3 × 3, 2 × 4) | Adjusts for table dimensions. Ranges from 0 to 1.0. Serves as the universal standard for multi-category tables. |
Step-by-Step SPSS Execution Protocol for Chi-Square
Interpreting SPSS Chi-Square Output Tables
Examine the table labeled Chi-Square Tests:
- Locate the top row: Pearson Chi-Square.
- Inspect the Value column (χ² statistic), the df column (degrees of freedom), and the Asymptotic Significance (2-sided) column (p-value).
- Rule: If Asymp. Sig. < 0.05, we reject H&sub0; and conclude there is a statistically significant association between the two variables.
- Inspect the footnote below the table: SPSS explicitly states: "0 cells (0.0%) have expected count less than 5. The minimum expected count is 14.20." If the percentage exceeds 20%, Cochran's assumption is violated, and the analyst should either combine categories or report Fisher's Exact Test.
2.3 Bivariate Correlation Analysis in SPSS
Theoretical Framework & Nature of Correlation
Correlation Analysis investigates the degree, strength, and direction of linear association between two numerical or ordinal variables. It does not imply cause-and-effect; rather, it quantifies co-variation—the tendency of two variables to vary systematically together.
Direction of Association: • Positive Correlation (+): As variable X increases, variable Y increases systematically (e.g., Advertising expenditure and Total sales revenue).
• Negative Correlation (-): As variable X increases, variable Y decreases systematically (e.g., Product price and Sales volume demanded; Employee stress and Job satisfaction).
• Zero / Negligible Correlation (0): No systematic relationship exists (e.g., Employee shoe size and Annual sales volume).
Magnitude Benchmark Guidelines: The correlation coefficient (r) spans strictly from -1.00 to +1.00:
• 0.00 to ±0.29: Weak / Negligible association.
• ±0.30 to ±0.49: Moderate association.
• ±0.50 to ±0.69: Substantial / Strong association.
• ±0.70 to ±0.99: Very strong / Very high association.
• ±1.00: Perfect deterministic linear relationship.
Pearson's Product-Moment Correlation Coefficient (r)
Pearson's r is the premier parametric measure of linear association for two continuous, metric (Scale) variables. It requires that both variables exhibit bivariate normality, linearity, and homoscedasticity without influential multivariate outliers.
r = ∑ [ (X - ¯X)(Y - ¯Y) ] / √[ ∑(X - ¯X)² × ∑(Y - ¯Y)² ]
The Coefficient of Determination (r²):
Squaring Pearson's r yields r², which quantifies the proportion of total variance in Y that is directly explained or accounted for by its linear relationship with X.
Example: If r = 0.80 between Training Hours and Employee Productivity, r² = (0.80)² = 0.64 (64% of the variance in productivity is explained by training).
Non-Parametric Alternatives: Spearman's Rank & Kendall's Tau-b
When research data fails the stringent parametric assumptions of normality, or when data is measured on an ordinal scale (such as Likert satisfaction ratings, rankings, or performance tiers), non-parametric correlation techniques are deployed:
- Spearman's Rank-Order Correlation (ρ - Rho / r&subs;): Computes Pearson's r on the converted rank orders of the data rather than raw scores. It evaluates monotonic relationships—whether variables change in the same general direction, even if non-linearly.
- Kendall's Tau-b (τ&subb;): A robust non-parametric measure based on concordant and discordant pairs of observations. It is mathematically superior to Spearman's rho when the dataset has a small sample size or a substantial number of tied ranks.
Step-by-Step SPSS Execution Protocol for Correlation
Interpreting the SPSS Correlation Matrix
The output presents a symmetric matrix where every variable is paired against every other variable:
- The diagonal cells show the correlation of each variable with itself (always Pearson Correlation = 1.000).
- Each off-diagonal cell displays three critical numbers:
- Pearson Correlation: The direction and magnitude of association (e.g.,
0.732**). - Sig. (2-tailed): The exact p-value (e.g.,
.000, which indicates p < 0.001). - N: The total number of valid paired cases included in the computation.
- Pearson Correlation: The direction and magnitude of association (e.g.,
- Academic Interpretation: "A Pearson product-moment correlation was run to determine the relationship between advertising spend and revenue. There was a strong, positive, statistically significant linear correlation between the two variables, r(98) = .732, p < .001."