Skip to content

feat(intermediate): add student feedback sentiment analyzer starter project - #10

Open
Hitesh-XS wants to merge 1 commit into
AIMLCLUBOCT:mainfrom
Hitesh-XS:feat/student-feedback-analyzer
Open

Hitesh-XS wants to merge 1 commit into
AIMLCLUBOCT:mainfrom
Hitesh-XS:feat/student-feedback-analyzer

Conversation

@Hitesh-XS

Copy link
Copy Markdown

Closes #8

📌 Summary

Adds a new intermediate starter project under intermediate/student-feedback-sentiment-analyzer/. This project trains, compares, and evaluates classic Machine Learning models to classify course feedback into Positive, Neutral, or Negative categories, and performs aspect-based sentiment analysis (which part of the course each opinion is about).


🛠️ What's Included

1. train_and_evaluate.py

  • Data Generation: Template-generated synthetic data (~900 comments across 3 balanced classes). The 16 hand-written campus comments are kept test-only as an independent gold set. A slice of phrase templates is also held out from training to measure generalization to unseen wording.
  • Preprocessing: Custom text normalization with negation marking ("not helpful" → not_helpful, scope ends at punctuation or "but"), contraction expansion, and hybrid TF-IDF features: word n-grams (1–2) plus character n-grams (3–5).
  • Model Comparison: Evaluates Logistic Regression, Calibrated Linear SVC, Complement Naive Bayes, and a soft-voting ensemble using 5-fold Stratified Cross-Validation (macro-F1). Optional --tune grid-searches Logistic Regression (n-grams, C).
  • Evaluation Metrics: Evaluates on three test sets (in-distribution, unseen phrasing, hand-written gold) with Precision, Recall, Macro F1, Accuracy, and a Confusion Matrix. Metrics can be exported with --metrics-out.
  • Theme & Aspect Extraction: Top n-grams per class, most influential words from Logistic Regression weights, and clause-level aspect-based sentiment (lectures, labs, assignments, instructor, exams, materials, pace). Mixed comments like "labs were helpful, but deadlines were too tight" get a sentiment per aspect, plus confidence and low-confidence flags.
  • Model Persistence: --save / --load to reuse a trained model, and --predict "..." to analyze a single comment.
  • Bring Your Own Data: --data feedback.csv (columns text,label) trains on real comments.
  • Interactive Mode: Optional --interactive CLI flag for testing custom comments in real time.

2. Supporting Files

  • requirements.txt: Standard dependencies (scikit-learn, pandas, numpy, joblib).
  • README.md: Covers project architecture, evaluation methodology, setup instructions, usage examples, sample results, and system limitations.
  • intermediate/README.md: Updated index linking to the new project.

📊 Sample Results (default synthetic run, macro-F1)

Model In-distribution Unseen phrasing Hand-written gold
Logistic Regression 1.00 0.79 0.94
Linear SVC (calibrated) 1.00 0.80 0.94
Complement NB 0.91 0.79 0.88
Ensemble (soft voting) 1.00 0.80 0.94

In-distribution and CV scores are optimistic because synthetic folds share phrase templates. The unseen phrasing and hand-written columns are the more honest generalization signals.


⚠️ Known Limitations

  • Vocabulary is limited by the synthetic data, so unseen words (e.g. "great") carry little signal. Training on real comments via --data is recommended.
  • Aspect detection is keyword-based and heuristic. Sarcasm and long multi-sentence comments may be misread.
  • Saved models must be loaded through train_and_evaluate.py (preprocessing functions are pickled by reference).

🧪 How to Test

cd intermediate/student-feedback-sentiment-analyzer
pip install -r requirements.txt

# Run full pipeline (Train, Cross-Validate, Evaluate, Feature Weights, ABSA samples)
python train_and_evaluate.py

# Save a model, then predict a single comment
python train_and_evaluate.py --save model.joblib
python train_and_evaluate.py --load model.joblib --predict "Labs were helpful but the deadlines were too tight"

# Optional: hyperparameter tuning, custom data, interactive mode
python train_and_evaluate.py --tune
python train_and_evaluate.py --data feedback.csv
python train_and_evaluate.py --interactive

✅ Checklist

  • Runs end to end with default settings
  • Includes README.md and requirements.txt
  • Updated intermediate/README.md index
  • No external datasets or network access required

@github-actions

Copy link
Copy Markdown

🎉 Thank you for your Pull Request, @Hitesh-XS!

Welcome to AIML Club OCT! 🚀

A maintainer will review your code shortly. In the meantime:

Happy coding! 💻✨

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(intermediate): Add student feedback sentiment analyzer starter project

1 participant