Skip to content

About

NLP project for TY sem 2

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

4 Commits

Folders and files

Repository files navigation

Python Streamlit NLTK OpenAI License

⚖️ Contract Clause Risk Explainer

An intelligent, rule-based NLP tool that analyzes legal contract clauses, quantifies risk through deterministic scoring, and provides actionable insights — all without requiring machine learning models.

Features • Architecture • Quick Start • How It Works • Project Structure • Commands • Deployment


🎯 Overview

Contract Clause Risk Explainer is an educational NLP-powered web application built with Streamlit that helps users understand and assess risk in legal contract clauses. The tool uses a fully deterministic, rule-based approach — no ML black boxes — ensuring every risk decision is transparent, explainable, and auditable.

Why Rule-Based?

Rule-Based (This Project) ML-Based
✅ 100% explainable decisions ❌ Black-box predictions
✅ No training data required ❌ Needs labeled datasets
✅ Instant, consistent results ❌ May vary between runs
✅ Full audit trail ❌ Hard to audit
✅ Works offline ❌ Often needs cloud APIs

✨ Features

Core Analysis

  • 📋 Clause Classification — Identifies clause type from 23 legal categories (Indemnification, Liability, Termination, IP, etc.) using weighted keyword matching
  • 🎯 Risk Scoring Engine — Deterministic, configurable risk scoring with full breakdown (Low / Medium / High)
  • 🔍 Risk Feature Extraction — Detects broad scope language, power imbalance, conditional traps, complexity, and one-sided obligations
  • 📊 Confidence Calculation — Transparent confidence metric based on keyword match strength and feature count

Explainability & Advisory

  • 📋 Detailed Risk Analysis — Per-category keyword breakdown showing exactly which verbs and phrases triggered each score
  • 📝 Risk Summary — Auto-generated natural language summary of the analysis
  • ⚠️ What to Watch Out For — Context-specific warnings based on detected risk features
  • 💡 Negotiation Tips — Actionable suggestions tailored to the clause type and detected risks
  • 🤖 AI-Powered Explanation — Optional OpenAI integration for plain-English explanations (with full offline fallback)

User Experience

  • 🎨 Polished Dark Theme UI — Custom CSS with gradients, animations, and responsive layout
  • 🏷️ Keyword Highlighting — Color-coded pills overlaid on the original clause text
  • 📊 Risk Gauge & Progress Bar — Visual risk indicators with smooth animations
  • 📱 Info Cards — At-a-glance metrics for clause type, risk level, and confidence

🏗 Architecture

┌─────────────────────────────────────────────────────┐
│                    STREAMLIT UI                      │
│              (app.py + ui/ modules)                  │
│  ┌──────────┐  ┌──────────┐  ┌───────────────────┐  │
│  │Risk Gauge│  │ Keywords │  │Highlighted Clause │  │
│  │Progress  │  │ Badges   │  │Analysis Cards     │  │
│  └──────────┘  └──────────┘  └───────────────────┘  │
└─────────────────────┬───────────────────────────────┘
                      │
         ┌────────────┴────────────┐
         ▼                         ▼
┌─────────────────┐     ┌──────────────────┐
│   NLP PIPELINE  │     │   GenAI MODULE   │
│   (nlp/)        │     │   (genai/)       │
│                 │     │                  │
│ preprocessing.py│     │ explanation.py   │
│ clause_classif. │     │ (OpenAI + local  │
│ risk_features.py│     │  fallback)       │
└────────┬────────┘     └──────────────────┘
         │
         ▼
┌─────────────────────┐     ┌──────────────────┐
│    RULES ENGINE     │     │   KNOWLEDGE BASE │
│    (rules/)         │◄────│   (data/)        │
│                     │     │                  │
│  risk_scoring.py    │     │ keywords.json    │
│  weights.py         │     │ clause_categories│
│  explainability.py  │     │ prompts.json     │
└─────────────────────┘     └──────────────────┘

Data Flow

User Input (clause text)
    │
    ▼
┌─ Preprocessing (NLTK tokenization, lemmatization, sentence splitting)
│       │
│       ▼
├─ Clause Classification (keyword matching against 23 categories)
│       │
│       ▼
├─ Risk Feature Extraction (broad scope, power imbalance, traps, complexity)
│       │
│       ▼
├─ Risk Scoring (weighted feature aggregation → Low/Medium/High)
│       │
│       ▼
├─ Explainability (audit trail, summaries, warnings, tips)
│       │
│       ▼
└─ Rendering (gauges, cards, highlights, AI explanation)

🚀 Quick Start

Prerequisites

  • Python 3.10+ (tested on 3.13)
  • pip package manager

Installation

# 1. Clone the repository
git clone https://github.com/YOUR_USERNAME/contract-clause-risk-explainer.git
cd contract-clause-risk-explainer

# 2. Create a virtual environment (recommended)
python -m venv venv
source venv/bin/activate        # macOS/Linux
# venv\Scripts\activate         # Windows

# 3. Install dependencies
pip install -r requirements.txt

# 4. Download NLTK data (one-time setup)
python -c "import nltk; nltk.download('punkt'); nltk.download('punkt_tab'); nltk.download('wordnet'); nltk.download('averaged_perceptron_tagger')"

# 5. (Optional) Set OpenAI API key for AI explanations
export OPENAI_API_KEY="sk-your-key-here"    # macOS/Linux
# set OPENAI_API_KEY=sk-your-key-here       # Windows

# 6. Run the app
streamlit run app.py

The app will open at http://localhost:8501.


🔬 How It Works

1. Text Preprocessing (nlp/preprocessing.py)

The input clause text goes through:

  • Tokenization — Splits text into individual words using NLTK's word_tokenize (with regex fallback)
  • Lemmatization — Reduces words to their base form (e.g., "indemnifies" → "indemnify") using NLTK's WordNetLemmatizer
  • Sentence Splitting — Splits text into sentences using sent_tokenize for complexity analysis
Input:  "The Provider shall indemnify and hold harmless..."
Tokens: ["provider", "shall", "indemnify", "hold", "harmless", ...]
Sentences: ["The Provider shall indemnify and hold harmless..."]

2. Clause Classification (nlp/clause_classifier.py)

A rule-based keyword matcher that scores each of the 23 categories:

  • Loads data/keywords.json containing verbs and phrases for each legal clause type
  • Scores each category: verbs get 2 points, phrases get 3 points (phrases are more specific)
  • Returns the highest-scoring category as the clause type
Example: "shall indemnify" →
  Indemnification: verb "indemnify" = 2pts → BEST MATCH
  Obligations:     verb "shall" = 2pts

3. Risk Feature Extraction (nlp/risk_features.py)

Extracts 6 types of risk signals:

Feature What It Detects Example Triggers
Triggered Keywords Verbs/phrases from all 23 categories "shall", "indemnify", "terminate"
Broad Scope Phrases Language that widens the clause's reach "any and all", "without limitation"
Power Imbalance One-sided control language "sole discretion", "unilateral"
Conditional Traps Exceptions that weaken protections "unless", "subject to", "notwithstanding"
Complexity Dense, hard-to-read language Word count > 35, heavy punctuation
One-Sided Obligations Disproportionate duty language "only", "solely", "exclusively"

4. Risk Scoring (rules/risk_scoring.py + rules/weights.py)

A deterministic scoring engine that converts features into a numeric risk score:

Weights (configurable in weights.py):

Feature Weight per Item
Triggered Keywords ×2
Broad Scope Phrases ×3
Power Imbalance ×3
Conditional Traps ×2
Complexity ×2
One-Sided Obligations ×2

Formula: risk_score = Σ (weight × count of items in each feature)

Thresholds:

Score Range Risk Level
0 – 14 🟢 Low
15 – 29 🟠 Medium
30+ 🔴 High

5. Classification Confidence Calculation

The confidence percentage reflects how sure the system is about its classification:

raw_confidence = min(match_score × 8 + keyword_count × 3 + feature_count × 5, 100)
confidence = max(raw_confidence, 15)  # Floor at 15%
Component Multiplier Reasoning
match_score ×8 Strongest signal — how well the clause matched its category
keyword_count ×3 More keywords detected = more evidence
feature_count ×5 More risk features triggered = more context for classification
Floor: 15% — Even with minimal matches, we have some classification basis
Cap: 100% — Prevent overflow

Example: match_score=5, 8 keywords, 2 features → min(40 + 24 + 10, 100) = 74%

6. Explainability Layer (rules/explainability.py)

Generates human-readable outputs:

  • explain_risk() — Structured per-category audit trail showing which verbs and phrases triggered each score
  • generate_risk_summary() — Natural language paragraph summarizing findings
  • generate_watch_out() — Context-aware warnings about specific detected patterns
  • generate_negotiation_tips() — Actionable suggestions based on clause type and features

7. GenAI Explanation (genai/explanation.py)

  • With API Key: Sends clause to OpenAI GPT-3.5-turbo for plain-English explanation, real-world example, risk assessment, and beneficiary analysis
  • Without API Key: Uses a built-in fallback system with pre-written educational explanations for common clause types (Indemnification, Obligations, Liability, etc.)

8. Frontend Rendering (app.py + ui/)

  • ui/visuals.py — Risk gauge badge (animated, color-coded) and progress bar
  • ui/output_formatter.py — Keyword highlighting with color-coded pills, explanation cards, warning cards, tip cards
  • app.py — Streamlit orchestration, global CSS, section rendering, all module integration

📂 Project Structure

contract-clause-risk-explainer/
│
├── app.py                          # Main Streamlit application entry point
├── requirements.txt                # Python dependencies
├── README.md                       # This file
├── Context.md                      # Project specification (Single Source of Truth)
│
├── data/                           # Knowledge base (JSON)
│   ├── keywords.json               # 23 categories × verbs + phrases
│   ├── clause_categories.json      # Clause category definitions
│   └── prompts.json                # Prompt templates for GenAI
│
├── nlp/                            # NLP pipeline
│   ├── __init__.py
│   ├── preprocessing.py            # Tokenization, lemmatization, sentence splitting
│   ├── clause_classifier.py        # Rule-based clause type identification
│   ├── risk_features.py            # Risk feature extraction engine
│   └── utils.py                    # NLP utilities
│
├── rules/                          # Rule engine
│   ├── __init__.py
│   ├── risk_scoring.py             # Deterministic risk scoring
│   ├── weights.py                  # Configurable weights & thresholds
│   └── explainability.py           # Audit trail, summaries, warnings, tips
│
├── genai/                          # GenAI integration
│   ├── __init__.py
│   └── explanation.py              # OpenAI integration + offline fallback
│
├── ui/                             # UI components
│   ├── __init__.py
│   ├── visuals.py                  # Risk gauge, progress bar
│   ├── output_formatter.py         # Keyword highlighting, card formatting
│   └── layout.py                   # Layout utilities
│
├── tests/                          # Test suite
│   ├── __init__.py
│   ├── test_classifier.py
│   ├── test_explainability.py
│   ├── test_preprocessing.py
│   └── test_risk_scoring.py
│
└── .streamlit/
    └── config.toml                 # Streamlit theme configuration

📖 Module Documentation

nlp/preprocessing.py

Function Input Output
preprocess_clause(text) Raw clause string (tokens: list[str], sentences: list[str])

nlp/clause_classifier.py

Function Input Output
classify_clause(tokens) Lemmatized tokens (clause_type: str, match_score: int, matched_keywords: list)

nlp/risk_features.py

Function Input Output
extract_risk_features(text, tokens, sentences) Raw text + tokens + sentences dict with keys: triggered_keywords, broad_scope_phrases, power_imbalance, conditional_traps, complexity, one_sided_obligations

rules/risk_scoring.py

Function Input Output
score_risk(features) Features dict (risk_level: str, risk_score: int, breakdown: dict)

rules/explainability.py

Function Input Output
explain_risk(features, breakdown) Features + breakdown list[dict] with title, score, details
generate_risk_summary(...) All analysis data str (HTML paragraph)
generate_watch_out(features, clause_type) Features + type list[str] (warnings)
generate_negotiation_tips(clause_type, features) Type + features list[str] (tips)

genai/explanation.py

Function Input Output
generate_explanation(clause_text, clause_type) Raw text + type str (formatted explanation)

🖥 Commands Reference

Starting the Application

# Start the app (opens browser automatically)
streamlit run app.py

# Start on a custom port
streamlit run app.py --server.port 8080

# Start without auto-opening browser
streamlit run app.py --server.headless true

# Start in wide layout mode
streamlit run app.py --theme.base dark

Stopping the Application

# If running in foreground: press Ctrl+C

# If running in background: kill the process
pkill -f "streamlit run"

# Or find and kill by PID
lsof -i :8501
kill <PID>

Development Commands

# Install dependencies
pip install -r requirements.txt

# Download NLTK data
python -c "import nltk; nltk.download('punkt'); nltk.download('punkt_tab'); nltk.download('wordnet')"

# Set OpenAI API key (optional)
export OPENAI_API_KEY="sk-..."

# Check installed packages
pip list | grep -E "streamlit|nltk|openai"

# Upgrade Streamlit
pip install --upgrade streamlit

# Clear Streamlit cache
streamlit cache clear

Git Commands

# Initialize git repo
git init
git add .
git commit -m "Initial commit: Contract Clause Risk Explainer"

# Push to GitHub
git remote add origin https://github.com/YOUR_USERNAME/contract-clause-risk-explainer.git
git branch -M main
git push -u origin main

🚢 Deployment

Option 1: Streamlit Community Cloud (Recommended — Free)

  1. Push your code to GitHub
  2. Go to share.streamlit.io
  3. Click "New app" → select your repo, branch, and app.py
  4. (Optional) Add OPENAI_API_KEY in Advanced Settings → Secrets
  5. Click Deploy — your app will be live at https://your-app.streamlit.app

Option 2: Docker

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN python -c "import nltk; nltk.download('punkt'); nltk.download('punkt_tab'); nltk.download('wordnet')"
COPY . .
EXPOSE 8501
CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]
docker build -t clause-risk-explainer .
docker run -p 8501:8501 clause-risk-explainer

Option 3: Railway / Render

  1. Push to GitHub
  2. Connect repo on Railway or Render
  3. Set start command: streamlit run app.py --server.port $PORT --server.address 0.0.0.0
  4. Add environment variables if using OpenAI

Note: Vercel is optimized for frontend/serverless apps and doesn't natively support long-running Streamlit processes. Use Streamlit Community Cloud, Railway, or Docker instead.


⚙️ Configuration

Risk Weights (rules/weights.py)

WEIGHTS = {
    "triggered_keywords": 2,
    "broad_scope_phrases": 3,
    "power_imbalance": 3,
    "conditional_traps": 2,
    "complexity": 2,
    "one_sided_obligations": 2,
}

RISK_THRESHOLDS = {
    "Low": 0,      # Score 0-14
    "Medium": 15,   # Score 15-29
    "High": 30,     # Score 30+
}

Streamlit Theme (.streamlit/config.toml)

[theme]
primaryColor = "#FF6B6B"
backgroundColor = "#0e1117"
secondaryBackgroundColor = "#262730"
textColor = "#fafafa"

Adding New Clause Categories

Edit data/keywords.json:

{
  "Your New Category": {
    "verbs": ["keyword1", "keyword2"],
    "phrases": ["multi word phrase"]
  }
}

The system will automatically pick up new categories — no code changes needed.


🔧 Requirements

streamlit>=1.29.0
openai>=1.3.0        # Optional — for AI explanations
nltk>=3.8.1

📜 License

This project is for educational purposes only. It does not provide legal advice. Always consult with a qualified attorney for legal matters.


🤝 Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Built with ❤️ using Python, Streamlit, and NLTK

About

NLP project for TY sem 2

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages