Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

# Utix

An open-source toolkit of small, focused utilities for Stellar developers —
Expand Down Expand Up @@ -29,6 +30,7 @@ Infrastructure docs: [Telemetry](./docs/TELEMETRY.md) ·
[Audit trail](./docs/AUDIT.md) ·
[Tamper-evident ledger](./docs/LEDGER.md) ·
[Fee calculation](./docs/FEES.md) ·
[Anomaly detection](./docs/ANOMALY_DETECTION.md) ·
[Authorization expiry](./docs/AUTHORIZATIONS.md) ·
[Public status feed](./docs/PUBLIC_STATUS.md) ·
[Seed data](./docs/SEED_DATA.md) ·
Expand All @@ -45,6 +47,9 @@ Infrastructure docs: [Telemetry](./docs/TELEMETRY.md) ·
| Transaction Lookup | Ledger, fee, memo, result and operations for a transaction hash |
| Freighter Connect | Detects the wallet and warns about a network mismatch |
| Testnet Faucet | Funds a testnet account through Friendbot |
| Anomaly Report | Read-only report of unusual volume, timing and actor behavior |
| | Grouped by severity and affected resource, with triage guidance |
| | Signals: volume spikes, timing anomalies, actor repetition, failures |

## Tech stack

Expand Down Expand Up @@ -83,12 +88,16 @@ npm run verify:issues # check 40+ independent issues and the advanced wav
npm run verify:telemetry # verify >= 5 operations are instrumented
npm run verify:reconciliation # every invariant is checked, the dry run is read-only, repair is dry-run-first
npm run verify:contract # run the API contract drift tests
npm run verify:anomaly # anomaly report identifies fixture spikes, repeats and suspicious actors
npm run verify:anomaly # validates severity grouping, resource grouping and triage output
npm run verify:replay # versioned event replay suite
npm run verify:replay-concurrency # deterministic concurrent-write and retry replay tests
npm run test:workers # run the background worker suite
npm test -- core/lifecycle # allowed and rejected record state transitions
npm test -- core/idempotency # retry, replay, expiry and key collisions
npm test -- core/reconciliation # drift, root-cause hints, repair plan/apply/rollback
npm test -- features/anomaly-report # baseline, spike, repeated failure and suspicious actor scenarios
npm test -- features/anomaly-report # also covers false-positive mitigation and severity grouping
npm run issues # preview the next five GrantFox issue payloads
npm run test # unit, hook, component and accessibility tests
npm run test:e2e # full Playwright suite
Expand Down Expand Up @@ -141,7 +150,16 @@ See documentation for operational workflows:
- [DISASTER_RECOVERY.md](./docs/DISASTER_RECOVERY.md) - Data validation after restore
- [OPERATIONAL_HEALTH.md](./docs/OPERATIONAL_HEALTH.md) - Health monitoring dashboard
- [BUNDLE_BUDGET.md](./docs/BUNDLE_BUDGET.md) - Per-feature bundle size enforcement
- [ANOMALY_DETECTION.md](./docs/ANOMALY_DETECTION.md) - Anomaly signals, severity triage and false-positive mitigation

## Security

See [SECURITY.md](./SECURITY.md). Utix is read-only by design.

The anomaly report is read-only and groups findings by severity and affected
resource. See [docs/ANOMALY_DETECTION.md](./docs/ANOMALY_DETECTION.md) for the
signal definitions and maintainer triage guidance.

False-positive mitigation is documented in the same guide: baseline windows,
minimum sample sizes, and per-signal suppression rules keep normal traffic
from being flagged while still surfacing genuine spikes, repeats and failures.
137 changes: 137 additions & 0 deletions core/analytics/__tests__/aggregator.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,141 @@ describe('AnalyticsAggregator', () => {
const insights = analyticsAggregator.getInsights();
expect(Array.isArray(insights)).toBe(true);
});

describe('anomaly detection', () => {
it('reports normal baseline without anomalies', async () => {
const now = Date.now();
for (let i = 0; i < 5; i++) {
analyticsAggregator.recordEvent({
category: 'ticket_issuance',
eventType: 'ticket_issued',
userId: `user-${i}`,
timestamp: now - i * 60_000,
});
}

const report = await analyticsAggregator.generateAnomalyReport();
expect(report.findings.length).toBe(0);
expect(report.summary.totalFindings).toBe(0);
});

it('detects volume spikes against baseline', async () => {
const now = Date.now();
// Baseline: a few events per hour over several hours
for (let h = 1; h <= 6; h++) {
for (let i = 0; i < 2; i++) {
analyticsAggregator.recordEvent({
category: 'ticket_redemption',
eventType: 'ticket_redeemed',
userId: `baseline-user-${h}-${i}`,
timestamp: now - h * 3_600_000,
});
}
}
// Spike: many events in the current hour
for (let i = 0; i < 40; i++) {
analyticsAggregator.recordEvent({
category: 'ticket_redemption',
eventType: 'ticket_redeemed',
userId: `spike-user-${i}`,
timestamp: now - (i % 60) * 1_000,
});
}

const report = await analyticsAggregator.generateAnomalyReport();
const volume = report.findings.find(f => f.signal === 'volume_spike');
expect(volume).toBeDefined();
expect(['medium', 'high', 'critical']).toContain(volume?.severity);
expect(volume?.resource).toBe('ticket_redemption');
});

it('detects repeated failures from an actor', async () => {
const now = Date.now();
for (let i = 0; i < 12; i++) {
analyticsAggregator.recordEvent({
category: 'error',
eventType: 'ticket_redemption_failed',
userId: 'repeat-failure-actor',
timestamp: now - i * 5_000,
});
}

const report = await analyticsAggregator.generateAnomalyReport();
const failure = report.findings.find(f => f.signal === 'repeated_failure');
expect(failure).toBeDefined();
expect(['medium', 'high', 'critical']).toContain(failure?.severity);
});

it('detects suspicious actor repetition', async () => {
const now = Date.now();
for (let i = 0; i < 30; i++) {
analyticsAggregator.recordEvent({
category: 'ticket_transfer',
eventType: 'ticket_transferred',
userId: 'suspicious-actor',
timestamp: now - i * 2_000,
});
}

const report = await analyticsAggregator.generateAnomalyReport();
const actor = report.findings.find(f => f.signal === 'actor_repetition');
expect(actor).toBeDefined();
expect(['medium', 'high', 'critical']).toContain(actor?.severity);
});

it('detects unusual timing patterns', async () => {
// Events clustered at an off-hours window (03:00 UTC)
const base = new Date();
base.setUTCHours(3, 0, 0, 0);
const offHours = base.getTime();
for (let i = 0; i < 25; i++) {
analyticsAggregator.recordEvent({
category: 'ticket_issuance',
eventType: 'ticket_issued',
userId: `off-hours-user-${i}`,
timestamp: offHours + i * 1_000,
});
}

const report = await analyticsAggregator.generateAnomalyReport();
const timing = report.findings.find(f => f.signal === 'timing_anomaly');
expect(timing).toBeDefined();
});

it('groups findings by severity and resource', async () => {
const now = Date.now();
for (let i = 0; i < 50; i++) {
analyticsAggregator.recordEvent({
category: 'ticket_redemption',
eventType: 'ticket_redeemed',
userId: `group-user-${i}`,
timestamp: now - i * 500,
});
}

const report = await analyticsAggregator.generateAnomalyReport();
expect(report.bySeverity).toBeDefined();
expect(report.byResource).toBeDefined();
for (const finding of report.findings) {
expect(report.bySeverity[finding.severity]).contains(finding);
expect(report.byResource[finding.resource]).contains(finding);
}
});

it('suppresses low-confidence noise from small samples', async () => {
// Only two events -- below minimum sample threshold
const now = Date.now();
for (let i = 0; i < 2; i++) {
analyticsAggregator.recordEvent({
category: 'ticket_redemption',
eventType: 'ticket_redeemed',
userId: `noise-user-${i}`,
timestamp: now - i * 1_000,
});
}

const report = await analyticsAggregator.generateAnomalyReport();
expect(report.findings.length).toBe(0);
});
});
});
9 changes: 8 additions & 1 deletion core/analytics/index.ts
Original file line number Diff line number Diff line change
@@ -1,2 +1,9 @@
export { analyticsAggregator } from './aggregator';
export { AnomalyDetector, anomalyDetector } from './anomaly';
export type {
AnomalyReport,
AnomalySignal,
AnomalySignalType,
AnomalySeverity,
AnomalyDetectionConfig,
} from './anomaly';
export * from './types';
144 changes: 144 additions & 0 deletions core/analytics/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,3 +60,147 @@ export interface MaintainerInsight {
data: Record<string, unknown>;
actionItems: string[];
}

/**
* Anomaly detection reporting for ticketing and event access operations.
* These types are purely descriptive and read-only: they describe what the detector
* found and how maintainers should triage it. They do not mutate any system state.
*/

export type AnomalySignalKind = 'volume' | 'timing' | 'actor_repetition' | 'failed_operations';

export type AnomalySeverity = 'info' | 'warning' | 'critical';

/**
* Resources that an anomaly can be attributed to. These map to Utix domain objects
* (events, tickets, organizers, actors) so maintainers can jump straight to the
* affected entity.
*/
export type AnomalyResourceType =
| 'event'
| 'ticket'
| 'organizer'
| 'actor'
| 'redemption_point'
| 'transfer_channel';

export interface AnomalyResource {
type: AnomalyResourceType;
// Stable identifier of the affected resource (e.g. event id, ticket id, hashed actor id).
id: string;
// Human-readable label for report display.
label?: string;
}

/**
* Evidence behind a single finding. Keep this numeric and reference-oriented so the
* report remains auditable and free of raw PII.
*/
export interface AnomalyEvidence {
// Metric that triggered the finding (e.g. 'tickets_issued').
metric: string;
// Observed value during the window.
observed: number;
// Baseline expected value for the same window/scope.
expected: number;
// Ratio of observed to expected (1.0 = on baseline).
ratio: number;
// Score in [0, 1] expressing how confident the detector is.
confidence: number;
// Optional sample of reference ids (event/ticket/actor ids) for drill-down.
sampleIds?: string[];
// Free-form details such as the detection window or threshold used.
details?: Record<string, number | string>;
}

export interface AnomalyFinding {
id: string;
// When the anomaly was observed.
timestamp: Date;
kind: AnomalySignalKind;
severity: AnomalySeverity;
// Short stable code for filtering/deduplication (e.g. 'volume_issue_spike').
code: string;
title: string;
description: string;
// The primary resource affected by this finding.
resource: AnomalyResource;
// Additional resources correlated with the finding (e.g. actor + event).
relatedResources: AnomalyResource[];
evidence: AnomalyEvidence;
// Ordered triage steps for maintainers.
triageSteps: string[];
// Explanation of why this may be a false positive and how to confirm.
falsePositiveNotes?: string[];
// Tags for grouping (e.g. 'spike', 'repeated_failure', 'single_actor').
tags: string[];
// Whether this finding was suppressed by false-positive mitigation rules.
suppressedByMitigation?: boolean;
}

export interface AnomalyReportSummary {
totalFindings: number;
bySeverity: Record<AnomalySeverity, number>;
byKind: Record<AnomalySignalKind, number>;
byResourceType: Record<AnomalyResourceType, number>;
}

/**
* Read-only anomaly report. Grouped by severity and affected resource so it can be
* rendered directly on organizer dashboards and in maintainer tooling.
*/
export interface AnomalyReport {
// Window covered by the report.
generatedAt: Date;
windowStart: Date;
windowEnd: Date;
summary: AnomalyReportSummary;
// Findings grouped by severity, each group sorted by confidence descending.
bySeverity: Record<AnomalySeverity, AnomalyFinding[]>;
// Findings grouped by affected resource type.
byResourceType: Record<AnomalyResourceType, AnomalyFinding[]>;
// Flat list of all findings for programmatic consumers.
findings: AnomalyFinding[];
}


// ---------------------------------------------------------------------------------
// Detection configuration
// ---------------------------------------------------------------------------------

/**
* Per-signal thresholds. Defaults are defined in the detector and are intentionally
* conservative to reduce false positives. Overrides are merged on top of defaults.
*/
export interface AnomalyThresholds {
// Multiple of baseline volume that triggers a volume finding.
volumeMultiple: number;
// Minimum absolute count before volume findings are considered (avoids noise on low traffic).
volumeMinimumCount: number;
// Multiple of baseline failure rate that triggers a failed-operations finding.
failureRateMultiple: number;
// Minimum number of failures in the window before a finding is raised.
failureMinimumCount: number;
// Number of repeated actions by a single actor that is considered suspicious.
actorRepetitionThreshold: number;
// Minimum number of distinct actors for a timing cluster to be meaningful.
timingClusterMinimumActors: number;
// Maximum number of events in a single minute that is considered normal for timing.
timingBurstThreshold: number;
// Minimum confidence [0, 1] for a finding to be included in the report.
minimumConfidence: number;
}

export interface AnomalyDetectionConfig {
// Window used to compute baselines and detect anomalies.
windowMs: number;
// How far back to look for baseline data.
baselineWindowMs: number;
// Minimum number of events required before a baseline is trusted.
minBaselineEvents: number;
thresholds: Partial<AnomalyThresholds>;
// Event types to ignore entirely (e.g. known background jobs).
ignoredEventTypes?: string[];
// Actor ids to ignore (e.g. internal automation accounts).
ignoredActorIds?: string[];
}
Loading