Skip to content

feat(FDSE-3324): add directory and usage modular inputs - #48

Open
Arthur-10-coder wants to merge 3 commits into
mainfrom
FDSE-3324-directory-inputs-and-usage
Open

Arthur-10-coder wants to merge 3 commits into
mainfrom
FDSE-3324-directory-inputs-and-usage

Conversation

@Arthur-10-coder

@Arthur-10-coder Arthur-10-coder commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds 5 new modular inputs to the Genesys Cloud TA: users_directory, queues_directory, usage_events, org_usage, and oauth_client_usage
  • Directory inputs provide daily full-snapshot ingestion for lookup-table correlation (user/queue ID → human-readable name)
  • Usage inputs replace the deprecated POST /api/v2/usage/query and POST /api/v2/oauth/clients/{clientId}/usage/query endpoints (removal deadline: Oct 16, 2026)

Context

Jira: https://splunk.atlassian.net/browse/FDSE-3324

Changes

File Description
globalConfig.json +5 input definitions (UCC framework) with 86400s default interval
package/bin/users_directory_helper.py GET /api/v2/users → sourcetype genesyscloud:directory:users
package/bin/queues_directory_helper.py GET /api/v2/routing/queues → sourcetype genesyscloud:directory:queues
package/bin/usage_events_helper.py POST /api/v2/usage/events/query → sourcetype genesyscloud:usage:events
package/bin/org_usage_helper.py POST /api/v2/usage/aggregates/query/jobs → sourcetype genesyscloud:usage:organization
package/bin/oauth_client_usage_helper.py POST /api/v2/usage/client/{id}/aggregates/query/jobs → sourcetype genesyscloud:usage:oauth_clients

Technical Notes

  • All inputs follow existing helper pattern (UCC modular input + genesyscloud_client.py)
  • Directory inputs: no checkpointing needed (full snapshot each run)
  • Usage inputs: KVStore checkpointing for incremental ingestion; async job polling pattern for aggregate endpoints
  • OAuth scopes required: users:readonly, routing:readonly, usage:readonly
  • SDK v238 required for new endpoint support

Follow-up (separate PR)

  • Custom Search Commands (| genesysusers, | genesysqueues) will be added to genesys_cloud_app_for_splunk — these live in the App, not the TA, per architectural decision

Implements the official solution approved by Ashley Hoang and Erica Pescio:
- users_directory: GET /api/v2/users snapshot (genesyscloud:directory:users)
- queues_directory: GET /api/v2/routing/queues snapshot (genesyscloud:directory:queues)
- usage_events: POST /api/v2/usage/events/query (genesyscloud:usage:events)
- org_usage: POST /api/v2/usage/aggregates/query/jobs (genesyscloud:usage:organization)
- oauth_client_usage: POST /api/v2/usage/client/{id}/aggregates/query/jobs (genesyscloud:usage:oauth_clients)

The usage inputs replace the deprecated POST /api/v2/usage/query endpoints
scheduled for removal on Oct 16, 2026. Directory inputs use daily snapshots
(86400s interval) to provide lookup-compatible ID-to-name correlation data.

Ref: https://splunk.atlassian.net/browse/FDSE-3324
@Arthur-10-coder
Arthur-10-coder force-pushed the FDSE-3324-directory-inputs-and-usage branch 2 times, most recently from d3b7680 to 8974d2b Compare August 11, 2026 21:22

@edro15 edro15 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Arthur-10-coder Please have a look at these changes and comments.

Comment thread docs/ConfigureOperationalInputs/index.md Outdated
- [Configure an integration application in Genesys Cloud for the Genesys Cloud Add-on for Splunk](../ConfigureGenesysCloud/index.md)
- [Configure an account in the Genesys Cloud Add-on for Splunk](../ConfigureAccount/index.md)

**OAuth scopes required (for usage inputs):** `usage:readonly`

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**OAuth scopes required (for usage inputs):** `usage:readonly`

Please move this information to the table at docs/ConfigureGenesysCloud/index.md. ALL necessary permissions for ALL inputs MUST be added to that table following given format

Comment thread docs/ConfigureOperationalInputs/index.md Outdated
Comment on lines +105 to +111
## Usage Inputs Reference

| Input | API Endpoint | Sourcetype |
|-------|-------------|------------|
| `operational_events` | `POST /api/v2/usage/events/query` | `genesyscloud:operational:events` |
| `org_api_usage` | `POST /api/v2/usage/aggregates/query/jobs` | `genesyscloud:operational:usage:api` |
| `clients_api_usage` | `POST /api/v2/usage/client/{id}/aggregates/query/jobs` | `genesyscloud:operational:usage:clients_api` |

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Usage Inputs Reference
| Input | API Endpoint | Sourcetype |
|-------|-------------|------------|
| `operational_events` | `POST /api/v2/usage/events/query` | `genesyscloud:operational:events` |
| `org_api_usage` | `POST /api/v2/usage/aggregates/query/jobs` | `genesyscloud:operational:usage:api` |
| `clients_api_usage` | `POST /api/v2/usage/client/{id}/aggregates/query/jobs` | `genesyscloud:operational:usage:clients_api` |

This is not aligned with other inputs documentation. To be removed.

Comment thread docs/Sourcetypes/index.md Outdated
Comment thread globalConfig.json
Comment thread globalConfig.json Outdated
Comment thread globalConfig.json Outdated
Comment thread globalConfig.json
Comment thread globalConfig.json Outdated
- Fix usage helpers to use valid SDK v238 UsageApi methods/models
  (post_usage_query, post_usage_aggregates_query_jobs,
  post_usage_client_client_id_aggregates_query_jobs and their results)
- Pass request body as 'body' kwarg so path params (client_id) work
- Make max_poll_attempts/poll_interval_seconds configurable per input
- Rename sourcetypes/titles to operational:* convention
- Clean input titles/descriptions, add poll fields, group in operational menu
- Consolidate OAuth scopes and document new inputs
- Add integration/functional tests and mock routes for the new inputs
@Arthur-10-coder
Arthur-10-coder force-pushed the FDSE-3324-directory-inputs-and-usage branch from 7ae144b to c9f4372 Compare September 8, 2026 16:57
@Arthur-10-coder

Copy link
Copy Markdown
Contributor Author

Update: review feedback applied + CI root cause fixed

Pushed c9f4372 addressing the review. Summary of what changed:

CI failure root cause (the 3 failing Deploy and test the TA jobs).
The UsageApi method/model names used by the helpers did not exist in the pinned SDK PureCloudPlatformClientV2==238.0.0, so the client's dynamic getattr returned None and assert response is not None failed. Corrected to the real v238 symbols:

  • events: post_usage_query (ApiUsageOrganizationQuery) + poll get_usage_query_execution_id_results
  • org: post_usage_aggregates_query_jobs (OrganizationPublicApiUsageQueryRequest) + get_usage_aggregates_query_job
  • client: post_usage_client_client_id_aggregates_query_jobs (ClientPublicApiUsageQueryRequest) + get_usage_client_client_id_aggregates_query_job
  • granularity fixed from "P1D" (invalid, coerced to outdated_sdk_version) to "Day".
  • genesyscloud_client.post now passes the payload as the body= kwarg so path params like client_id are honored.

Verified locally against Mockoon + SDK 238: integration suite 26/26 passing (previously 3 failed / 20 passed).

Review comments addressed.

  • Sourcetypes renamed to operational:* (operational:events, operational:usage:api, operational:usage:clients_api) across helpers, docs and props.conf.
  • Input titles/descriptions cleaned (no endpoint/scope/deprecation) with the requested titles.
  • max_poll_attempts / poll_interval_seconds are now user-configurable (like audit_query), added to globalConfig.json and docs; removed hard-coded constants.
  • Readability: inverted if job_response and compact to_dict() forms; continue on proxy errors.
  • Permissions consolidated in ConfigureGenesysCloud; version kept at 0.4.1.
  • Tests consolidated into tests/integration/test_genesyscloud_client.py and tests/modinput_functional/test_genesyscloud_ta.py with mock routes for the new endpoints.

Note on scope: this revision keeps the users_directory and queues_directory inputs and consolidates the tests (rather than the separate test_operational_usage*.py files). Happy to adjust if you prefer dropping the directory inputs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants