Repository navigation
Quota charges attached volumes twice since 4.22.1 (cumulative + per-VM usage_volume records from #11531) #14550
Description
problem
Since #11531 (4.22.1), Usage intentionally writes two kinds of usage_volume helper rows for an attached volume: a cumulative row (vm_id NULL) and a per-VM row (vm_id set). Both generate an hourly usage_type = 6 (VOLUME) record in cloud_usage, one with and one without vm_instance_id.
The Quota plugin treats both records as billable, so every attached volume is charged twice.
versions
- CloudStack 4.22.1.1 (management, usage)
- KVM, Ceph RBD primary storage
- Quota enabled with a volume tariff (
usage_type = 6)
steps to reproduce
- Enable Quota and configure a tariff for usage type 6.
- Create a data volume and attach it to a running VM (events
VOLUME.CREATE, thenVOLUME.ATTACH). - After the next Usage run, check
cloud_usage.usage_volume. There are two rows withdeleted IS NULL, one withvm_idNULL and one withvm_idset to the VM. - Check
cloud_usage.cloud_usagefor that volume and hour. There are twousage_type = 6records with the sameraw_usage: one withvm_instance_idNULL and one withvm_instance_idset. - Check
cloud_usage.quota_usage. There is one row percloud_usagerecord, both with the samequota_used, so the account balance is debited twice.
Example, one volume and one hour:
| cloud_usage.vm_instance_id | records | quota_used |
|---|---|---|
| NULL (cumulative) | 1 | 0.0081 |
| set (per-VM) | 1 | 0.0081 |
expected
A volume should be charged once per hour. Quota should bill only one of the two record kinds, for example the cumulative record (vm_instance_id NULL), since that is the one that is consistent with pre-#11531 behaviour.
notes
- Today, root volumes on new VMs don't show the double charge, because the second
usage_volumeinsert fails on the old(volume_id, created)unique key (usage: widen usage_volume unique key to include vm_id #13909 / backport: usage: widen usage_volume unique key to include vm_id (#13909) #13933). Once that key is widened in 4.22.2, root volumes will get both rows too, and will also be double-charged by Quota. - Our workaround: an hourly job sets
deleted = createdon per-VMusage_volumerows whenever an active cumulative row exists for the same volume. Per-VM records then stop being generated for that volume, and the cumulative row keeps billing it through detach and re-attach. Records written before we found this had to be removed by hand.
Activity
boring-cyborg commentedon Nov 1, 2026
Thanks for opening your first issue here! Be sure to follow the issue template!
github-actions commentedon Nov 2, 2026
Warning
threat detection engine error
The threat detection engine encountered an error and could not complete analysis. This is a tooling failure, not a security finding.
What happened
The threat detection engine failed to produce results.
Review the workflow run logs for details.
🎯 Triage report
Since #11531 (4.22.1), Usage writes two
usage_volumehelper rows for an attached volume: a cumulative row (vm_idNULL) and a per-VM row. Each generates an hourlyusage_type = 6record, and Quota bills both, so attached volumes are charged twice. The reporter describes a workaround that setsdeletedon per-VM rows, and notes that the double charge will also reach root volumes once the unique key is widened in 4.22.2 (#13909 / #13933).📊 Assessment
🔗 Similar issues
usage_volume, referenced in the report as the reason root volumes are not yet affectedvm_instance_idis null for volume usage (usage_type = 6)💡 Notes and suggestions
cloud_usageandquota_usagerows will need a cleanup or credit decision; the reporter removed theirs by hand.Add this agentic workflow to your repo
To install this agentic workflow, run
Daniel-Pereira-Linux commentedon Nov 5, 2026
Hi! I'd like to work on this issue.
My initial idea is to make Quota bill only the cumulative record
(vm_instance_id NULL), which matches the behaviour before #11531.
Before I start, I'd like to confirm with you:
@DaanHoogland
cloud_usage and quota_usage by affected versions?
I can open a PR with the fix and tests once we agree on the approach.
Thanks!
winterhazel commentedon Nov 5, 2026
@MacroHarry @Daniel-Pereira-Linux this was already addressed in 4.23 via #13936, which makes Quota ignore the per-VM row. We can just backport the change to branch 4.22. Please feel free to open the pull request. I can review once I get some time.
There should not be affected cloud_usage and quota_usage records in any environment yet, as Usage was unable to generate these records before the changes in #13933 unless operators manually changed the database. I would just ignore them.
DaanHoogland commentedon Nov 5, 2026
feel free to backport as @winterhazel describes, @Daniel-Pereira-Linux . I would rather you add on top of his fixes in main if needed, but that is no rule by any means.
Daniel-Pereira-Linux commentedon Nov 5, 2026
Thanks for the green light @DaanHoogland. I'll do the backport and send it as a separate PR, to keep things organized.