Feedback on evaluation methodology for VeloGraphX dynamic graph analytics

[sauravsingla]

Hi GraphScope community,

I’m working on VeloGraphX, an open-source C++20 research engine for exact analytics on dynamically changing graphs:

https://github.com/sauravsingla/VeloGraphX

VeloGraphX combines mutable graph storage, exact localized repair, and workload-aware selection between incremental execution and full recomputation.

Given GraphScope’s experience with large-scale graph analytics and graph systems, I would greatly value technical feedback on our evaluation methodology.

In particular:

• Which datasets and graph scales should a convincing systems evaluation include?
• Which dynamic/update workloads would you consider representative?
• What should be included in end-to-end latency measurements?
• Which scalability experiments would you expect?
• Which external systems or baselines would make the evaluation stronger?
• Are there any methodological choices that you would challenge as a reviewer?

Our objective is not to maximize headline speedups. We want the experiments to clearly identify the conditions under which exact incremental repair is beneficial and the conditions under which full recomputation remains preferable.

Critical feedback, including experiments that might expose weaknesses in VeloGraphX, would be particularly valuable.

Thank you.