Filed by @ygoldfeld pre-open-source:
(Various services) can output logs directly in gzip form. (gzip? And/or other compression formats too?) Async_file_logger, at least, should support the same.
The naive thing, which is to output one gzip stream, should be easy with Boost.stream which has a gzip filter (EDIT: but consider other possibilities). The ostream writer in flow.log could be easily adapted to use it.
(colleague) also tells me that (some services) don't output one gzip stream (or maybe it is optional). (They) output little gzip streams (one per message? one per certain # of lines or messages? check), each one valid and then followed by some sentinel or something (check it). The idea is that if the file is corrupted, the non-corrupted parts can still be decoded fine.
So do that too.
All details TBD.
But the general feature is clearly a good one.
Filed by @ygoldfeld pre-open-source:
(Various services) can output logs directly in gzip form. (gzip? And/or other compression formats too?) Async_file_logger, at least, should support the same.
The naive thing, which is to output one gzip stream, should be easy with Boost.stream which has a gzip filter (EDIT: but consider other possibilities). The ostream writer in flow.log could be easily adapted to use it.
(colleague) also tells me that (some services) don't output one gzip stream (or maybe it is optional). (They) output little gzip streams (one per message? one per certain # of lines or messages? check), each one valid and then followed by some sentinel or something (check it). The idea is that if the file is corrupted, the non-corrupted parts can still be decoded fine.
So do that too.
All details TBD.
But the general feature is clearly a good one.