Implement block-based cache in RAF?

[haileyajohnson]

So I am thinking of making an opt-in block-based cache in the base RandomAccessFile implementation for a Hacktoberfest contribution - I think it would help with repeat reads for record netcdf3 datasets, and I don't think it would negatively impact any other reads...

Am I overlooking anything important or any considerations I should be aware of? Am I heading down totally the wrong path??

[lesserwhirls]

Hey Hailey! That could be useful! Are you thinking remote access specifically, or local as well? We have something similar in RemoteRandomAccessFile. The idea behind that one was that it was supposed to help reduce remote (HTTP, S3) requests when trying to figure out the appropriate IOSP to use, so somewhat small in size by design and a short time-to-live in the cache to clear the memory quick (although both configurable).

[haileyajohnson]

Oh interesting.. I was actually thinking of the base class. Since we mount an EFS file system on our EC2 instances, I'm pretty sure it is treated as local. Maybe I could extend/generalize what's in Remote to the base class?

[lesserwhirls]

Reads from files on EFS will be going through RandomAccessFile for sure, so that's where you'd want this cache. A cache will help reduce your IOPS, which can only be a good thing! I think a lot of what is in the RemoteRandomAccessFile class with regards to the loading cache could be lifted up to RandomAccessFile, as it's already designed to be a cache of buffers for RandomAccessFile's use of buffered reads. The big difference is that we use a much larger buffer size (256 KiB) for remote files than for local (8 KiB).

Perhaps the tricky part will be in how to make the cache in RandomAccessFile configurable (with the default being disabled), while having RemoteRandomAccess and subclasses default on, without double caching the data when both are enabled. Might not be too terrible, just not super clear off the top of my head.

[haileyajohnson]

I have a rough implementation set up, but two things that still need working out, in regards to the above are:

• picking sane defaults for buffer size/cache size/ttl when a RemoteRandomAccessFile also wants a block cache
• whether buffer size and cache item size should be different to accommodate a sane default
• or if everything should just be documented with suggested config for performance and we let the user figure it out for themselves

So let me know if you have thoughts!

[lesserwhirls]

Those are great questions! The goal of the current block cache was to minimize the number of repeat remote requests for the same blocks when netCDF-Java was trying to figure out which IOSP to use. In that sense, a modest read buffer size (bigger than RandomAccessFile) and block cache size (enough to hold what is needed for most isValidFile IOSP checks) with a short-ish ttl was sufficient. Users could crank those up for better overall performance against remote files as needed (depending on access patterns), but the default size was easier to judge based on the goal of minimizing the impact of the IOSP selection process. For local files, the goal is different.

For your case, the selection sort of depends on the number of items in/size of each record, so it's a bit different and way harder to judge what would be helpful in terms of buffer size/cache size/ttl. I'm wondering if using a similar cache (tech-wise) but at a slightly higher level, directly in the netcdf-3 IOSP as a "record cache", would be a better option than a block cache in RandomAccessFile, or at least easier to reason about from an end-user point of view? I hate to suggest that after you have a rough implementation up and running as a the block cache in RandomAccessFile.

[haileyajohnson]

Honestly yeah , that's a better idea. I think I was trying to do something that was not purely selfish for our case and would generally help out users, but yes I think I lean toward your suggestion for a more foolproof guard against this specific problem. Don't worry about what I've done, it's just generic caching! Also the more I think about it the better it is, it can be on by default for record datasets and there's way less piping into the API... 💯

[haileyajohnson]

Edit: my take after an hour of looking at it is that a cache at the iosp level is the most robust answer for us specifically, BUT a raf level cache with block-sized items would also probably solve the cloud I/O cost problem, regardless of record size, as long as the end user could set the cache size high enough. The only benefit to that is it might be more versatile. But at the same time, netcdf3 is the only format that really has this problem reading record.

TL;DR - I want whatever I do to be helpful and low maintenance, so whichever YOU think is better is what I'm gonna do.

[lesserwhirls]

BUT a raf level cache with block-sized items would also probably solve the cloud I/O cost problem, regardless of record size, as long as the end user could set the cache size high enough.

Does the existing buffer cache in RemoteRandomAccessFile (and subclasses) help address the cloud cost issue in this way (for example, by adjusting the S3RandomAccessFile properties)? Although a TTL of 30 seconds might be too short and need to be configurable as well.