Parquet cache memory management and back-pressure

[hiltontj]

Problem statement

The parquet cache is limited in its ability to prevent an influx of cache requests from causing overuse of system memory.

Proposed solution(s)

• Require file size in the CacheRequest so that the cache oracle can decide if it needs to prune before fetching a file, or if the file is too big for the cache entirely
• Provide an EvictionRequest to explicitly evict entries from the cache that are no longer needed

[hamzzy]

can i work on this @hiltontj

[hiltontj]

@hamzzy - we appreciate your offer to contribute. There are some downstream changes that depend on this work and so we are currently hashing out the details of those. That may shape the requirements for this work, so for now, I would hold off working on this. If you keep an eye on our issue tracker for issues with the good-first-issue label, those would be better candidates to pick up if you are looking to contribute. If I end up refining this or breaking it down into separate issues, they may be good as first issues.