portable-files: local and remote file IO, on the JVM and JS, for "src/"s and "test/"s

Java’s (N)IO interfaces are insoluble in idiomatic Scala, so I wrote an alternative in portable Scala.

It reads local files, HTTP URLs, and cloud-store objects, bakes in block-LRU-caching, captures effects with the tagless final pattern, and more.

I’ll demo it and discuss designs and alternatives!

Java NIO is a standard way to read and write remote files using familiar local-IO interfaces, but it leaves a lot of type-safety and ergonomics on the table: - various URI schemes are supported via brittle SPI classloader tricks - the API surface is large, and not applicable to many file stores; UnsupportedOperationExceptions ensue - excessive mutability is assumed, e.g. channels typically have both read and write capabilities, but only use one - etc.

After years of frustration, I made an attempt at a pure, portable, Scala NIO replacement. I use it in server- and web-apps, and it lets my portable libraries’ tests assert against “expected” values serialized to local files, removing an awkward asymmetry between JVM- and JS-based tests. It solves many points I had, that also seem close to but not covered by http4s or fs2.

I’ll show a range of prior art I studied and drew inspiration from (buffering, caching, ahead-looking Java InputStreams, fs and request Node libraries, etc.), and discuss the pros and cons of bringing heavy Scala abstractions (cats-effect!) to bear on the classic task of reading and writing files.

As a newcomer to the FP-Scala world, I think I can make this accessible and interesting to a wide audience. Some of the use cases and demo possibilities (e.g. cloud-store IO) should also make it feel practical. Thanks!