JarCastingVirtual File Framework (VFF)
A File System abstraction for scala inspired by FlySystem.
Currently two adapters are provided:
- Local (included with VFF)
- S3 (as of version 0.6.0 must be installed separately
"io.onema" % "vff-s3-adapter_2.12" % "VERSION")
Getting started
Install
for the latest version please visit maven.org
libraryDependencies += "io.onema" % "vff_2.12" % "LATEST_VERSION"
Create a file system object
import io.onema.vff.FileSystem
import io.onema.vff.adapter.{Local, AwsS3Adapter}
// Local file system
val fs = new FileSystem(new Local)
// S3 file system
val s3fs = new FileSystem(AwsS3Adapter("bucket-name"))
Reading a file
All read operations return an Option:
read
val data: Option[InputStream] = fs.read("path/to/file.txt")
val str: String = data.mkString
Other convenience methods are provided to read the data as a string or as an iterator of strings
readAsString
val strOption: Option[String] = fs.readAsString("path/to/file.txt")
readAsIterator
val iterOption: Option[Iterator[String]] = fs.readAsIterator("path/to/file.txt")
Creating a file
VFF uses byte iterators to write to the destination
write
import scala.io.Source
import io.onema.vff.extensions.StreamExtension._
// If you have an input stream you can convert it using the stream extensions
fs.read("path/to/image0.png").foreach((image: InputStream) => {
val result = fs.write("path/to/image1.png", image.toBytes)
val copyResult = fs.copy("path/to/image1.png","path/to/image1.png")
})
// A convenience method is included to write strings directly
fs.write("path/to/file2.txt", "Some text")
Extension Methods
InputStream
Using the InputStream methods and extensions provided in the you can quickly convert the data returned by the read method into other types
import io.onema.extensions.StreamExtension._
val data = fs.read("path/to/file.txt")
val str: String = data.mkString
val byteIter: Iterator[Byte] = data.toBytes
val byteArray: Array[Byte] = data.toByteArray
val lines: Iterator[String] = data.getLines
Async API
As of version 0.6.0 an Async file system is provided, the method names remain the same but all calls async and return a scala Future Future[ReturnType]. Implementations for each of the adapters are provided.
Example:
import java.util.UUID
import scala.concurrent.Await
import scala.concurrent.duration._
import io.onema.vff.FileSystemAsync
// Arrange
val fs = FileSystemAsync()
val uuid = UUID.randomUUID().toString
val dir = s"/tmp/$uuid"
// create a single file, this will not be part of the test
println(dir)
Await.result(fs.write(s"$dir/tmp", "0"), 500.millis)
// Act
val results: Seq[Boolean] = (0 to 100000).map { i =>
fs.write(s"$dir/$i", i.toString)
}.map(Await.result(_, 5000.millis))
// Assert
results.foreach(x => {
x should be (true)
})