Wednesday, April 17, 2013 at 9:25AM
Tachyon is a fault tolerant distributed ﬁle system enabling reliable file sharing at memory-speed across cluster frameworks, such as Spark and MapReduce.It offers up to 300 times higher throughput than HDFS, by leveraging lineage information and using memory aggressively. Tachyon caches working set files in memory, and enables different jobs/queries and frameworks to access cached files at memory speed. Thus, Tachyon avoids going to disk to load datasets that is frequently read.
It has a Java-like File API, native support for raw tables, a pluggable file system, and it works with Hadoop with no modifications.
It might work well for streaming media too as you wouldn't have to wait for the complete file to hit the disk before rendering.