Friday, January 27, 2012

Should file systems use the disk...?

... as their main storage?

We are in the process of creating a new file server program to Nix. The main assumption underlying nix is that the hardware has changed so much that we should revisit all "well known" facts about system design.

File servers are not an exception. The test machine in use for Nix at lsub has 64 Gbytes of memory. With that much memory, the machine can easily host in memory the entire file tree we have in our main file server. However, our standard file server program insists on using the memory as a cache of the disk, considering the disk as the "master" copy of the file tree.

Instead, I'm advocating for file servers that keep the main file tree at memory, using the disk mostly to survive during machine halts and power outages, and to keep those files that are big enough that cannot be held easily on RAM.

More on the IX file server in the near future.

2 comments:

  1. Are remote file systems also a variable to take into account? That is, can part of a file system be mounted in RAM in another node? Would it be handled in a different way?

    ReplyDelete
    Replies
    1. Remote trees are going to me mounted using the file protocol, so you don't care where
      the servers keep the files as long as they speak the protocol properly.

      Delete