We'll publish it soon.
This is the documentation for the Go binding of ix plus a few examples.
gr := ns.GetReader("/foo/dockingframescore.pdf")
of, err := os.Create("/tmp/file")
if err != nil {
t.Fatal(err)
}
for {
b := make([]byte, 8192)
n, err := gr.Read(b)
if n > 0 {
of.Write(b[0:n])
}
if err != nil || n == 0 {
break
}
}
of.Close()
And as a result of running this...
2013/02/07 21:00:34 -> Tfid[7] fid 1
2013/02/07 21:00:34 -> Tclone[7] newfid 7
2013/02/07 21:00:34 -> Tclunkon[7] err
2013/02/07 21:00:34 -> Twalk[7] name 'dockingframescore.pdf'
2013/02/07 21:00:34 -> Topen[7] r
2013/02/07 21:00:34 -> Tread[7] off 0 count 18446744073709551615
2013/02/07 21:00:34 -> Tend[7]
2013/02/07 21:00:34 <- Rok[7]
2013/02/07 21:00:34 <- Rok[7]
2013/02/07 21:00:34 <- Rok[7]
2013/02/07 21:00:34 <- Rok[7]
2013/02/07 21:00:34 <- Rok[7]
2013/02/07 21:00:34 <- Rread[7] off 0 data[8192]
2013/02/07 21:00:34 <- Rread[7] off 8192 data[8192]
2013/02/07 21:00:34 <- Rread[7] off 16384 data[8192]
2013/02/07 21:00:34 <- Rread[7] off 24576 data[8192]
2013/02/07 21:00:34 <- Rread[7] off 32768 data[8192]
2013/02/07 21:00:34 <- Rread[7] off 40960 data[8192]
2013/02/07 21:00:34 <- Rread[7] off 49152 data[8192]
2013/02/07 21:00:34 <- Rread[7] off 57344 data[8192]
And the stream goes on and on.
This is an example of how to use the protocol directly from Go:
// Read a directory
r = c.NewRpc()
r.Wc <- &Call{Kind: Tfid, Fid: 1}
r.Wc <- &Call{Kind: Tclone, Newfid: 2}
r.Wc <- &Call{Kind: Tclunkon, When: End}
r.Wc <- &Call{Kind: Twalk, Name: "foo"}
r.Wc <- &Call{Kind: Topen, Mode: Read}
r.Wc <- &Call{Kind: Tread, Off: 0, Count: ^uint64(0)}
r.Wc <- nil
b := make([]byte, 0)
for rep := range r.Rc {
fmt.Printf("got rep %s\n", rep)
if rep.Kind == Rread {
b = append(b, rep.Data...)
}
}
r.Close()
Now the promised go doc. There are two packages, ix (the raw protocol interface) and ns (a canned interface more convenient for client code).
package ix
import "/Users/nemo/gosrc/src/lsub.org/ix"
IX protocol messages.
This implements pack, unpack, and printing of IX messages.
Authentication and encryption is left out of the protocol. The
underlying transport must be secured by caller before using the
protocol.
The underlying transport is assumed to be a reliable stream (e.g. TCP).
Messages are exchanged by sending the message length (4 bytes in big
endian) and then the message data. The resulting transport thus
preserves message boundaries.
All integers are sent in big endian format, using either 4 bytes or 8
bytes. These are noted as [4] and [8] in what follows. All strings use
UTF-8. They are sent by sending the string size[4] in bytes and then so
many bytes in UTF-8. A string on the wire is noted str[s]. Messages that
carry raw data send such data as the last field, without including the
field length in the message. The total message size indicates how many
data bytes are included. Such data fields are noted data[].
Messages sent on the wire have the format:
length[4] type[4] tag[4] ...
Here, type identifies the message type.
Requests belong to a group (also known as a channel), which is
identified by a tag. A group is a series of requests with the same tag,
and finishes with a Tend request. It is ok to send messages for
different groups at the same time (one message at a time, of course).
A server processes requests for a group one at a time, in order.
Requests in a group are terminated as soon as one fails or the final
Tend request is reached. Clients are expected to send the Tend request
even if the group has already failed, for simplicity.
Each group operates on a fid that is implicit for the group. The first
request usually initializes the implicit fid. Requests like Tclone,
which allocates a new fid, make that fid implicit for the rest of the
group. We use ``implicit file'' to refer to the file pointed to by the
implicit fid.
All requests have replies, which carry the tag found in the request. Two
generic replies are used almost everywhere: Rok indicates that the
request was performed fine; Rerror indicates that it failed and conveys
the error message. Most requests use these as their replies; some
require specific reply messages.
These are the messages in the protocol. The length, type, and tag are
not shown; instead, a symbolic name for each message type is shown.
tattach fid[4] afid[4] uname[s] tname[s] msize[4]
rattach msize[4] afid[4]
Attaches the fid to the tree with name tname for the user with name
uname. The fid becomes implicit for the rest of the group. All data[]
fields in the conversation will be no larger than msize bytes. Auth is
left out of the protocol, thus it's up the the client and the server to
authenticate in a way that makes the server trust the client on behalf
of uname, and vice-versa. Afid is kept as a provision to add auth fids
similar to those of 9p.
Negotiation of the protocol version is also left out of the protocol.
twalk name[s]
walks the implicit fid to the given name. Name is a single path element,
not a full path. Servers are free to refuse walks to ".." and ".".
tclone newfid[4]
allocates a new fid and makes it implicit for the rest of the group.
tclunkon end|err
makes Tend and/or Rerror get rid of the implicit fid. Otherwise, the fid
remains allocated after the entire group is processed. Clunking on end
implies clunking also on errors.
topen mode[1]
opes the implicit fid for I/O according to mode.
Directories can only be open for reading.
tcreate type[4] perm[4] name[s]
creates a file (or directory, depending on type) named "name" under the
implicit file, with the given permissions. Note that the implicit fid
still refers to the directory after this request, and not to the new
file. Also, the new file is not open. This request only creates a file,
nothing else. The values for type should be the character used in UNIX
to list the file type, by convention.
tremove
removes implicit file.
trattr name[s]
rrattr name[s] data[]
Reads an attribute of the implicit file. All attributes have UTF-8 names
and have data[] as values. Values are usually encoded as UTF-8 strings,
which is useful for other requests shown later.
The special name "?" may be used as an attribute name to obtain the list
of attribute names for the implicit file.
The special name "*" may be used to obtain one reply per existing
attribute, followed by one reply with empty name and data.
By convention, the following attributes are always available:
id: file id or path
name: file name
length: length in bytes for files; number of children for directories.
mode: file mode; conventional rwxrwxrwx bits are expected at least.
uid: user id
mtime: last modified, seconds since epoch
type: file type: "d" for directories, "-" for regular files
twattr name[s] data[]
writes a new value for the named attribute.
tmove tofid[4] name[s]
moves the implicit file to the directory implied by tofid[4], using
name[s] as the new name for the file in the destination directory.
tfid fid[4]
sets the implicit fid.
tread off[8] count[8]
rread off[8] data[]
asks for replies conveying up to count[8] bytes from the implicit file
(starting at the indicated offset). Here, count may be larger that the
maximum data[] size negotiated. The server will reply with one or more
replies until either count bytes are returned or EOF is reached in the
file (signaled as an reply with 9 bytes). Note that this is equivalent
to readn, not to read.
To issue a single read request to the server, count should be 0, which
means that up to the negotiated msize bytes might be sent.
The result from reading a directory is a series of names, encoded as
name[s], for each of the files contained in the directory by the time of
the open.
treplace off0[8] off1[8] data[]
rreplace off[8] count[4]
replaces the range off0:off1 in the implicit file with new data.
tcond op[1] name[s] data[]
applies the condition op (LT,LE,EQ,GE,GT,NE) to the named attribute
using the supplied data as the second operand. If the condition holds,
Rok is replied; Rerror otherwise. If the attribute value is not a valid
UTF-8 string, the request fails. Otherwise, it is compared as a string
unless both strings are valid as printed integers, in which case they
are compared as integer values.
tforall rec[1]
rforall data[] // one sent per file, carrying the file id. then one with data[0]
This request applies the rest of the requests in the group to each one
of the files contained in the implicit file (which must be a directory).
If rec is non-zero, the request recurres to the entire file tree rooted
at the implicit file. If rec is >1, then the requests are applied using
tail recursion.
This is useful to remove entire trees, to retrieve entire trees, etc.
See the examples.
Before applying the rest of the group to each one of the files, an
Rforall is sent back, carrying the value of the "id" attribute for the
file. Then, the rest of the group is applied to a temporary implicit fid
that is set to point to the file (in each case). If the group fails,
processing continues on the next file.
tend
rend
terminates the group.
rok
is a generic reply to requests that do not need specific replies.
rerror err[s]
is a generic reply for a request that failed.
tflush
flushes the group. It is Ok to send a flush after Tend. Flush requests
are always replied with Rerror containing "flushed", as are flushed
requests.
Example, remove a tree:
Tfid 13, set the fid 13
Tforall 2
Tremove
Tend
Example, retrieve everything about a file:
Tfid 13
Tclone
Tclunkon end
Twalk a
Twalk b
Trattr *
Topen OREAD
Tread 0 ~0 8192
Tend
Example, retrieve an entire tree:
Tfid 13
Tclone
Tclunkon end
Twalk a
Twalk b
Tforall 1
Trattr *
Topen OREAD
Tread 0 ~0 8192
Tend
Example, retrieve all metadata for a tree:
Tfid 13
Tclone
Tclunkon end
Twalk a
Twalk b
Tforall 1
Trattr *
Tend
Example, retrieve all metadata and data for files newer than 1234
Tfid 13
Tclone
Tclunkon end
Twalk a
Twalk b
Tforall 1
Tcond GT mtime 1234
Trattr *
Topen OREAD
Tread 0 ~0 8192
Tend
CONSTANTS
const (
None = CallKind(iota)
Tattach
Twalk
Tclone
Tclunkon
Topen
Tcreate
Tremove
Trattr
Twattr
Tmove
Tfid
Tread
Tcond
Tforall
Tflush
Tend
Rattach
Rrattr
Rread
Treplace
Rreplace
Rok
Rforall
Rerror
Rend
)
Message types.
const (
Read = 1
Write = 2
Trunc = 4
)
Known modes for Call.Mode.
const (
End = 0
Err = 1
)
Known flags for Tclunkon when
const (
Tdir = uint8('d')
Tfile = uint8('-')
)
Known file types for Tcreate.Ftype
const (
LT = iota
LE
EQ
GE
GT
NE
)
Known ops for Tcond.Op.
const (
Dontrec = iota
Topdown
Bottomup
)
Known flag values for Tforall.Rec.
VARIABLES
var (
// PEM and Key files used.
DefaultPem = "certs/client.pem"
DefaultKey = "certs/client.key"
)
FUNCTIONS
func AddWalks(calls []*Call, path string) []*Call
Add enough Twalk requests to calls to walk to the given path.
func DirEnts(b []byte) ([]string, error)
Convert data read from a directory read into directory entries
func NetDialTLS(network, addr string, cfg *tls.Config) (io.ReadWriteCloser, error)
Dial a server using tls and return a rwc for it. You can create a config
given paths to pem and key files using MakeTLSCfg(pem, key). If cfg is
nil, default files with pem and key are used.
func NextElem(path string) (string, string)
Return the first path element in path and the rest of it.
func ReadMsg(r io.Reader) ([]byte, error)
Read a message from the given reader, allocating a buffer to hold the
message bytes. Use ReadMsgBuf if you want to use your own buffer.
func ReadMsgBuf(r io.Reader, buf []byte) ([]byte, error)
Read a message from the given reader using the given buffer. The buffer
should have enough capacity for the largest expected message.
func ReadStringFrom(r io.Reader) (string, error)
Read a string from r using the IX string encoding.
func WriteMsg(w io.Writer, buf []byte) error
Write a packed message to the given writer.
func WriteStringTo(w io.Writer, s string) error
Write a string into w using the IX string encoding.
TYPES
type Call struct {
Kind CallKind // all
Tag uint32 // all
Fid uint32 // Tattach, Tfid
Afid uint32 // Tattach, Rattach
Uname string // Tattach
Tname string // Tattach
Msize uint32 // Tattach, Rattach
Name string // Twalk, Tcreate, Trattr, Twattr, Tmove, Tcond, Rrattr
Newfid uint32 // Tfid
Mode uint8 // Topen
Ftype uint8 // Tcreate
Perm uint32 // Tcreate
Data []byte // Twattr, Tcond, Rrattr, Rread, Treplace, Rforall
Tofid uint32 // Tmove
Off uint64 // Tread, Rread, Rreplace
Count uint64 // Tread, Rreplace (uint32 on the wire for Rreplace)
Op uint8 // Tcond
Rec uint8 // Tforall
Off0 uint64 // Treplace
Off1 uint64 // Treplace
Err string // Rerror
When uint8 // Tclunkon
// nothing: Tremove, Tflush, Tend, Rok, Rend
}
Unpacked IX message operation (or reply).
func Unpack(buf []byte) (*Call, error)
Unpack a message from buf. The buffer must contain exactly one message.
func (m *Call) Pack(buf []byte) ([]byte, error)
Pack the message in the given buffer. The packed message does not
include the message length. That should be written prior to writing the
buffer, eg. using WriteMsg.
func (m *Call) PackedSize() (int, error)
Return the size of the packed message by writing it into a memory buffer
and returning its length.
func (m *Call) String() string
Return a string representation for the message, for debugging.
func (m *Call) WriteTo(w io.Writer) (int64, error)
Write the message to the given writer by packing the message into a
newly allocated buffer and writing the message length and then the
message bytes to w. Use Pack and WriteMsg if you want to provide your
own buffer.
type CallKind uint32
Message kind type.
func (k CallKind) ReplyKind() CallKind
Return the expected reply type (besides Rerror) or Tnone if there is no
reply.
func (k CallKind) String() string
Return the name for the message type.
type Cli struct {
sync.Mutex
// contains filtered or unexported fields
}
A client connection to an IX server. There should be only one per open
connection. This is useful to keep track of fid and tag allocation and
to allocate Rpcs to be sent to a server.
func Dial(network, addr string) (*Cli, error)
Dial a server and return a client for it
func DialTLS(network, addr string, cfg *tls.Config) (*Cli, error)
Dial a server using tls and return a client for it. You can create a
config given paths to pem and key files using MakeTLSCfg(pem, key). If
cfg is nil, default files with pem and key are used.
func NewCli(rw io.ReadWriteCloser) *Cli
Build a new Cli for the given connection.
func (c *Cli) Close()
Hangup the client and wait until hung up.
func (c *Cli) NewFid() uint32
Get a new unique fid number.
func (c *Cli) NewRpc() *Rpc
Return an ongoing call to an IX server, made out of different
operations.
func (c *Cli) Rpc(ts []*Call) ([]*Call, error)
Convenience method to issue a series of requests in a new RPC and return
the replies.
type Rpc struct {
// Channel to write requests within this RPC.
// Send nil and close to terminate the RPC.
Wc chan<- *Call
// Channel to read replies from.
Rc <-chan *Call
// contains filtered or unexported fields
}
An ongoing rpc to an IX server.
func (r *Rpc) Close()
Terminate an RPC request by sending nil through r.Wc and closing it, and
then wait for it to actually terminate.
func (r *Rpc) Discard(n int) (int, error)
Receive and discard up to n calls, indicating any error received and how
many did we receive up to the error, if any.
func (r *Rpc) Flush()
Flush this RPC.
func (r *Rpc) Send(calls []*Call)
Send the given requests terminating the Rpc after doing that.
SUBDIRECTORIES
certs
ns
package ns
import "/Users/nemo/gosrc/src/lsub.org/ix/ns"
A name space implemented out of connections to IX servers
CONSTANTS
const (
Read = ix.Read
Write = ix.Write
Trunc = ix.Trunc
)
modes for Open
const (
// Maximum message (data) size
Msize = 8 * 1024
)
FUNCTIONS
func Attr2Int(b []byte) int64
Return the int value for the given attribute value.
func Int2Attr(i int64) []byte
Return the attribute value for the given int value.
TYPES
type Dir map[string][]byte
A directory entry: named attribute values.
func NewDir() Dir
Make a new, empty, directory entry.
func (d Dir) Gid() string
func (d Dir) Id() string
func (d Dir) Length() int64
func (d Dir) Mode() int
func (d Dir) Mtime() int64
func (d Dir) Name() string
func (d Dir) SetGid(s string)
func (d Dir) SetId(id string)
func (d Dir) SetLength(l int64)
func (d Dir) SetMode(m int)
func (d Dir) SetMtime(mt int64)
func (d Dir) SetName(n string)
func (d Dir) SetType(s string)
func (d Dir) SetUid(s string)
func (d Dir) Type() string
func (d Dir) Uid() string
type File struct {
// contains filtered or unexported fields
}
Represents a file in use.
func (f *File) Close() error
Close a file in use.
func (f *File) Get() (<-chan Dir, <-chan []byte, <-chan error)
Get everything about this file, its directory entry and all data. The
file must be open for reading. Caller must receive from the Dir channel
to retrieve the Dir, and from the []byte channel until a null slice is
received. Upon reception of a null slice, the error channel should be
checked out to discover any error (none if nil).
func (f *File) GetData(off, count uint64) (<-chan []byte, <-chan error)
Stream data from the file. Caller must receive from the channel of
[]byte to get the data until the total count is received or a null slice
is received. Upon reception of a null slice, the error channel should be
checked out to discover any error. The file must be open for reading.
func (f *File) GetReader() *GetReader
Return a reader to read the entire file.
func (f *File) Pread(count uint32, off uint64) ([]byte, error)
Pread a file, issuing a single IX read request.
func (f *File) Pwrite(data []byte, off uint64) (uint32, uint64, error)
Pwrite a file. May issue multiple IX write requests.
func (f *File) Rattr(name string) ([]byte, error)
Read the value of an attribute.
func (f *File) Read(buf []byte) (int, error)
func (f *File) ReadDir() ([]Dir, error)
Read all directory entries of files contained in a directory, open for
reading. Should not be used on files that are not directories.
func (f *File) ReadDirNames() ([]string, error)
Read all names of files contained in a directory (open for reading).
Should not be used on files that are not directories.
func (f *File) Remove() error
Remove this file.
func (f *File) Stat() (Dir, error)
Stat a file and return its Dir entry.
func (f *File) String() string
Return a printable version of File, for debugging.
func (f *File) Wattr(name string, value []byte) error
Write the value of an attribute.
func (f *File) Write(data []byte) (int, error)
func (f *File) Wstat(m Dir) ([]string, error)
Wstat a file. The names returned correspond to attributes not udpated
due to errors.
type GetReader struct {
// contains filtered or unexported fields
}
To be used as a io.Reader with Ns.GetData dn Ns.Get.
func NewGetReader(c <-chan []byte, ec <-chan error) *GetReader
Return an io.Reader retrieve data resulting from Get or GetData calls.
func (r *GetReader) Close() error
Close the reader for the Get of the file. Beware that this would still
receive all the data streamed in the background. The implementation
should flush the request instead.
func (r *GetReader) Read(data []byte) (int, error)
Read data retrieved and report any error condition.
type Ns struct {
sync.RWMutex // don't use this; implementation only.
// contains filtered or unexported fields
}
Represents a name space attaching to IX servers, implemented by a prefix
mount table.
func NewNs(uname string) *Ns
Make a new name space for the given user name.
func (ns *Ns) Chdir(p string) error
Change the current working directory to this named file.
func (ns *Ns) Close()
Dismantle the Ns, closing all connections held. Each client connection
is closed once, even if it's shared among multiple mount point entries.
func (ns *Ns) Create(p string, perm uint32) (*File, error)
Create a new file in the name space and return a fid for it, open in RW
mode.
func (ns *Ns) CreateDir(p string, perm uint32) error
Create a new directory in the name space.
func (ns *Ns) Cwd() string
Return the absolute path of the current working directory.
func (ns *Ns) Get(file string) (<-chan Dir, <-chan []byte, <-chan error)
Get everything about this file, its directory entry and all data. Caller
must receive from the Dir channel to retrieve the Dir, and from the
[]byte channel until a null slice is received. Upon reception of a null
slice, the error channel should be checked out to discover any error
(none if nil).
func (ns *Ns) GetData(file string, off, count uint64) (<-chan []byte, <-chan error)
Stream data from the file. Caller must receive from the channel of
[]byte to get the data until the total count is received or a null slice
is received. Upon reception of a null slice, the error channel should be
checked out to discover any error (none if nil).
func (ns *Ns) GetReader(path string) *GetReader
Return a reader to read the entire file.
func (ns *Ns) GetTree(file string, tcond *ix.Call) (<-chan Dir, <-chan []byte, <-chan error)
Retrieve an entire subtree, including metadata. If filter is not null,
then it's placed as a Tcond filter such that only files matching are
retrieved, otherwise everything is retrieved. The caller should wait for
a dir to be posted on the Dir channel, then read the file data until
eof, receive another Dir, etc.
func (ns *Ns) List() []string
Return the set of mount points.
func (ns *Ns) Mount(fd io.ReadWriteCloser, spec string, mnt string) error
Mount an ix server reached through fd into ns. Spec indicates the tree
and path to mount and mnt is the mount point. The first path element in
spec identifies the tree name. Mounting on the current working directory
updates it so it sees the mount. Mounting a suffix of the cwd is always
seen, because it's a prefix mount table. Mounting on a prefix of the cwd
has no effect (behaving as if an entry for dot was always present).
func (ns *Ns) NewPutWriter(file string, d Dir) *PutWriter
do a ns.Put() for the given arguments, returning an io.WriteCloser to
stream data writes to the new file.
func (ns *Ns) Open(path string, mode uint8) (*File, error)
Open a file in the name space.
func (ns *Ns) Put(file string, d Dir, data <-chan []byte) <-chan error
Create a file, update it's attributes to those in Dir (if given), and
its data to be that received from the data channel. The data ends when
nil is sent through the data channel. The error channel is sent the
final error condition once the data channel has been closed and the
streaming process completes. It is also posted an error condition if an
error reply is noticed while streaming data, in which case all further
data updates are discarded. Caller might check the error channel
non-blocking during streaming to see if it should stop sending data due
to errors.
func (ns *Ns) Remove(p string) error
Remove a file in the name space.
func (ns *Ns) Stat(path string) (Dir, error)
Stat a file and return its Dir entry.
func (ns *Ns) Unmount(mnt string) error
Remove an entry in the mount table. The prefix must be exactly the given
path. If dot is unmounted, it is left as it was. You could ChdDir(Cwd())
to resolve dot again in the ns after the unmount has taken effect.
func (ns *Ns) Wstat(path string, m Dir) ([]string, error)
Wstat a file. The names returned correspond to attributes not udpated
due to errors.
type PutWriter struct {
// contains filtered or unexported fields
}
To be used as a io.WriteCloser with Ns.PutData and Ns.Put.
func (w *PutWriter) Close() error
func (w *PutWriter) Write(data []byte) (int, error)
The count returned does not actually indicate that the bytes were
written, because of streaming.
