Tuesday, October 14, 2014

Simple things are more powerful. The clive cmd/opt package.

The golang flag package is very powerful. It makes it easy to process your command arguments.
We have been using it for a long time to process Clive command arguments.

But, to say it in a few words, it is not able to operate on different argument vectors in a reasonable way (i.e., using the same interface used for processing the actual command line), it cannot handle combined -abc style flags instead of the utterly verbose -a -b -c, it cannot handle flags that repeat, and it is far more complex than needed. The clive opt package does all this by actually doing less.

For example, the flag package operates on os.Args to retrieve the arguments.  Using another array requires using another interface. Instead, opt takes always a []string and works on it. This makes it work on any given set of arguments, which makes it unnecessary to provide any functionality to process "sub-arguments" or "different command lines". Instead, the interface is used against the desired set of arguments.

Looking at the interface of flag provides more examples of what this post is about. There are multiple StringVar(), String(), IntVar(), Int(), ... functions to define new flags. There are many such functions and they define flags globally. Instead, opt provides a single method

    func (f *Flags) NewFlag(name, help string, vp interface{})

to define new flags. It can be used as in

        debug := false
        opts.NewFlag("d", "debug enable", &debug)
        odir := "/tmp"
        opts.NewFlag("o", "output dir, defaults to /tmp", &odir)

which has several advantages:

  1. The flag is defined on a set of flags, not globally. This permits using different sets of flags yet the code is the same that it would be to define global flags, as discussed above.
  2. There is a single way of defining a new flag. This is a benefit, not a drawback.
  3. There is a single method to call for any flag, not many to remember.
  4. It does not re-do things the language is doing. For example, a default value is simply the value of the variable unless the options change it. Go already has assignment and initialisation, thus, it is not necessary to supply that service in the flag definition. 
  5. It is not possible to introduce errors due to initialisation of option variables not matching the default value given to the flag definition call.
The implementation has also important differences. The flag package is an elaborate set of internal types defined to provide methods to handle each one of the flag types. Adding a new flag requires defining types and methods and must be done with care. A consequence is that Flag has 850 lines and is harder to understand. Opt has 356 lines. In both cases including all the documentation.

Instead of the approach used in flag to define and implement the arguments, opt relies on a type switch on the pointers given when defining each one of the flags. All the package does during parsing is to set the pointer to the value found in the argument, using the type of the pointer to determine how to parse the value by default. This is the an excerpt of the code:

switch vp := d.valp.(type) {
case *bool:
*vp = true
if len(argv[0]) == 0 {
argv = argv[1:]
} else { // put back the "-" for the next flag
argv[0] = "-" + argv[0]
case *string:
nargv, arg, err := optArg(argv)
if err != nil {
return nil, fmt.Errorf("option '%s': %s", d.name, err)
argv, *vp = nargv, arg
case *[]string:
nargv, arg, err := optArg(argv)
if err != nil {
return nil, fmt.Errorf("option '%s': %s", d.name, err)
argv, *vp = nargv, append(*vp, arg)

And that suffices. There is another type switch when a flag is defined to make sure that the pointer type can be handled later by the package during parsing. Because of this, the code walking the arguments trying to parse each option can be fully shared and is not highly coupled with the per-option parsing code.

What about more complex arguments? Easy: the program may define string arguments and then parse them as desired. Or, if the new argument type becomes very popular, it can be added to opt by

  1. Adding an empty case to the type switch of NewFlag (to check the pointer type for validity)
  2. Adding a new case to the type switch of Parse (the excerpt shown above) to actually do the parsing.

All this is not to say that opt is the greatest argument processing package. In fact, it is just born and it is likely that there are still bugs and things to improve. All this is to say that more can be done by doing less in the interface and in the implementation of the package considered.

Go is a very nice language. I'd like its interfaces to be as clean, tiny and powerful as possible. If we want complex interfaces and implementations, we know where to find Java and C++.