api

Swagger as you Go

Gregory Golberg romana , , , ,

As someone who offers a REST API, we at Romana project wanted to provide documentation on it via Swagger. But writing out JSON files by hand seemed not just tedious (and thus error-prone), but also likely to result in an outdated documentation down the road.

Why not automate this process, sort of like godoc? Here I walk you through an initial attempt at doing that — a Go program that takes as input Go-based REST service code and outputs Swagger YAML files.

It is not yet available as a separate project, so I will go over the code in the main Romana code base.

NOTE: This approach assumes the services are based on the Romana-specific REST layer on top of Negroni and Gorilla MUX. But a similar approach can be taken without Romana-specific code — or Romana approach can be adopted.

The entry point of the doc tool for generating the Swagger docs is in doc.go.

The first step is to run Analyzer on the entire repository. The Analyzer:

  1. Walks through all directories and tries to import each one (using Import()). If import is unsuccessful, skip this step. If successful, it is a package.
  2. For each *.go and *.cgo file in the package, parse the file into its AST
  3. Using the above, compute docs and collect godocs for all methods

The next step is to run the Swaggerer — the Swagger YAML generator.

At the moment we have 6 REST services to generate documentation for. For now, I’ll explicitly name them in main code. This is the only hardcoded/non-introspectable part here.

From here we, for each service, Initialize Swaggerer and call its Process() method, which will, in turn, call the main workhorse, the getPaths() method, which will, for each Route:

  1. Get its Method and Pattern — e.g., “POST /addresses”
  2. Get the Godoc string of the Handler (from the Godocs we collected in the previous step)
  3. Figure out documentation for the body, if any. For that, we use reflection on the MakeMessage field — the mechanism we used for sort-of-strong-dynamic typing when consuming data, as seen in a prior installment on this topic.

This is pretty much it.

As an example, here is Swagger YAML for the Policy service.

See also

Building on Gorilla Mux and Negroni

Gregory Golberg romana , , , ,

The combination of Negroni and Gorilla MUX is a useful combination for buildng REST applications. However, there are some features I felt were necessary to be built on top. This has not been made into a separate project, and in fact I doubt that it needs to — the world doesn’t need more “frameworks” to add to the paralysis of choice. I think it would be better, instead, to go over some of these that may be of use, so I’ll do that here and in further blog entries.

This was borne out of some real cases at Romana; here I’ll show examples of some of these features in a real project.

Overview

At the core of it all is a Service interface, representing a REST service. It has an Initialize() method that would initialize a Negroni, add some middleware (we will see below) and set up Gorilla Mux Router using its Routes. Overall this part is a very thin layer on top of Negroni and Gorilla and can be easily seen from the above-linked source files. But there are some nice little features worth explaining in detail below.

In the below, we assume that the notions of Gorilla’s Routes and Handlers are understood.

Sort-of-strong-dynamic typing when consuming data


While we have to define our route handlers as taking interface{} as input, nonetheless, it would be nice if the handler received a struct it expects so it can cast it to the proper one and proceed, instead of each handler parsing the provided JSON payload.

To that end, we introduce a MakeMessage field in Route. As its godoc says, “This should return a POINTER to an instance which this route expects as an input”, but let’s illustrate what it means if it is confusing.

Let us consider a route handling an IP allocation request. The input it needs is an IPAMAddressRequest, and so we set its MakeMessage field to a function returning that, as in


func() interface{} {
    return &api.IPAMAddressRequest{}
}

Looks convoluted, right? But here is what happens:

The handlers of routes are not called directly, they are wrapped by wrapHandler() method, which will:

  1. Call the above func() to create the pointer to IPAMAddressRequest
  2. Use the unmarshaller (based on Content-Type) to unmarshal the body of the request into the above struct pointer.

Voila!

Rapid prototyping with hooks

Sometimes we prototype features outside of the Go-bases server — as we may be calling out to various CLI utilities (iptables, kubectl, etc), it is easier to first ensure the calls work as CLI or shell scripts, and iterate there. But for some demonstration/QA purposes we still would like to have this functionality available via a REST call to the main Romana service. Enter the hooks functionality.

A Route can specify a Hook field, which is a structure that defines:

  • Which executable to run
  • Whether to run it before or after the Route‘s Handler (When field)
  • Optionally, a field where the hook’s output will be written (which can then be examined by the Handler if the When field is “before”). If not specified, the output will just be logged.

Hooks are specified in the Romana config, and are set up during service initialization (this is in keeping with the idea that no Go code needs to be modified for this prototyping exercise).

Then during a request, they are executed by the wrapHandler() method (it has been described above).

That’s it! And it allows for doing some work outside the server to et it right, and only then bother about adding the functionality into the server’s code.

If this doesn’t seem like that much, wait for further installments. There are several more useful features to come. This just sets the stage.

Of course you have an API!

Gregory Golberg holding forth , , , , ,

The following is a dramatization of actual events.

“I need access to these reports.”

“Well, here they are, in the UI.”

“But I need programmatic access.”

“We don’t have an API yet.”

“Fine, I’ll scrape this… Wait… This is Flex. Wait, let me just run Charles… Flex is talking to the back-end using AMF. So what do you mean you don’t have an API? Of course you do — it is AMF. A little PyAMF script will do the trick.”

“Please don’t show it to anyone!”

P. S. That little script was still running (in “stealth production”) months, if not years, later.