When I review Go code at work, I write the same comments again and again. “Please wrap the error with our helper, not fmt.Errorf.” “You must call Init on this client before you use it.” These are not bugs. The code compiles and the tests pass. They are only our own rules.
Tools like go vet, staticcheck, and golangci-lint are very good. But they check common rules that fit any Go project. They do not know the rules that only my team has. So I keep checking these by hand in review. It is boring, and sometimes I miss them. When the number of rules grows, my attention spreads out, and I cannot focus on the more important comments.
So I made a small tool called goast for this. This post explains why I made it, and how it works.
Why I did not use the usual tools #
Go already has a way to write your own checks. You can write a go/analysis pass, or a custom plugin for golangci-lint. I did not choose them, for a simple reason. Both need real Go code and a build step.
For go/analysis, I write Go code that walks the AST for each rule. For a golangci-lint custom plugin, I have to build golangci-lint with my plugin inside it. This is fine when a check is big and shared by many teams. But my rules are small, and I only have a few of them. Writing Go code and keeping a build for three or four small rules felt too heavy.
So for a serious and reusable linter, I still use these tools. goast is for the lighter layer.
The idea: split the linter in two #
goast cuts the linter into two parts.
One part is the engine. It parses the Go file and gives each AST node to the rule. This part does not change. The other part is the rules. I write the rules as data, in Rego (the policy language of OPA). Not as Go code, and not as a build.
So the person who makes the tool and the person who writes the rules can be different. The rule is data, not code. This is the main point of goast.
How it works #
For every node in the AST, goast evaluates the rule fail in the Rego package goast. The node is passed as input. If fail returns something, it is a violation.
Each input looks like this:
{
"Path": "sample.go",
"FileName": "sample.go",
"DirName": ".",
"Kind": "CallExpr",
"Node": { }
}Kind is the node type name, like "CallExpr" or "FuncDecl". Node is the AST node as JSON, with the same field names as Go. The rule returns an object with three fields: msg, pos, and sev. pos is a byte offset in the file, and goast turns it into a line and column for me.
I do not guess the JSON shape. goast has a dump command that prints it. I look at the dump, and then I write the rule against it.
Example 1: forbid a function call #
The simple case is to ban one call. Say I want to forbid fmt.Println. First I write a small file:
package main
import "fmt"
func main() {
fmt.Println("hello")
}Then I dump the line to see its shape:
$ goast dump --line 6 sample.go | jq
The output is long, so here is the part I need (trimmed):
{
"Kind": "ExprStmt",
"Node": {
"X": {
"Fun": {
"X": { "Name": "fmt", "NamePos": 44 },
"Sel": { "Name": "Println", "NamePos": 48 }
}
}
}
}The node is an ExprStmt, and the call sits under .X. X.Fun.X.Name is "fmt", and X.Fun.Sel.Name is "Println". Now I write the rule:
package goast
fail contains res if {
input.Kind == "ExprStmt"
input.Node.X.Fun.X.Name == "fmt"
input.Node.X.Fun.Sel.Name == "Println"
res := {
"msg": "use the logging package, not fmt.Println",
"pos": input.Node.X.Fun.X.NamePos,
"sev": "ERROR",
}
}Then I run it:
$ goast eval -p no_println.rego sample.go
[sample.go:6] - use the logging package, not fmt.Println
fmt.Println("hello")
~~~~~~~~~~~~~~~~~~~~
Detected 1 violations
For CI, goast eval --fail returns a non-zero exit when it finds something, and -f json prints the reviewdog format.
Example 2: call Init before Run #
The next rule is harder. I want to say: you must call sdk.Init() before sdk.Run(). This is a rule about order.
goast looks at one node at a time, so it cannot really follow the program and check if Init ran before. But I can get close. I anchor the rule on the FuncDecl node. A function declaration carries its whole body as input.Node.Body.List, and this is an ordered list. So inside one function, I can check if an Init call comes before a Run call, by comparing their byte positions.
package goast
# Flag sdk.Run() when no sdk.Init() call comes before it in the same function.
fail contains res if {
input.Kind == "FuncDecl"
input.Node.Body != null
some stmt in input.Node.Body.List
stmt.X.Fun.X.Name == "sdk"
stmt.X.Fun.Sel.Name == "Run"
run_pos := stmt.X.Fun.X.NamePos
not init_before(input.Node.Body.List, run_pos)
res := {
"msg": "sdk.Init() must be called before sdk.Run()",
"pos": run_pos,
"sev": "ERROR",
}
}
init_before(list, run_pos) if {
some stmt in list
stmt.X.Fun.X.Name == "sdk"
stmt.X.Fun.Sel.Name == "Init"
stmt.X.Fun.X.NamePos < run_pos
}The rule finds a sdk.Run() call in the body, and remembers its position. The helper init_before checks if any sdk.Init() call has a smaller position. If there is no such Init, it is a violation.
A function without a body (an external function) has Body equal to null, so I check that first.
Here is a bad file with only Run:
package main
import "example.com/sdk"
func handler() {
sdk.Run()
}$ goast eval -p init_before_run.rego bad.go
[bad.go:6] - sdk.Init() must be called before sdk.Run()
sdk.Run()
~~~~~~~~~
Detected 1 violations
If I add sdk.Init() before sdk.Run(), the file is clean (Detected 0 violations). And if I put Init after Run, it is a violation again, because I compare the positions.
But this is only a shape, not the real run. For example, if I move Init inside an if:
func handler(ready bool) {
if ready {
sdk.Init()
}
sdk.Run()
}goast still reports a violation, even though Init is there. My rule reads the top-level statements of the function only. It does not go inside the if block. The other side is also missed: if I put Run inside an if, the rule does not check it at all. And even at the top level, it cannot know if a branch really runs. So this is not real scope analysis.
One thing is safe. Because the rule works on one FuncDecl at a time, Init and Run are always in the same function. An Init in another function does not count. But inside that function, I only see the flat top-level list. So I keep this rule for simple cases, and I know it is not perfect. I say more about this limit below.
Testing the rules so they do not rot #
A rule can rot. Someone changes the code, or the AST shape changes a little, and the rule stops matching. Nobody notices, because a rule that never fires and a rule that is broken look the same.
goast has a test command for this. I write sample files and the expected result in .goast.toml:
[test]
policy = [".goast"]
[[test.cases]]
name = "forbids fmt.Println"
source = [".goast/testdata/bad.go"]
expect = "fail"
count = 1
[[test.cases]]
name = "allows fmt.Fprintf"
source = [".goast/testdata/good.go"]
expect = "pass"Then I run it:
$ goast test
ok forbids fmt.Println (1 violation)
ok allows fmt.Fprintf (0 violations)
2 cases: 2 passed, 0 failed
It exits non-zero if any case is wrong, so I run it in CI. Now the rule and its samples stay together, and I notice when a rule breaks.
What it cannot do #
goast matches one node at a time. Each node is checked alone. A rule cannot see other nodes, other files, or type information. The false positive in Example 2 is a small case of this. Looking at the whole FuncDecl still shows me only the shape of the code, not the real path of the run.
So goast is not good for rules that need types, or that follow a value across nodes or functions. Rules like “how was this variable changed” or “did we really call Init on this path” are out of reach.
It is good for rules about syntax and shape: banned calls, banned imports, function signatures, file-level rules, and simple order inside one function like Example 2. I wrote more about this trade-off, in Japanese, here.
Closing #
goast is a small tool. It cannot do type-based or data-flow rules, and I do not try to make it do them. But for the small house rules that I used to repeat in every review, it works well enough, and I can write them as data instead of Go code.
The code is here: github.com/m-mizutani/goast.