Innovating with Protogen: A Guide to Custom Protoc Plugin Development

mkdir struct-tag-plugin
cd struct-tag-plugin && mkdir cmd/protoc-gen-structtag
go mod init github.com/theluckiestsoul/protoc-gen-structtag
go get -u google.golang.org/protobuf 

Let’s write the code that will generate structs with JSON and ORM(gorm/bun) tags.

Create a file main.go and put the following code.

package main

import (
    "errors"
    "flag"
    "google.golang.org/protobuf/compiler/protogen"
)

var dbtag * string

func main() {
    var flags flag.FlagSet
    dbtag = flags.String("dbtag", "", "db tag to use for generated structs")
    options: = protogen.Options {
        ParamFunc: flags.Set,
    }
    options.Run(func(gen * protogen.Plugin) error {
        if *dbtag == "" {
            gen.Error(errors.New("dbtag must be set"))
        }
        for _, f: = range gen.Files {
            if !f.Generate {
                continue
            }
            generateStructTags(gen, f)

        }
        return nil
    })
}

func generateStructTags(gen * protogen.Plugin, file * protogen.File) * protogen.GeneratedFile {
    filename: = file.GeneratedFilenamePrefix + "_structtags.pb.go"
    g: = gen.NewGeneratedFile(filename, file.GoImportPath)
    g.P("// Code generated by protoc-gen-go-structtags. DO NOT EDIT.")
    g.P()
    g.P("package ", file.GoPackageName)
    g.P()
    for _,
    message: = range file.Messages {
        g.P("type ", message.GoIdent, " struct {")
        for _, field: = range message.Fields {
            g.P(field.GoName, " ", field.Desc.Kind(), " `json:\"", field.Desc.Name(), "\" ", * dbtag, ":\"", field.Desc.Name(), "\"`")
        }
        g.P("}")
    }

        return g
}

A Deep Dive into Custom Code Generation

The above code snippet is related to how `protogen`, a part of the Go protocol buffer compiler (`protoc`), generates code from `.proto` files. Let’s break down the snippet to understand its functionality:

Setting Up Options:

options := protogen.Options{
    ParamFunc: flags.Set,
}

Here, an instance of `protogen.Options` is created. `ParamFunc` is set to `flags.Set`, which is a function that will be used to parse command-line parameters. This setup is crucial for handling any flags or additional parameters passed to the plugin.

Running the Plugin:

options.Run(func(gen *protogen.Plugin) error {
// ...
})

The `Run` method of `options` is called with a function that takes a `*protogen.Plugin` as an argument. This function is where the main logic of the plugin resides. The `*protogen.Plugin` object (`gen`) represents the entire run of the plugin and contains information about all the files to be generated, among other things.

Checking for `dbtag`:

if *dbtag == "" {
    gen.Error(errors.New("dbtag must be set"))
}

Inside the function, there’s a check to ensure that the `dbtag` option is set. If `dbtag` is not provided, the plugin reports an error using `gen.Error`. The `dbtag` is likely a custom flag used to specify database tags for struct fields in the generated code, which is essential for ORM  (Object-Relational Mapping) compatibility.

Iterating Over Files:

for _, f := range gen.Files {
   if !f.Generate {
      continue
   }
   generateStructTags(gen, f)
}

The plugin iterates over all the files (`gen.Files`) that are part of the code generation process. For files that require generation, it calls `generateStructTags(gen, f)`, for adding struct tags to the generated Go structs, based on the `.proto` file definitions and the `dbtag` option.

Now, create the following folders:

• proto
• internal

Create a proto file with the following content and put the file in the proto folder.

syntax = "proto3";
package entity;

option go_package = "github.com/theluckiestsoul/protoc-gen-structtag/pkg/proto";

message HelloRequest{
   string name = 1;
}

message HelloResponse{
   string message = 1;
}

message User{
   int32 id = 1;
   string name = 2;
   string email = 3; 
   string password = 4;
   string phone = 5;
   string address = 6;
   string city = 7;
   string state = 8;
   string country = 9;
   string zip = 10;
   string createdAt = 11;
   string updatedAt = 12;
   string deletedAt = 13;
   string lastLogin = 14;
   string lastLoginIp = 15;
   string lastLoginLocation = 16;
}

Then execute the following commands which will install the plugin locally.

cd cmd/protoc-gen-structtag && go install && cd ..

If there is no error message, then you can check the bin folder of your $GOPATH directory for a binary named protoc-gen-structtag.

Once we are done with the installation, we can run the below command to execute the plugin to generate Go structs for each message present in the above proto file with JSON and ORM(bun/gorm) tags.

protoc --structtag_out=internal --structtag_opt=paths=source_relative,dbtag=bun proto/*.proto

The command operates as follows:

1.  `protoc` first parses and validates all the `.proto` files provided to it.
2. It then searches for any arguments formatted as `<plugin>_out`. Flags matching this format are recognized as plugins. `protoc` looks for a plugin named `protoc-gen-<plugin>` within the `$GOPATH` and executes it. In our case, the plugin is named `protoc-gen-structtag`.
3. After that, `protoc` checks for `<plugin>_opt` arguments, known as options. These options allow passing multiple arguments to the plugin. In our scenario, the options include `path`, an inbuilt option that informs the compiler about the file’s path relative to the output folder. Additionally, we have a custom option named `dbtag`, where we specify the ORM for which we want to generate tags (e.g., `dbtag=bun` for Bun ORM tags). If there are more options, they can be passed, separated by commas.
4. Finally, protoc creates a code generation request and passes it to the plugin protogen via stdin which will return the code generation response via stdout.

The above command would produce a file with the following content in the internal folder.

// Code generated by protoc-gen-go-structtags. DO NOT EDIT.

package proto

type HelloRequest struct {
   Name string `json:"name" bun:"name"`
}
type HelloResponse struct {
   Message string `json:"message" bun:"message"`
}
type User struct {
   Id int32 `json:"id" bun:"id"`
   Name string `json:"name" bun:"name"`
   Email string `json:"email" bun:"email"`
   Password string `json:"password" bun:"password"`
   Phone string `json:"phone" bun:"phone"`
   Address string `json:"address" bun:"address"`
   City string `json:"city" bun:"city"`
   State string `json:"state" bun:"state"`
   Country string `json:"country" bun:"country"`
   Zip string `json:"zip" bun:"zip"`
   CreatedAt string `json:"createdAt" bun:"createdAt"`
   UpdatedAt string `json:"updatedAt" bun:"updatedAt"`
   DeletedAt string `json:"deletedAt" bun:"deletedAt"`
   LastLogin string `json:"lastLogin" bun:"lastLogin"`
   LastLoginIp string `json:"lastLoginIp" bun:"lastLoginIp"`
   LastLoginLocation string `json:"lastLoginLocation" bun:"lastLoginLocation"`
}

As demonstrated, we’ve successfully automated the generation of structs with bun ORM tags. This solution can be tailored to meet a variety of other specific requirements, offering customizable versatility to fit our needs.

Conclusion

In conclusion, the `protoc-gen-structtag` plugin effectively showcases the versatility of the `protogen` library in Go for custom code generation. By utilizing `protogen`, this plugin efficiently automates the addition of struct tags to Protocol Buffer generated code, thereby enhancing the functionality of Go structs for JSON serialization and database operations. This not only streamlines the development process but also highlights the power of `protogen` in extending Protocol Buffers’ capabilities, making it a valuable asset in the toolkit of any Go developer working with Protocol Buffers.