
For more content like this, buy my in-progress eBook, Data Serialization in Go, and get updates immediately as they are added!
I’ve done a lot of JSON handling in Go. In the process, I’ve learned a number of tricks to solve specific problems. But one pattern in particular I find myself repeating ad infinitum. It’s evolved over the years, and now I want to share it, in hopes that others may benefit. Or perhaps you can tell me that I’m a moron for doing things this way, and then I can learn from your superior experience!
The problem
The general problem I aim to solve here, is a struct (or any data type, really), that implements a custom MarshalJSON
method, but that method needs to access the non-custom JSON representation of itself. (The same general problem exists for UnmarshalJSON
, too, but I’ll focus on just marshaling for demonstration purposes.)
But why?
Why would you ever want this? If you’ve not run into the problem, it may seem like insanity. If you have run into this problem, you’ve probably pulled out a little hair trying to find a good solution (I know I would have, if I had any hair to begin with…)
There are many reasons you may want to do this, but a few off the top of my (bald) head:
- You want to special-case some value (such as an empty struct) to a specific output (such as
null
) - You need to add to, or modify the output before returning it
- You need to unmarshal the data more than once (obviously only applies to the
UnmarshalJSON
case)
To demonstrate, let me use a simple example: That of rendering an empty struct as null
.
Default behavior
To demonstrate, let’s imagine a data type that contains a blog post ID, and a list of tags. Not all blog posts have tags, so sometimes this struct should marshal as null
.
type Metadata struct {
ID string `json:"id"`
Tags []string `json:"tags"`
}
If we marshal an empty instance of this with the default behavior, we get the following output:
{ "id": "", "tags": null }
A first attempt
To achieve our stated goal of producing null
for an empty value, we need a custom marshaler:
func (m Metadata) MarshalJSON() ([]byte, error) {
if m.ID == "" && len(m.Tags) == 0 {
return []byte("null"), nil
}
// TODO
return nil, nil
}
Now an empty instances of Metadata
marshals as we desire:
null
However, we still need to solve the general case of marshaling the non-empty case. The naïve approach is simply to add a call to json.Marshal
:
func (m Metadata) MarshalJSON() ([]byte, error) {
if m.ID == "" && len(m.Tags) == 0 {
return []byte("null"), nil
}
return json.Marshal(m)
}
If the problem with this approach isn’t obvious, it will be as soon as you run this code, and discover the infinite loop caused by MarshalJSON
(indirectly) calling itself.
Breaking the cycle
To break this infinite loop, we need to call json.Marshal
on a different type–one that doesn’t have the same MarshalJSON
method. This is easy enough to accomplish, by defining a copy of the type. We could declare the copy at the package level, but I like to keep my package-level symbols as tidy as possible, and since this type should only ever be used in the MarshalJSON
method, I like to define it within the method itself.
func (m Metadata) MarshalJSON() ([]byte, error) {
if m.ID == "" && len(m.Tags) == 0 {
return []byte("null"), nil
}
type metadataCopy struct {
ID string `json:"id"`
Tags []string `json:"tags"`
}
myCopy := metadataCopy{
ID: m.ID,
Tags: m.Tags,
}
return json.Marshal(myCopy)
}
This works nicely! But it’s very verbose. Especially for a struct with many fields. We can do better.
If our original type, and the local “copy” have the same fields and types, we can do a simple conversion, rather than assigning each field individually:
func (m Metadata) MarshalJSON() ([]byte, error) {
if m.ID == "" && len(m.Tags) == 0 {
return []byte("null"), nil
}
type metadataCopy struct {
ID string `json:"id"`
Tags []string `json:"tags"`
}
myCopy := metadataCopy(m)
return json.Marshal(myCopy)
}
More improvements
Now we’re getting somewhere. But we can still do better. There’s actually no need to list all of the fields in our definition of metadataCopy
, if we use Metadata
as the TypeSpec in our type declaration:
func (m Metadata) MarshalJSON() ([]byte, error) {
if m.ID == "" && len(m.Tags) == 0 {
return []byte("null"), nil
}
type metadataCopy Metadata
myCopy := metadataCopy(m)
return json.Marshal(myCopy)
}
This version is not only shorter, it’s also much more future-proof, as now any change made to the Metadata
type will automatically be supported by our MarshalJSON
method.
Note that this is not an alias declaration. A type alias would not serve our purpose, as it’s just a new name for an existing type, meaning the infinite loop would still be a problem.
A final revision
With one final edit to shorten things a bit, and add some test cases, we have a final version (See it on the Go Playground)
package main
import (
"encoding/json"
"fmt"
)
type Metadata struct {
ID string `json:"id"`
Tags []string `json:"tags"`
}
func (m Metadata) MarshalJSON() ([]byte, error) {
if m.ID == "" && len(m.Tags) == 0 {
return []byte("null"), nil
}
type metadataCopy Metadata
return json.Marshal(metadataCopy(m))
}
func main() {
empty := Metadata{}
emptyMarshaled, err := json.Marshal(empty)
if err != nil {
panic(err)
}
fmt.Println("empty:", string(emptyMarshaled))
populated := Metadata{ID: "abc", Tags: []string{"def", "hij"}}
populatedMarshaled, err := json.Marshal(populated)
if err != nil {
panic(err)
}
fmt.Println("populated:", string(populatedMarshaled))
}
The output produced is:
empty: null
populated: {"id":"abc","tags":["def","hij"]}
A json.Marshaler example
For the sake of completeness, let me briefly show how to do the reverse with a self-referencing json.Unmarshaler
implementation on the same type:
func (m *Metadata) UnmarshalJSON(p []byte) error {
if string(p) == "null" {
*m = Metadata{}
return nil
}
type metadataCopy Metadata
var result metadataCopy
err := json.Unmarshal(p, &result)
*m = Metadata(result)
return err
}
Conclusion
This basic pattern can be extended and adapted to a large variety of situations. I’ll be writing about more of these here, so be sure to subscribe to be notified of new posts.
Questions?
Let me know what challenges you face with JSON in Go in the comments section, and I’ll try to answer in an upcoming post.