This post is an excerpt from my in-progress book, Data Serialization in Go, available on LeanPub.
Back in 2016, when I was still fairly new to Go, I asked a question on StackOverflow about how to properly marshal a struct which embeds a struct with a custom MarshalJSON method. I got a few answers that helped point me in the right direction, but to this day I never received a completely satisfactory answer, that allows extending the existing MarshalJSON method, without duplicating it.
Now with approximately four more years of experience under my belt, I’m back to try to answer my own question.
Background
To begin, let me lay out the problem to be solved.
Imagine we’re using a data type with its own MarshalJSON method. I’ll use a contrived example to illustrate:
// File represents an arbitrary file’s contents.
type File struct {
Filename string
ContentType string
Content []byte
}
// MarshalJSON satisfies the json.Marshaler interface, and outputs
// the file’s name, content, type, content, and MD5 sum.
func (f File) MarshalJSON() ([]byte, error) {
h := md5.New()
h.Write(f.Content)
md5sum := hex.EncodeToString(h.Sum(nil))
return json.Marshal(map[string]interface{}{
"filename": f.Filename,
"content_type": f.ContentType,
"content": f.Content,
"md5sum": md5sum,
})
}
In this example, we rely on the custom MarshalJSON method to calculate the MD5 sum of the file’s contents. Given the following input:
f := &File{
Filename: "test.txt",
ContentType: "text/plain",
Content: []byte{"This is a test"},
}
We would naturally expect the following output (with some added whitespace for readability, of course):
{
"content": "VGhpcyBpcyBhIHRlc3Q=",
"content_type": "text/plain",
"filename": "test.txt",
"md5sum": "ce114e4501d2f4e2dcea3e17b546f339"
}
The problem
The problem arises when we embed this type in another, with the goal of adding fields to the struct:
// Image is a special case of File, and includes dimension
// metadata.
type Image struct {
File
Height int `json:"height"`
Width int `json:"width"`
}
If the File type were defined with standard JSON tags and no MarshalJSON method, this would work very nicely with no additional changes. But since the MarshalJSON method on the embedded type is promoted, any attempt to marshal this type as-is will ignore our new fields:
i := &Image{
File: File{
Filename: "test.jpg",
ContentType: "image/jpeg",
Content: []byte("not really an image"),
},
Height: 640,
Width: 480,
}
data, _ := json.MarshalIndent(i, "", " ")
fmt.Println(string(data))
This will output the following (note the conspicuous absence of the height and width fields):
{
"content": "bm90IHJlYWxseSBhbiBpbWFnZQ==",
"content_type": "image/jpeg",
"filename": "test.jpg",
"md5sum": "b15301000bc458c348a12fc66e5ede74"
}
Wrapping the custom marshaler
The solution to this problem is, of course, to create our own MarshalJSON method on our Image type. But we don’t want to duplicate the logic in File.MarshalJSON, which could lead to bugs if the logic ever changes.
func (i Image) MarshalJSON() ([]byte, error) {
fileJSON, err := i.File.MarshalJSON() // Step 1
if err != nil {
return nil, err
}
type img struct { // Step 2
Height int `json:"height"`
Width int `json:"width"`
}
imageJSON, err := json.Marshal(img{ // Step 3
Height: i.Height,
Width: i.Width,
})
if err != nil {
return nil, err
}
imageJSON[0] = ','
return append(fileJSON[:len(fileJSON)-1], imageJSON...), nil
}
Let’s examine the details of this example closely:
First, we call the embedded MarshalJSON method explicitly (Step 1), storing the result in fileJSON. This avoids duplicating the logic (which may not even be possible to duplicate, if the embedded type contains unexported fields).
Next we define a method-local type img (Step 2), which contains only the unique fields of the exported Image type. Unfortunately, this tight coupling between Image and img is necessary, so make sure that any fields added to Image are also added to img, or they’ll be excluded when marshaling JSON.
Then we marshal the unique Image fields into a separate variable, imageJSON (Step 3).
And finally, we replace the first character of imageJSON (which should be an opening curly brace ({)), with a comma, in preparation to join our two JSON strings. Then as a final step, we combine all but the last byte of fileJSON (which is a closing curly brace (})) with the contents of imageJSON, to produce a single JSON object, and return it.
If we now re-attempt the marshal example from above, we should get the correct output:
{
"content": "bm90IHJlYWxseSBhbiBpbWFnZQ==",
"content_type": "image/jpeg",
"filename": "test.jpg",
"md5sum": "b15301000bc458c348a12fc66e5ede74",
"height": 640,
"width": 480
}
That’s it!
I’ve used this approach several times in Kivik and elsewhere. I hope you find it useful, as well.
The complete code
import (
"crypto/md5"
"encoding/hex"
"encoding/json"
)
// File represents an arbitrary file's contents.
type File struct {
Filename string
ContentType string
Content []byte
}
// MarshalJSON satisfies the json.Marshaler interface, and outputs
// the file's name, content, type, content, and MD5 sum.
func (f File) MarshalJSON() ([]byte, error) {
h := md5.New()
h.Write(f.Content)
md5sum := hex.EncodeToString(h.Sum(nil))
return json.Marshal(map[string]interface{}{
"filename": f.Filename,
"content_type": f.ContentType,
"content": f.Content,
"md5sum": md5sum,
})
}
// Image is a special case of File, and includes dimension
// metadata.
type Image struct {
File
Height int `json:"height"`
Width int `json:"width"`
}
// MarshalJSON satisfies the json.Marshaler interface by appending
// i.Height and i.Width to the output of i.File.MarshalJSON.
func (i Image) MarshalJSON() ([]byte, error) {
fileJSON, err := i.File.MarshalJSON()
if err != nil {
return nil, err
}
type img struct {
Height int `json:"height"`
Width int `json:"width"`
}
imageJSON, err := json.Marshal(img{
Height: i.Height,
Width: i.Width,
})
if err != nil {
return nil, err
}
imageJSON[0] = ','
return append(fileJSON[:len(fileJSON)-1], imageJSON...), nil
}
