Parsing Common Data Formats With Go

3 minute read
, , , , ,

Intro

Hello, 🌎! There are many popular data formats used for many different purposes such as transferring data between microservices, storing configuration files, storing and transferring metadata, etc. In this blog, I’ll provide a few quick examples demonstrating how you’d go about parsing some of the more common data formats like JSON, TOML, YAML, XML, and CSV, using Go. Note: I created this post mostly for reference. With that said, let’s parse some data!

Prerequisites

  1. Need Go >= v1.11 installed
  2. You know how to use Go modules. Makes working with Go easier

Parsing Data Formats with Go

Parsing JSON

For parsing JSON-formatted data, we can use Go’s standard library package encoding/json.

Here is an example parsing a JSON-formatted string with Go:

package main

import (
	"encoding/json"
	"fmt"
)

type UserData struct {
	Profile struct {
		FirstName string   `json:"first_name"`
		LastName  string   `json:"last_name"`
		Age       int      `json:"age"`
		Sex       string   `json:"sex"`
		Hobbies   []string `json:"hobbies"`
	} `json:"profile"`
}

var data string = `
{
	"profile": {
		"first_name": "John",
		"last_name": "Doe",
		"age": 45,
		"sex": "male",
		"hobbies": [
			"reading",
			"programming",
			"watching movies"
		]
	} 
}
`

func main() {
	var user UserData
	err := json.Unmarshal([]byte(data), &user)
	if err != nil {
		fmt.Println(err.Error())
	}

	fmt.Printf("%+v\n", user)
}

Parsing TOML

For parsing TOML-formatted data, we will use BurntSushi’s toml Go package. With Go modules, we can pull the package using go get:

go get -u -v github.com/BurntSushi/[email protected]

Here is an example parsing a TOML-formatted string with Go:

package main

import (
        "fmt"

        "github.com/BurntSushi/toml"
)

type Profile struct {
        Data Data `toml:"profile"`
}

type Data struct {
        FirstName string   `toml:"first_name"`
        LastName  string   `toml:"last_name"`
        Age       int      `toml:"age"`
        Sex       string   `toml:"sex"`
        Hobbies   []string `toml:"hobbies"`
}

var data string = `
[profile]
first_name = "John"
last_name = "Doe"
age = 27
sex = "M"
hobbies = ["reading", "programming", "movie-watching"]
`

func main() {
        var profile Profile
        if _, err := toml.Decode(data, &profile); err != nil {
                panic(err)
        }

        fmt.Printf("%+v\n", profile)
}

Parsing YAML

For parsing YAML-formatted data, we can the go-yaml package. We can pull the package using go get:

go get -u -v gopkg.in/yaml.v3

Here is an example parsing a YAML-formatted string with Go:

package main

import (
        "fmt"

        "gopkg.in/yaml.v3"
)

type Profile struct {
        Data struct {
                FirstName string   `yaml:"first_name"`
                LastName  string   `yaml:"last_name"`
                Age       int      `yaml:"age"`
                Sex       string   `yaml:"sex"`
                Hobbies   []string `yaml:"hobbies"`
        } `yaml:"profile"`
}

var data string = `
profile:
  first_name: John
  last_name: Doe
  age: 45
  sex: male
  hobbies:
  - reading
  - programming
  - watching movies
`

func main() {
        var profile Profile
        if err := yaml.Unmarshal([]byte(data), &profile); err != nil {
                panic(err)
        }

        fmt.Printf("%+v\n", profile)
}

Parsing XML

For parsing XML-formatted data, we can use Go’s standard library package encoding/xml.

Here is an example parsing a XML-formatted string with Go:

package main

import (
	"encoding/xml"
	"fmt"
)

type Profile struct {
	XMLName   xml.Name `xml:"profile"`
	FirstName string   `xml:"firstName"`
	LastName  string   `xml:"lastName"`
	Age       int      `xml:"age"`
	Sex       string   `xml:"sex"`
	Hobbies   struct {
		XMLName xml.Name `xml:"hobbies"`
		Hobby   []string `xml:"hobby"`
	} `xml:"hobbies"`
}

var data string = `
<profile>
	<firstName>John</firstName>
	<lastName>Doe</lastName>
	<age>30</age>
	<sex>M</sex>
	<hobbies>
		<hobby>reading</hobby>
		<hobby>reading</hobby>
		<hobby>reading</hobby>
	</hobbies>
</profile>
`

func main() {
	var profile Profile
	if err := xml.Unmarshal([]byte(data), &profile); err != nil {
		panic(err)
	}

	fmt.Printf("%+v\n", profile)
}

Parsing CSV

For parsing CSV-formatted data, we can use Go’s standard library package encoding/csv.

Here is an example parsing a CSV-formatted string with Go:

package main

import (
        "encoding/csv"
        "fmt"
        "io"
        "strings"
)

var data string = `ProfileID,FirstName,LastName,Age,Sex,Hobbies
1,John,Doe,25,M,reading programming movies-watching
2,Louise,Belcher,15,F,video-games travelling
3,Jack,Reacher,34,M,shooting-range reading hiking
`

func main() {
        r := csv.NewReader(strings.NewReader(data))

        //ignore first line
        _, err := r.Read()
        if err != nil {
                fmt.Println(err.Error())
        }

        for {
                row, err := r.Read()
                if err == io.EOF {
                        break
                }
                if err != nil {
                        fmt.Println(err.Error())
                        break
                }
                for col := range row {
                        fmt.Printf("%s\n", row[col])
                }
        }
}

And there you have it - 5 of the most common data formats parsed using Go. Hope you learned something while reading this blog! Happy Hacking!

EOF

If you enjoyed reading this blog and learned something, keep an eye out for more of my posts and maybe consider following me on GitHub, where I work on cybersecurity projects. And if you are feeling really generous, consider buying me a coffee!

References

comments powered by Disqus