Zog is a schema builder for runtime value parsing and validation. Define a schema, transform a value to match, assert the shape of an existing value, or both. Zog schemas are extremely expressive and allow modeling complex, interdependent validations, or value transformations.
Killer Features:
- Concise yet expressive schema interface, equipped to model simple to complex data models
- Zod-like API, use method chaining to build schemas in a typesafe manner
- Extensible: add your own validators, schemas and data providers
- Rich error details, make debugging a breeze
- Almost no reflection when using primitive types
- Built-in coercion support for most types
- Zero dependencies!
- Three Helper Packages
- zenv: parse environment variables
- zhttp: parse http forms & query params
- i18n: Opinionated solution to good i18n zog errors
API Stability:
- I will consider the API stable when we reach v1.0.0
- However, I believe very little API changes will happen from the current implementation. The APIs are are most likely to change are the data providers (please don't make your own if possible use the helpers whose APIs will not change meaningfully) and the ParseCtx most other APIs should remain the same
- Zog will not respect semver until v1.0.0 is released. Expect breaking changes (mainly in non basic apis) until then.
1 Install
go get github.com/Oudwins/zog
2 Create a schema & a struct
import (
z "github.com/Oudwins/zog"
)
var nameSchema = z.Struct(z.Schema{
// its very important that schema keys like "name" match the struct field name NOT the input data
"name": z.String().Min(3, z.Message("Override default message")).Max(10),
})
var ageSchema = z.Struct(z.Schema{
"age": z.Int().GT(18).Required(z.Message("is required")),
})
// Merge the schemas creating a new schema
var schema = nameSchema.Merge(ageSchema)
type User struct {
Name string `zog:"firstname"` // tag is optional. If not set zog will check for "name" field in the input data
Age int
}
3 Parse the struct
func main() {
u := User{}
m := map[string]string{
"firstname": "", // won't return an error because fields are optional by default
"age": "30", // will get casted to int
}
errsMap := schema.Parse(m, &u)
if errsMap != nil {
// handle errors -> see Errors section
}
u.Name // ""
u.Age // 30
}
4 You can also parse individual fields
var t = time.Time
errsList := Time().Required().Parse("2020-01-01T00:00:00Z", &t)
5 And do stuff before and after parsing
var dest []string
Slice(String().Email().Required()).PreTransform(func(data any, ctx z.ParseCtx) (any, error) {
s := val.(string)
return strings.Split(s, ","), nil
}).PostTransform(func(destPtr any, ctx z.ParseCtx) error {
s := val.(*[]string)
for i, v := range s {
s[i] = strings.TrimSpace(v)
}
return nil
}).Parse("foo@bar.com,bar@foo.com", &dest) // dest = [foo@bar.com bar@foo.com]
6 Use the zhttp package to parse JSON, Forms or Query Params
err := userSchema.Parse(zhttp.Request(r), &user)
- All fields optinal by default. Same as graphql
- When parsing into structs, private fields are ignored (same as stdlib json.Unmarshal)
- The struct parser expects a
DataProvider
(although if you pass something else to the data field it will try to coerce it into aDataProvider
), which is an interface that wraps around an input like a map. This is less efficient than doing it directly but allows us to reuse the same code for all kinds of data sources (i.e json, query params, forms, etc). - Errors returned by you can be the ZogError interface or an error. If you return an error, it will be wrapped in a ZogError. ZogError is just a struct that wraps around an error and adds a message field which is is text that can be shown to the user.
- You should not depend on test execution order. They might run in parallel in the future
A WORD OF CAUTION. ZOG & PANICS Zog will never panic due to invalid input but will always panic if invalid destination is passed to the
Parse
function (i.e if the destination does not match the schema).
var schema = z.Struct(z.Schema{
"name": z.String().Required(),
})
// This struct is a valid destionation for the schema
type User struct {
Name string
Age int // age will be ignored since it is not a field in the schema
}
// this struct is not a valid destination for the schema. It is missing the name field. This will cause a panic even if the input data is map[string]any{"name": "zog"}
type User2 struct {
Email string,
Age int
}
Changes from zod:
- Zog is Zod inspired, we adhere to the Zod API whenever possible but there are significant differences because:
- Go is statically typed and does not allow optional function params
- I have chosen to make Zog prioritize idiomatic Golang over the Zod API. Meaning some of the schemas & tests (validation rules) have changed names,
z.Array()
isz.Slice()
,z.String().StartsWith()
isz.String().HasPrefix
(to follow the std lib). Etc. - When I felt like a Zod method name would be confusing for Golang devs I changed it
- Some other changes:
- The refine method for providing a custom validation function is renamed to
schema.Test()
- schemas are optional by default (in zod they are required)
- The
z.Enum()
type from zod is removed in favor ofz.String().OneOf()
and is only supported for strings and numbers string().regex
is renamed toz.String().Match()
as that is in line with the regexp methods from the standard library (i.eregexp.Match
andregexp.MatchString()
)
- The refine method for providing a custom validation function is renamed to
Most of these things are issues we would like to address in future versions.
- Structs do not support pointers at the moment
- slices do not support pointers
- maps are not a supported schema type
- structs and slices don't support catch, and structs don't suppoort default values
- Validations and parsing cannot be run separately
- It is not recommended to use very deeply nested schemas since that requires a lot of reflection and can have a negative impact on performance
For convenience zog provides three helper packages:
zhttp: helps parse http requests
import (
z "github.com/Oudwins/zog"
"github.com/Oudwins/zog/zhttp"
)
var userSchema = z.Struct(z.Schema{
"name": z.String().Required(),
"age": z.Int().Required().GT(18),
})
func handlePostRequest(w http.ResponseWriter, r *http.Request) {
var user := struct {
Name string
Age int
}
// if using json (i.e json Content-Type header):
errs := userSchema.Parse(zhttp.Request(r), &user)
// if using form data (i.e Content-Type header = application/x-www-form-urlencoded)
errs := userSchema.Parse(zhttp.Request(r), &user)
// if using query params (i.e no http Content-Type header)
errs := userSchema.Parse(zhttp.Request(r), &user)
if errs != nil {
}
user.Name // defined
user.Age // defined
}
zenv: helps validate environment variables
import (
z "github.com/Oudwins/zog"
"github.com/Oudwins/zog/zenv"
)
var envSchema = z.Struct(z.Schema{
"PORT": z.Int().GT(1000).LT(65535).Default(3000),
"Db": z.Struct(z.Schema{
"Host": z.String().Default("localhost"),
"User": z.String().Default("root"),
"Pass": z.String().Default("root"),
}),
})
var Env = struct {
PORT int // zog will automatically coerce the PORT env to an int
Db struct {
Host string `zog:"DB_HOST"` // we specify the zog tag to tell zog to parse the field from the DB_HOST environment variable
User string `zog:"DB_USER"`
Pass string `zog:"DB_PASS"`
}
}{}
// Init our typesafe env vars, panic if any envs are missing
func Init() {
errs := envSchema.Parse(zenv.NewDataProvider(), &Env)
if errs != nil {
log.Fatal(errs)
}
}
// if you want to always panic on error
var Env = parse()
func Parse() env {
var e env
errs := envSchema.Parse(zenv.NewDataProvider(), &e)
if errs != nil {
fmt.Println("FAILURE TO PARSE ENV VARIABLES")
log.Fatal(z.Errors.SanitizeMap(errs))
}
return e
}
zi18n: helps with having error messages in multiple languages There are two use cases for zi18n:
- You just want to change the language of your error messages
- You want to support having errors in multiple languages because your app is in 2+ languages
USE CASE 1:
// import one of our supported languages
import (
"github.com/Oudwins/zog/i18n/es"
"github.com/Oudwins/zog/i18n/en"
"github.com/Oudwins/zog/conf" // import the zog configuration packag
)
// override the default error map
conf.DefaultErrMsgMap = es.Map // now all errors will be in spanish!
USE CASE 2:
// Somewhere when you start your app
import (
"github.com/Oudwins/zog/i18n" // import the i18n library
"github.com/Oudwins/zog/i18n/es" // import any of the supported language maps or build your own
"github.com/Oudwins/zog/i18n/en"
)
i18n.SetLanguagesErrsMap(map[string]i18n.LangMap{
"es": es.Map,
"en": en.Map,
},
"es", // default language
i18n.WithLangKey("langKey"), // (optional) default lang key is "lang"
)
// Now when we parse
schema.Parse(data, &dest, z.WithCtxValue("langKey", "es")) // get spanish errors
schema.Parse(data, &dest, z.WithCtxValue("langKey", "en")) // get english errors
schema.Parse(data, &dest) // get default lang errors (spanish in this case)
Zog uses a ParseCtx
to pass around information related to a specific schema.Parse()
call. Currently use of the parse context is quite limited but it will be expanded upon in the future.
uses
- Pass custom data to custom functions
Here is an example with a pretransform
nameSchema := z.String().Min(3).PreTransform(func(data any, ctx z.ParseCtx) (any, error) {
char := ctx.Get("split_by")
return strings.Split(data.(string), char), nil
})
nameSchema.Parse("Michael Jackson", &dest, z.WithCtxValue("split_by", " "))
- Change the error formatter
nameSchema := z.String().Min(3)
nameSchema.Parse(data, &dest, z.WithErrFormatter(MyCustomErrorMessageFormatter))
Zog creates its own error interface called ZogError
which also implements the error interface.
type ZogError interface {
Code() ZogErrCode // unique error code
Value() any // the value that caused the error
Dtype() ZogType // type of the destionation
Params() map[string]any // params for the error as defined by
Message() string
SetMessage(string)
Error() string // returns the string representation of the ZogError. The same as String(). Printf will not print correctly otherwise.
String() string // returns the string representation of the ZogError. Example: "ZogError{Code: '', ...}"
Unwrap() error // returns the wrapped error.
}
This is what will be returned by the Parse
function. To be precise:
- Primitive types will return a list of
ZogError
instances. - Complex types will return a map of
ZogError
instances. Which uses the field path as the key & the list of errors as the value.
For example:
// ! WARNING EXAMPLES HERE ARE SHOWN AS IF ZogError IS A STRUCT FOR EASIER READING BUT IT IS AN INTERFACE
errList := z.String().Min(5).Parse("foo", &dest) // can return []z.ZogError{z.ZogError{Message: "min length is 5"}} or nil
errMap := z.Struct(z.Schema{"name": z.String().Min(5)}).Parse(data, &dest) // can return map[string][]z.ZogError{"name": []z.ZogError{{Message: "min length is 5"}}} or nil
// Slice of 2 strings with min length of 5
errsMap2 := z.Slice(z.String().Min(5)).Len(2).Parse(data, &dest) // can return map[string][]z.ZogError{"$root": []z.ZogError{{Message: "slice length is not 2"}, "[0]": []z.ZogError{{Message: "min length is 5"}}}} or nil
Additionally, z.ZogErrMap
will use the field path as the key. Meaning
// ! WARNING EXAMPLES HERE ARE SHOWN AS IF ZogError IS A STRUCT FOR EASIER READING BUT IT IS AN INTERFACE
errsMap := z.Struct(z.Schema{"inner": z.Struct(z.Schema{"name": z.String().Min(5)}), "slice": z.Slice(z.String().Min(5))}).Parse(data, &dest)
errsMap["inner.name"] // will return []z.ZogError{{Message: "min length is 5"}}
errsMap["slice[0]"] // will return []z.ZogError{{Message: "min length is 5"}}
$root
& $first
are reserved keys for both Struct & Slice validation, they are used for root level errors and for the first error found in a schema, for xample:
errsMap := z.Slice(z.String()).Min(2).Parse(data, &dest)
errsMap["$root"] // will return []z.ZogError{{Message: "slice length is not 2"}}
errsMap["$first"] || errsMap.First() // (equivalent) will return the same in this case []z.ZogError{{Message: "slice length is not 2"}}
If you want to return errors to the user without the possibility of exposing internal errors, you can use the Zog sanitizer functions z.Errors.SanitizeMap(errsMap)
or z.Errors.SanitizeSlice(errsSlice)
. These functions will return a map or slice of strings of the error messages (stripping out the internal error).
Please also look at the section on the i18n package above.
You have many different options for handling the formatting of error messages. Here are a few examples:
OPTION 1: Use z.Message() or z.MessageFunc() to customize the error messages
userSchema := z.Struct(z.Schema{
"name": z.String().Min(3, z.Message("min length is 3")),
"age": z.Int().GT(18, z.MessageFunc(func(e z.ZogError, p z.ParseCtx) {
e.SetMessage("age must be greater than 18")
})),
})
This is the simplest option but it has some limitations, for example you cannot customize the error messages for coercion errors.
OPTION 2: Iterate over the returned errors and create custom messages
errs := userSchema.Parse(data, &user)
msgs := formatZogErrors(errs)
func FormatZogErrors(errs z.ZogErrMap) map[string][]string {
// iterate over errors and create custom messages based on the error code, the params and destination type
}
OPTION 3: Override the default error formatter
Under the conf package you will find ErrorFormatter
which is a function that is in charge of formatting the error messages. You can override this function to customize the error messages for all zog schemas.
You can also override the DefaultErrMsgMap
which is a map[zogType][ErrCode]
or just specific keys to customize the error messages for specific zog types or codes.
For more information on this see the overriding defaults section
OPTION 4: Override the error formatter for a specific parsing execution
userSchema.Parse(data, &user, z.WithErrFormatter(conf.ErrorFormatter)) // set the error formatter for this parse to be the default formatter. This doesn't make a lot of sense but you can pass a custom one
What do I recommend for i18n? If you are serius about i18n and want to translate all your messages and return meaningful error messages I recommend you choose option 3 and do something like this:
zconf.ErrorFormatter = func(e p.ZogError, p z.ParseCtx) {
lang := p.Get("lang").(string)
// generate a different error message based on the language
}
// then when you parse a schema you can pass the language
userSchema.Parse(data, &user, z.WithCtxValue("lang", "en"))
(In this example I'll use go templ, but you can use any template engine)
Example use case: simplified Signup form validation Imagine our handler looks like this:
type SignupFormData struct {
Email string
Password string
}
schema := z.Struct(z.Schema{
"email": z.String().Email().Required(),
"password": z.String().Min(8).Required(),
})
func handleSignup(w http.ResponseWriter, r *http.Request) {
var signupFormData = SignupFormData{}
errs := schema.Parse(zhttp.NewRequestDataProvider(r), &signupFormData)
if errs != nil {
www.Render(signupFormTempl(&signupFormData, errs))
}
// handle successful signup
}
templ signupFormTempl(data *SignupFormData, errs z.ZogErrMap) {
<input type="text" name="email" value={data.Email}>
// display only the first error
if e, ok := errs["email"]; ok {
<p class="error">{e[0].Message()}</p>
}
<input type="text" name="password" value={data.Password}>
// display only the first error
if e, ok := errs["password"]; ok {
<p class="error">{e[0].Message()}</p>
}
}
PS: If you are using go html templates & tailwindcss you might be interesting in my port of tailwind-merge to go.
Zog providers a helper function called z.Errors.SanitizeMap(errsMap)
that will return a map of strings of the error messages (stripping out the internal error). So, if you do not mind sending errors to your users in the same form zog returns them, you can do something like this:
errs := schema.Parse(zhttp.Request(r), &userFormData)
if errs != nil {
sanitized := z.Errors.SanitizeMap(errs)
// sanitize will be map[string][]string
// for example:
// {"name": []string{"min length is 5", "max length is 10"}, "email": []string{"is not a valid email"}}
// ... marshal sanitized to json and send to the user
}
Zog supports two main ways of creating custom tests a simple function and a struct.
OPTION 1: Full Test Definition
nameSchema := z.String().Test(z.Test{
ErrCode: "custom_error_code", // this is the error code for all errors constructed from this test
Params: map[string]any{"foo": "bar"}, // these will be passed to the error formatter via the constructed error
ErrFmt: func(e z.ZogError, p z.ParseCtx) // This is what z.Message() sets, its a function to format the error message
ValidateFunc: func(data any, ctx z.ParseCtx) bool{...}, // this is the validate func
})
Beware that if using the default error formatter you will get the default error message for any error code that is not defined see error formatting. You can define the ErrFmt function or use z.Message() like so:
nameSchema := z.String().Test(MyCustomTest(), z.Message("hello world") ) // MyCustomTest() returns the z.Test so it can be reused
OPTION 2: use z.TestFunc Helper
// here I will set a custom message so I don't have to add my message to the error messages map
nameSchema := z.String().Test(z.TestFunc("error_code", func (data any, ctx z.ParseCtx)bool{...}), z.Message("custom msg"))
These are methods that can be used on most types of schemas
// marks the schema as required. Remember fields are optional by default
schema.Required(z.Message("message or function"))
schema.Optional() // marks the schema as optional (default)
// optional & required are mutually exclusive
schema.Required().Optional() // marks the schema as optional
schema.Default(val) // sets the default value. See Zog execution flow
schema.Catch(val) // sets the catch value. A value to use if the validation fails. See Zog execution flow
schema.PreTransform(func(data any, ctx z.ParseCtx) (any, error) {}) // transforms the value before validation. returned value will override the input value. See Zog execution flow. errors returned from this will be wrapped in a ZogError under the "custom" code (You may also return a ZogError from this function)
schema.PostTransform(func(destPtr any, ctx z.ParseCtx) error {}) // transforms the value after validation. Receives a pointer to the destination value. errors returned from this will be wrapped in a ZogError under the "custom" code (you may also return a ZogError from this function)
// Primtives. Calling .Parse() on these will return []ZogError
String()
Int()
Float()
Bool()
Time()
// Complex Types. Calling .Parse() on these will return map[string][]ZogError. Where the key is the field path ("user.email") & $root is the list of complex type level errors not the specific field errors
Struct(Schema{
"name": String(),
})
Slice(String())
// Validations
String().Min(5)
String().Max(10)
String().Len(5)
String().Email()
String().URL()
String().UUID()
String().Match()
String().Contains(string)
String().ContainsUpper()
String().ContainsDigit()
String().ContainsSpecial()
String().HasPrefix(string)
String().HasSuffix(string)
String().OneOf([]string{"a", "b", "c"})
// Validators
Int().GT(5)
Float().GTE(5)
Int().LT(5)
Float().LTE(5)
Int().EQ(5)
Float().OneOf([]float64{1.0, 2.0, 3.0})
Bool().True()
Bool().False()
Use Time to validate time.Time
instances
Time().After(time.Now())
Time().Before(time.Now())
Time().Is(time.Now())
s := z.Struct(z.Schema{
"name": String().Required(),
"age": Int().Required(),
})
user := struct {
Name string `zog:"firstname"` // name will be parsed from the firstname field
Age int // since zog tag is not set, age will be parsed from the age field
}
s.Parse(map[string]any{"firstname": "hello", "age": 10}, &user)
s := Slice(String())
Slice(Int()).Min(5)
Slice(Float()).Max(5)
Slice(Bool()).Length(5)
Slice(String()).Contains("foo")
Zog uses internal functions to handle many aspects of validation & parsing. We aim to provide a simple way for you to customize the default behaviour of Zog through simple declarative code inside your project. You can find the options you can tweak & override in the conf package (github.com/Oudwins/zog/conf
).
Lets go through an example of overriding the float64
coercer function, because we want to support floats that use a comma as the decimal separator.
import (
// import the conf package
"github.com/Oudwins/zog/conf"
)
// we override the coercer function for float64
conf.Coercers.Float64 = func(data any) (any, error) {
str, ok := data.(string)
// identify the case we want to override
if !ok && strings.Contains(str, ",") {
return MyCustomFloatCoercer(str)
}
// fallback to the original function
return conf.DefaultCoercers.Float64(data)
}
There are two different things you can override from the errors configuration:
- The error formatter function
- The error messages map. Which is a
map[zogType][ErrCode]
or just specific keys to customize the error messages for specific zog types or codes.
// override the error formatter function
conf.ErrorFormatter = func(e p.ZogError, p z.ParseCtx) {
// do something with the error
...
// fallback to the default error formatter
conf.DefaultErrorFormatter(e, p)
}
// override specific error messages
// For this I recommend you import `zod/zconst` which contains zog constants
conf.DefaultErrMsgMap[zconst.TypeString]["my_custom_error_code"] = "my custom error message"
conf.DefaultErrMsgMap[zconst.TypeString][zconst.ErrCodeRequired] = "Now all required errors will get this message"
- Pretransforms
- On error all parsing and validation stops and error is returned.
- Can be caught by catch
- Default Check -> Assigns default value if the value is nil value
- Optional Check -> Stops validation if the value is nil value
- Casting -> Attempts to cast the value to the correct type
- On error all parsing and validation stops and error is returned
- Can be caught by catch
- Required check ->
- On error: aborts if the value is its nil value and returns required error.
- Can be caught by catch
- Tests -> Run all tests on the value (including required)
- On error: validation errors are added to the errors. All validation functions are run even if one of them fails.
- Can be caught by catch
- PostTransforms -> Run all postTransforms on the value.
- On error you return: aborts and adds your error to the list of errors
- Only run on valid values. Won't run if an error was created before the postTransforms
These are the things I want to add to zog before v1.0.0
- For structs & slices: support pointers
- Support for schema.Clone()
- support for catch & default for structs & slices
- Add additional tests
- Better docs
- Big thank you to @AlexanderArvidsson for being there to talk about architecture and design decisions. It helped a lot to have someone to bounce ideas off of
- Credit for all the inspiration goes to /colinhacks/zod & /jquense/yup
- Credit for the initial idea goes to anthony -> /anthdm/superkit he made a hacky version of this idea that I used as a starting point, I was never happy with it so I inspired me to rewrite it from scratch
- Credit for the zod logo goes to /colinhacks/zod
- To @anthonyGG for the idea to do a Zog like validation library for golang. I wouldn't have started working on it if it weren't for him
This project is licensed under the MIT License - see the LICENSE file for details.