Where should I define the errors in a Go project?

I am trying to standardize an architecture for my Go projects, so I have a file structure like this:

    ├── go.mod
    ├── go.sum
    ├── internal
    │   ├── domain
    │   │   ├── models
    │   │   │   └── user.go
    │   │   └── services
    │   │       └── user.go
    │   └── repositories
    │       └── mongodb
    │           ├── base.go
    │           ├── config.go
    │           └── user.go

File services/user.go:

 https://go.dev/play/p/ZYWhgHRLU5V

In the services I have business logic and in another folder called repositories I have implementations in different technologies as needed. For example I can have an implementation of UserStore in Mongo DB... or Postgres etc.... And the idea is then to have a layer on top of these services... that can be, an APIRest, a serverless etc....
I have doubts about where I should define the errors in my system. For example, a common error could be *ErrRecordNotFound*, but depending on the repository/technology, this error will be different.
I was thinking of defining the errors inside the domain folder, and then in the repositories, the errors are mapped to my domain errors.... for example if it was in mongo, my code would look like this:

    if errors.Is(err, mongo.ErrNoDocuments) {
    return nil, domain.ErrDocumentNotFound
    }
   
(or in some cases wrap the library specific error in my domain error).
This way my business logic would not depend on specific technologies, but the implementations/libraries depend on my business logic. A possible disadvantage is that my repositories code may no longer be reusable because it is “smeared” with specific business logic.

Is this approach correct? Or should I approach it differently?

I am going to explain what my experience has led me to believe works well but cannot say for sure that others will agree as there seems to me many different opinions on this subject. And that said, I would be very interested to hear what others have to say in hopes that I could learn from them, too.

First, I do not think it a good idea to standardize a directory structure in Go, at least not entirely. I think it is better to standardize a pattern of directory structures in Go. I think Go projects grow their directory structure best when the grow organically. This will hopefully make more sense as I elaborate.

Next. remember that a directory defines a package in Go. THIS IS KEY. Thus packages should (IMO) be cohesive aka " consisting of parts that fit together well and form a united whole." That means that things that are highly coupled by nature should typically live in the same directory/package, and things that are loosely coupled from one another can exist in different directories/packages.

In your example use-case I would question whether or not everything in /internal/domain should not be in its own single directory/package instead of many separate subdirectories. Even some of the aspects that you have in repositories should even be in that same directory; the aspects that are specific to your project.

And that directory/package would have a name highly SPECIFIC to your app, and in my strong opinion one that is unlikely to conflict with any other package name your will need to be using. And if you are planning to offer as a package for others to use, then a name that is unlikely to conflict with anything anyone else would publish, i.e. something that relates to your brand.

From a simplistic perspective — and by simplistic I mean that in reality it would end up being more complicated than this — I would consider starting something like instead, this where `/cmd` is someone standardized in Go when more than one executable is needed by of a project/repo (alternately having go.mod and main.go in the root still makes sense when you will only ever need one executable):

├── cmd
│ ├── acmewidgetsd
│ │ ├── go.mod
│ │ ├── go.sum
│ │ └── main.go
│ ├── acmewidgets-cli
│ │ ├── go.mod
│ │ ├── go.sum
│ │ └── main.go
├── internal
│ ├── acmewidgets
│ │ ├── user.go
│ │ ├── repository.go
│ │ └── services.go
│ ├── storage
│ │ ├── mongodb
│ │ │ └── mongodb.go
│ │ └── postgres
│ │ └── postgres.go

Personally I tend to try and design my packages to be reusable — if not for others than at least for myself — so I don't use /internal all that often and mine would look more like this (note that `/storage/` is often an empty directory and thus not a package so having a generic name here is fine):

├── cmd
│ ├── acmewidgetsd
│ │ ├── go.mod
│ │ ├── go.sum
│ │ └── main.go
│ ├── acmewidgets-cli
│ │ ├── go.mod
│ │ ├── go.sum
│ │ └── main.go
├── acmewidgets
│ ├── user.go
│ ├── repository.go
│ └── services.go
├── storage
│ ├── mongodb
│ │ └── mongodb.go
│ └── postgres
│ └── postgres.go

When developing a Go app I like to restrict myself to only creating a new package/directory when I feel like I really must create a new package/directory, when it can stand on its own independently of others. When it can have its own well-defined identity.

To illustrate I have a project I am currently working on to provide a CLI tool to manage macOS preferences, especially for when setting up a new macOS system: https://github.com/mikeschinkel/prefsctl — It is still very much a work in progress. I only mention this because it allows me to provide concrete examples rather than speak in generic terms.

`macprefs` is the core, reusable package that `/cmd/prefsctl` leverages to perform its CLI tasks. IOW, `macprefs` could also be used by a service, if I decide to add one.

Under `macprefs` there is a `kvfilters` and `prefdefaults` package that provide generic grouped, key-value filtering and preference defaults functionality, respectively, where the latter has defaults specific to each version of macOS. The fact these are subdirectories of `macprefs` is an arbitrary choice; I could have easily moved them to the root and other than the import path used Go wouldn't care either way.

Then I have ~uniquely named `stdlibex` and `sliceconv` package where the former is "standard library extensions (that have functionality I really wish where in the Go standard library) and the latter so I can have commonly called generic funcs with easy to remember named like `sliceconv.ToStrings()`, `sliceconv.ToAny()`, `sliceconv.ToPtrs()`, and `sliceconv.Func()`. These are also things I really wish were in the Go standard library, but I pulled them out of my `stdlibex` package so I could create easier to use names, e.g. vs. `stdlibex.ConvertSliceToStrings()`, etc.

Then there is `macosutils`, `logutil`, `errutil`, and `cobrautil` which are all cohesive reusable packages I could see publishing as their own repo but for now I am learning the emergent shape of them in this and prior projects. I named them generically and so I can use names like `errutil.Multierr` in my code while they are still specific to my code. BTW, the former I created to I put all cgo/Objective-C code for interacting with the macOS API, and the latter has extensions to the Cobra CLI project that requires less boilerplate to creating a complex CLI.

Funnily enough, as I wrote to describe it I realized `logutil` had project specific content, so I moved that into a `logargs` package as a subdirectory of `macprefs`.

I want to circle back and make some key points and provide some rule of thumbs. When starting a Go project:

1. Start with a very simple directory structure, maybe just this:

├─ cmd
│ ├─ acmewidgetsd
├─ acmewidgets

2. Start writing your code in your main package, e.g. `acmewidgets`

3. Then, when you feel you have enough reason to create a new package, do so. But that package should have a clear purpose, be able to stand on its own, and either be independent of your existing packages or provide some key functionalities for another existing package while needing to be separate to make naming and referencing of funcs and types less complex.

Point #3 is really important for Go because Go does not support circular dependencies, and that means packages should either be independent, or depend in only one direction.

To handle circular dependencies you need to use interfaces, which can add complexity. The other approach is to create a struct in each package that is unique to the package's needs EVEN IF you have two structs that appear to duplicate functionality. Do not be afraid to use lots of similar structs, e.g. one might be a User struct in a JSON serialization package whereas a different User struct is in your main package designed to work with Users in general. For example, my `macosutils` package has a `Preference` struct and my `macprefs` package has its own `Pref` struct because I realized that they both had slightly different needs and that to use only one would mean having to implement an interface. As such, I just have a func to copy macosutil.Preference to a macprefs.Pref thus needing to only have a one-directional dependency.

And finally, that brings me to a rule of thumb and litmus test: Pay close attention to your import statements. Strive for the following:

1. Minimize the number of import statements per file,
2. Minimize or better eliminate the need for import aliases, and
3. Consider isolating import statements from each external package to just one package in your project

Elaborating:
1. Having lots of import statements in your Go files is (IMO) a code smell and indication that you should reconsider your directory layout/package structure.

2. Needing to use aliases in your project to import your own project's packages — where in my experience aliases are never consistently named unless only one developer is on a project leading to lots of dissonance when reading other code — means your own packages should probably be renamed.

3. Isolating external references into a package is a sure way to improve package cohesiveness and often leads to better package names. As two examples, I refactored all cgo/Objective-C macOS API code out of `macosprefs` and into `macosutils` as I saw I could consolidate all macOS API code into a single package and while I was writing this I realized that `cobrautil` was a better name than `cliutil`.

I hope this helps.

-Mike
> --
> You received this message because you are subscribed to the Google Groups "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/85b7fa7d-1c01-4281-be20-4837f6019743n%40googlegroups.com.