DEV Community: Sam Newby

Path-Based Reverse Proxying with Caddy

Sam Newby — Wed, 30 Oct 2024 20:12:38 +0000

Caddy is a great web server built with Go and can be used for a multitude of things. Here at Vizalo we use it on nearly all of our servers that power our network.

This guide explains how to set up Caddy as a reverse proxy that routes traffic to different backend services based on URL paths. This is useful when you have multiple services running on different ports and want to expose them under a single domain.

Basic Setup

Create a Caddyfile in your project directory:

example.com {
    handle /api/* {
        reverse_proxy localhost:3000
    }

    handle /admin/* {
        reverse_proxy localhost:8080
    }

    handle /* {
        reverse_proxy localhost:5000
    }
}

This configuration will:

Send /api/* requests to a service running on port 3000
Send /admin/* requests to a service running on port 8080
Send all other requests to port 5000

A More Complete Example

Here's a more practical example that includes common settings you might need:

example.com {
    # API Service
    handle /api/* {
        reverse_proxy localhost:3000 {
            header_up Host {upstream_hostport}
            header_up X-Real-IP {remote_host}
            header_up X-Forwarded-For {remote_host}
        }
    }

    # Admin Dashboard
    handle /admin/* {
        reverse_proxy localhost:8080 {
            # Health checks
            health_uri /health
            health_interval 30s

            # Timeout settings
            timeout 30s
        }
    }

    # Frontend App
    handle /* {
        reverse_proxy localhost:5000 {
            # Load balancing
            lb_policy round_robin
            lb_try_duration 30s
        }

        # Enable compression
        encode gzip
    }

    # Global options
    log {
        output file /var/log/caddy/access.log
        format json
    }
}

Running Multiple Backend Services

For testing, you might run these simple backend services:

# API Service (Node.js/Express)
node api.js        # Runs on :3000

# Admin Dashboard (Go)
go run admin.go    # Runs on :8080

# Frontend (React)
npm start         # Runs on :5000

Verifying the Setup

Test your configuration:

# Test API endpoint
curl example.com/api/users

# Test admin endpoint
curl example.com/admin/dashboard

# Test frontend
curl example.com

Common Patterns

Stripping Path Prefixes

If your backend service doesn't expect the /api prefix:

handle /api/* {
    uri strip_prefix /api
    reverse_proxy localhost:3000
}

Adding Headers

Add authentication headers or API keys:

handle /api/* {
    reverse_proxy localhost:3000 {
        header_up X-API-Key {env.API_KEY}
    }
}

That's it. The beauty of Caddy is that it handles HTTPS certificates automatically and has sensible defaults for most settings. Most of the "extra" configurations shown above are only needed for specific use cases.

Building a simple REST API with Go

Sam Newby — Sun, 27 Oct 2024 21:17:54 +0000

Go is a great language for systems programming but it also shines on the web, especially when building REST APIs. This guide walks through creating a simple REST API using Go's standard library. We'll build an API to manage a list of servers, letting us add, remove, and view server records. We're also going to use Go 1.22 new router enhancements for method matching which allows us to have cleaner routes and handlers.

This guide assumes you have a basic understanding of Go and it installed on your machine.

Setting Up

Create a new directory for your project and initialize a Go module:

mkdir server-api
cd server-api
go mod init server-api

The Code

Create a file called main.go. We'll use Go's standard http package - it has everything we need for a basic API.

package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"
)

First, let's define what a server looks like. We'll keep it simple - just an ID, name, IP address, and region:

type Server struct {
    ID     string `json:"id"`
    Name   string `json:"name"`
    IP     string `json:"ip"`
    Region string `json:"region`
}

We'll store our servers in memory using a slice. In a real application, you'd probably use a database:

var servers = []Server{
    {ID: "srv1", Name: "prod-1", IP: "10.0.1.1", Region: "us-east"},
    {ID: "srv2", Name: "prod-2", IP: "10.0.1.2", Region: "eu-west"},
}

Creating the router

Next, we'll set up our routes. Go 1.22 introduced a new routing syntax that makes this pretty straightforward:

func main() {
    mux := http.NewServeMux()

    mux.HandleFunc("GET /servers", listServers)
    mux.HandleFunc("GET /servers/{id}", showServer)
    mux.HandleFunc("POST /servers", createServer)
    mux.HandleFunc("DELETE /servers/{id}", deleteServer)

    fmt.Println("Server starting on port 8080...")
    log.Fatal(http.ListenAndServe(":8080", mux))
}

Handlers

Now let's implement each handler. First, listing all servers:

func listServers(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(servers)
}

Getting a single server by ID:

func showServer(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    id := r.PathValue("id")

    for _, server := range servers {
        if server.ID == id {
            json.NewEncoder(w).Encode(server)
            return
        }
    }

    http.Error(w, "Server not found", http.StatusNotFound)
}

Creating a new server:

func createServer(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")

    var server Server
    if err := json.NewDecoder(r.Body).Decode(&server); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

    servers = append(servers, server)
    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(server)
}

And finally, deleting a server:

func deleteServer(w http.ResponseWriter, r *http.Request) {
    id := r.PathValue("id")

    for i, server := range servers {
        if server.ID == id {
            servers = append(servers[:i], servers[i+1:]...)
            w.WriteHeader(http.StatusNoContent)
            return
        }
    }

    http.Error(w, "Server not found", http.StatusNotFound)
}

Using the API

Once you've got the code in place, run it:

go run main.go

Here's how to interact with each endpoint using cURL:

List all servers:

curl localhost:8080/servers

Get a specific server:

curl localhost:8080/servers/srv1

Add a server:

curl -X POST localhost:8080/servers   -H "Content-Type: application/json";   -d '{"id":"srv3","name":"prod-3","ip":"10.0.1.3","region":"ap-south"}';

Delete a server:

curl -X DELETE localhost:8080/servers/srv1

What's Next?

This is a basic API, but there's a lot you could add:

Input validation
Proper error handling
Persisting servers with a database such as PostgreSQL
Authentication
Request logging
Unit tests

The standard library is surprisingly capable for building APIs. While there are more full-featured frameworks available, starting with the standard library helps you understand the basics without any magic happening behind the scenes.

Getting started with h3 by unjs

Sam Newby — Mon, 21 Oct 2024 12:19:54 +0000

Modern Express.js

h3 is a modern alternative to express.js built by the amazing unjs team. It's built with TypeScript so you get that typed magic straight out of the box. But enough of talking let's build an API with h3.

Install

We're going to use pnpm as our package manager but you could use npm or yarn if you like. Let's start by scaffolding a new project with:

pnpm init

That will create a base package.json for us. Now, to install h3, run:

pnpm i h3

Let's also create a base file structure, create a new directory and an app.ts: mkdir src && touch src/app.ts. And finally let's add a script to run our server using listhen which is another amazing package by the unjs team. Add the following to the scripts object in package.json:

npx --yes listhen -w --open ./src/app.ts

Building the server

Now we can create our base server in the app.ts file:

// Import h3 as npm dependency
import { createApp, createRouter, defineEventHandler } from h3";; 

// Create an app instance
export const app = createApp(); 

// Create a new router and register it in app 
const router = createRouter(); app.use(router); 

// Add a new route that matches GET requests to / path 
router.get( 
        "/", defineEventHandler((event) =>; { 
                return { message: "Tadaa!"; }; 
        }), 
);

Let's break down what we've just created:

We're importing a number of exports from the h3 package
We're going to create an app using the createApp() function we imported in step 1
We're then creating a router using the createRouter() function we imported in step 1, and then telling our app we created in step 2 to use that router
We're then creating our first route which matches a GET request to the path /, if any request matches that criteria the server will respond with the value returned from the defineEventHandler, which in this case the function returns an object which contains a message. h3 will automatically convert that to JSON for us
Now if we run pnpm run dev listhen will automatically start our server and serve it at port 3000 We can now hit our server and see that amazing Tadaa! response!

Wrapping up

We've built a basic server with h3, get in there! But there is still so much we can do such as using adapters to make it into a Node.js server, or creating middleware or building session based auth with h3 but we will discuss in future content.

Getting Started with Nixpacks

Sam Newby — Tue, 15 Oct 2024 21:37:00 +0000

Building images without a Dockerfile with Nixpacks

Nixpacks is a tool created by the Railway team which allows you to build OCI compliant images without having to write a Dockerfile. Essentially you can run Nixpacks in your repository, it will automatically figure out how to build your repo into an image and then build that image that you can then run with docker run. Pretty amazing right?

Installation

Installing Nixpacks is pretty easy, you can just run:

curl -sSL https://nixpacks.com/install.sh | bash

on any platform and it will install.

Building our first image

Right now that we've got Nixpacks installed with that incredibly easy install, we can build our first image. If you don't have a repo to hand then you can clone this demo repo built by the Vizalo team to showcase our new managed application service. Clone the repo to your machine by running:

git clone git@github.com:vizalo/app-0.git

Now we can run nixpacks build . --name app-0 to let nixpacks build the image. Let's break this command down:

We're running the nixpacks build command
We're telling nixpacks to use this directory by using .
We're then giving the image a name which is easy for us to remember with the --name flag

Give it a minute or two and we will have an image which we can run with docker run -it app-0

But wait! What's that nixpacks.toml file? Well that is a configuration file that we can add to our repo to provide nixpacks with some custom instructions. Why do we need it? Well our repo doesn't actually have a start command for production, so nixpacks won't be able to define a command to start our code in the image. So we manually tell it to run node ./dist/index.mjs to start our code. If your repo has a command to start in production, maybe you've defined a npm run start command then you should be good and nixpacks will automatically use that.

Finishing up

This has been a whirlwind introduction to nixpacks, and to cleanup our machine we can remove Nixpacks by running rm "$(command -v 'nixpacks')".

How to hash a password in Go

Sam Newby — Mon, 08 Nov 2021 21:32:23 +0000

When you're storing a password in a database the worst thing that you could do as a software developer is store that password in plain text. We have to store and treat sensitive data as what it is, sensitive!

Luckily, in Go we can do this really easily using the Bcrypt package. Once, we retrieved the package with go get we can use it straight away. Hashing a can be done with one function, yes I'll repeat that, one function.

Let's start by importing the package in the file we want to use it in:

import (
    "golang.org/x/crypto/bcrypt"
)

Now we have our package we can hash our package using the GenerateFromPassword() function like so:

hashed, _ := bcrypt.GenerateFromPassword([]byte(u.Password), 8)

u.Password = string(hashed)

In our example we have a hypothetical user struct which has a plain text password assigned to the Password field, we was to use that to generate our password and then reassign our hashed password back to the struct field.

So we pass the password as a byte array to the GenerateFromPassword() function firstly and then pass the cost, which in this example we have set to an arbitrary value of 8. We get back the hashed password as a byte array and a possible error, which for the example we have ignored with the underscore. Finally, we convert the hashed password to a string and reassign it back to the password field on the user struct. Really simple, really nice, and a perfect solution for storing user passwords in a database.

Thank you for reading! I'll be back with more Go tutorials in the future.

Using Chi as a router for Go APIs

Sam Newby — Sun, 17 Oct 2021 18:55:45 +0000

The Go net/http package is great, it provides Go developers with a fantastic build block for building super fast APIs with Go.

However, it doesn't have a built in router that most developers would be used to. Luckily, a lot of community members have built really simple routers that can be used with the standard net/http package. There is a lot of options out there, but my personal favourite is Chi

Disclaimer: I'm not going to go into a lot of detail why I personally prefer Chi as I believe that those who are new to Go should be allowed to freely make there own choice on their favourite router without influence from others.

Right, let's take a look at how we can build a small API using Chi as our router.

Firstly, let's install Chi

go get -u github.com/go-chi/chi/v5

Then, we'll define our main.go and start using Chi straight away. We're going to declare a new router and create an initial route to receive a GET request and then start our http server. That will look like this:

package main

import (
    "net/http"
    "github.com/go-chi/chi/v5"
)

func main() {
    router := chi.NewRouter()

    r.Get("/", testHandler)

    http.ListenAndServe(":8000", r)
}

To recap this we're importing chi and net/http then defining a main function and creating a new Chi router and assigning it a new variable called router. We're then defining a new route at the path / that will receive a GET request and will run the testHandler function, we haven't defined that handler and we won't to keep this tutorial short. We then start the http server on port 8000 and pass the Chi router.

This has just touched the surface on Chi and it has a lot more features to offer when building an API and I plan to write more on the power of Chi in the future.

If you'd like to stay up to date with my content, make sure to follow me on Twitter

How to create Go files from Stub files like Laravel does

Sam Newby — Sat, 25 Sep 2021 12:26:46 +0000

I love Laravel, it's great for getting a full-stack app off the ground in groundbreaking time. It also has a great feature called stub files that provide the boilerplate for files like models and controllers and these stub files can actually be customised for each project. Meaning everytime you create a model you could customise your app to use your custom stub file as it's boilerplate.

How can we recreate this functionality in Go? The answer is quite simple and we can do it in 31 lines of code. I've even built a package that allows this functionality to be used in other Go applications, if you want to check it out you can find it at https://github.com/NWBY/go-stubs and if you like it please think about starring it.

Defining our stub

Anyway back to the functionality. We want to be able to define a file that we will call a stub and at it's core it is a Go text template and then use that stub to create a new file with the variables filled in correctly.

Firstly, let's create a stubs directory in our Go project and define a stub called model.go.stub and in that file we're going to add the following code:

package models

type {{.Model}} struct {

}

Let's break it down, the first line we're defining what package our file would end up in. Yes it's in the stubs directory which would mean the package should be stubs however, when we turn this stub into a real Go file it will be placed in a directory called models so the package will be models. Next, we're defining a new struct to represent our Model, the {{.Model}} part is a variable in the Go templating engine, meaning whatever value we pass to the variable will be rendered in the new file.

Defining our StubDetails type

We've got our stub, now we need to use it to create a new Go file. To start, we're going to define a new struct to represent a stub in our main.go, this struct will receive specific data that will help us to create our file.

type StubDetails struct {
    Name, FileName, Destination string
    Values                      map[string]string
}

Again, let's break it down, we're defining fields Name, FileName and Destination which are all type string, and a Values field that is a map with key value types of string.

Name is the path and name of the stub we want to use
FileName is the name of our new file
Destination is the path that our new file will be created at
Values is the data (variables) that we want to pass to our stub

Creating files from the stub

Now we have a simple way to represent our stub files and all the other required information using our StubDetails struct we can move onto creating our .go files.

We're going to do all the work inside our main function of our main.go as there isn't a huge amount to this so we don't need to abstract it into another package. So let's take a look at how to do it:

func main() {
    stub := StubDetails{
        Name:        "./stubs/model.go.stub",
        FileName:    "user.go",
        Destination: "./",
        Values: map[string]string{
            "Model": "User",
        },
    }

    contentsBuff, err := os.ReadFile(s.Name)
    if err != nil {
        log.Fatalf("Unable to read file: %s", s.Name)
    }

    f, err := os.OpenFile(s.Destination+s.FileName, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0755)
    if err != nil {
        log.Fatalf("Unable to open file: %s", s.FileName)
    }
    defer f.Close()

    template, err := template.New(s.FileName).Parse(string(contentsBuff))
    if err != nil {
        log.Fatalf("Unable to parse template: %s", s.Name)
    }
    template.Execute(f, s.Values)
}

So this can look a bit complicated at first but I promise it's not.

Firstly, we're creating a new struct literal of our StubDetails struct and passing it the values to it's field. The Name contains the path to the stub we want to use. Then we're saying what we want our new file to be called and saying where we want it to be created. The finally, we're passing a map of values which will be passed to the stub.
We're then reading the stub file into a byte array and then checking if it errors.
Then we're opening our new file at the destination path and passing it the new file name and setting some permissions. os.OpenFile() will create the file if it doesn't exist. We're also checking for any errors again and then deferring closing the file.
We then move onto using Go's template package to create a new template and parse the contents of the stub file. Since the stub file is a byte array we need to convert it to a string using the string() function. Again, we're checking for any errors.
Finally, we're executing the template, passing in the file which will write the contents to our new file and the Values field from our struct which will automatically insert our variables into the template.

That's it, we're done. I hope you've found this article useful. If you have and would like to stay up to date with my latest posts then why not follow me on Twitter.

Taking a different approach to building in public

Sam Newby — Wed, 28 Apr 2021 18:28:50 +0000

Logidev is a new projected focused on learning, education and content creation all centered around a new cryptocurrency.

The landing page I've built is intentionally minimal and basic and doesn't give to much away as I hope it will intrigue users. Additionally, I want to share every step of the process and update the landing page as I made progress on the application. Sort of like the landing page becomes more informative about the project as the project progresses.

Some people may love this approach, some will hate it, and some I guess will just find it boring. But either way I would really appreciate any feedback on the idea and if you would like to subscribe to updates on Logidev then you can subscribe at: https://logidev.app

Lucid is live!

Sam Newby — Mon, 26 Apr 2021 11:56:02 +0000

Last week I launched Lucid, and launched it on Product Hunt and it received 21 upvotes. 21 upvotes isn't a lot, I know that. But when I launched Lucid on Product Hunt I didn't even expect 1. To get 21 after a week and to have a couple of users actively engage with a comment is unbelievable to me.

Lucid is a micro-CRM that helps small business owners manage their business leads, manage their customers and grow their business by removing a lot of the bloat that modern CRMs provide and going back to basics. And removes the drama of having to configure Mail records to use it, whereas most CRMs require Mail records to be set up. I have so many ideas and features that I will be working on and adding to the product over the next couple of weeks and months, so keep an eye out for updates!

If you have a spare minute and would like to show your support for Lucid, I would really appreciate any support on Product Hunt

Enums in PHP8.1

Sam Newby — Mon, 15 Feb 2021 11:36:15 +0000

At University I had used Java quite a bit and found Enums a great way to express data that I knew wasn't going to change and was super useful in many types of applications. However, when I got started with PHP and it didn't have Enums I was slightly surprised. However, that is going to change with PHP8.1 as Enums are coming to PHP!

To use Enums in our applications we will be able to do something like this:

enum Case {
  case Open;
  case Active;
  case Closed;
}

And then in a class that uses the Case Enum we would be able to do something like this: $caseStatus = Case::Open;

I know for sure I will be using Enums in my applications when I can.

Hope you enjoyed, catch you later!

Why I'm giving Python and Django another chance

Sam Newby — Sun, 20 Sep 2020 15:38:34 +0000

I work mostly with PHP and Laravel at work and in my own projects. I love PHP and Laravel, they both get a lot of criticism from the wider developer community at times, but I feel like I want to learn something new and give another language and framework a chance. By no means am I downing tools with PHP and Laravel, I will still very much use them.

Whilst studying at University, I wrote some Python. However, this was only to implement Algorithms and Data Structures and to be honest I think this put me off Python. At the time I just didn't enjoy writing the code that I was required to and subconsciously I think I developed some sort of dislike towards Python and I started to develop this bias that I would dislike any Python frameworks too. Now that I have completed my course at University I've decided that giving Python another chance and diving into Django might be the challenge I'm looking for.

So, where do I start? Well, I already have a grasp of the basics of Python and I know that really I need to use something like virtualenv when developing projects. But, I don't really have any knowledge of Django and how it handles things such as databases, routing, and views.

Sometimes I think I dwell too much on what I should learn next, but in this scenario I think I have an idea of what path I will take.
Firstly, I'm going to spend a bit of time brushing up on my Python basics and learning some more advanced Python core features such as how Python handles object oriented development and using tools like virtualenv.
Secondly, I think I'm going to move onto getting a basic grasp of Django and getting a simple Django project up and running locally, maybe even using Docker if I can.
Finally, I'll probably look into building some sort of functional app with Django and seeing how I can deploy it to AWS Lambda, maybe using a tool like Apex Up or Zappa.

Thank you for reading, I hope you enjoyed. Catch you later!

Disabling services in Laravel Vapor

Sam Newby — Wed, 06 May 2020 18:04:41 +0000

I’m a huge fan of serverless and Laravel and then Taylor Otwell came along with Laravel Vapor. Allowing us to deploy our Laravel apps to serverless infrastructure on AWS.

We deploy a production and staging environment for each project. Really we don’t want to use Queues, Command Scheduler or Mail in our staging environment. We only want to use these features for our production environment. Vapor makes it really simple for us to do this by adding some configuration options to our Vapor config file: vapor.yml.

id: 19
name: serverless-rules
environments:
    staging:
        queues: false
        mail: false
        scheduler: false

That’s all we need to is set these three config options to false in our staging environment and then it will remove Laravel’s Queue, scheduler and Mail sending functionality from AWS.

That’s all for this post, I use Vapor to deploy all my projects nowadays. If you would like to find out more about it then you can find it here: Vapor. Enjoy.